AI and Deep Learning in Biometric Security Artificial Intelligence (AI): Elementary to Advanced Practices Series Editors: Vijender Kumar Solanki, Zhongyu (Joan) Lu, and Valentina E. Balas In the emerging smart city technology and industries, the role of artificial intelli- gence is getting more prominent. This AI book series will aim to cover the latest AI work, which will help the naïve user to get support to solve existing problems, and for the experienced AI practitioners, it will shed light on new avenues in the AI domains. The series will cover the recent work carried out in AI and associated domains and it will also cover a broad scope of application areas such as biometric security, Pattern recognition, NLP, Expert Systems, Machine Learning, Block-Chain, and Big Data. The work domain of AI is quite deep, so it will be covering the latest trends that are evolving with the concepts of AI, and it will be helpful to those who are new to the field, practitioners, students, and researchers to gain some new insights. Cyber Defense Mechanisms Security, Privacy, and Challenges Gautam Kumar, Dinesh Kumar Saini, and Nguyen Ha Huy Cuong Artificial Intelligence Trends for Data Analytics Using Machine Learning and Deep Learning Approaches K. Gayathri Devi, Mamata Rath, and Nguyen Thi Dieu Linh Transforming Management Using Artificial Intelligence Techniques Vikas Garg and Rashmi Agrawal AI and Deep Learning in Biometric Security Trends, Potential, and Challenges Gaurav Jaswal, Vivek Kanhangad, and Raghavendra Ramachandra For more information on this series, please visit: https://www.crcpress.com/ Artificial-Intelligence-AI-Elementary-to-Advanced-Practices/book-series/ CRCAIEAP AI and Deep Learning in Biometric Security Trends, Potential, and Challenges Edited by Gaurav Jaswal, Vivek Kanhangad, and Raghavendra Ramachandra MATLAB® and Simulink® are trademarks of The MathWorks, Inc. and are used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® and Simulink® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® and Simulink® software. First edition published 2021 by CRC Press 6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742 and by CRC Press 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN © 2021 Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, LLC Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, access www.copyright. com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermissions@ tandf.co.uk Trademark notice : Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Jaswal, Gaurav, editor. | Kanhangad, Vivek, editor. | Ramachandra, Raghavendra, editor. Title: AI and deep learning in biometric security : trends, potential, and challenges / edited by Gaurav Jaswal, Vivek Kanhangad, and Raghavendra Ramachandra. Description: First edition. | Boca Raton, FL : CRC Press, 2021. | Series: Artificial intelligence (AI) : elementary to advanced practices | Includes bibliographical references and index. Identifiers: LCCN 2020032531 (print) | LCCN 2020032532 (ebook) | ISBN 9780367422448 (hardback) | ISBN 9781003003489 (ebook) Subjects: LCSH: Biometric identification. | Artificial intelligence. Classification: LCC TK7882.B56 A53 2021 (print) | LCC TK7882.B56 (ebook) | DDC 006.2/48—dc23 LC record available at https://lccn.loc.gov/2020032531 LC ebook record available at https://lccn.loc.gov/2020032532 ISBN: 978-0-367-42244-8 (hbk) ISBN: 978-0-367-67251-5 (pbk) ISBN: 978-1-003-00348-9 (ebk) Typeset in Times by codeMantra v Contents Preface......................................................................................................................vii Editors .......................................................................................................................ix Contributors ..............................................................................................................xi Chapter 1 Deep Learning-Based Hyperspectral Multimodal Biometric Authentication System Using Palmprint and Dorsal Hand Vein .........1 Shuping Zhao, Wei Nie, and Bob Zhang Chapter 2 Cancelable Biometrics for Template Protection: Future Directives with Deep Learning .......................................................... 23 Avantika Singh, Gaurav Jaswal, and Aditya Nigam Chapter 3 On Training Generative Adversarial Network for Enhancement of Latent Fingerprints......................................................................... 51 Indu Joshi, Adithya Anand, Sumantra Dutta Roy, and Prem Kumar Kalra Chapter 4 DeepFake Face Video Detection Using Hybrid Deep Residual Networks and LSTM Architecture..................................................... 81 Semih Yavuzkiliç, Zahid Akhtar, Abdulkadir Sengür, and Kamran Siddique Chapter 5 Multi-spectral Short-Wave Infrared Sensors and Convolutional Neural Networks for Biometric Presentation Attack Detection ....... 105 Marta Gomez-Barrero, Ruben Tolosana, Jascha Kolberg, and Christoph Busch Chapter 6 AI-Based Approach for Person Identification Using ECG Biometric .......................................................................................... 133 Amit Kaul, A.S. Arora, and Sushil Chauhan Chapter 7 Cancelable Biometric Systems from Research to Reality: The Road Less Travelled .................................................................. 155 Harkeerat Kaur and Pritee Khanna vi Contents Chapter 8 Gender Classification under Eyeglass Occluded Ocular Region: An Extensive Study Using Multi-spectral Imaging ......................... 175 Narayan Vetrekar, Raghavendra Ramachandra, Kiran Raja, and R. S. Gad Chapter 9 Investigation of the Fingernail Plate for Biometric Authentication using Deep Neural Networks...................................205 Surabhi Hom Choudhury, Amioy Kumar, and Shahedul Haque Laskar Chapter 10 Fraud Attack Detection in Remote Verification Systems for Non-enrolled Users .......................................................................... 239 Ignacio Viedma, Sebastian Gonzalez, Ricardo Navarro, and Juan Tapia Chapter 11 Indexing on Biometric Databases .................................................... 257 Geetika Arora, Jagdiah C. Joshi, Karunesh K. Gupta, and Kamlesh Tiwari Chapter 12 Iris Segmentation in the Wild Using Encoder-Decoder-Based Deep Learning Techniques .............................................................. 283 Shreshth Saini, Divij Gupta, Ranjeet Ranjan Jha, Gaurav Jaswal, and Aditya Nigam Chapter 13 PPG-Based Biometric Recognition: Opportunities with Machine and Deep Learning............................................................ 313 Amit Kaul and Akhil Walia Chapter 14 Current Trends of Machine Learning Techniques in Biometrics and its Applications .......................................................................... 333 B. S. Maaya and T. Asha Index ...................................................................................................................... 361 vii Preface With the growth of data and the increasing awareness about the sensitivity of per- sonal information, people have started to treat their privacy more seriously. Biometric systems have now significantly improved person identification and verification, play- ing an important role in personal, national, and global security. The recently evolved deep neural networks (DNN) learn the hierarchical features at intermediate layers automatically from the data and have shown many inspiring results for biometric applications. With this motivation, the text offers a showcase of cutting-edge research on the use of DNN in face, nail, finger knuckle, iris, ECG, palm print, fingerprint, vein, and medical biometric systems, and hence focuses on two parts: “Biometrics” and “Deep Learning for Biometrics”. This text highlights original case studies to solve real-world problems on biomet- ric security and presents a broad overview of advanced deep learning architectures for learning domain-specific feature representation for biometrics-related tasks. The book aims to provide an in-depth overview of the recent advancements in the domain of biometric security using artificial intelligence (AI) and deep learning techniques, enabling readers to gain a deeper insight into the technological background of this domain. The text acts as a platform for the decision on the use of advanced archi- tectures of convolutional neural networks, generative adversarial networks, auto- encoders, recurrent convolutional neural networks, and graph convolution neural networks for various biometric security tasks such as indexing, gender classification, recognition in the wild, spoofing attacks/liveness detection, quality analysis, ROI segmentation, cross-sensor matching, and domain adaptation. In the text, feasibility studies on medical modalities (ECG, EEG, PPG) have been investigated using AI and deep learning. This book also examines the potential and future perspectives of AI and deep learning towards biometric template protection and multi-spectral biometrics. Overall, the reference provides better readability to readers through its chapter organisation and contains fourteen chapters only. This text/reference is an edited volume by prominent academic researchers and industry professionals in the area of AI and biometric security. It will be essential read- ing for prospective undergraduate/postgraduate students, young researchers, and tech- nology aspirants who are willing to research in the field of AI and biometric security. Gaurav Jaswal Vivek Kanhangad Raghavendra Ramachandra viii Preface MATLAB ® is a registered trademark of The MathWorks, Inc. For product information, Please contact: The MathWorks, Inc. 3 Apple Hill Drive Natick, MA 01760-2098 USA Tel: 508-647-7000 Fax: 508-647-7001 E-mail: info@mathworks.com Web: www.mathworks.com ix Editors Dr. Gaurav Jaswal is currently working as post-doctoral researcher at Indian Institute of Technology Delhi, India since January 2020. Before this, he served as Project Scientist (Electrical Engineering) at National Agri-Food Biotechnology Institute Mohali, India. He was research associate at School of Computing and Electrical Engineering, Indian Institute of Technology Mandi, India. He received MTech and PhD degrees in Electrical Engineering from National Institute of Technology Hamirpur in 2018. His research interests are in the areas of multimodal biometrics, medical imaging, and deep learning. He regularly reviews papers for various interna- tional journals including IEEE Transactions on Information Forensics and Security (TIFS), IEEE Transactions on Biometrics, Behavior, and Identity Science (T-BIOM), IET Biometrics , and Pattern Recognition Letters Dr. Vivek Kanhangad is currently working as associate professor, Department of Electrical Engineering, Indian Institute of Technology Indore, India since February 2012. Prior to this, he was visiting assistant professor, International Institute of Information Technology Bangalore, India (June 2010–December 2012). He received PhD from the Hong Kong Polytechnic University in 2010. Prior to joining Hong Kong PolyU, he received MTech degree in Electrical Engineering from Indian Institute of Technology Delhi in 2006 and worked for Motorola India Electronics Ltd, Bangalore for a while. His research interests are in the overlapping areas of digital signal and image processing, pattern recognition with a focus on biometrics and biomedical applications. He regu- larly reviews papers for various international journals including IEEE Transactions on Information Forensics and Security (TIFS), IEEE Transactions on Cybernetics , IEEE Transactions on Human-Machine Systems , and Elsevier journals – Pattern Recognition and Pattern Recognition Letters Dr. Raghavendra Ramachandra is currently working as a professor in Department of Information Security and Communication Technology (IIK). He is a member of Norwegian Biometrics Laboratory (http://nislab.no/biometrics_lab) at NTNU Gjøvik. He received B.E. (Electronics and Communication) from University of Mysore, India; MTech (Digital Electronics and Advanced Communication Systems) from Visvesvaraya Technological University, India; and PhD (Computer Science with specialisation of Pattern Recognition and Image Processing) from the University of Mysore, India, and Telcom SudParis, France. His research inter- est includes pattern recognition, image and video analytics, biometrics, human behaviour analysis, video surveillance, health biometrics, and smartphone authentication. xi Contributors Zahid Akhtar Department of Computer Science University of Memphis Memphis, Tennessee Adithya Anand Indian Institute of Technology Delhi Delhi, India A.S. Arora Department of Electrical & Instrumentation Engineering Sant Longowal Institute of Engineering and Technology Longowal, India Geetika Arora Department of Computer Science and Information Systems Birla Institute of Technology and Science Pilani Pilani, India T. Asha Department of CSE Banglore Institute of Technology Bengaluru, India Christoph Busch da/sec – Biometrics and Internet Security Research Group Hochschule Darmstadt Darmstadt, Germany Sushil Chauhan Department of Electrical Engineering National Institute of Technology Hamirpur Hamirpur, India Surabhi Hom Choudhury Department of Electronics & Instrumentation Engineering National Institute of Technology Silchar Silchar, India R.S. Gad Department of Electronics Goa University Taleigao-Plateau, India Marta Gomez-Barrero Fakultät Wirtschaft Hochschule Ansbach Ansbach, Germany Sebastian Gonzalez R + D TOC Biometrics Labs Santiago, Chile Divij Gupta Department of Electrical Engineering Indian Institute of Technology Jodhpur Jodhpur, India Karunesh K. Gupta Department of Electrical and Electronics Engineering Birla Institute of Technology and Science Pilani Pilani, India Gaurav Jaswal Department of Electrical Engineering Indian Institute of Technology Delhi Delhi, India xii Contributors Ranjeet Ranjan Jha School of Computing and Electrical Engineering Indian Institute of Technology Mandi Mandi, India Indu Joshi Indian Institute of Technology Delhi Delhi, India Jagdiah C. Joshi Department of Electrical and Electronics Engineering Birla Institute of Technology and Science Pilani Pilani, India Prem Kumar Kalra Indian Institute of Technology Delhi Delhi, India Amit Kaul Department of Electrical Engineering National Institute of Technology Hamirpur Hamirpur, India Harkeerat Kaur Indian Institute of Technology Jammu Pritee Khanna PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur Jabalpur, India Jascha Kolberg da/sec – Biometrics and Internet Security Research Group Hochschule Darmstadt Darmstadt, Germany Amioy Kumar Client Computing Group Intel Corporation Bangalore Bangalore, India Shahedul Haque Laskar Department of Electronics & Instrumentation Engineering National Institute of Technology Silchar Silchar, India B.S. Maaya Department of CSE Banglore Institute of Technology Bengaluru, India Ricardo Navarro R + D TOC Biometrics Labs Santiago, Chile Wei Nie Department of Computer and Information Science University of Macau Macau, China Aditya Nigam School of Computing and Electrical Engineering Indian Institute of Technology Mandi Mandi, India Kiran Raja Norwegian Biometrics Laboratory Norwegian University of Science and Technology (NTNU) Gjøvik, Norway Raghavendra Ramachandra Norwegian Biometrics Laboratory Norwegian University of Science and Technology (NTNU) Gjøvik, Norway Sumantra Dutta Roy Indian Institute of Technology Delhi Delhi, India xiii Contributors Shreshth Saini Department of Electrical Engineering Indian Institute of Technology Jodhpur Jodhpur, India Abdulkadir Sengür Department of Electrical and Electronics Engineering Fırat University Elazig, Turkey Kamran Siddique Department of Information and Communication Technology Xiamen University Malaysia Sepang, Malaysia Avantika Singh School of Computing and Electrical Engineering Indian Institute of Technology Mandi Mandi, India Juan Tapia R + D TOC Biometrics Labs Santiago, Chile Kamlesh Tiwari Department of Computer Science and Information Systems Birla Institute of Technology and Science Pilani Pilani, India Ruben Tolosana Biometrics and Data Pattern Analytics – BiDA Lab Universidad Autonoma de Madrid Madrid, Spain Narayan Vetrekar Department of Electronics Goa University Taleigao-Plateau, India Ignacio Viedma R + D TOC Biometrics Labs Santiago, Chile Akhil Walia Department of Electrical Engineering National Institute of Technology Hamirpur Hamirpur, India Semih Yavuzkiliç Department of Electrical and Electronics Engineering Fırat University Elazig, Turkey Bob Zhang Department of Computer and Information Science University of Macau Macau, China Shuping Zhao Department of Computer and Information Science University of Macau Macau, China 1 1 Deep Learning-Based Hyperspectral Multimodal Biometric Authentication System Using Palmprint and Dorsal Hand Vein Shuping Zhao, Wei Nie, and Bob Zhang University of Macau 1.1 INTRODUCTION Biometric recognition system has been widely used in the construction of a smart society. Many types of biometric systems, including face, iris, palmprint, palm vein, dorsal hand vein, and fingerprint, currently exist in security authentication. Palmprint CONTENTS 1.1 Introduction ......................................................................................................1 1.2 Device Design ...................................................................................................5 1.3 System Implementation .................................................................................... 6 1.3.1 ROI Extraction ......................................................................................6 1.3.1.1 Hyperspectral Palmprint ROI Extraction ..............................6 1.3.1.2 Hyperspectral Dorsal Hand Vein ROI Extraction .................8 1.3.2 Feature Extraction .............................................................................. 10 1.3.3 Feature Fusion and Matching ............................................................. 13 1.4 Experiments .................................................................................................... 13 1.4.1 Multimodal Hyperspectral Palmprint and Dorsal Hand Vein Dataset.....14 1.4.2 Optimal Pattern and Band Selection .................................................. 14 1.4.3 Multimodal Identification ................................................................... 17 1.4.4 Multimodal Verification ..................................................................... 17 1.4.5 Computational Complexity Analysis .................................................. 18 1.5 Conclusions ..................................................................................................... 19 Acknowledgements .................................................................................................. 19 References ................................................................................................................ 19 2 AI and Deep Learning in Biometric Security recognition system is a kind of reliable authentication technology, due to the fact that palmprint has stable and rich characteristics, such as textures, local orientation features, and lines. In addition, a palmprint is user-friendly and cannot be easily captured by a hidden camera device without cooperation from the users. However, palmprint images captured using a conventional camera cannot be used in liveness detection. Palm vein is a good remedy for the weakness of palmprint acquired using a near-infrared (NIR) camera. The vein pattern is the vessel network underneath human skin. It can successfully protect against spoofing attacks and impersonation. Similar to palm vein, dorsal hand vein also has stable vein structures that do not change with age. Besides vein networks, some related characteristics to palmprint such as textures and local direction features can also be acquired. Up to now, palmprint and dorsal hand vein-based recognition methods have achieved competitive performances in the literature. Huang et al. [1] put forward a method for robust principal line detection from the palmprint image, even if the image contained long wrinkles. Guo et al. [2] presented a binary palmprint direction encoding schedule for multiple orientation representation. Sun et al. [3] presented a framework to achieve three orthogonal line ordinal codes. Zhao et al. [4] constructed a deep neural network for palmprint feature extraction, where a convolutional neural network (CNN)-stack was constructed for hyperspectral palmprint recognition. Jia et al. presented palmprint-oriented lines in [5]. Khan et al. [6] applied the principle component analysis (PCA) to achieve a low-dimensionality feature in dorsal hand vein recognition. Khan et al. [7] obtained a low-dimensionality feature representa- tion with Cholesky decomposition in dorsal hand vein recognition. Lee et al. [8] encoded multiple orientations using an adaptive two-dimensional (2D) Gabor filter in dorsal hand vein feature extraction. The palmprint and dorsal hand vein recognition is usually carried out by conven- tional and deep learning-based methods. The conventional methods need to design a filter to extract the corresponding feature, i.e., local direction, local line, principal line, and texture. These hand-crafted algorithms usually require rich prior knowl- edge based on the specific application scenario. PalmCode [9] encoded palmprint features on a fixed direction by using a Gabor filter. Competitive code [10] extracted the dominant direction feature by using six Gabor filters. Xu et al. [11] encoded a competitive code aiming to achieve the accurate palmprint dominant orientation. Fei et al. [12] detected the apparent direction from the palmprint image. In addi- tion, Huang et al. [13] put forward a centroid-based circular key-point grid (CCKG) pattern in dorsal hand vein recognition, which extracts local features based on key- points detection. Deep learning-based algorithms require a mass of training data to train the parameters in the deep convolutional neural network (DCNN). Afterwards, the optimal DCNN can be utilised for classification or convolution feature extrac- tion. However, a mass of training data is usually unavailable for a palmprint or dorsal hand vein recognition task. Especially, the transfer learning technology with DCNN supports an approach that a pretrained DCNN can be fine-tuned with a few train- ing samples for classification in a specific application. Zhao et al. [14] proposed a deep discriminative representation method, which extracted palmprint features from deep discriminative convolutional networks (DDCNs). DDCNs contain a pretrained DCNN and a set of lightened CNNs corresponding to the global and local patches 3 Multimodal Biometric Authentication System segmented from the palmprint image. Wan et al. [15] trained the VGG depth CNN to extract dorsal hand vein features and used the logistic regression for identification. Deep learning-based methods can be widely applied in generic application scenarios. Increasing research studies have moved to the area of hyperspectral imagery tech- nology in the past decades. Contrary to the traditional imagery technology, not only skin texture but also vascular networks are imaged using the designed hyperspectral imagery system with the specific spectrum setup. In the phase of imaging palmprint or dorsal hand combined hyperspectral technology, more discriminative information from the palmprint or dorsal hand image can be captured achieving a high recogni- tion rate. With more than 60 bands covered in hyperspectral palmprint, the three- dimensional (3D) feature was extracted through 3D Gabor filters [16]. Due to the redundant data, hyperspectral palmprint authentication improved but not remarkably when every spectral data were considered in the feature extraction phase. Based on band combination, Shen et al. [17] clustered typical bands in hyperspectral palm- print images for authentication, which performed better compared with in Ref. [16], while Guo et al. [18] applied an approach of k means algorithm for representative band selection in hyperspectral palmprint database to improve performance. What’s more, the band clustering method can decrease computation and increase efficiency in hyperspectral biometrics. As is known, dorsal hand vein and palmprint are con- centrated in one hand, which makes it more convenient to collect these two different modalities simultaneously. Based on this observation, the combination of hyperspec- tral palmprint and dorsal hand biometrics is developed to meet a higher security requirement and to guarantee an exceptional recognition performance. In addition, unimodal biometrics recognition based on a single trait easily suffers from spoofing and other attacks as stated in the literature [19,20]. Table 1.1 illustrates the survey of the current multimodal biometric recognition algorithms. First, it is observed from this table that palmprint and dorsal hand vein have been fused before [21]. However, Ref. [21] and the other methods in Table 1.1 used only two single-spectrum images (one for each modality) to improve the recognition performance. Different from the literature in Table 1.1, this work will study and implement the merging hyperspectral palmprint feature into dorsal hand vein feature to develop a novel hyperspectral multimodal biometric authentication system, which is demon- strated by a flow diagram (refer to Figure 1.1). A hyperspectral acquisition device was utilised for collecting hyperspectral palmprint and dorsal hand images. Then, region of interest (ROI) is detected from hyperspectral palmprint, and dorsal hand images resulted in two corresponding ROI cubes. After ROI extraction, the optimal feature pattern, i.e., local binary pattern (LBP) [22], local derivative pattern (LDP) [9], 2D-Gabor [2], and deep convolutional feature (DCF) [23], is selected for the palmprint and dorsal hand vein, correspondingly. In the pattern selection procedure, each image in the ROI cube is extracted and its features are used in recognition. Thus, the pattern and band which can achieve the highest recognition are treated as the optimal pattern and band for hyperspectral palmprint and dorsal hand images. Afterwards, the feature corresponding to the optimal pattern from palmprint on the optimal band and the feature concerning to the optimal pattern from dorsal hand vein on the optimal band are merged as one feature vector. At last, this fused multimodal feature vector is directly used in matching with the 1-NN classifier. 4 AI and Deep Learning in Biometric Security TABLE 1.1 The Survey of Multimodal Biometric Recognition Algorithms Literature Algorithms Modalities Features Year [19] Concatenation Palmprint and hand-geometry Line features; hand lengths and widths 2003 [20] Combined face-plus-ear image Face and ear PCA 2003 [24] Concatenation Face and hand PCA, linear discriminant analysis (LDA) and 9-byte features 2005 [25] Concatenation Face and palmprint 2D-Gabor PCA 2007 [26] Concatenation Fingerprint and face Minutia features 2007 [27] Concatenation Side face and gait PCA 2008 [28] Fusion Palmprint and fingerprint Discrete cosine transforms 2012 [29] Fusion Profile face and ear Speeded up robust features (SURF) 2013 [30] Concatenation Palmprint and fingerprint Bank of 2D-Gabor 2014 [31] Weighted concatenation Face and ear PCA 2015 [32] Feature level Iris, face and fingerprint Group sparse representation- based classifier (GSRC) 2016 [21] Score level Palmprint and dorsal hand vein Mean and average absolute deviation (AAD) features 2016 [33] Bayesian decision fusion Face and ear CNN features 2017 [34] Score level Finger-vein and finger shape CNN features 2018 [35] Concatenation Face and ear CNN features 2017 FIGURE 1.1 The flowchart of the designed system-merged hyperspectral palmprint feature with dorsal hand feature. 5 Multimodal Biometric Authentication System The major contributions in the chapter are briefly introduced as follows: 1. A novel real-time hyperspectral multimodal biometric authentication sys- tem is conceived. It captures hyperspectral hand images by the proposed hyperspectral imaging acquisition device under 53 spectrums in the range of 520–1040 nm with intervals of 10 nm. 2. We collected a big multimodal dataset containing hyperspectral palmprint and dorsal hand images using the designed device. More information about this dataset can be found in Section 1.4.1. The remaining work is organised as follows. In Section 1.2, the designed capture device is introduced. Following this, the designed system is illustrated in Section 1.3, including ROI and feature extraction as well as multimodal fusion and matching. Extensive experiments and analysis are included in Section 1.4, while Section 1.5 concludes the proposed system. 1.2 DEVICE DESIGN The hyperspectral imaging acquisition system consists of two halogen lamps made by Osram, Inc., one charge coupled device (CCD) camera produced by Cooke, Inc., and one liquid crystal tunable filter manufactured by Meadowlark, Inc. The cost of the setup is approximately USD 6,500.00. The prototype of this acquisition system is illustrated in Figure 1.2. The CCD camera is placed in the middle with one halogen lamp on either side. The halogen lamps produce both visible light and NIR with spectra ranging from 520 to 1,040 nm. The light from the two halogen lamps irradiates on the palm or dorsal hand, and then reflects to the camera sensor for capturing images. A tunable filter is settled ahead of the camera lens and allows a single band to pass through its settings. To obtain stable spectral images, 10 nm is set as the spectral distance in the tunable filter. Therefore, this hyperspectral FIGURE 1.2 Schematic of our designed hyperspectral imaging device.