This article proposed a multimodal biometric enrolment and authentication system (MBEAS) with modified score-level fusion and TriBlendNN-based template matching approach. It consists of four phases: (i) enrolment phase, (ii) security phase, (iii) storage phase, and (iv) authentication phase. Initially, the raw data of the iris, face, hand, speech, signature, handwriting, fingerprint, and keystroking are collected from the BiosecurID database, and then the raw images are pre-processed via resizing and cropping. Raw signal  of speech is pre-processed via wavelet denoising and spectral subtraction. The raw data of key stroking is pre-processed via Z-score normalization. Then the pre-processed images of iris, face, signature, hand, handwriting, and fingerprints are segmented via optimized watershed segmentation. The pre-processed speech signal is segmented via VAD. From the segmented data, the optimal features are extracted for iris (LBP), face (IncepV3), signature and handwriting (shape features and GLCM), speech (MFCC), fingerprints (minutiae extraction), hand (palm print features), and keystroking. From the feature-extracted data, the feature fusion is then performed via modified score-level fusion. After the enrolment phase, the feature fused data is secured using watermarking. Then the watermarked data is stored in cloud storage. The final stage is the authentication stage, wherein the template matching is processed via the newly proposed TriBlendNN model. The proposed TriBlendNN model is a combination of the CNN, RNN, and Bi-LSTM. The final outcome comes from template matching. The proposed model is implemented in Python and its accuracy at learning rates of 70% and 80% are 96.89% and 97.76%, respectively.

multimodal biometric, score-level fusion, TriBlendNN model, template matching, enrolment and authentication system, neural network, face recognition

Received: August 22, 2024; Revised: October 18, 2024; Accepted: October 23, 2024; Published: November 30, 2024

How to cite this article: Abdulgader Zaid Almaymuni, Ahmad Raza Khan, Azan Hamad Alkhorem and Mohammed A. Saleh, Multimodal biometric enrolment and authentication system (MBEAS) with modified score-level fusion and TriBlendNN-based template matching, Advances and Applications in Discrete Mathematics 42(2) (2025), 113-149. https://doi.org/10.17654/0974165825009

This Open Access Article is Licensed under Creative Commons Attribution 4.0 International License

References:
[1] R. Ryu, S. Yeom, S. H. Kim and D. Herbert, Continuous multimodal biometric authentication schemes: a systematic review, IEEE Access 9 (2021), 34541-34557.
[2] A. Al Abdulwahid, N. Clarke, I. Stengel, S. Furnell and C. Reich, Continuous and transparent multimodal authentication: reviewing the state of the art, Cluster Computing 19 (2016), 455-474.
[3] M. Singh, R. Singh and A. Ross, A comprehensive overview of biometric fusion, Information Fusion 52 (2019), 187-205.
[4] S. Aleem, P. Yang, S. Masood, P. Li and B. Sheng, An accurate multi-modal biometric identification system for person identification via fusion of face and finger print, World Wide Web 23 (2020), 1299-1317.
[5] V. Talreja, M. C. Valenti and N. M. Nasrabadi, Deep hashing for secure multimodal biometrics, IEEE Transactions on Information Forensics and Security 16 (2020), 1306-1321.
[6] P. Szczuko, A. Czyżewski, P. Hoffmann, P. Bratoszewski and M. Lech, Validating data acquired with experimental multimodal biometric system installed in bank branches, Journal of Intelligent Information Systems 52 (2019), 1-31.
[7] I. A. Ogbodo, Exploring access to EHR by emergency patients using multimodal biometrics, Int. J. Latest Technol. Eng. Manag. Appl. Sci. 9 (2020), 44-50.
[8] M. Rukhiran, Wong-In S, Netinant P. IoT-based biometric recognition systems in education for identity verification services: Quality assessment approach, IEEE Access. 2023 Mar 6; 11:22767-87.
[9] R. Brown, G. Bendiab, S. Shiaeles and B. Ghita, A novel multimodal biometric authentication system using machine learning and blockchain, Selected Papers from the 12th International Networking Conference: INC 2020, Springer, Vol. 12, 2021, pp. 31-46.
[10] T. Vijayakumar, Synthesis of palm print in feature fusion techniques for multimodal biometric recognition system online signature, Journal of Innovative Image Processing (JIIP) 3(2) (2021), 131-143.
[11] M. Labayen, R. Vea, J. Flórez, N. Aginako and B. Sierra, Online student authentication and proctoring system based on multimodal biometrics technology, IEEE Access 9 (2021), 72398-72411.
[12] M. Sabri, M. S. Moin and F. Razzazi, A new framework for match-on-card and match-on-host quality-based multimodal biometric authentication, Journal of Signal Processing Systems 91 (2019), 163-177.
[13] R. Das, E. Piciucco, E. Maiorana and P. Campisi, Convolutional neural network for finger-vein-based biometric identification, IEEE Transactions on Information Forensics and Security 14(2) (2018), 360-373.
[14] S. S. Ali, I. I. Ganapathi, S. Prakash, P. Consul and S. Mahyo, Securing biometric user template using modified minutiae attributes, Pattern Recognition Letters 129 (2020), 263-270.
[15] C. Kant and S. Chaudhary, A watermarking-based approach for the protection of templates in a multimodal biometric system, Procedia Computer Science 167 (2020), 932-941.
[16] O. Najah Kadhim, M. Hasan Abdulameer and Y. Mahdi Hadi Al-Mayali, A Multimodal Biometric System for Iris and Face Traits Based on Hybrid Approaches and Score Level Fusion, In BIO Web of Conferences, Vol. 97, EDP Sciences, 2024, p. 00016.
[17] H. Purohit and P. K. Ajmera, Optimal feature level fusion for secured human authentication in multimodal biometric system, Machine Vision and Applications 32(1) (2021), 24.
[18] T. S. Rao and E. S. Reddy, Multimodal optimal score level fusion method in multimodal biometric system, Webology 19(2) (2022), 8565-8586.
[19] C. Medjahed, A. Rahmoun, C. Charrier and F. Mezzoudj, A deep learning-based multimodal biometric system using score fusion, IAES Int. J. Artif. Intell. 11(1) (2022), 65.
[20] A. J. Prakash, K. K. Patro, S. Samantray, P. Pławiak and M. Hammad, A deep learning technique for biometric authentication using ECG beat template matching, Information 14(2) (2023), 65.
[21] F. Ahamed, F. Farid, B. Suleiman, Z. Jan, L. A. Wahsheh and S. Shahrestani, An intelligent multimodal biometric authentication model for personalised healthcare services, Future Internet 14(8) (2022), 222.
[22] S. P. Singh and S. Tiwari, A dual multimodal biometric authentication system based on woa-ann and ssa-dbn techniques, Sci. 5(1) (2023), 10.
[23] F. Cherifi, K. Amroun and M. Omar, Robust multimodal biometric authentication on IoT device through ear shape and arm gesture, Multimedia Tools and Applications 80(10) (2021), 14807-14827.
[24] H. Byeon, V. Raina, M. Sandhu, M. Shabaz, I. Keshta, M. Soni and T. V. Lakshmi, Artificial intelligence-enabled deep learning model for multimodal biometric fusion, Multimedia Tools and Applications (2024), 1-24.
[25] S. Tharewal, T. Malche, P. K. Tiwari, M. Y. Jabarulla, A. A. Alnuaim, A. M. Mostafa and M. A. Ullah, Score‐level fusion of 3D face and 3D ear for multimodal biometric human recognition, Computational Intelligence and Neuroscience 2022(1) (2022), 3019194.
[26] N. Alay and H. H. Al-Baity, Deep learning approach for multimodal biometric recognition system based on fusion of iris, face, and finger vein traits, Sensors 20(19) (2020), 5523.
[27] S. Daas, A. Yahi, T. Bakir, M. Sedhane, M. Boughazi and E. B. Bourennane, Multimodal biometric recognition systems using deep learning based on the finger vein and finger knuckle print fusion, IET Image Processing 14(15) (2020), 3859-3868.
[28] B. Pathak and D. Barooah, Texture analysis based on the gray-level co-occurrence matrix considering possible orientations, International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering 2(9) (2013), 4206-4212.
[29] Y. Tang, F. Gao, J. Feng and Y. Liu, FingerNet: an unified deep network for fingerprint minutiae extraction, 2017 IEEE International Joint Conference on Biometrics (IJCB), IEEE, 2017, pp. 108-116.
[30] L. Muda, M. Begam and I. Elamvazuthi, Voice recognition algorithms using Mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques, 2010. arXiv preprint arXiv:1003.4083.
[31] A. Siguenza, J. Garrido-Salas, E. Anguiano, G. Gonzalez-de-Rivera, R. Ribalda, M. Faundez-Zanuy, J. A. Ortega and V. Cardenoso-Payo, BiosecurID: a multimodal biometric database, Pattern Anal. Appl. 13(2) (2010), 235-246. https://doi.org/10.1007/s10044-009-0151-4.
[32] A. K. Yadav, Deep learning approach for multimodal biometric recognition system based on fusion of iris, fingerprint and hand written signature traits, Turkish Journal of Computer and Mathematics Education 12(11) (2021), 1627-1640.
[33] S. A. Haider, Y. Rehman and S. U. Ali, Enhanced multimodal biometric recognition based upon intrinsic hand biometrics, Electronics 9(11) (2020), 1916.
[34] K. W. Tse and K. Hung, Framework for user behavioural biometric identification using a multimodal scheme with keystroke trajectory feature and recurrent neural network on a mobile platform, IET Biometrics 11(2) (2022), 157-170.
[35] M. J. Hamayel and A. Y. Owda, A novel cryptocurrency price prediction model using GRU, LSTM and Bi-LSTM machine learning algorithms, AI 2(4) (2021), 477-496.
[36] M. Z. Alom, M. Hasan, C. Yakopcic, T. M. Taha and V. K. Asari, Improved inception-residual convolutional neural network for object recognition, Neural Computing and Applications 32 (2020), 279-293.
[37] S. A. El_Rahman and A. S. Alluhaidan, Enhanced multimodal biometric recognition systems based on deep learning and traditional methods in smart environments, PloS ONE 19(2) (2024), eo291084.