C2 A_Framework_for_Multimodal_Biometric_Authentication_Systems_With_Template_Protection

Received 11 August 2022, accepted 25 August 2022, date of publication 8 September 2022, date of current version 19 September 2022.
Digital Object Identifier 10.1109/ACCESS.2022.3205413
A Framework for Multimodal Biometric

Authentication Systems With Template Protection
ZHENG HUI GOH1 , YANDAN WANG2 , LU LENG 3 , (Member, IEEE), SHIUAN-NI LIANG1 ,
ZHE JIN 4 , (Member, IEEE), YEN-LUNG LAI5 , AND XIN WANG 1 , (Senior Member, IEEE)
1 School of Engineering, Monash University Malaysia, Subang Jaya 47500, Malaysia
2 Facultyof Computer Science and Technology, Wenzhou University, Wenzhou 325035, China
3 Key Laboratory of Jiangxi Province for Image Processing and Pattern Recognition, Nanchang Hangkong University, Nanchang 330063, China
4 School of Artificial Intelligence, Anhui University, Hefei 230039, China
5 Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Kajang 43200, Malaysia
Corresponding author: Zhe Jin (jinzhe@ahu.edu.cn)

This work was supported in part by the Ministry of Higher Education (MOHE) through the Fundamental Research Grant Scheme (FRGS)
under Grant FRGS/1/2018/ICT02/MUSM/03/3; in part by the National Natural Science Foundation of China under Grant 61866028; in
part by the Technology Innovation Guidance Program Project (Special Project of Technology Cooperation, Science and Technology
Department of Jiangxi Province) under Grant 20212BDH81003; and in part by the Malaysia Universiti Tunku Abdul Rahman Research
Grant UTARRF/2021-C2/L01.
1 ABSTRACT A multimodal biometric authentication framework based on Index-of-Max (IoM) hashing,

2 Alignment-Free Hashing (AFH), and feature-level fusion is proposed in this paper. This framework enjoys
3 three major merits: 1) Biometric templates are secured by biometric template protection technology (i.e.,
4 IoM hashing), thus providing strong resistance to security and privacy invasion; 2) It flexibly adopts a
5 variety of biometric feature representations (e.g., binary, and real-valued), thus generalizing to a wide range
6 of biometric features for fusion; 3) Feature-level fusion, which has low template storage, low matching
7 computational complexity, and low privacy risks, which can be accomplished without alignment via AFH.
8 Specifically, the proposed framework works as a drag-and-drop model that can quickly adopt all popular
9 biometric modalities with different feature distributions for feature-level fusion. The fused templates are
10 produced using operators AND, OR and XOR in binary domain. To evaluate the proposed framework,
11 benchmarking datasets from four widely deployed biometric modalities (i.e., FVC 2002 for fingerprint,
12 LFW for face, CASIA-v3-Interval for iris, and UTFVP for finger-vein) are used. The experimental results
13 presented in Table 5 suggest that the proposed framework can achieve state-of-the-art performance in most
14 of the datasets while offering additional folds, such as template protection and generalization to variable
15 features. Moreover, biometric template protection criteria (irreversibility, unlinkability, and revocability) are
16 also analyzed. The results of the analysis indicate satisfaction in terms of the security and privacy of the
17 templates generated from the proposed framework.
18 INDEX TERMS Feature-level fusion, multimodal biometrics, privacy and security.
19 I. INTRODUCTION widely used in various applications ranging from identity 26
20 Biometric recognition systems utilize the physical or behav- identification to marketing tools like recognizing and per- 27
21 ioral characteristics of an individual for identity manage- sonalizing ads to specific customers [2]. For example, elec- 28
22 ment. These characteristics include fingerprint, face, iris, trocardiogram (ECG) can be used to identify and authorize 29
23 hand/finger geometry, retina, gait, palmprint, voice pattern, users [3]. 30
24 signature, ear, hand/finger-vein, odor, or even the DNA Employing biometric data as an identity credential can 31
25 information of an individual [1]. Recently, it has become result in improved comfort and convenience, like removing 32
the need of remembering a password. However, this can 33
The associate editor coordinating the review of this manuscript and sometimes bring additional security risks [4]. The main con- 34
approving it for publication was Mehdi Sookhak . cern of the public is the privacy risks associated with the 35
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
96388 VOLUME 10, 2022
Z. H. Goh et al.: Framework for Multimodal Biometric Authentication Systems With Template Protection
36 biometric system [5], which are then aggravated by the fact face and hand images. In exchange for its secureness, feature- 92
37 that biometric data cannot be changed. If the unprotected level fusion sacrifices on simplicity and requires a generic 93
38 biometric data are compromised, the updated data cannot be framework to establish a common representation of biometric 94
39 issued. features [8]. While some of the current multimodal schemes 95
40 The present commercial systems either leave biomet- use novel fusion techniques like internal fusion [15], most 96
41 ric templates unprotected or employ classical cryptographic of them rely on simple concatenation or binary operations 97
42 techniques [6]. However, even if the stored data are kept (i.e., AND, OR, and XOR) to fuse the biometric data at 98
43 secure when not used, decryption must be performed during feature-level. Since multimodal biometric systems contain 99
44 authentication, thus leaving biometric information unpro- more than one biometric modality, the importance of BTP is 100
45 tected after the decryption phase. One way to combat these much more prevalent here in keeping the system from being 101
46 shortcomings is to use biometric template protection (BTP) compromised. 102
47 schemes, like cancellable biometrics and biometric cryp- In this paper, a multimodal biometric authentication 103
48 tosystem. Cancellable biometrics protect the original tem- scheme is proposed for biometric templates with different dis- 104
49 plate by creating a ‘‘cancellable’’ template that can be tributions and alignment-dependent modalities. Fingerprint, 105
50 revoked if compromised through intentional and repeatable face, iris, and finger-vein are used as example biometrics 106
51 distortion on a biometric signal. On the other hand, biometric traits due to their popularity and ability in covering various 107
52 cryptosystems use key generated from helper data related to scenarios and representations [16]. 108
53 the biometric templates. A BTP scheme should satisfy the The rest of this paper is structured as follows: 109
54 following criteria [7]: Section 1 revisits existing BTP works and explains the moti- 110
55 • Irreversibility: It should be computationally infeasible vations and contributions of this paper. Section 2 describes 111
56 to derive the underlying biometric template from the the preliminaries of this paper. Section 3 introduces the 112
57 protected template. This prevents privacy attacks, such proposed multimodal biometric authentication framework. 113
58 as inversion attack and Attack via Record Multiplicity Section 4 presents the experimental results. Section 5 presents 114
59 (ARM). the security and privacy analysis of the proposed scheme. 115
60 • Revocability: A new protected template should be able Lastly, conclusions are drawn in Section 6. 116
61 to be revoked or reissued from the underlying biometric

62 template when the old protected template is compro- A. RELATED WORKS 117
63 mised. Due to the advantages mentioned previously, this paper will 118
64 • Unlinkability: It should be computationally difficult focus on feature-level fusion schemes with biometric tem- 119
65 to differentiate protected templates derived from the plate protection. In [17], Paul and Gavrilova proposed a 120
66 same underlying biometric template. This can prevent feature-level fusion BTP scheme for face and ear features. 121
67 cross-matching across different applications and pre- At first, the face and ear images are randomly separated 122
68 serve the users’ privacy. into two parts and combined to produce two fused images. 123
69 • Performance preservation: The accuracy performance Later, the dimension of the two fused images is reduced 124
70 of the protected templates should be preserved when using user-specific random projections and Principal Compo- 125
71 compared to the unprotected templates. nent Analysis (PCA). After that, a distance-based feature is 126
72 There is a trade-off between performance and security for extracted from each low-dimensional feature using k-Mean 127
73 many of the earlier schemes, which are often unimodal (i.e., Clustering, and a cancellable template is produced by com- 128
74 use only a single biometric cue). One way to handle this issue bining them using linear discriminate analysis. The paper 129
75 is to use multi-biometric template protection schemes [8]. showed the proposed scheme with fused cancellable template 130
76 This is because the fusion of a range of different biometric has a better matching accuracy when compared with uni- 131
77 characteristics in BTP generally leads to higher accuracy and modal cancellable template and that multimodal cancellable 132
78 robustness [9]. biometric system is a viable option. 133
79 As defined in ISO/IEC TR 24722 on multimodal and other Later in [18], Rathgeb et al. proposed a generic frame- 134
80 multi-biometric fusion [10], such fusion can be performed at work for multimodal biometric template protection based 135
81 feature, score, and decision levels in a conventional biometric on Bloom filters for face and iris. In short, Bloom filter 136
82 system [11]. However, not all methods are created equal. Both transformation is performed on the binary face feature and 137
83 score-level and decision-level fusions require separate stor- IrisCode separately to obtain cancellable face and iris tem- 138
84 age of templates, which need high storage cost and matching plates. These two templates are then fused via a bitwise-OR 139
85 computational complexity as well as high privacy risks [12], operation. After that, Gomez et al. improved this scheme 140
86 therefore, feature-level fusion is identified to be the safer and extended it to include finger-vein as a new biometric 141
87 option [13]. Feature-level fusion fuses the features from the modality [19]. To enhance the recognition performance of 142
88 same or different input data of a given individual into a single the scheme, a weighted fusion strategy was proposed. During 143
89 representation [14]. These features can come from the same the transformation, a weight αi is allocated to each can- 144
90 biometric trait, like the textural and structural features of a cellable template Ci where i = 1, . . . , n represent the num- 145
91 face image, or they can come from different modalities, like ber of included biometric characteristics. The weighted sum 146
VOLUME 10, 2022 96389

147 (i.e., weight × similarity score) is then calculated as the fused transformation key r is used to encrypt the hashed template, 203
148 score. nCharac templates extracted from different instances resulting in a final cancellable template c. To safely store the 204
149 are fused by OR-ing the original Bloom filter-based tem- transformation key, an encrypted string e is also generated 205
150 plates, randomly allocating them over the largest template of by XOR-ing it with the hashed template (i.e., e = r ⊕ h), 206
151 dimensions nBlocksmax × 2nBitsmax according to the positions and is stored alongside the cancellable template. During ver- 207
152 vector pos. Each characteristic contributes with a weight αi . ification, the probing hashed template h0 is used to decrypt 208
153 To obtain a weighted fusion, multiple bits are activated with the encrypted string via XOR and retrieve the transforma- 209
154 each word w using a Multi-Key XOR (MK-XOR) approach tion key r0 . The key is then used to generate the probing 210
155 for the Bloom filter computation of one characteristic. In the cancellable template c0 and compared against the stored can- 211
156 end, the authors concluded that their framework is able to cellable template c using Hamming distance. The experi- 212
157 improve the users’ privacy while preserving the recognition mental results for unimodal cases showed improvement in 213
158 accuracy of an unprotected score-level fusion. terms of matching accuracy when compared to other existing 214
159 In [20], Chin et al. proposed a feature-level fusion BTP unimodal schemes, and the matching accuracy of the multi- 215
160 system based on fingerprint and palmprint. In the scheme, the modal case outperforms the unimodal case under the same 216
161 normalized fingerprint and palmprint templates are first fused parameter setting. 217
162 into a single feature using concatenation. The fused feature is Khoo et al. also proposed a feature-level fusion scheme 218
163 then transformed into another form (i.e., RT feature) using a utilizing Indexing-First-One (IFO) hashing and integer value 219
164 parameterized function known as Random Tiling (RT). The mapping for iris and fingerprint [23]. IFO is a row-wise hash- 220
165 RT function extracts a set of random rectangles from the ing function inspired by Min-hashing and improved secu- 221
166 fused template using a user-specific key, and a feature vector rity wise through P-order Hadamard Product and Modula 222
167 is formed by concatenating the standard deviation of each threshold function. The biometric feature is first permuted 223
168 rectangle. This key allows the scheme to be revocable by with n number of random permutation sets. The n sets of 224
169 replacing the key once there is a leak or breach of security. permuted features then undergo a product rule by AND-ing 225
170 Finally, a bit-string template is formed as the final cancellable them together to produce a combined vector. A window of 226
171 template by binarizing the RT feature using an equal-probable size K is then applied to the vector, and the index of the 227
172 2N discretization scheme. In the experiment, this scheme first ‘1’ observed is recorded. A modular threshold function 228
173 was shown to have better recognition performance when is also applied to prevent leakage. These steps are repeated 229
174 compared to the unimodal counterparts in the stolen-token for Q number of times with different permutation vectors. 230
175 scenario and approaches 0% Equal Error Rate (EER) in the During the fusion stage, the integer matrices for iris and 231
176 genuine-token scenario. fingerprint are first mapped onto a Bloom filter and later 232
177 Another implementation of feature-level fusion can be seen OR-ed together to produce a single fused template. While the 233
178 in [21], where Huang et al. proposed a multimodal biometric proposed scheme did not outperform the unimodal iris system 234
179 framework based on Sparse Representation (SR) for face and in terms of matching accuracy, it did provide strong security 235
180 ear. To start, a novel index named Sparse Coding Error Ratio for the biometric data. IFO hashing is proven to survive major 236
181 (SCER) is built to represent the dependability distinction privacy attacks like attack via record multiplicity (ARM), not 237
182 among face and ear query samples. Later in the multimodal to mention that integer mapping and fusion via logical OR 238
183 characterization stage, SR-based classification schemes like can also provide additional protection. 239
184 Multimodal Sparse Representation based Classification with In [24], Yang et al. proposed a feature-level fusion scheme 240
185 feature Weighting (MSRCW) and Multimodal Robust Sparse for fingerprint and finger-vein based on enhanced partial 241
186 Coding with feature Weighting (MRSCW) are used. The discrete Fourier transform (EP-DFT). Given a binary feature 242
187 features are concatenated together using a proposed adaptive vector f with K elements, a K -by-K DFT matrix W can be 243
188 feature weighting scheme. The experimental results showed obtained. To introduce irreversibility, only k out of K rows of 244
189 that both MSRCW and MRSCW are superior to the existing the DFT matrix are selected to generate a P-DFT matrix M. 245
190 unimodal and multimodal schemes that use face and ear in The transformation is then calculated by y0 = Mf, where y0 246
191 terms of recognition at the time. In addition, MSRCW and is a complex vector of k elements, resulting in an underdeter- 247
192 MRSCW are robust against image degeneration on one of the mined system. To further enhance the security of the system, 248
193 modalities. EP-DFT added a wavelet transformation to f before P-DFT. 249
194 Recently, Lee et al. introduced a token-less cancellable This converts the binary feature into a real-valued feature, 250
195 multimodal biometric scheme named Multimodal Extended increasing randomness and search space of solutions. For 251
196 Feature Vector (M·EFV) Hashing for face and finger- fusion, the authors introduced three options: 1) the fingerprint 252
197 print [22]. This scheme uses an XOR encryption/decryption and finger-vein features are concatenated before EP-DFT; 253
198 method to store the transformation key safely. At first, the fea- 2) the fingerprint and finger-vein features are concatenated 254
199 ture vectors are normalized and fused into a unified template after EP-DFT; 3) the fingerprint and finger-vein features are 255
200 using concatenation. After that, Index-of-Max (IoM) hashing XORed together before EP-DFT. The authors reported the 256
201 is applied to the fused template to obtain a binary hashed tem- best matching accuracy of 0.12% EER using fusion options 257
202 plate h with distance preserving properties. Finally, a random 1 and 2, with fusion option 3 performing the worst ranging 258
96390 VOLUME 10, 2022

TABLE 1. Summary of reviewed feature-level fusion schemes. storage cost and matching computational complexity, as well 285
as low privacy risks [13]. To ensure the successful imple- 286
mentation of the framework, the following issues have to be 287
handled first: 288
• Incompatibility of feature spaces: The feature spaces of 289
different biometric modalities or features extracted from 290
different extractors may not be compatible. For example, 291
the real-valued representation of face and fingerprint 292
features, and the binary presentation of the iris and 293
finger-vein features make fusion difficult if they do not 294
have a common representation. 295
• Large dimension of fused features: The fusion of features 296
by simple concatenation is not preferred, since this can 297
result in large dimensionality of the features, dramati- 298
cally increasing the required storage space and running 299
time. 300
• Alignment issue of biometric features: For some biomet- 301
ric features, alignment is needed to obtain optimal per- 302
formance during matching. However, performing such 303
acts can be challenging when all the features are fused 304
together [26]. This issue cannot be ignored if optimal 305
performance is to be obtained from a system. This can 306
be seen in [19], where the authors used pre-aligned 307
and alignment-free templates for their scheme. How- 308
ever, the scheme itself do not have the ability to deal 309
with unaligned templates. Example of modalities that 310
requires alignment for feature matching includes the iris 311
and finger-vein feature used in this paper [18]. 312
In order to address the challenges mentioned above, 313
259 from 0.55% to 0.69% EER. While no significant difference this paper proposed a multimodal biometric authentication 314
260 is observed between the proposed EP-DFT and the original scheme for biometric templates with different representations 315
261 P-DFT in terms of matching performance, EP-DFT provides and alignment-dependent modalities. The contributions of 316
262 better security, with fusion option 3 having the best security this paper can be summarized as follows: 317
263 overall. • A general framework for multimodal biometric authen- 318
264 Recently in 2022, Kim et al. proposed a deep-learning- tication scheme that is applicable to biometric templates 319
265 based multimodal BTP scheme named cancellable Softmax- of different sizes, origins, and representations, fulfilling 320
266 Out fusion network (CSMoFN) for face and the periocular the requirements for a cancellable BTP scheme estab- 321
267 region [25]. This scheme is comprised of three modules: a lished in [5] (i.e., irreversibility, revocability, unlinka- 322
268 feature extraction and fusion module, a permutation Soft- bility, and performance preservation). The adoption of 323
269 maxOut transform (PSMoT) module, and a multiplication- the Alignment-Free Hashing (AFH) on IrisCode and 324
270 diagonal compression (MDC) module. Firstly, the scheme finger-vein allows the proposed scheme to remove the 325
271 extracts biometric features from images using multiple con- need for alignment during matching. The adoption of 326
272 volutional blocks and fuses them using concatenation at Index-of-Max (IoM) hashing provides template protec- 327
273 feature-level. After that, PSMoT transforms the fused feature tion and the ability to generate a common cancellable 328
274 into a discrete hash vector, which is further compressed by representation for different biometric features that are 329
275 the MDC module to produce a cancellable template. A user- either pre-aligned or unaligned. This allows for easier 330
276 specific token is needed to generate a random seed for permu- fusion since all the templates have a common repre- 331
277 tation in the PSMoT module and for binary random projection sentation, and a complex fusion process is not needed. 332
278 in the MDC module. The authors reported the best recogni- Matching can also be done by simply calculating the 333
279 tion performance of 2.12% EER using the Facescrub dataset Jaccard similarity between two cancellable templates 334
280 and an average of 6.67% EER over six different datasets. with minimum computation cost. 335
281 A summary of the reviewed schemes is listed in Table 1. • Extensive testing on different combinations of 336
parameters and biometric modalities to discover the 337
282 B. MOTIVATIONS AND CONTRIBUTIONS relationship between the different parameters and their 338
283 While multimodal BTP scheme can be done at different recognition performance, and validate the feasibil- 339
284 levels, feature-level fusion is often preferred due to its low ity of the proposed scheme. To be specific, different 340
VOLUME 10, 2022 96391

341 combinations of the three parameters bn , bm , and m are TABLE 2. Nomenclature.

342 applied on 18 different fusion combinations, with this
343 process repeated for 10 times to obtain an averaged
344 result. The experimental result demonstrated that the
345 proposed scheme was able to improve the result when
346 compared with unimodal scenarios, and outperformed
347 most of the existing multimodal schemes, especially in
348 the modalities combinations of FP+F, FP+I, FP+V, and
349 F+V (0% EER using AND for fusion).
350 • Security and privacy analysis are performed thoroughly
351 on the proposed scheme through the testing of its unlink-
352 ability, revocability, and irreversibility properties. Major
353 security and privacy-related attacks are performed on all
B. INDEX-OF-MAX (IoM) HASHING 390
354 18 fusion combinations, and the worst-case scenario is
First proposed in [29], Index-of-Max or IoM hashing is 391
355 assumed for the proposed scheme. The scheme is shown
a ranking-based locality sensitive hashing (LSH) inspired 392
356 to satisfy the four criteria for BTP and is robust against
cancellable biometric technique for BTP. With the aid of 393
357 the subjected attacks.
random parameters generated externally, IoM hashing can 394
transform a real-valued biometric feature into discrete index 395

358 II. PRELIMINARIES (max ranked) hashed code. In our proposed scheme, IoM 396
359 A. ALIGNMENT FREE HASHING (AFH) hashing is used to provide protection and binarization to the 397
360 For some biometric characteristics, a proper alignment of the biometric data. 398
361 templates is critical to obtaining optimal recognition perfor- The Gaussian Random Projection (GRP) based IoM hash- 399
362 mance. However, this is computationally costly, especially ing can be condensed into the following procedure [29]: 400
363 when the process is performed on a large database, not
364 to mention that alignment is often not practical after BTP • Given a fingerprint vector x ∈ Rd , q number of random 401
365 scheme is applied. Therefore, [27] tries to solve this issue by Gaussian projection vectors are generated for m times, 402
366 introducing an alignment-free hashing (AFH) for finger-vein {wij ∈ R|i = 1, . . . , m, j = 1, . . . , q} ∼ N (0, Id ) 403
and i
367 template protection. The AFH is adopted and expanded to h hence a irandom Gaussian projection matrix W = 404
368 include both finger-vein and iris templates in our proposed w1 , . . . , wq can be formed.
i i 405
369 scheme. • The m indices of the maximum value computed from 406
370 Let d(i, j) = |i − j| be the local distance between the i-th ϕi (x) = argmaxj=1,...,q hwij , xi are recorded as ti . The 407
371 and j-th locations in a binary vector V = (v1 , . . . , vn ) where GRP-based IoM hashed code is hence t GRP = {ti ∈ 408
372 vi , vj ∈ [0, 1], and Kronecker delta functions δ(vi , vj ): [1, q]|i = 1, . . . , m}. 409
( With the use of GRP, the fingerprint vector is embedded onto 410
0 if vi 6 = vj a q-dimension Gaussian random subspace, and the index of
373 δ(vi , vj ) 411
1 if vi = vj max-valued projected feature is taken. This is equivalent to 412
a hashed entry h (·) in the rank space from the viewpoint 413
374 Combining the two equations above, we get the alignment- of LSH. This process is then repeated with m number of 414
375 free transformation, coined as T over a vector V which is independent Gaussian random matrices, producing m num- 415
376 described as: ber of IoM indices. Thus, the GRP hashed codes enjoy the 416
X Euclidean pairwise distance preservation property in the pro- 417

377 Tk (V ) = δ(d (i, j) − k)δ(vi , 1)δ(vj , 1) jected subspace Rq . IoM hashing is used in this paper to 418
i>j provide template protection and to generate binary templates 419
for fusion.
where k ∈ {1, 2, . . . , n} and n represents the length of the
420
378
379 given binary vector V . Based on the equation above, we can

380 see that T is designed to measure the number of pairs of III. METHODOLOGY 421
381 1s, which represents vein information in the binary feature In this section, an overview of the proposed scheme as well as 422
382 template that has a local distance k. The local distance is used its details are presented. Table 2 tabulates the notations used 423
383 as an invariant feature descriptor to replace the alignment in this paper. 424
384 required by global features used in algorithms like in [28].

385 This scheme offers a promising trade-off between recogni- A. PROPOSED MULTIMODAL BIOMETRIC 425
386 tion performance loss and unlinkability while producing a AUTHENTICATION SCHEME 426
387 biometric template protection scheme that is alignment-free. In this paper, the fusion between two different biometric 427
388 Features that require alignment (i.e., iris and finger-vein) will modalities is used. The overall flow of the scheme can be seen 428
389 undergo AFH in this paper. in Figure 1. 429
96392 VOLUME 10, 2022

FIGURE 2. Example of rearrangement process for bn = 10, bm = 20,

m = 100.
not required, m in IoM hashing is the only variable 457
parameter for them, and directly define the nbit of the 458
fused template: mFP = mF = nbitf = m. A pseudocode 459
FIGURE 1. Overall flow of proposed scheme. for this scenario can be seen in Algorithm 1. 460
• Scenario 2: Alignment required for both modalities. 461
This only applies to the case of I+V. Let L be the larger 462
430 The blue blocks denote the templates or parameters, while template and S be the smaller template (both templates 463
431 the green blocks denote the processes. Modality 1 (x1 ) and are AFH-ed), the scaling factor between them, α is 464
432 Modality 2 (x2 ) refer to the two modalities to be fused. With calculated: 465
4 different available modalities, there are 6 possible fusion

& '
433
nbitĥL
434 combinations: Fingerprint and face (FP+F), fingerprint and α= 466
435 iris (FP+I), fingerprint and finger-vein (FP+V), face and iris nbitĥS
436 (F+I), face and finger-vein (F+V), and iris and finger-vein After that, m is tweaked such that mL = m, mS = α · m, and 467
437 (I+V). Firstly, the need for alignment is determined for the mS ≥ mL . If the nbit of both templates are the same (i.e., the 468
438 input features. In our case, alignment is needed for the input template sizes are identical), then both templates will use the 469
439 features of both iris and finger-vein. AFH is used on the input same m. After IoM, hS will be rearranged such that hS and 470
440 features of iris and finger-vein to produce an alignment-free hL have the same number of columns. The extra rows of hS 471
441 template before IoM hashing. On the other hand, the input will be discarded so that both hS and hL have the same size 472
442 features of fingerprint and face used do not require alignment before fusion. 473
443 (i.e., they are already in alignment-free form after feature An example of the rearrangement process can be seen in 474
444 extraction), so AFH is skipped and goes directly into IoM Figure 2. The pseudocode for this scenario is divided into 475
445 hashing. two parts (i.e., Algorithm 2 and Algorithm 3) for easier 476
446 Before fusion, the imbalance of different template sizes understanding. Part 1 (Algorithm 2) determines the larger 477
447 between different modalities must first be addressed. To mit- template ĥL from the smaller template ĥS and calculates mL 478
448 igate this problem, the value of m during IoM hashing is and mS . Using parameters from Part 1, Part 2 (Algorithm 3) 479
449 increased for the smaller modality. This essentially increased performs IoM hashing to both templates, rearrangement of 480
450 the number of hashes done on the smaller modality and hS , and the fusion of both templates. 481
451 increases its hashcode length. However, this operation is • Scenario 3: Alignment required for only one of the 482
452 dependent on the necessity of alignment on the input tem- modalities. The remaining 4 fusion combinations fall 483
453 plates. Specifically, this can be divided into three different into this category (i.e., FP+V, FP+I, F+V, and F+I). 484
454 scenarios: m of fingerprint and face depends on the template size 485
455 • Scenario 1: Alignment not required for both modalities. of finger-vein and iris. Let A be the template whose 486
456 This only applies to the case of FP+F. Since AFH is modality does not require alignment (i.e., FP or F) and 487
VOLUME 10, 2022 96393

488 B be the template whose modality requires alignment Algorithm 2. Scenario 2 Part 1
489 (i.e., I or V). For a given value of m, mB = m and Input (From User): Iris AFH-ed template ĥI ∈ Rp×q , Finger-vein
490 mA = nbithB . After IoM, hA is rearranged into the same AFH-ed template ĥV ∈ Rr×s
491 dimension as hB before fusion. This ensures that the Parameter: Projection round m
Output: Projection rounds mL ∈ N, mS ∈ N
492 dimension of A and B always agree for an easier fusion.
1: if size ĥI ≥ size ĥV
493 Just like Scenario 2, the pseudocode for this scenario is
2: Set ĥL = ĥI
494 also divided into two parts (i.e., Algorithm 4 and Algo-
3: Set ĥS = ĥV
495 rithm 5). Since iris or finger-vein template will always 4: else
496 be the larger template ĥ, its mL will set to be equal to 5: Set ĥL = ĥV
497 m while mS for the smaller template x (fingerprint or 6: Set ĥS = ĥI
498 face) is set to be equal to p × m in Part 1, where p is 7: end if
size(ĥL )
499 the number of rows of ĥ. Finally, Part 2 performs IoM 8: compute α =
size(ĥS )
500 hashing to both templates, rearrangement of hS (which 9: compute mS = α · m
501 is the IoM hashed template of x), and the fusion of both 10: Set mL = m
11: return mL , mS , ĥL , ĥS
502 templates.
503 After the difference in template sizes is addressed and the
504 IoM hashing is done, the templates will undergo fusion. Algorithm 3. Scenario 2 Part 2
505 We decided to use logical AND, OR, and XOR as the fusion Input (From User): Larger template ĥL ∈ Rp×q , Smaller template
506 method for this project. Specifically: ĥS ∈ Rr×s , Projection rounds mL ∈ N, mS ∈ N
Parameter: Gaussian random projection matrices WL = {W1 , . . . , WmL }
507 AND : h1 ∧ h2 → f where each Wi ∈ Rq×2 and WS = {W1 , . . . , WmS } where each
Wj ∈ Rs×2
508 OR : h1 ∨ h2 → f Output: Fused bio vectors fAND ∈ [0, 1]p×m , fOR ∈ [0, 1]p×m ,
509 XOR : h1 ⊕ h2 → f fXOR ∈ [0, 1]p×m
1: Initialize hL = [0]p×m , hS = [0]r×m
510 where h1 , h2 , f ∈ [0, 1]i×j and j = m. 2: for i ← 1 to mL
511 After fusion is done, the fused template f will enter the 3: ĥL = ĥL Wi
512 testing phase together with other fused templates. 4: Set v equal to index of ArgMax ĥL
5: hL = v − 1
6: end for
Algorithm 1. Scenario 1 7: for i ← 1 to mS
Input (From User): Fingerprint vector xFP ∈ Rd1 , Face vector xF ∈ Rd2 8: ĥS = ĥS Wj
Parameter: Projection round m, Gaussian random projection matrices 9: Set v equal to index of ArgMax ĥS
WFP = {W1 , . . . , Wm } where each Wi ∈ Rd1 ×2 and WF = 10: hS = v − 1
{W1 , . . . , Wm } 11: end for
where each Wj ∈ Rd2 ×2 12: Set hS = hS ∈ [0, 1]p×m
Output: Fused bio vectors fAND ∈ [0, 1]m , fOR ∈ [0, 1]m , fXOR ∈ [0, 1]m 13: compute fAND = hL ∧ hS
1: Initialize hFP = [0]m , hF = [0]m 14: compute fOR = hL ∨ hS
2: for i ← 1 to m 15: compute fXOR = hL ⊕ hS
3: xFP = xFP Wi 16: return fAND , fOR , fXOR
4: xF = xF Wj
5: Set vFP equal to index of ArgMax(xFP )
6: Set vF equal to index of ArgMax(xF )
7: hFP,i = vFP − 1 on a PC with the specifications: Intel Core i5-3470 CPU @ 521
8: hF,i = vF − 1
3.20GHz, Memory DDR3 @ 8GB, Hard Disk Drive (HDD) 522
9: end for
10: compute fAND = hFP ∧ hF @ 500GB. 523
11: compute fOR = hFP ∨ hF

12: compute fXOR = hFP ⊕ hF A. DATABASES SELECTION AND MATCHING PROTOCOL 524
13: return fAND , fOR , fXOR
For fingerprint, FVC 2002 DB1 (Set A) dataset [30] is 525
used. While the dataset contains 100 fingers with 8 samples 526
each (800 templates in total), only the first 60 fingers with 527
513 IV. EXPERIMENTS 4 samples each are used for this experiment. For face, the 528
514 In this section, a comprehensive evaluation of the recognition templates are extracted from the Labelled Faces in the Wild 529
515 performance of the proposed scheme is presented. The bio- or LFW database [31]. This database contains 13233 images 530
516 metric templates that are adopted in this paper are as follows: of 5749 unique individuals with highly variable image con- 531
517 fingerprint vector xFP ∈ R299 , face vector xF ∈ R512 , iris ditions, but only the first 60 individuals with at least 4 face 532
518 template xI ∈ [0, 1]20×512 , and finger-vein template xV ∈ images are used. For iris, the CASIA-v3-Interval iris database 533
519 [0, 1]111×316 . They are generated using the techniques dis- is used. There are a total of 2639 iris images taken from 534
520 cussed in Sections 2.1 to 2.4. The experiments are carried out 249 subjects in this dataset, but only the first 60 images of 535
96394 VOLUME 10, 2022

Algorithm 4. Scenario 3 Part 1 first template of the remaining individuals to compute 560
Input (From User): Fingerprint or face vector x ∈ Rd , Finger-vein or iris the imposter matching scores. This will generate a total 561
AFH-ed template ĥ ∈ Rp×q of n2 imposter scores in the dataset. 562
Parameter: Projection round m This is similar to an exhaustive cross-validation scheme, 563
Output: Projection rounds mL ∈ N, mS ∈ N
where all possible ways to divide the original dataset into a 564
1: compute mS = p · m
2: Set mL = m
training and validation set are learned and tested. Note that 565
3: return mL , mS both the attempts mentioned above avoid the case of sym- 566
metric matching. A total of 10 repetitions using 10 different 567
sets of GRPs are performed for each parameter combination 568

Algorithm 5. Scenario 3 Part 2 to obtain the average matching accuracy. The GRP used in 569
Input (From User): Fingerprint or face vector x ∈Rd , Finger-vein or a certain repetition and parameter combination is the same 570
iris AFH-ed template ĥ ∈ Rp×q , Projection rounds mL ∈ N, mS ∈ N for all individuals, resulting in a stolen-token scenario and 571
Parameter: Gaussian random projection matrices WL = {W1 , . . . , WmL }
where each Wi ∈ Rq×2 and WS = {W1 , . . . , WmS } where each
removing the influence of GRP on recognition performance. 572
Wj ∈ Rd
Output: Fused bio vectors fAND ∈ [0, 1]p×m , fOR ∈ [0, 1]p×m , fXOR ∈ B. PREPARATION OF TEMPLATES 573
[0, 1]p×m This section describes the extraction process of the vari- 574
1: Initialize hL = [0]p×m , hS = [0]mS ous biometric modalities. There are 4 biometric modalities 575
2: for i ← 1 to mL used in this paper, namely fingerprint, face, iris, and finger- 576
3: ĥ = ĥWi vein. The technique used in [33] was adopted to generate a 577
4: Set v equal to index of ArgMax ĥ
5: hL = v − 1 fixed-length feature vector fingerprint minutia using Minu- 578
6: end for tiae Cylinder Code (MCC) descriptor. The length of the 579
7: for i ← 1 to mS fingerprint vector is fixed at the default dimension of 299 580

8: x = xWj (i.e., d = 299). 581
9: Set v equal to index of ArgMax (x)
For face data, a pre-trained convolution neural network 582
10: hS = v − 1
11: end for known as InsightFace (also known as ArcFace) [34] is 583
12: Reshape hS → hS ∈ [0, 1]p×m adopted. After alignment, the facial images are cropped to 584
13: compute fAND = hL ∧ hS a size of 112 × 112 for InsightFace. Images that failed to 585
14: compute fOR = hL ∨ hS be aligned are discarded. By utilizing InsightFace pre-trained 586
15: compute fXOR = hL ⊕ hS
with MS-Celeb-1M [35], a dataset and benchmark developed 587
16: return fAND , fOR , fXOR
for large-scale face recognition, a face vector of size 512 can 588
be obtained from the LFW database. 589
The method for generating iris templates or IrisCode used 590
536 the left eye with at least 4 samples per subject are used. in this paper is adopted from [36]. Using the weighted adap- 591
537 For finger-vein, University of Twente Finger Vascular Pat- tive Hough and ellipsopolar transforms, the iris region can 592
538 tern Database (UTFVP) [32] is used. The database contains be detected and segmented from the pupil boundaries. The 593
539 60 individuals, 6 fingers per individual, and 4 images per iris region is then unwrapped and averaged to produce the 594
540 finger, totaling up to 1440 images. Images of the first finger iris feature. The iris feature is then extracted using a method 595
541 of all 60 individuals (with 4 images per finger) are used. proposed by Masek [37]. Each vector is convoluted with the 596
542 Equal Error Rate or EER is used to measure the matching 1D Log-Gabor filter to produce a complex iris Gabor feature 597
543 performance. EER is a single value that indicates the point of size 10 × 512. The phase angle of the resulting complex 598
544 where FAR is equal to FRR (i.e., the intersection of genuine value for each pixel of the features is phase-quantized into 599
545 and imposter score distributions). Basically, the lower the 2 bits to generate a binary IrisCode X ∈ {0, 1}n1 ×n2 , where 600
546 value of EER, the better the matching performance of the n1 = 20 and n2 = 512, with a total of 10240 bits per iris 601
547 system. This protocol is widely used among researchers in template. 602
548 the field of biometric system; thus, the outcome of this project During pre-processing, the region of interest (ROI) con- 603
549 can be easily compared with other works. Given n individuals taining the finger-vein patterns is extracted. The visibility of 604
550 with m fingerprint images each (a total of n × m templates) the vein patterns inside the ROI is then enhanced using High 605
551 in a dataset, the attempts at generating genuine / imposter Frequency Emphasis Filtering (HFE) [38], Circular Gabor 606
552 matching scores can be described as: Filter (CGF) [39], and CLAHE (local histogram equaliza- 607
553 • Genuine Matching Attempt: For every individual, each tion). After that, vein feature extraction method is applied 608
554 generated template will be matched against the remain- on the enhanced image. While there are several well-known 609
555 ing templates generated by the scheme to compute the techniques that can output a binary vessel structure, only 610
556 genuine matching scores. This will generate a total of Maximum Curvature (MC) [28] is used for this paper due to 611
557 n×m 2 genuine scores in the dataset. its superior performance [27]. 612
558 • Impostor Matching Attempt: For each dataset, the first Referring to the above, the generated fingerprint and face 613
559 template of an individual will be matched against the features are of vector form with fixed length. Hence, there 614
VOLUME 10, 2022 96395

615 is no alignment needed for the face and fingerprint features. TABLE 3. Raw recognition performance of all modalities used.
616 This means that in our experiments, any input face or fin-
617 gerprint features will go straight into IoM hashing without
618 first going through AFH. On the other hand, the iris feature
619 is extracted using the method proposed by Masek [37], while
620 the finger-vein feature is extracted based on the visibility of
621 the vein pattern inside the ROI. Both of them are subjected
622 to alignment issues due to head tilt and rotation (for iris) and
TABLE 4. Best unimodal recognition performance.
623 finger rotation (for finger-vein). Therefore, any input iris or
624 finger-vein features have to go through AFH first before IoM
625 hashing can take place.
626 C. EFFECT OF PARAMETERS

627 In this experiment, there are three parameters that can be
628 tuned, namely bn , bm , and m. The effect of each parameter
629 is discussed in this sub-section.
tilt during acquisition need to be addressed. A pre-alignment 666
630 1) EFFECT OF PARAMETERS bn AND bm is performed by shifting the query IrisCode ±2 bits horizon- 667
631 The parameters bn and bm determine the window size in tally before evaluating its performance with Jaccard similar- 668
632 AFH, with bn representing the number of rows (or the height) ity. All the tests above are performed on templates of 60 indi- 669
633 and bm representing the number of columns (or the width) viduals with 4 samples each. Finally, the raw performance 670
634 in a window. These two parameters are only applicable to for finger-vein is taken from [27]. Table 3 shows the raw 671
635 modalities that require alignment (i.e., iris and finger-vein). performance of all the modalities. 672
636 The maximum value of bn is bounded by the height of the After that, the raw features undergo AFH and IoM hash- 673
637 iris, which is 20. To maintain consistency, 20 is also set as ing, and their unimodal performance is collected. The range 674
638 the maximum bn value for finger-vein. Suggested by [27], the of parameters used are m = [20, 3000] for fingerprint; 675
639 maximum bm value is set to 60, which is a reasonable trade- m = [100, 10000] for face; bn = [5, 20], bm = [10, 60], 676
640 off between experimental accuracy and time requirement. and m = [20, 200] for iris; and bn = [10, 20], bm = [10, 60], 677
641 Due to the nature of the AFH algorithm, different bn and bm and m = [20, 200] for finger-vein. All tests are repeated for 678
642 values will result in an AFH-ed template ĥ of different sizes. 10 times, and the average is taken. The best results from all 679
643 Besides that, the size of ĥ also depends on the raw template x, the different modalities and the parameters used to obtain it 680
644 so even with the same bn and bm values, the ĥ of iris and are listed in Table 4. 681
645 finger-vein can have different dimensions due to the different The raw and unimodal performances are the baseline used 682
646 sizes of their x. Because of how the scheme is designed, the to measure the validity of the scheme (i.e., the multimodal 683
647 parameters bn and bm together influence the number of rows performance). 684
648 in the final hashed template h.

2) MULTIMODAL PERFORMANCE 685
649 2) EFFECT OF PARAMETER m To find the optimum parameters for AFH and IoM, the tests 686
650 The parameter m determines the number of hashes performed are run with bn = [10, 20], bm = [10, 60], and m = 687
651 during IoM hashing. While there is no hard limit on the range [20, 200] for modalities that require alignment (i.e., iris and 688
652 of m in IoM hashing, the equidistance m values in the range finger-vein). For modality combinations that do not require 689
653 of [20, 200] are selected for iris and finger-vein in accordance alignment (i.e., FP+F), only IoM hashing is used with m = 690
654 with [27]. These two modalities are chosen because they are [20, 10000]. The results are represented in the form of EER, 691
655 the larger modality in terms of template size when fused with and the best results are listed in Table 5 alongside their param- 692
656 either fingerprint or face. The parameter m influences the eters. All the test results are the average of over 10 iterations. 693
657 number of columns in the final hashed template h. Due to the results having a similar trend, only one of them 694
is shown in this paper as an example. Shown below is a 695
658 D. RECOGNITION PERFORMANCE detailed test of FP+V. This test is one of the four fusion 696
659 This sub-section presents the recognition performance of the combinations mentioned in Scenario 3, where only one of the 697
660 proposed scheme and its comparison with the baseline. modalities requires alignment (finger-vein in this case) while 698
the other modality does not require alignment (fingerprint 699
661 1) BASELINE PERFORMANCE in this case). Firstly, finger-vein templates xV ∈ [0, 1]p×q 700
662 Firstly, the recognition performance of the raw features is will undergo AFH with parameters bn = [10, 20] and bm = 701
663 collected. Since fingerprint and face templates are real-valued [10, 60], resulting in an AFH-ed finger-vein template ĥV ∈ 702
664 vectors, cosine distance is used to evaluate their performance. Rr×s . After that, IoM hashing is performed on both ĥV and 703
665 For iris, rotational inconsistencies of the images due to head the fingerprint vector xFP . For finger-vein, the normal IoM 704
96396 VOLUME 10, 2022

FIGURE 3. Recognition performance of FP+V using fusion type OR with parameter bn = 10.
FIGURE 4. Recognition performance of FP+V (OR) with parameter FIGURE 5. Recognition performance of FP+V (OR) with parameter
bm = 60. bm = 60.
705 hashing procedure is performed with m = [20, 200], resulting

706 in the IoM hashed finger-vein template hV ∈ [0, 1]r×m . For
707 fingerprint however, its m (denoted as mFP ) is tweaked such
708 that mFP = r × m in order for the IoM hashed fingerprint
709 template hFP ∈ [0, 1]r×m to match the size of the finger-vein
710 template for easier fusion. Finally, hV and hFP are fused
711 with either logical AND, OR, or XOR to produce the fused
712 template f ∈ [0, 1]r×m .
713 From Figure 4, we can see that the lower the value of bm ,
714 the better the recognition performance in general. The same
715 is true for the case of bn as seen in Figure 5. In addition,
FIGURE 6. Scatter plot of FP+V (OR) up until nbit = 10000.
716 we can also observe that under the same bn and bm , the
717 increase of the value of m can also improve the recognition
718 performance. This is because smaller bn and bm (i.e., a smaller size only yields minimal improvement in performance while 733
719 AFH window) and larger m resulted in a larger final template greatly increasing the runtime. As such, Table 5 lists only the 734
720 size, implying that more information is contained within it. parameters of the smallest template size required for the best 735
721 This is also supported by the theory of LSH, where more recognition performance. 736
722 comparison can result in a better recognition performance It can be seen that in all the tests, the fusion type of AND 737
723 by increasing the probability of collision. This phenomenon demonstrated the best recognition performance while XOR 738
724 can be seen more clearly when a scatter plot of EER against has the worst result. Let x be the biometric template and 739
725 template size (nbit) is plotted. W = wi1 , . . . , wiq be the Gaussian random projection
i 740
726 As illustrated in Figure 6, the recognition performance (GRP) matrix, where i= 1, . . . , m and q = 2. The maximum 741
727 tends to improve in general as the template size increases, value xkj = max Wk x is calculated and its index hi = j−1 is 742
728 regardless of the parameters. However, the largest template is found, where k = 1, . . . @commam. Since q is set to be 2, 743
729 not always the optimal solution, as the EER can plateau long Wk x will result in a vector with only 2 inputs, and the −1 in 744
730 before that. Based on Figure 6, we can see that increasing the equation for hi is done so that index 1 is denoted as 745
731 the template size has significant improvement on the EER 0 while index 2 is denoted as 1 in the IoM hashed template h, 746
732 before nbit = 4000. But after that, increasing template making it a binary template. Because of this, if hi = 1 under 747
VOLUME 10, 2022 96397

TABLE 5. Best multimodal performance and its corresponding TABLE 6. Comparison between proposed scheme and other multimodal
parameters. schemes.
748 index of max, then hi = 0 will be the result under index of

749 min (due to having only two inputs in the projected vector
750 Wk x). In the other words, if h represents the index-of-max
751 hashed template, then ¬h represents the index-of-min hashed
752 template of the same underlying biometric template. This
753 means that both 0 and 1 in h should theoretically contain FIGURE 7. Average time taken per run for FP+V.
754 the same amount of information about the template. But
755 with the matching protocol used in this project (i.e., Jaccard Table 6 shows the comparison with other multimodal 784
756 similarity), which only considers the information contained in schemes. Do note that the difference in databases used and 785
757 1’s in a binary template, it is fair to say that the information techniques applied makes the comparison between different 786
758 contained within the 0’s is lost. multimodal schemes difficult. Even then, it can be seen that 787
759 Using OR, if either one of the templates to be fused the proposed scheme produced comparable or even better per- 788
760 has 1 in a certain location, that location in the fused template formance in terms of recognition accuracy when compared 789
761 will also have a 1 as input. This results in a many-to-one with other relevant schemes. 790
762 mapping in the sense that it is impossible to know if a given
763 input of 1 in the fused template came from which of the
E. TEMPLATE SIZE & COMPUTATIONAL EFFICIENCY 791
764 unfused templates or both of them. Since AND only retains
Figure 7 shows the average runtime of FP+V. It can be 792
765 information where both templates have 1 as input after fusion,
observed that a larger template size results in a longer average 793
766 this makes it lose some of the uncertainty presented in OR,
runtime for the system. The average runtime (including the 794
767 allowing it to be more discriminative. On the other hand, XOR
genuine and impostor tests for all 60 individuals) increases 795
768 further increases the uncertainty by inverting the area where
linearly as the value of nbit increases. Thus, the best results 796
769 both templates have 1 as input, thus losing the information
and parameters listed in Table 5 may not be the most practical 797
770 on that area. Therefore, we can say that the different fusion
choice, as it does not consider the runtime of the scheme. 798
771 types exchange recognition performance for better security
In practical settings, users may want to consider the diminish- 799
772 and vice versa, and none of them is truly superior. With these,
ing return of larger nbit, and selects a suitable set of parame- 800
773 it is possible to explain the difference in performance among
ters that has enough recognition performance and acceptable 801
774 the different fusion types seen in the experiment as a natural
runtime. 802
775 consequence of the protocols used in this scheme.
776 In a nutshell, the scheme is able to convert biometric
777 features of different representations into a common space V. SECURITY AND PRIVACY ANALYSIS 803
778 through the use of AFH and IoM hashing, allowing fusion A. IRREVERSIBILITY ANALYSIS 804
779 to be performed relatively easily at feature-level. Com- The irreversibility property of the scheme with respect to 805
780 pared with baseline performance in Table 3 and Table 4, several attacks against it is analyzed in this sub-section. 806
781 while differences do exist between the three fusion methods

782 (i.e., AND, OR, and XOR), all of them do nonetheless provide 1) REVERSE ATTACK VIA SINGLE RECORD 807
783 good performance as shown in Table 5. The model for this attack is outlined as follows: 808
96398 VOLUME 10, 2022

809 • Adversary’s goal: The adversary aims to recover the • Adversary’s goal: The adversary aims to recover the 862
810 original biometric template x by reversing the compro- original biometric template x by estimating the origi- 863
811 mised cancellable fused template f. nal biometric templates x1 and x2 . x1 and x2 are then 864
812 • Adversary’s knowledge: The adversary possesses the transformed into the fake cancellable template f∗ and 865
813 knowledge of the proposed scheme, including AFH, matched to a compromised template f. 866
814 IoM hashing, and the fusion protocol. • Adversary’s knowledge: The adversary possesses the 867
815 • Adversary’s capability: The adversary compromised the knowledge of the proposed scheme, including AFH, 868
816 GRP matrix W and has access to the parameters used, IoM hashing, and the fusion protocol. In addition, the 869
817 i.e., {bn , bm , m}. On the other hand, the adversary has no adversary also knows the nature of x (i.e., real-valued or 870
818 information on the templates before fusion {h1 , h2 } and binary) and the range of the real-valued templates. 871
819 the scaling factor α since they are not stored. However, • Adversary’s capability: The adversary compromised the 872
820 α can be inferred from the parameters {bn , bm , m}. cancellable template f and has access to the parameters 873
821 Recall the methodology: To recover x, the compromised f used, i.e., {bn , bm , m}. 874
822 must first be un-fused to reveal the individual hashed tem- First, we use the case of I+V as an example due to 875
823 plates of h1 and h2 since they are not stored. However, the the x of both modalities being binary in nature and is 876
824 fusion protocol (AND, OR, and XOR) is a lossy transfor- easier to calculate. Given that the size of the original iris 877
825 mation. While some parts of the templates can be revealed template xI and the original finger-vein template xV are 878
826 by analysing f, a definite h1 and h2 cannot be determined. 20 × 512 and 111 × 316 respectively, the attack complex- 879
827 Nevertheless, the adversary may attempt to guess h1 and h2 ity can be calculated to be 220×512 × 2111×316 = 245316 880
828 based on f, but he or she has no way in verifying the guessed guesses. For real-valued modalities like fingerprint and face, 881
829 h1 and h2 definitely. Even if the adversary somehow is able some amount of leeway in guessing the templates is avail- 882
830 to recover h1 and h2 , it would still be infeasible to recover able. The lower and upper bounds of fingerprint and face 883
831 the AFH-ed templates ĥ1 and ĥ2 even with the help of the templates used in the project are [−0.1828, 0.1593] and 884
832 compromised W. This is because IoM hashing is a one-way [−0.1880, 0.1711] respectively when expressed in a pre- 885
833 transformation. cision of 4 decimal places. The attack s complexity can be 886
834 Alternatively, the adversary may attempt to guess all pos- expressed as kUB − LBk × 10dp + 1 guesses, where UB 887
835 sible ĥ1 and ĥ2 for the binary modalities (i.e., iris and finger- and LB are the upper and lower bounds of the template respec- 888
836 vein) and perform a pre-image attack with the help of the tively, dp is the decimal places used or the precision, and s is 889
837 compromised W and parameters to generate a ‘fake’ can- the template size. For the case of FP+F, the attack complexity 890
299
838 cellable fused template f∗ . After the adversary verified f∗ can be calculated to be k0.1593 + 0.1828k × 104 + 1 × 891
with the compromised f, they can use ĥ∗1 and ĥ∗2 to breach the 4
512 299 512
839
k0.1711 + 0.1880k × 10 + 1 = 3422 × 3592 ≈ 892
840 system and act as if they are the genuine user. However, the 2.15 × 101820 guesses with a precision of 4 decimal places. 893
841 adversary still cannot recover x1 and x2 using ĥ∗1 and ĥ∗2 , this In short, we can conclude that recovering x with the use of 894
842 is because like IoM hashing, AFH used to generate ĥ1 and ĥ2 attack by IE is computationally infeasible. 895
843 from x1 and x2 is a one-way transformation. Not to mention
844 that the real-valued modalities (i.e., fingerprint and face) do B. UNLINKABILITY ANALYSIS 896
845 not undergo AFH and that the adversary need to directly guess To test the unlinkability property of the scheme, a benchmark 897
846 the original biometric template x, which will be discussed in analysis framework developed by Gomez-Barrero et al. is 898
847 Section 3) Attack by Input Enumeration. used [40]. In particular, the assessment of this framework is 899
based on the mated/non-mated samples score distributions: 900
848 2) REVERSE ATTACK VIA MULTIPLE RECORD • Mated samples matching attempt: This is done by 901
849 This attack is related to Attack via Record Multiplic- cross-matching multiple cancellable templates c’s gen- 902
850 ity (ARM), which is a more dreadful instance of privacy erated from the same biometric instance x using different 903
851 attack where multiple compromised templates are involved. GRPs. For this part, up to 4 c’s are generated for each x 904
852 According to [29], the determination of the original biometric (i.e., user), and matching is done among the same user. 905
853 template x using ARM is computationally challenging as • Non-mated samples matching attempt: This is done by 906
854 the stored templates protected by IoM hashing have been matching c’s generated from different x’s using different 907
855 transformed into the rank space, which has no correlation to GRPs. For this part, only 1 c is generated for each x (i.e., 908
856 the original feature space. Thus, it can be inferred that the user), and matching is done between different users. 909
857 attack complexity is identical to that presented in Section 1) The score distributions are then statistically calculated to 910
858 Reverse Attack via Single Record. produce the following two indicators: 911
• Local measure D↔ (s): Measures the linkability of the 912
859 3) ATTACK BY INPUT ENUMERATION (IE) templates on a score-wise basis. 913

sys
860 Apart from reversing f, the adversary may attempt to recover • Global measure D↔ : Measures the overall linkability 914
861 x by enumeration. The attack model can be outlined as follow: of the system, independent of the score domain of the 915
VOLUME 10, 2022 96399

FIGURE 8. Unlinkability analysis of FP+F with fusion type AND (left), OR (center), and XOR (right).
FIGURE 9. Revocability analysis of FP+F with fusion type AND (left), OR (center), and XOR (right).
sys
TABLE 7. Global measure D↔ for unlinkability test with ω = 1. template by re-issuing a different GRP and regenerating a new 937
IoM hash code using it. To further examine the revocability of 938
the scheme, a quantitative experiment built upon 3 score dis- 939
tributions, namely the genuine, impostor, and mated-samples 940
score distribution, is performed. 941
Different GRP matrices W’s are used to generate 51 can- 942
cellable fused templates f (1 as reference template and the 943
remaining 50 as probing templates) for the same input origi- 944
916 system. This allows for benchmarking among different nal template x. Under this model, the mated-samples attempt 945
917 systems is assumed to be acquired by matching old cancellable tem- 946
sys plates to various renewed cancellable templates. Therefore, 947

918 In summary, both D↔ (s) and D↔ which ranged the mated-samples similarity scores should be similar to that 948
919 from 0 to 1, measure the linkability of the system, with of impostor scores. Specifically, the revocability of a system 949
920 0 being fully unlinkable and 1 being fully linkable. In this is validated when the mated-samples and impostor scores 950
921 section, the unlinkability test is conducted using the best distributions are highly overlapped. 951
922 performing parameters listed in Table 5 for all six fusion Using the optimal parameters listed in Table 5, a revoca- 952
923 combinations, with ω set to 1 so that the test is performed bility test is done on the scheme. An example of the results 953
924 under the worst-case scenario [40]. can be seen in Figure 9, where the mated-samples and impos- 954
sys
925 Table 7 shows the global measure D↔ for all modality tor scores distributions are highly overlapped across all the 955
926 combinations and fusion types using the best performing different fusion methods. This suggests that the regenerated 956
927 parameters. In general, the linkability of the system is weak as cancellable templates are unique to each other even though 957
sys
928 most of the D↔ are below 0.1. An example of the mated/non- they are generated from the same x. Other fusion combina- 958
929 mated sample distributions can be seen in Figure 8. We can tions also show similar results, inferring that the revocability 959
930 see that the mated and non-mated sample distributions are criteria are met by the scheme. 960
931 highly overlapped in all of the fusion types, which yields a
932 ‘‘nearly unlinkable’’ scenario according to [40].
VI. CONCLUSION 961
933 C. REVOCABILITY ANALYSIS In this paper, a multimodal biometric authentication scheme 962
934 As described in [29], randomly constructed Gaussian matrix for biometric templates with different representations and 963
935 or GRP are used to generate IoM hash codes. Therefore, alignment-dependent modalities is proposed. We outlined a 964
936 a new template can be generated to replace the compromised procedure that accepts both aligned and unaligned features of 965
96400 VOLUME 10, 2022

966 various origins and representations, and transforms them into [18] C. Rathgeb, M. Gomez-Barrero, C. Busch, J. Galbally, and J. Fierrez, 1034
967 a unified binarized cancellable template. Extensive evaluation ‘‘Towards cancelable multi-biometrics based on Bloom filters: 1035
A case study on feature level fusion of face and iris,’’ in Proc. 1036
968 of the recognition performance of the proposed scheme and 3rd Int. Workshop Biometrics Forensics, Mar. 2015, pp. 1–6, doi: 1037
969 assessment of its unlinkability and revocability are carried 10.1109/IWBF.2015.7110225. 1038
970 out. Security and irreversibility analysis of the scheme have [19] M. Gomez-Barrero, C. Rathgeb, G. Li, R. Ramachandra, J. Galbally, and 1039
C. Busch, ‘‘Multi-biometric template protection based on 1040
971 also been formulated into several attack models for evalua- Bloom filters,’’ Inf. Fusion, vol. 42, pp. 37–50, Jul. 2018, doi: 1041
972 tion. The proposed scheme has been shown to satisfy the four 10.1016/j.inffus.2017.10.003. 1042
973 primary design criteria of a biometric template protection [20] Y. J. Chin, T. S. Ong, A. B. J. Teoh, and K. O. M. Goh, ‘‘Integrated bio- 1043
metrics template protection technique based on fingerprint and palmprint 1044
974 scheme. Our future works consist of a novel key-based fusion feature-level fusion,’’ Inf. Fusion, vol. 18, pp. 161–174, Jul. 2014, doi: 1045
975 technique similar to that in [41], and a deep-learning-based 10.1016/j.inffus.2013.09.001. 1046
976 biometric cryptosystem based on this proposed scheme. [21] Z. Huang, Y. Liu, C. Li, M. Yang, and L. Chen, ‘‘A robust face 1047
and ear based multimodal biometric system using sparse representa- 1048
tion,’’ Pattern Recognit., vol. 46, no. 8, pp. 2156–2168, Aug. 2013, doi: 1049
977 REFERENCES 10.1016/j.patcog.2013.01.022. 1050
[22] M. J. Lee, A. B. J. Teoh, A. Uhl, S.-N. Liang, and Z. Jin, ‘‘A tokenless 1051
978 [1] J. Wayman, A. Jain, D. Maltoni, and D. Maio, Biometric Systems. London,
cancellable scheme for multimodal biometric systems,’’ Comput. Secur., 1052
979 U.K.: Springer, 2005.
vol. 108, Sep. 2021, Art. no. 102350, doi: 10.1016/j.cose.2021.102350. 1053
980 [2] Facial Recognition: Who’s Tracking You in Public?—Consumer
[23] Y.-H. Khoo, B.-M. Goi, T.-Y. Chai, Y.-L. Lai, and Z. Jin, ‘‘Multi- 1054
981 Reports. Accessed: Feb. 25, 2022. [Online]. Available:
modal biometrics system using feature-level fusion of iris and finger- 1055
982 https://www.consumerreports.org/privacy/facial-recognition-who-is-
print,’’ in Proc. 2nd Int. Conf. Adv. Image Process., 2018, pp. 6–10, doi: 1056
983 tracking-you-in-public1-a7157224354/
10.1145/3239576.3239599. 1057
984 [3] M. Hammad, G. Luo, and K. Wang, ‘‘Cancelable biometric authenti-
[24] W. Yang, S. Wang, J. Hu, G. Zheng, and C. Valli, ‘‘A fingerprint and finger- 1058
985 cation system based on ECG,’’ Multimedia Tools Appl., vol. 78, no. 2,
vein based cancelable multi-biometric system,’’ Pattern Recognit., vol. 78, 1059
986 pp. 1857–1887, Jan. 2019, doi: 10.1007/s11042-018-6300-2.
pp. 242–251, Jun. 2018, doi: 10.1016/j.patcog.2018.01.026. 1060
987 [4] X. Wang, K. Fan, K. Yang, X. Cheng, Q. Dong, H. Li, and Y. Yang,
[25] J. Kim, Y. G. Jung, and A. B. J. Teoh, ‘‘Multimodal biometric template 1061
988 ‘‘A new RFID ultra-lightweight authentication protocol for medical privacy
protection based on a cancelable SoftmaxOut fusion network,’’ Appl. Sci., 1062
989 protection in smart living,’’ Comput. Commun., vol. 186, pp. 121–132,
vol. 12, no. 4, p. 2023, Feb. 2022, doi: 10.3390/app12042023. 1063
990 Mar. 2022, doi: 10.1016/j.comcom.2022.01.014.
[26] L. Leng, A. Beng, and J. Teoh, ‘‘Alignment-free row-co-occurrence 1064
991 [5] R. M. Bolle, J. H. Connell, and N. K. Ratha, ‘‘Biometric perils and
cancelable palmprint fuzzy vault,’’ Pattern Recognit., vol. 48, no. 7, 1065
992 patches,’’ Pattern Recognit., vol. 35, no. 12, pp. 2727–2738, 2002, doi:
pp. 2290–2303, Jul. 2015, doi: 10.1016/j.patcog.2015.01.021. 1066
993 10.1016/S0031-3203(01)00247-3.
[27] S. Kirchgasser, C. Kauba, Y.-L. Lai, J. Zhe, and A. Uhl, ‘‘Finger vein tem- 1067
994 [6] G. Paul and J. Irvine, ‘‘IEDs on the road to fingerprint authentica- plate protection based on alignment-robust feature description and index- 1068
995 tion: Biometrics have vulnerabilities that PINs and passwords don’t,’’ of-maximum hashing,’’ IEEE Trans. Biometrics, Behav., Identity Sci., 1069
996 IEEE Consum. Electron. Mag., vol. 5, no. 2, pp. 79–86, Apr. 2016, doi: vol. 2, no. 4, pp. 337–349, Oct. 2020, doi: 10.1109/TBIOM.2020.2981673. 1070
997 10.1109/MCE.2016.2521978.
[28] N. Miura, A. Nagasaka, and T. Miyatake, ‘‘Extraction of finger-vein 1071
998 [7] K. Nandakumar and A. K. Jain, ‘‘Biometric template protection: patterns using maximum curvature points in image profiles,’’ IEICE 1072
999 Bridging the performance gap between theory and practice,’’ IEEE Trans. Inf. Syst., vol. 90, no. 8, pp. 1185–1194, Aug. 2007, doi: 1073
1000 Signal Process. Mag., vol. 32, no. 5, pp. 88–100, Sep. 2015, doi: 10.1093/ietisy/e90-d.8.1185. 1074
1001 10.1109/MSP.2015.2427849. [29] Z. Jin, J. Y. Hwang, Y.-L. Lai, S. Kim, and A. B. J. Teoh, ‘‘Ranking-based 1075
1002 [8] A. Nagar, K. Nandakumar, and A. K. Jain, ‘‘Multibiometric cryptosystems locality sensitive hashing-enabled cancelable biometrics: Index-of-max 1076
1003 based on feature-level fusion,’’ IEEE Trans. Inf. Forensics Security, vol. 7, hashing,’’ IEEE Trans. Inf. Forensics Security, vol. 13, no. 2, pp. 393–407, 1077
1004 no. 1, pp. 255–268, Feb. 2012, doi: 10.1109/TIFS.2011.2166545. Feb. 2018, doi: 10.1109/TIFS.2017.2753172. 1078
1005 [9] A. Ross and A. Jain, ‘‘Information fusion in biometrics,’’ Pattern [30] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain, 1079
1006 Recognit. Lett., vol. 24, no. 13, pp. 2115–2125, Sep. 2003, doi: ‘‘FVC2002: Second fingerprint verification competition,’’ in 1080
1007 10.1016/S0167-8655(03)00079-5. Proc. Int. Conf. Pattern Recognit., Aug. 2002, pp. 811–814, doi: 1081
1008 [10] ISO/IEC JTC1 SC37 Biometrics, Multimodal and Other Multibiometric 10.1109/ICPR.2002.1048144. 1082
1009 Fusion, ISO, Standard ISO/IEC TR 24722:2007, 2007. [31] G. B. Huang, M. Mattar, T. Berg, E. Learned-Miller, and 1083
1010 [11] A. Ross, A. K. Jain, and K. Nandakumar, Handbook of Multibio- E. Learned-Miller. Labeled Faces in the Wild: A Database for 1084
1011 metrics, vol. 6. Boston, MA, USA: Kluwer, 2006, doi: 10.1007/ Studying Face Recognition in Unconstrained Environments Labeled 1085
1012 0-387-33123-9. Faces in the Wild: A Database for Studying Face Recognition in 1086
1013 [12] L. Leng, M. Li, C. Kim, and X. Bi, ‘‘Dual-source discrimination Unconstrained Environments. Accessed: Mar. 14, 2022. [Online]. 1087
1014 power analysis for multi-instance contactless palmprint recognition,’’ Available: https://hal.inria.fr/inria-00321923 1088
1015 Multimedia Tools Appl., vol. 76, no. 1, pp. 333–354, Jan. 2017, doi: [32] B. T. Ton and R. N. J. Veldhuis, ‘‘A high quality finger vas- 1089
1016 10.1007/S11042-015-3058-7. cular pattern dataset collected using a custom designed capturing 1090
1017 [13] E. J. C. Kelkboom, X. Zhou, J. Breebaart, R. N. J. Veldhuis, and C. Busch, device,’’ in Proc. Int. Conf. Biometrics (ICB), Jun. 2013, pp. 1–5, doi: 1091
1018 ‘‘Multi-algorithm fusion with template protection,’’ in Proc. IEEE 3rd 10.1109/ICB.2013.6612966. 1092
1019 Int. Conf. Biometrics, Theory, Appl., Syst., Sep. 2009, pp. 1–8, doi: [33] Z. Jin, M.-H. Lim, A. B. J. Teoh, B.-M. Goi, and Y. H. Tay, ‘‘Generating 1093
1020 10.1109/BTAS.2009.5339045. fixed-length representation from minutiae using kernel methods for fin- 1094
1021 [14] M. Singh, R. Singh, and A. Ross, ‘‘A comprehensive overview of gerprint authentication,’’ IEEE Trans. Syst., Man, Cybern., Syst., vol. 46, 1095
1022 biometric fusion,’’ Inf. Fusion, vol. 52, pp. 187–205, Dec. 2019, doi: no. 10, pp. 1415–1428, Oct. 2016, doi: 10.1109/TSMC.2015.2499725. 1096
1023 10.1016/j.inffus.2018.12.003. [34] J. Deng, J. Guo, J. Yang, N. Xue, I. Cotsia, and S. P. Zafeiriou, ‘‘Arc- 1097
1024 [15] M. Hammad, Y. Liu, and K. Wang, ‘‘Multimodal biometric authentication Face: Additive angular margin loss for deep face recognition,’’ IEEE 1098
1025 systems using convolution neural network based on different level fusion Trans. Pattern Anal. Mach. Intell., early access, Jun. 9, 2021, doi: 1099
1026 of ECG and fingerprint,’’ IEEE Access, vol. 7, pp. 26527–26542, 2019, 10.1109/TPAMI.2021.3087709. 1100
1027 doi: 10.1109/ACCESS.2018.2886573. [35] Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, ‘‘MS-Celeb- 1101
1028 [16] N. K. Ratha, A. Senior, and R. M. Bolle, ‘‘Automated biometrics,’’ 1M: Challenge of recognizing one million celebrities in the real 1102
1029 in Proc. Int. Conf. Adv. Pattern Recognit., 2001, pp. 447–455, doi: world,’’ Electron. Imag., vol. 28, no. 11, pp. 1–6, Feb. 2016, doi: 1103
1030 10.1007/3-540-44732-6_46. 10.2352/ISSN.2470-1173.2016.11.IMAWM-463. 1104
1031 [17] P. P. Paul and M. Gavrilova, ‘‘Multimodal cancelable biometrics,’’ in [36] C. Rathgeb, F. Breitinger, and C. Busch, ‘‘Alignment-free cancelable iris 1105
1032 Proc. IEEE 11th Int. Conf. Cognit. Informat. Cognit. Comput., Aug. 2012, biometric templates based on adaptive Bloom filters,’’ in Proc. Int. Conf. 1106
1033 pp. 43–49, doi: 10.1109/ICCI-CC.2012.6311208. Biometrics (ICB), Jun. 2013, pp. 1–8, doi: 10.1109/ICB.2013.6612976. 1107
VOLUME 10, 2022 96401

1108 [37] L. Masek, ‘‘Recognition of human iris patterns for biometric identi SHIUAN-NI LIANG received the B.Sc. degree in 1156
1109 cation,’’ M.S. thesis, Univ. Western Aust., 2003. [Online]. Available: mathematics from the National Central University, 1157
1110 https://www.peterkovesi.com/studentprojects/libor/index.html Taiwan, in 2000, the Ph.D. degree in physics from 1158
1111 [38] J. Zhao, H. Tian, W. Xu, and X. Li, ‘‘A new approach to hand vein image Monash University, in 2012, and the M.Sc. degree 1159
1112 enhancement,’’ in Proc. 2nd Int. Conf. Intell. Comput. Technol. Autom., in physics from the National Central University, 1160
1113 2009, pp. 499–501, doi: 10.1109/ICICTA.2009.127. in 2013. She is currently a Lecturer at the School of 1161
1114 [39] J. Zhang and J. Yang, ‘‘Finger-vein image enhancement based on Engineering, Monash University, Malaysia Cam- 1162
1115 combination of gray-level grouping and circular Gabor filter,’’ in
pus. Her research interests include biophysics, 1163
1116 Proc. Int. Conf. Inf. Eng. Comput. Sci., Dec. 2009, pp. 1–4, doi:
biomedical engineering, nonlinear dynamics, and 1164
1117 10.1109/ICIECS.2009.5363431.
1118 [40] M. Gomez-Barrero, J. Galbally, C. Rathgeb, and C. Busch, ‘‘General machine learning. 1165
1119 framework to evaluate unlinkability in biometric template protection sys-

1120 tems,’’ IEEE Trans. Inf. Forensics Security, vol. 13, no. 6, pp. 1406–1420,
1121 Jun. 2018, doi: 10.1109/TIFS.2017.2788000.
1122 [41] H. Guesmi, H. Trichili, A. M. Alimi, and B. Solaiman, ‘‘Novel biometric
1123 features fusion method based on possibility theory,’’ in Proc. IEEE 14th
1124 Int. Conf. Cognit. Informat. Cognit. Comput., Jul. 2015, pp. 418–425, doi:
1125 10.1109/ICCI-CC.2015.7259419.
ZHE JIN (Member, IEEE) received the Ph.D. 1166
degree in engineering from the Universiti Tunku 1167
Abdul Rahman (UTAR), Malaysia. He visited the 1168
University of Salzburg, Austria, and the Univer- 1169
sity of Sassari, Italy, respectively, as a Visiting 1170
Scholar under the EU Project IDENTITY. He is 1171
1126 ZHENG HUI GOH received the B.Eng. degree currently a Professor at the School of Artificial 1172
1127 (Hons.) in mechanical engineering from the Uni- Intelligence, Anhui University, China. He has pub- 1173
1128 versity of Malaya, Malaysia, in 2018. He is cur- lished more than 50 refereed journals and confer- 1174
1129 rently pursuing the Ph.D. degree with Monash ence papers, including the IEEE TRANSACTIONS ON 1175
1130 University Malaysia. His research interests include INFORMATION FORENSICS AND SECURITY, IEEE TRANSACTIONS ON SYSTEMS, MAN, 1176
1131 information security, biometrics, and machine AND CYBERNETICS: SYSTEMS, DSC, and PR. His research interests include 1177
1132 learning. biometrics, pattern recognition, computer vision, and multimedia security. 1178
He was awarded Marie Skłodowska-Curie Research Exchange Fellowship. 1179
1133 YANDAN WANG received the Ph.D. degree in YEN-LUNG LAI received the B.Sc. degree in 1180
1134 information technology from Monash University, physics from the Universiti Tunku Abdul Rahman 1181
1135 Malaysia. She is currently working as a Lecturer (UTAR), Malaysia, in 2015, and the Ph.D. degree 1182
1136 at Wenzhou University, China. Her research inter- from Monash University Malaysia, in 2021. He is 1183
1137 ests include multimedia indexing and retrieval, currently an Assistant Professor at the Lee Kong 1184
1138 machine vision, and pattern recognition. Chian Faculty of Engineering and Science, UTAR. 1185
His research interests include information security 1186
and biometrics. 1187
1139 LU LENG (Member, IEEE) received the Ph.D.

1140 degree from Southwest Jiaotong University, XIN WANG (Senior Member, IEEE) received 1188
1141 Chengdu, China, in 2012. the Ph.D. degree from Nanyang Technological 1189
1142 He performed his postdoctoral research at University (NTU). 1190
1143 Yonsei University, Seoul, South Korea, and the She was a Research Fellow at the Robotics 1191
1144 Nanjing University of Aeronautics and Astronau- Research Center, NTU, from 2007 to 2009. She 1192
1145 tics, Nanjing, China. He was a Visiting Scholar was an Assistant Professor at the Faculty of 1193
1146 at West Virginia University, USA, and Yonsei Engineering and Science, Universiti Tunku Abdul 1194
1147 University. He is currently a Professor at Nan- Rahman, from 2009 to 2012. She was a Visit- 1195
1148 chang Hangkong University. He has published ing Professor at the National Metrology Centre, 1196
1149 more than 100 international journals and conference papers and has been Singapore, from November to December 2011. 1197
1150 granted several scholarships and funding projects for his academic research. She was a Visiting Professor at the Centre for Optical and Laser Engineering, 1198
1151 He is a reviewer of several international journals and conferences. His NTU. She is currently an Associate Professor at the School of Engineering, 1199
1152 research interests include computer vision, biometric template protection, Monash University Malaysia. She has published over 30 articles in peer- 1200
1153 and biometric recognition. He is a member of the Association for Computing reviewed journals. Her research interests include optical metrology, remote 1201
1154 Machinery (ACM), the China Society of Image and Graphics (CSIG), and sensing, non-destructive testing, and machine vision. 1202
1155 the China Computer Federation (CCF). 1203
96402 VOLUME 10, 2022

C2 A_Framework_for_Multimodal_Biometric_Authentication_Systems_With_Template_Protection

Uploaded by

Copyright:

Available Formats

You might also like

C2 A_Framework_for_Multimodal_Biometric_Authentication_Systems_With_Template_Protection

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

C2 A_Framework_for_Multimodal_Biometric_Authentication_Systems_With_Template_Protection

Uploaded by

Copyright:

Available Formats

Received 11 August 2022, accepted 25 August 2022, date of publication 8 September 2022, date of current version 19 September 2022.

Digital Object Identifier 10.1109/ACCESS.2022.3205413

A Framework for Multimodal Biometric

Corresponding author: Zhe Jin (jinzhe@ahu.edu.cn)

1 ABSTRACT A multimodal biometric authentication framework based on Index-of-Max (IoM) hashing,

18 INDEX TERMS Feature-level fusion, multimodal biometrics, privacy and security.

19 I. INTRODUCTION widely used in various applications ranging from identity 26

23 hand/finger geometry, retina, gait, palmprint, voice pattern, users [3]. 30

the need of remembering a password. However, this can 33

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

39 issued. features [8]. While some of the current multimodal schemes 95

46 shortcomings is to use biometric template protection (BTP) compromised. 102

61 to be revoked or reissued from the underlying biometric

78 robustness [9]. biometric system is a viable option. 133

VOLUME 10, 2022 96389

193 modalities. EP-DFT added a wavelet transformation to f before P-DFT. 249

96390 VOLUME 10, 2022

as low privacy risks [13]. To ensure the successful imple- 286

mentation of the framework, the following issues have to be 287

handled first: 288

• Incompatibility of feature spaces: The feature spaces of 289

different biometric modalities or features extracted from 290

different extractors may not be compatible. For example, 291

the real-valued representation of face and fingerprint 292

features, and the binary presentation of the iris and 293

finger-vein features make fusion difficult if they do not 294

have a common representation. 295

• Large dimension of fused features: The fusion of features 296

by simple concatenation is not preferred, since this can 297

result in large dimensionality of the features, dramati- 298

cally increasing the required storage space and running 299

• Alignment issue of biometric features: For some biomet- 301

ric features, alignment is needed to obtain optimal per- 302

formance during matching. However, performing such 303

together [26]. This issue cannot be ignored if optimal 305

performance is to be obtained from a system. This can 306

be seen in [19], where the authors used pre-aligned 307

and alignment-free templates for their scheme. How- 308

with unaligned templates. Example of modalities that 310

requires alignment for feature matching includes the iris 311

and finger-vein feature used in this paper [18]. 312

In order to address the challenges mentioned above, 313

263 overall. • A general framework for multimodal biometric authen- 318

parameters and biometric modalities to discover the 337

VOLUME 10, 2022 96391

341 combinations of the three parameters bn , bm , and m are TABLE 2. Nomenclature.

transform a real-valued biometric feature into discrete index 395

1 if vi = vj max-valued projected feature is taken. This is equivalent to 412

X Euclidean pairwise distance preservation property in the pro- 417

379 given binary vector V . Based on the equation above, we can

384 required by global features used in algorithms like in [28].

389 undergo AFH in this paper. in Figure 1. 429

96392 VOLUME 10, 2022

FIGURE 2. Example of rearrangement process for bn = 10, bm = 20,

not required, m in IoM hashing is the only variable 457

fused template: mFP = mF = nbitf = m. A pseudocode 459

• Scenario 2: Alignment required for both modalities. 461

4 different available modalities, there are 6 possible fusion

VOLUME 10, 2022 96393