AssamUniversityJournalofScience& TechnologJ' ISS\ 097i-277i

PhysicalSciencesandTechnology
Vol.SNumber II
2l- 31.201I

Informationfusionin MultimodalBiometricSystems
ShahinAra Begum* ond Th. Churjit Meetei
Department
ofComputerScience, Silchar- 78801l, tndia
AssamUniversity,
tCorrespondence;
e:mail:shahin.begum.ara@gmail.com

Abstract
Biometric systen con identifi individuals based on lheir inherent properties. Multimodql biometic systenr
can alleviate nany of the limitalions of unimodal biomelric systens os the different biometric evidences
compensatefor the inherent limitations of the olher sources. Multimodql biometric systent clesign depends on
vqriousfaclors s ch as sources ofinforma-tion, acquisition qnd processing architeclurc, level ofinformation
fusion anclftrsion nethodologt In this paper we present the state-of-art strategies of infornation fusion in
mult imodqI biom etric system.

Keywords: Biometric,MultimodalBiometric, InformationFusion

Introduction
Biometricsdealswith automatically identifying biometric)suchasgender,ethnicity,age,eyecolor,
individuals
basedon theirinherentphysicaland/or skin color, scars and tatoos also provide some
behavioralcharacteristics (Jainera/.,2004b).By information about the identity of a person.
usingbiometrics, it is possibleto establish an However, soft biometric traits do not provide
identitybasedon lrhowe are,ratherthanbywhat sufficient evidenceto preciselydeterminethe
we possess,suchas a passport,tokenor key,or identity(Jaine/ a/., 2004a).Unlike the possession
what we rentenrbe,,, suchas a password,PIN. basedandknowledgebasedidentityauthentication
schemes,the biometric identifiers cannot be
A numberofphysicalandbehavioral bodytraits
misplacedforgotten guessedor be easily forged.
canbeusedfor biometricrecognition asshownin
However,eachbiometric trait has its advantages
figurel. Examples of physicaltraitsincludeface,
and limitations,andno singletrait is expectedto
fingerprint,iris, palmprint,handgeometryear
effectively meet all the requirementssuch as
shape,fingervein,gait, signatureandkeystroke
accuracy,practicalityand cost imposedby all
d y n a m i cs a r e some of the behavioral
applications(Jainet al.,2004b).Hence,thereis
characteristics that can be usedfor person
no universallybestbiometrictraitandthe choice
authentication. Voicecanbeconsidered eitheras
of biometric depends on the nature and
anphysicalor asa behavioraltrait because certain
requirements of the application.
characteristics ofa person'svoice such as pitch,
bass/tenor andnasalityaredueto physicalfactors Despite the inherent advantages,biometric
like vocaltractshape,andothercharacteristicssystemsthat operateusingany singlebiometric
suchas word or phonemepronunciation (e.g. characteristichavethe following limitations(Ross
dialect),useof characteristic wordsor phrases et a|.,2006): (i) Norse in senseddala: The
and conversational stylesare mostlylearned. biometricdata beingpresentedto the systemmay
Ancillarycharacteristics (also knownas soft be contaminated by noise due to imperfect
 Infor \aIionfits ion i n.\ |ultintotltt1.....

acquisitionconditionsor subtlevariationsin the matchingscorelevel is generallypreferreddue

biometric itself (i.e. accumulationof dirty on to the easein accessing andcombiningmatching
sensoror poorly illuminatedenvironrnent).(ii) scores.
Inlra-class vuriutions: The biometric data
The restofthe paperis organizedas follows: in
acquiredfrom an individualduringauthentication
the section2, we presenta brief introductionof
may be very differentfrom that duringenrollment,
multimodalbiometricsystem.In section3, we
thereby affecting the matching process.This
presentdifferent strategiesof information fusion
variation is typically causedby a user r,vhois
at different levels, with an application to
incorrectly interactingwith the sensor(i.e.
multimodalbiometricsystem.The conclusionis
incorrect facial pose) or when sensor
in section4.
characteristics are modified (e.g., by changing
sensor)duringthe verificationphase.(iii) irler- Multimodal Biometric
clos.ssintilcn"ity:Every biometric trait has some
A systemtltat usesmore than one behavioural
theoretical upper bound in terms of its
and/or physiological characteristicsto verify
discrirnirratiorrcapability.ln a biometricsystern whethera person
is who he/sheclaims to be is
Iravinglargernurnbersofusers,therernaybe inter-
called a multimodalbiometricauthentication
classsirnilaritiesor overlap.(iv) 1y'oz
-universality: system(Norman Poher a|.,2005).ln
otherwords,
The biomen'icsystemmay not be ableto acquire
systemstltatconsolidateevidenceoblainedfrom
meaningfulbiornetricdata from a subsetof
multiple biometric modalsin order to reliably
individuals.For example,it may not possibleto
detenninethe identityofan individualareknown
obtaina goodqualityfingerprintfrom a subsetof
as rnultimodalbiometric systems(Ross e/.j/.,
peoplewith hand-related disabilities,manual 2006).Multimodal
biometricsystemscanalleviate
rvorkerslvith many cuts and bruiseson their
manyof the limitationsof unibiometricsystems
fingertips,andpeoplewith very oily or dry fingers.
because the differentbiometricevidences usually
Theseleadin a failure-to-enroll(FTE)error.(v)
compensatefor the inherentlimitationsof the
Spoof attucks: Behavioraltraits such as voice
othersources(Honget al., 1999).lJse of multiple
and signatureare vulnerableto spoof attacksby
sourceof biometricinformationfrom substantially
an impostor attemptingto mimic the traits
independentbiometric sensors,algorithms,or
corresponding to Iegitimatelyenrolledsubjects. modalities
typically gives improvedtechnical
Physicaltraits such as fingerprintscan also be
performance andreducesrisk (ANSI, 2006).The
spoofed by inscribingridge-like structureson
overallaccuracyofthe biometricsystemdepends
syntheticmaterialsuchas gelatineand play-doh
on an effective fusion schemethat is used to
(Matsumoto?/ (/1, 2002).
combinethe evidencesobtainedfrom different
Multimodalbiornetricsystemcan addresssome modalitiesusing an effectivefusion scheme.A
ol'the abovernentionedissues.In a multimodal multirnodalbiometricsystemcanreducethe False
biometric systernthat usesdifferent biometric AcceptanceRate (FAR)iFalseRejection Rate
traits,inlonnatiorlfrom rnultiplesources (modals) (FRR) and provide more resistanceagainst
can be integratedat various levels:fusion at the spoofingbecauseit is diflicult to simultaneously
settsor level, the feoture extraction level, spoof multiple biometricmodals.Multimodal
ntotchittgscore level and,decision level.It is biornetricsystemscanalsoprovidethecapability
difficult to consolidateinformationatthe feature to searcha largedatabasein an efficient and fast
level becausetl'le feature sets used by different manner.This can be achievedby usinga relatively
biometricmodalitiesmay eitherbe inaccessible simple but lessaccuratemodality to prunethe
or incompatible. Fusionatthedecisionlevelis too databasebefore using the more complex and
rigid sinceonly a limitedamountof informationis accuratemodality on the remainingdata to
availableat thislevel.Therefore, integrationat the performthe final identificationtask.
 I nfor mat i onrtrs ion i n I I uI t i n oclct1....

& (b)

F (e)

lJlu/e,s
(k)
Fig. l. Examplesof biometric
ffi fl)

traits:(a)Fingerprint;(b) Hand-geornetry; (c) Iris;(d) Retina;

(e)Face;(f) Palmprint;(g) Earstructure; (h) DNA; (i) Voice;C) cait; (k) Signature; (l) Keystroke
dynamics; (m) FacialThennogram; (n) blinkingand(o) lip movement (Rosseral, 2006).

InlormationFusion

Pre-classificatioo

r' -,.-..\ --\ DlnamicClJssitier

S(lcction
SensorLevel FcatureLevel
I
Y
i) WeightedSummation
ii) Concatenation

Rank level Abstract/DecisionIev€l

II II
Y Y
CombinationApproach i) tlighest Rank r) Majority Voting
+
i) NeuralNetrvorks
+
i) Nomralization+ Linear
ii) BordaCount
iii) Logistic
ii) BehaviorKnorvledge
Space
iii) DempstcrShalerTheory
ii) k-NN combinationofscores+ Regression
ol-Evidence
iii) DecisionTrees Thresholding
iv) AND Rule
iv) SVM ii) Nonnalization+ (Suol.
v) OR Rule
Max. Min) Rules+
Product.
Thresholding

Fig. 2. Approachesto informationfusion(Jainer a/., 2005)

Horvever,multimodalbiometricsystems
alsohave storage than unimodal biometric s)stems.
some disadvantages.They are more expensive Multimodalbiometricsystemsgenerallrrequire
and requiremore resources
for computationand moretimefor enrollmentandverification.
causins

-23 -
I nlormati onfus i on i n llul t i ntoda1.....

someinconvenience totheuser.Finally,
thesystem approaches fusionis presented
to information in
accuracy canactuallydegrade compared to the figure2.
unimodalsystemif a propertechniqueis not Fourtypesof informationcanbe obtainedfrom
followedfor cornbining theinformationprovided
thebiometricsources, namely, (i) rawbiometric
by the differentmodalities.However,the samples, (ii) featuresets,(iii) matchscoresand
advantages of multimodal systemsfar outweigh (iv) decisionlabels.In multimodalbiometric,
thelimitations andhence, suchsystems arebeing
accordingto thetypeof information usedin fusion,
i n c r e asi ng lyd e p loyedin security-critica I
thefusionscheme canbeclassified as(a)sensor
applications. level(b) featurelevel,(c) scoreleveland (d)
Level of Information Fusion decisionlevelfusiondepending on the typeof
provided information thatis fused(Mishra,2010).Various
Theprocess of integratinginformation
levelsoffusionin a biometric system is shownin
by differentbiometricsourcesis knownas
figure3.Amongtheselevels,it is relatively easy
biometricfusion.Oneof the mainissuesin the
andcombinethematchscoresgenerated
to access
designof a multimodalbiometricsystemis to
thatshouldbe by differentbiometricmatchers.Fusionat
detennine thetypeof information
matchingscorelevelneedsrelativelylesswork
fused. Jain et al. (2005) have classified
to carry out on testingand evaluatingthe
information fusionin biometricsystems intotwo
performanceof multiple-biometricsystems.
broadcategories: pre-classiJication
ftrsion and Consequently, scorelevel fusion is the most
posl-cI assificat ion fus ion. Pre-classifi cati on
commonlyusedapproach ln multimodalbiometric
fision (Fusionbeforematching)refersto systems.
combining information priortotheapplication of
any classifieror matchingalgorithm.In posl- FusionPrior to Matching
c lassif cationfusi oz (Fusionaftermatching),the Priorto matching,
integration
of information
from
infonnation is combined afterthedecisions ofthe multiplebiometricsourcescantakeplaceeither
classifiershave been obtained.Different at the sensorlevelor at the featurelevel.

Se n9o r
L q v eI
Fuslon
D oc is lon
Lov €l
F us ion

F oa l urc
L € v el
Fuslon ffi

FE : featureextractionmodule
MM: matchingmodule
DM : decision-making module
FM : fusionmodule

(Rosset a1.,2006)
Fig.3. Fusionlevelsin a biometricsystem
I nlbrnai i oi fus i on.i n M ul t i n oda1.....

Sensor Level Fusion . than $or.rl d be possi bl e usi ng JU S tscore-l e\el

In sensorlevel fusion,the informationiiom the combinationto give betteroverallperformance.
different sensorsmust.be compatibleand the Fusionat featurelevel is difficult to achievein
correspondences betweenpointsin the raw data practiagdue lo the following reasons(Jainelal..
mustbe eitherknownin advance(e.g.,calibrated 2005):
camerasystems)or reliably estimated.So, only
Jhe relationshipbetrveenthe featurespaces
fe* researchers practicefusion at this leVelfor
of different biometric sourcesmay not be
multimodalbiometric.Changel a/. (2003)usethe
linorvn.In the casewherethe relationshipis
normalizedmaskedear and face images of a
kno$ri in advance,care needsto be takento
subjectto form a combinedface-plus-ear image
discard those features that are highly
and suggestthai that the multi-modalbiometric
co'rrelated.This requiresthe applicationof
offers substantial performance gain. The
[eat.r.rre sclection a lgorithms prior t!)
differencein.therank-onerecognitionratesfcr
blassificaticn.
tfre day variationexperimentusingthe 197-vnagd
trainingsetsis 90.9%for themulti-modal biometric The featuresets may be.incompatible.F'or
versts 72.7Vof6i the ear and 69.3% for the tbce. .example, the minutiadsetof finger'-printiand
In anotherwork, Kai'anwalel a/. (2010)taketrvc eigenface coefficientscannot be directly
imagesoffrice andfinger.Waveletdecornposiiion combinedbecausethe former is a variable
is performed oir'both images using I)iscretc length featureset B,hoseindividual values
Wavelet'frahsform(DWT), followed by o l'usion represent theattributes ofa minutiapointwhile
ofdecompositionof two imageswhich produces .!he laiter is a fixed length feature set whose
fused image of low resolution.Ttren. scale individrralvaluesarescalareniitics.
invariant feature transform (SIFT) fearures are Thc 'curseofdimensionality'problem,themost
extractedfrom the fused image.After e;rtractir,.g colnmon problem of pattern recognition
SIFT features, matching is perforrneci. applicationsmay occur while concatenating
Experimental Results shows that the method t'.vofeaturevectorsresultsin a fearurevector
produced95Yo accuracy- rvith iargerdimen-sionality. It is moresevere
Feature Level Fusion in biornet-ricapplications because ofthe time,
cftbrt and cost involved in collectinglarge
In feature level fusion. different fbature setsthat
auttuntsof biometric(training)data.
are extractedfrom muitiple biometric sollrce.:ilre
combined befole matching.Featurcscan be A significantlymorecomplexmatchermight
combined in several ways. In nrultimodal be required in order to operate on the
biometric, the feature sets are generally non- concatenatedfeature vector
homogeneous(e.g., feature sets of dif-ferent Most commercialbiometricsystemsdo not
biometricmodalitiesIikefaceandhandgeometry) nrovideaccessto the featuresetsusedin their
or setsifthere is no implicit correspondence, we
frroductsdue to proprietary reasons.
can combine the featurevectorsto form a single
featureset, the combinedfeaturevector. Feature L) e to the constraints mentionedabove,most
selectionschemescan then be appliedto reduce of the attemptsat feature level fusion have
the dimensionalityof the resultantfeatureset. met only limited success.Hence,very few
Concatenationis not possiblewhen the feature researchers have studied integrationat the
setsare incompatible(e.g.,fingerprintminutiae feature level in a multibiometricsystemand
and eigenfacecoe{ficients). Where the features fusion schemesat the match score and
are not independent,between those from the decision levelsaregenerallypreferred.
contributingmultibiometrics, good feature-level Examplesof featurelevel fusion schemes
combinationshouldallow thedependencies to be proposedin the literaturecan be found in
exploitedmorefully, at leastin somecircumstances, Chibefuslri
et al., 1997(voiceand lip shape):
Inlor t.ttiottli$ion it1\lultinto4a!.....
Clrang cl al.. 2003 (ear and face); Ktmar el al.,
2005 (hand geornetryand palmprint);Rossand
Govindarajan,2005 (face and handgeometry);
Yongshen et u|.,2005 (tace and palmprint);
Rattatliet o1.,2007(Face and fingerprint);Yanel
a/., 2008 (face and palrnprint);AloysiusGeorge,
2008(Faceandfingerprint)andChin et a|.,2009
(palnrprintandfingerprint).
Fusiou after Matching
stage,schemes
Atter the classii'ication/matcher
for informationfusion can be divided into four
categories:(i) dynamicclassifierselection,(ii)
fusionat tlrernatchscorelevel,(iii) fusionat the
rank leveland(iv) fusiouat the decisionlevel.
Dynamic Classifier Selection
clussiJier-selectiottschemechooses
A tlynn1111,
thebiornetricsourcethatis rnostIikelyto givethe
correct decision for the specific input pattern
(Woodse1ul.. 1997)and is lirnitedto biometric
applications.
Match Score Level Fusion
Match score is a rneasureof the similarity
betweenthe inputandternplatebiometricfeature
vectors.In rnLrltirnodalsystem,eachsub-system
providesnratching scoresindicatingtheproximity
of the featurevector with the templatevector.
'l-hesescorescan then be combinedto improve
thc rnatchingperform:urce. When match scores
outpr.rtby different biornetric matchersare
con-solidated in order to arrive at a final
recognitiondecision,fusion is saidto be doneat
thernatchscorelcvel.This is alsoknorvnasfusion
at the n!cd.\urcnk:ntlevl or confdence level.
fl1"'r'l,,r. ljl
r r.: I \ !l'r:rr,,rrrrk
fbr scorelevel firsionu'ith Scorcnorrrtalization
Theefficientrnechanism for combinationof scores
r,vithina multirnodalbiometricsystemare(i) each
biometricprocessmust producea score,rather
than a hard accept/reject decision,and make it
availableto the fusionmodule.(ii) [n advanceof
operationaluse, each biometric processmust
rnakeavailableto thefusionmodule,its technical
performancein the appropriatefonn.
Sone comrnoncharacteristics of the matching
scoresin a multimodalsystemcanbesumrnarized
as: (i) the matching score of the individual
matclrersare not in a dornain.(ii)The matching
scoresat the output of the individualmatchers
may not be homogeneous.For example, one
matchermay output a distance(dissimilarity)
measurewhile anothermay output a proximity
(similarity) measure;(iii) The outputs of the
individual matchersneed not be on the same
numericalscale(range);(iv) The matchingscores
at the output ofthe matchersmay follow different
statistical
distributions.
Score normalizarion refers to changing the
locationand scale parametersof the matching
scoredistributionsat theoutputsofthe individual
matclrers,sothatthe matchingscoresofdifferent
matchersaretransformedinto a commondomain
(Jainet o1.,2005).Scorenormalizationis closely
relatedto matchscorelevel fusion since it affects
how scoresarecombinedandinterpreted in terms
of biometricperfonnance.The pararneters used
for nonnalizationcanbedetenninedusinga fixed
training set or adaptivelybasedon the current
feature vector.
i.,\:r'SI,2006)
 Infornation fus ion i n l{ul tinocltl.....

Hence,scoresare generallynormalizedprior to Based

on the different rnodalities,the fusion
fusioninto a commondomain.FigLrre 4 depictsa methodologies are listedin table L
scorelevel fusionfrarneworklor processingtwo
biometricsamples,taking normalizationinto Rank Level Fusion
account.There are two types of normalization When the output of eachbiometric
systentis a
schemes(Snelibket a1.,2005):(i) schernesthat subsetofpossiblematches (i.e.,iden-tities)sorted
modiflzthe locationand scaleparameters ofthe in decreasing orderofconfidence,the firsioncau
score distributione.g. the min-max, z-score, be doneat the rzrk tevel(Jai.rcr ai., 2005).
This
Median absolutedeviation (MAD) and tanh is relevantin an identificationsystentwhere
a
techniques;and (ii) schernesthat consideronlv rank may be assignedto the top !natcl.tin-q
the overlapregionof the gentrineand irposto, identities. Ho eial ( I 994)describe threemerhods
scorese.g.Two-quadrics (QQ) andeuadric-line_ to combine the ranks assignedb1,diflerent
quadric(QLQ).Generally, category( ii) techniques matchers.In the highest rank method, each
are usedafter havingappliedoneofthe category possibleidentityis assignedthe hest(minimunr)
(i) schemes. It canbe notedthat,sorn"s.or. l-"uel ofall rankscomputedby differcntsystenrs. Ties
lusi onrnethods useprobabilitydensitlfuncriorrs are brokenrandornlyto arriveat a strict ranking
(PDFs)directlyanddo not reouirenormalization orderandthe t'inaldecisionis rnade
basedon the
methods. consolidated ranks.The Bordacountrnethoduses
Therearetwo distinctapproaches the sum ofthe ranksassignedby the individual
to matchscore
levelfusion in the contextofverification(Jainar systemsto a pafticularidentiw in orderto calculatc
al.,2005). They are (i) clossifcation crpprotrch the fusedrank.The logisticregression methodis
and (ii) contbinarion upproach. In the a generalization ofthe Borda countmethodrvhere
classificationapproach,a fearure vector ts a u,eightedsum ofthe individualrarrksis Lrsed.
formulatedusingthematchingscoresourpurgtven The weights are deter-minedusine logisric
by the individualmatchers; regressron. AnotlrertechniqLre for rarrkleveliirsion
andit is classifiedinto
oneoftwo classes: is the mixed group ranks approach (Meluir c; a1..
"Accept"(genuine)or...Reject.'
(impostor).Generally, 2004), which atternpts to find a tradeoll bcir.r,c-en
the classifier(e.g.decision
tree,neuralnetwork,supportvectorrnachine,K- the general preference for specifrc nratchers and
nearestneighbour,randomforest,etc.) usedfor theconfidencein specificresults(as indicatedbv
this approachis capableof learningthe decision the ranks).Nandakutnarc,/n/. (2009) o!5sn,g
boundaryirrespectiveof how the featurevector thatthe recognitionperformanccoi the sirrplcst
rsgenerated. ranklevelfusionscheme, namely.tirchighcstrauh
ln thecombination approach, a single
scalar score is generatedby combining the method,is cornparableto the perfrrrmancetrf'
individualmatchingscoresarrdthe final decision complexfusion strategies. especiallyrrhen the
goal ls not to obtainthe bestrank-| acourac\l)r,l
is made based on this. Generally, score
normalizationis perfbnnedbeforefusion of thc tojust rctrievcrhetop fiJ\\rnrtclrcs. RankL,:rcl
scoresfrom the differentmodalitiesto cnsurea tirsiou is a metlrettl of decision Ievel tirsion
(Gokbcrker a/.. 2005:Chin et ot..200()).
meaningfulfusionofthe scoreslrom thediff'erent
modalities.
 lnlot nmttonJu.\ionin .ll ltitnoJol . .

Reference FusionMethodology Modalities Fused

Brunellieral., 1995 Geometric
weighted
average;
HyperBF
Bignn et ql.,199'7 Statisticalmodel basedon Bayesiantheory
Kinleretol.,1998 Sum.product. min.maxandmedian rules
Ben-Yacoub eta/.,1999 Support VectorMachine;
multilayer
perceptron;C4.5decision
tree;Fisher's Faceand voice
lineardiscriminant;
Bayesianclassi0er
Snelicke,aL,2005 Sum.product. min,maxandmedian rures
Yajariaeta|.,2007 Sumrule;maxrules,min-maxandz-score Faceand voice
normalization
Frischholz
eta1.,2000 Weightedsum rule; majorityvoting Face,voiceand lip movement
Snelick
e/aL,2005 Sumrule,Weighted sumrule
Rossetal,200i Sumrule;decisiontrees, Face,fingerprint
and
lineardiscriminant
function handgeometry
Ja inet ql. , 1999 Face.fingerprintand voice
Wanget d|.,2003 Sumrule;weighted sumrule;
Fisher'sIineardiscriminant;
neuralnetwork Faceand iris
Shakhnarovich et a1.,2001 Sumrule Faceand gait
Kale et a1.,2004 Sumandproductrules
Toh et q1.,20Q4 Weighted sumrule Fingerprint,handgeometry
andvoice
Tohet a|.,2003 Reducedmultivariate polynomialmodel Fingerprintand handgeometry
Ricardoera/.,2009 Likelihoodratiousingfuzzylogic. Face,fingerprint
Raghavendra
et al, 2010 Gaussian
MixtureModel (CMM);
MonteCarlosampling
basedhypothesistesting Face,palmprint
Darwishel al, 2009 Principal
Component Analysis Face,ear
Razzaket al., 2010 WeightedFuzzyfusion Face,fingerveins

TableI Matchscorelevelfusionmethodolosies
DecisionLevel Fusion outcomes sothatfusionrulescanbeformulated
In a multimodal biometricsystem,thedecisions aslogicalfunclions.It is basedonthebinaryresult
outputbytheindividual biometric
matchersshould values match and non-ntalch output by the
be availablein orderto carryout fusionat the decision modules. Voting schemes havebeen
obstractor decisionlevel.Decisionlevelfusion establishedasfusion rules, themost commonof
isonly feasible
solutionwhencommercial-off-the-whichis rnajorityvotingrule.TheAND andOR
shelf (COTS) matchersare usedto build a arespecific examples ofvotingschemes. Methods
multimodal biometricsystern
asCOTSbiometric Pro.po_sed in theliterature for decision
level
fusion
matchersprovideaccessonly to the final include "AND"and"OR"rules(Daugman,2000;
recognitiondecisiorr. Andreweta1.,2002;Alkootetal-,2007),majorit,
Decision
levelrusion
canbecarried
onatthe J:llHllj;:;:fl;:!f.,')$lr:""ntedmajoritv
followinglevels:
(a)Sintpte
Levet:-For
murtimodar.biometric
fl,-':#:i^::i"l;i1j$:]"r1ffi1i::fi:f
systems,logical values are assignedto match sample.The two
sub-groups
ofadvanceddecision

-28-
l nfor mationfns lotl itr lrlul tinoda l.....

level fusion are (i) layered and (ii) cascoded summary,

(ANSI, 2006).A layeredsystemusesindividual
biometric scoresto determinethe pass/fail
thresholdsfor other biometricdata processing.
Cascadedsystemsuse pass/fail thresholdsof
modality-specific biometric samplesto determine
if additional biometric samples from other
modalitiesare requiredto reachan overall system(ii) Fusionat thematchscorelevelhasreceived
decision.
Someof the classifierDecisionfusion methods
are Bayesiandecisionfusion (Xu et al., 1992;
Kryszczuk et a1.,2007), the Dempster-Shafer
theory of evidence (Xu er al., 1992), behavior
knowledge space (Huang et al., 1995),
MultivariatePolynomialModel (Kar-Ann Toh,
2004) andk-NN neighborhoodclassifier(Andrew
et al.,2002\.
Summary
MultimodalBiometricis a very promisingapproach
to person verification and identification. It
combinesthe advantagesof different techniques
and may perform betterthan unimodal biometric
system. The design of multimodal biometric
system is governed by several different factors
includingthe (a) sourcesof information,(b) the
acquisitionandprocessingsequence (c) the type
of information and (d) the fusion strategy.In
thefollowingisconcluded:
(i) lnformationfusionat the featurelevelis a
difficult taskbecausethe featuresetsusedby
differentbiometricmodalitiesmay eitherbe
inaccessible or incompatible and 'curseof
dimensionality'problemoccurs.
the maximumattentionfrom the biometrics
community. However,mostofthe proposed score
levelfusionschemes involvead-hoctechniques
lor normaliz-ing thematchscores andassig;ing
optimalweights to differentmatchers.Scorefusion
in a multimodal biometric verificationsystemcan
beformulated asa hvo-class problem
classification
anda sig-nificant number oftrainingsamples are
usuallyavailable for boththegenuine and impostor
classes.
(iii) Fusionatthedecisionlevelis toorigidsince
onlya limitedamount (accept/
of information not
accept) is available
atthislevel.Butit isthemost
suitabletechniquefor multimodalbiometric system
if commercial-off-the-shelf(COTS)matchers are
used-
Employingappropriatefusion strategies can
further improvethe accuracyof the system.
Hence,further evaluationof each of the
informationfusionstrategies
is required.

References
Alkoot,F.;Qasem, H. (2007)."AComparison ofSoft Ben-Yacoub, S.;Abdeljaoued, Y.;Mayoraz,E.(1999)-
FusionMethodsUnderDifferentBaggingScenarios", "FusionofFaceandSpeech Datafor Person Identity
6th WSEASInt. Conference on Computational Verification",IEEETransactions onNeuralNetworks,
lntelligence,Man-Machine Systems andCybernetics, l0(5),pp.l065-107s.
Tenerife, Spain. Bigun,E.S.;Bigun,J.; Duc,B.; Fischer, S. (1997).
AloysiusGeorge(2008)."BizarreApproaches for "Expert Conciliationfor Multi-modalPerson
MultimodalBiometrics',IJCSNSlntemational Joumal Authentication Systems usingBayesian Statistics",
ofComputerScienceandNetworkSecurity,Vol.8(7), In Proceedings of First InternationalConference on
pp.64-69. Audio- and Video-BasedBiometric Person
AndrewBengJin Teoh;SalinaAbdul Samad;Aini Authentication (AVBPA).pp. 291-300.Crans-
Hussain (2002)."DecisionFusionComparison for a Montana, Switzerland.
BiometricVerification
SystemUsingFaceandSpeech", Brunelli,R.;Falavigna,D. ( 1995).
"Personldentification
Malaysian Journal
ofComputer Science,
Vol.l5 No.2, UsingMultipleCues".IEEETransactions on Pattem
pp.t7-27. AnalysisandMachinelntelligence, l7(10),pp.955-
ANSI, 1506 (2006). "Multimodal and Other 96r.
Multibiometric Fusion",ISO/lECPDTR24722.ISO/ Chang,K.; BowyegK.t Bamabas, S.(2003).
V.;Sarkar.
IEC. "Comparison and.combination ofearandfaceimages

-29 -
Infofilotion filsi on in lhtltintodul.....

in appearance basedbiometrics",IEEE Trans'on (BioAW),Vol.LNCS3087,pp.259-269, Prague, Czech

PatternAnalysisand MachineIntelligence 25, pp. Republic,Springer.
l 160.1165. Jain,A.K.; Ross,A; Prabhakar, S. (2004b)."An
Chang,K.; Bowyer,K.W.;Sarkar,S.;Victor,B. (2003). Introduction to Biometric Recognition", IEEE
"Comparison andCombination ofEarandFaceImages Transactions on Circuits and Systems for Video
in Appearance-based Biometrics",IEEETransactions Technology, SpecialIssueon lmage-andVideo-Based
on PatternAnalysisandMachinelntelligence,25(9), Biometrics, l4(t):+20.
pp.t 160-1165. Jain,A.K.; Nandakumar, K.; Ross,A. (2005)."Score
Chibelushi, C.C.;Mason,J.S.D.;Deravi, F. (1997). Normalization in Multimodal BiometricSystems",
"Feature-level DataFusionfor Bimodal Recognition,
Pattern 38(12), pp.227 0-2285.
PersonRecognition", [n Proceedings of the Sixth Kale,A.; RoyChowdhury A.K.;Chellappa, R.(2004).
lntemational Conference on Image Processing andlts "Fusion ofcait and Face for Human Identification", tn
Applications, Vol. l, pp.399-403, Dublin, Ireland. IEEE International Conference on Acoustics, Speech,
SignalProcessing (ICASSP), Vol.5,pp.901'904,
ChinYJ.; OngT.S.;GohM.K.O.;HiewB'Y.(2009) and
Montreal, Canada.
"lntegratingPalmprintand Fingerprintfor ldentity
Verification",Third lnternationalConference on Kar-AnnToh (2004).'A ReducedMultivariate
NetworkandSystemSecurity. PolynomialModel for MultimodalBiometricsand
R.;FawziAli,A' (2009). Fusion",IEEETransactions
Classifiers on Circuisand
Darwish, A.A.;AbdElghafar,
Systems forVideo Technologr, Vol. l4(2),pp-224-233.
"Multimodal Faceand Ear lmages",Journalof
Computer Science 5 (5),pp.374-379. Karanwal, S.;Kumar,D.; Maurya, R. (2010). "Fusion
andFaceby UsingDWT and SIFT",
Daugman, J.(2000)."CombiningMultipleBiometrics", of Fingerprint
Journalof Computer Applications (0975
Availableathttp://www.cl.cam. ac.uk/users/jgdl000/ lnternational
-8887),Volume 2 (5),pages 33-36.
combine/combine.html.
U. (2000)."BiolD: A Kittler, J.; Hatef, M.; Duin, R.P.;Matas,J.C.(1998).
Frischholz, R.: Dieckmann,
"On Combining Classifiers", IEEETransactions on
MultimodalBiometricIdentificationSystem",IEEE
PatternAnalysis and Machine lntelligence,20(3), pp.
Computer, 33(2),pp.64-68.
226-239.
Cokberk,B.; Salah, A.A.;Akarun,L. (2005)."Rank-
Face Kryszczuk, K.; Richiardi,l.; Prodanov, P.;Drygajlo,A.
BasedDecision Fusion for 3D Shape-Based
pp.t0l9'1028. (2007). "Reliability-Based Decision Fusion in
Recognition," LNCS3546:AVBPA,
Multimodal BiometricVerificationSystems",
Ho, T.K.;Hull, J.J.;Srihari,S.N.(1994)."Decision EURASIPJournalon Advancesin SignalProcessing
Combination in MultipleClassifier Systems", IEEE
Volume,ArticleID 86572.
Transactions on PatternAnalysisand Machine
l6(l), pp.66-75. KumaqA.;Zhang,D. (2005)."Personal Authentication
lntelligence,
usingMultiplePalmprintRepresentation". Pattern
Hong, L.; Jain, A.; Pankanti,S' (1999)."Can Recognition, 38(l0),pp.1695- l 704.
Multibiometrics Improveperformance?", Proceedings
NJ,pp.59-64. Kuncheva, L.l. (2004)."CombiningPattemClassifiers
AutolD'99, Summit,
- MethodsandAlgorithms".
Huang, YS.;Suen, C.Y.(1995). "MethodofCombining
T.; Matsumoto, H.;Yamada, K.; Hoshino,
MultipleExpertsfor theRecognition ofUnconstrained Matsumoto,
(2002). "lmpact ofArtificial "Gummy" Fingers on
Handwritten Numerals",IEEETransactions on Patlern S.
Systems", tn Optical Security and
Analysis andMachine I ntelligence,I 7(I )' pp.90-94. Fingerprint
Counterfeit Deterrence Techniques lV Proceedings of
Jain,A.K.; Hong,L.;Kulkami, Y.(1999). "AMultimodal
4677,pp.275-289.
SPIE,volume
BiometricSystem usingFingerprint, FaceandSpeech",
on Audio-and MelnikO.;Vardi,Y.;Zhang, C.H.(2004)."MixedGroup
ln Second International Conference
Ranks: Preference and Confidence in Classifier
Video-basedBiometricPersonAuthentication
Combination", IEEE Transactions on Pattern Analysis
(AVBPA). pp.182-187,Washington D.C.'USA.
andMachine Intelligence, 26(8).pp.973-981.
Jain,A.K.lNandakumar. K.; Lu,X ; Park.U. (2004a).
(2010)."MultimodalBiometrics it is:
"lntegrating Faces.Fingerprints andSoft Biometric MishraAshish
ofECCV. Need for Future Systems", Intemational Journal of
TraitstbrUscrRecognition", In Proceedings
Authcntication Computer Applications (0975 - 8887), Vol. -
-.i No.4.
Iotcrnntion{lWorkshop on Biometric

-30-
 Informati ohfus i on in l4ultimodal.....

Nandakumar, K.; Jain,A.K.;Ross, A. (2009).,,Fusion Systems", IEEETransactions on pattemAnalrsisand

in MultibiometricIdentificationSystems: Whatabout Machine Intelligence,27(J),pp.450455.
theMissingData?",Toappearin proc.oflCB, Alghero.
Toh,K.A.,JiangX.,Yau,W.y (2004)..,Exptoiting Gtobal
NormanPoh;SamyBengio(2005)..,Can Chimeric and Local Decisionsfor MultimodalBiomerrics
PersonsBe Used in Multimodal Biometric Verification", IEEETransactions on Signalprocessing-
Authentication Experiments", IDIAp Research Report (Supplementon SecureMedia), 52(10), pp.3059-j07i.
05-20.
Toh,K.A.;Xiong,W; yau, Wy.; Jiang,X. (200J).
Raghavendra R; RaoA; Hemantha KumarG. (2010). "Combining Fingerprintand Hand-Geom etr\.
"MultimodalbiometricscorefusionusingCaussian VerificationDecisions".
ln Fourthlnternational
mixturemodelandMonteCarlomethod.'lJournal of Conference on Audio-and Video-based Biometric
Computer Science andTechnologr 25(4),pp.77IJBZ. PersonAuthentication (AVBpA),pp. 688-696.
Raftani,A.; Kisku,D. R.; Bicego,M.; Tistarelli, M. Guildford,UK.
(2007)."FeatureLevelFusionofFaceandFingerprint Ulery B.; Hicklin,
A.R.; Warson,C.; Fellner.W.:
Biometrics",978-l-4244-t 59't-7/07,2007IEEE Hallinan,P (2006).,,Studies of BiometricFusion".
RazzakMd. tmran; Rubiyahyusof; MarzukiKhalid Technical ReportlR 7346,NIST.
(2010)."Multimodalfaceandfingerveinsbiometric Vajaria,H.; lslam,
T.; Mohanty,p; Sarkar,S.;Sankar,
authentication", ScientificResearch andEssays.Vol. R.; Kasturi,R. (2007).,,Evaluation andanalysisofa
5(t7).pp.2s2g-2s34 face and voice outdoormulti-biometric svstem"-
RicardoN. Rodrigues;Lee Luan Ling; Venu PanemRecognition Leners28,pp. l5'12-l586.
Govindaraju(2009)..,Robustness of multimodal Wang, Y.;Tan,T.;Jain,A.K. (2003). ,.Combining
Face
biometricfusionmethodsagainstspoof attacks", andIris Biometrics for ldentityVerification",
In Fourth
JournalofVisual Language andComputing. International
Conference on Audio-andVideo-based
Ross,A.;Govindarajan, R.(2005).Feature LevelFusion BiometricPersonAuthentication (AVBpA),pp. g05_
UsingHandand FaceBiometrics.In proceedines of 813,Guildford,UK.
SPIEConference on BiometricTechnology for Hriman Woods,K.; Bowyer,K.; Kegelmeyer, Wp. (1997).
ldentificationll, volume5779,pp. 196-204, Orlando, "Combination of MultipleClassifiers UsingLocal
USA. AccuracyEstimates",IEEE Transactions on pattem
Ross,A.; Jain,A.K. (2003),..lnformationFusionin Analysis andMachine Inrelligence, l9(4),pp.405410.
Biometrics",PatternRecognition Letters,2a(r, pp- Xu, L.; Krzyzak, A.; Suen,C.y.( 1992).,,Methods for
2u52t25. CombiningMultipleClassifiersandtheirApplications
Ross,A.; Nandakumar, K.; Jain,A.K. (2006). to Handwriting Recognition". IEEETraniactions on
"Handbook of Multibiometrics". Springer Systems,Man, and Cybemetics, 22(3),pp.418_435.
Shakhnarovich, G.; Lee, L.; Darrell,T.J. (2001). Yongsheng, G; Michael,M. (2005).,.Feature-Level
"[ntegrated ,Face andGaitRecognition fromMultiple Fusionin Personal ld€ntification",proceedinss ofthe
Views",tn IEEEConference on Compuler Visionan 2005IEEEComputerSociety Conference onComputer
Pattem Recognition (CVpR),pp.439-446, Hawaii,USA. VisionandPattern Recognition.
Snelick,R.;Uludag,U.;Mink,A.; tndovina, M.; Jain, Yan Yan and Yu-JinZhang(2008).,.Multimodal
A.K. (2005)."LargeScaleEvaluation of Multimodal Biomerics Fusion UsingConelation FilterBank..,
IEEE
BiometricAuthenticationUsing State-of-the-Art -978-t4244-2t75-6/08.

-i l