Professional Documents
Culture Documents
18юcamera and Radar Sensor
18юcamera and Radar Sensor
0XOWLOHYHO)XVLRQ6FKHPHIRU7DUJHW&ODVVLILFDWLRQXVLQJ
&DPHUDDQG5DGDU6HQVRUV
(XJLQ+\XQDQG<RXQJ6HRN-LQ
$57$GYDQFHG5DGDU7HFKQRORJ\/DE&RQYHUJHQFH5HVHDUFK&HQWHUIRU)XWXUH$XWRPRWLYH7HFKQRORJ\
'*,67'DHJX*\HRQJEXN,QVWLWXWHRI6FLHQFH 7HFKQRORJ\'DHJX6RXWK.RUHD
EUDKDP#GJLVWDFNUFRUUHVSRQGLQJDXWKRU
7KXV WKH FRQFHSW RI
VHQVRU IXVLRQ
UHIHUULQJ WR D
$EVWUDFW - We propose a multi-level fusion scheme for target
FRPELQDWLRQ RI UDGDU DQG FDPHUDV KDV UHFHQWO\ DWWUDFWHG WKH
detection using camera and radar sensors. For the proposed
DWWHQWLRQ RI UHVHDUFKHUV ZKR DWWHPSW WR HQKDQFH WKH WDUJHW
scheme, the radar sensor provides the target track
LQGLFDWLRQ SHUIRUPDQFH FDSDELOLWLHV RI DXWRPRWLYH
information of the range, velocity, angle, and track ID. This
DSSOLFDWLRQV (VSHFLDOO\ FRQVLGHULQJ WKDW WKH (XUR 1&$3
data is applied during the vision processing step as the ROI
1HZ &DU $VVHVVPHQW 3URJUDP UHTXLUHV WKH GHWHFWLRQ RI
(region of interest) where the target may exist. Next, for
SHGHVWULDQV DQG ELF\FOHV VHQVRU IXVLRQ ZLOO EH QHFHVVDU\ LQ
feature-level fusion, the Doppler spectrum of the ROI is
WKHIXWXUH>@
provided to the sensor-fusion-based target classifier. In the
classifier, we then determine the class of the target using an
$V QRWHG DERYH UDGDU VHQVRUV KDYH KLJK UDQJH DQG
image database and a Doppler pattern database. In the
YHORFLW\ GHWHFWLRQ SHUIRUPDQFH FDSDELOLWLHV EXW DQJOH
experimental results, we verify the proposed processing
GHWHFWLRQ UHVROXWLRQ LV ORZ ,Q WKH FDPHUD VHQVRU WKH
scheme using a 24 GHz FMCW transceiver with a single
HVWLPDWLRQRXWFRPHVRIWKHSRVLWLRQRIDKRUL]RQWDOWDUJHWDUH
antenna.
YHU\JRRGEXWWKHUDQJHGHWHFWLRQHUURULVYHU\KLJK
.H\ZRUGVVHQVRUIXVLRQUDGDUVHQVRUDXWRPRWLYHUDGDU
%DVHG RQ WKHVH FRQFHSWV ZH LOOXVWUDWH WKH H[SHFWHG
IXVLRQUHVXOWVRIV\VWHPVLQFRUSRUDWLQJERWKFDPHUDDQGUDGDU
VHQVRUV LQ )LJXUH 7KH RYHUODS RI ERWK VHQVRUV LV WKH ILQDO
,QWURGXFWLRQ GHWHFWHGWDUJHWSRVLWLRQ>@
:KLOHWKHUROHVRIYHKLFOHVLQFOXGHGRQO\WUDQVSRUWDWLRQ
LQ WKH SDVW VDIHW\ KDV EHFRPH DQ LPSRUWDQW UHTXLUHPHQW RI
YHKLFOHV 7KXV YDULRXV $'$6V DGYDQFHG GULYHU DVVLVWDQW
V\VWHPV KDYH GHYHORSHG DQG PDQ\ VXFK DSSOLFDWLRQV KDYH
EHHQLPSOHPHQWHGLQWRFRPPHUFLDOYHKLFOHVUHFHQWO\)RUWKH
HIIHFWLYH RSHUDWLRQ RI DQ $'$6 WKH VHQVRUV DUH YHU\
LPSRUWDQW ZLWK WKH PRVW SRSXODU VHQVRUV EHLQJ FDPHUD DQG
UDGDUVHQVRUV>@
2Q WKH RWKHU KDQG UDGDU VHQVRUV DUH UREXVW WR H[WHUQDO
FRQGLWLRQV DQG WKH GHWHFWLRQ UDQJH DQG YHORFLW\ DUH KLJKO\
DFFXUDWH7KXVUDGDUVHQVRUVKDYHEHHQDSSOLHGLQUHODWLRQWR
$&&DGDSWLYHFUXLVHFRQWURO/&$ODQHFKDQJHDVVLVWDQFH
)&: IRUZDUG FROOLVLRQ ZDUQLQJ DQG %6' EOLQG VSRW
GHWHFWLRQV\VWHPV+RZHYHUWKHDQJOHSRVLWLRQRIWKHWDUJHW
LV HVWLPDWHG DW D ORZ UHVROXWLRQ DQG WKH WDUJHW FODVVLILFDWLRQ
SHUIRUPDQFHLVOLPLWHGLQWKHVHV\VWHPVDWSUHVHQW>@ )LJXUH 7KH W\SLFDO LQVWDOODWLRQ SRVLWLRQV RI FDPHUD DQG
UDGDUVHQVRUV
)LJXUH VKRZV WKH W\SLFDO LQVWDOODWLRQ SRVLWLRQV RI FRPSXWDWLRQ FRPSOH[LW\ LV UHTXLUHG GXH WR WKH ODUJH DPRXQW
FDPHUDDQGUDGDUVHQVRUV+HUH5WDUJHWDQG9WDUJHWDUHWKHUDQJH RIGDWDDQGWKHORQJSURFHVVLQJWLPHV
DQGYHORFLW\RIWKHWDUJHWDVGHWHFWHGE\WKHUDGDUVHQVRU2Q
WKHRWKHUKDQGWKHFDPHUDVHQVRUSUHGLFWVWKHGLVWDQFHRIWKH 7KXV LQ WKLV SDSHU ZH SURSRVHG D PXOWLOHYHO IXVLRQ
WDUJHW EDVHG RQ WKH SHUVSHFWLYH ,Q DGGLWLRQ WKH GLVWDQFH VFKHPH EDVHG RQ UDGDU DQG FDPHUD VHQVRUV DV LOOXVWUDWHG LQ
EHWZHHQ WKH FDPHUD DQG UDGDU VHQVRUV LV 5RIIVHW 7KXV WR )LJXUH 7KH SURSRVHG PHWKRG KDV WKUHH IXQFWLRQV UDGDU
UHDOL]HVHQVRUIXVLRQFRRUGLQDWHPDSSLQJLVQHFHVVDU\GXHWR VLJQDOSURFHVVLQJYLVLRQSURFHVVLQJDQGIXVLRQSURFHVVLQJ
WKH GLIIHUHQW FRRUGLQDWHV RI WKH VHQVRUV 0RUHRYHU WLPLQJ
V\QFKURQL]LQJVKRXOGDOVREHFRQVLGHUHGGXHWR WKH GLIIHUHQW
IUDPHUDWHVRIWKHVHQVRUV
)LQDOO\LQORZOHYHOIXVLRQWKHRULJLQDOLPDJHVDQGWKH
UHFHLYHG UDGDU VLJQDO DUH XVHG ,Q WKLV FDVH KRZHYHU WKH
GHWDLOHGIXVLRQGHVFULSWLRQVKDYH\HWWREHLQWURGXFHG
)LJXUH7KHSURSRVHGPXOWLOHYHOVHQVRUIXVLRQVFKHPH
52, 7KLV LQIRUPDWLRQ LV IHG LQWR FDPHUD VHQVRUV WKURXJK )RU WKH UDGDU VHQVRU ZH PHDVXUHG WKH UDQJH SURILOH DV
FRRUGLQDWH PDSSLQJ SURFHVVLQJ %DVHG RQ WKH 52,V WKH UHIOHFWHGIURPDOOWDUJHWVLQFOXGLQJWKHVWDWLRQDU\WDUJHWVVXFK
YLVLRQ VHQVRU H[WUDFWV WKH WDUJHW IHDWXUH ZLWK WKH GHWHUPLQHG DV WKRVH VKRZQ LQ )LJXUHV D +RZHYHU LQ WKLV FDVH ZH
ZLQGRZGHWHFWVREMHFWVDQGSURFHVVHVWKHWUDFNLQJRXWSXW FRXOG GLVWLQJXLVK WKH PRYLQJ KXPDQ EHFDXVH RI WKH VWURQJ
VWDWLRQDU\ WDUJHWV LQFOXGLQJ FOXWWHU 7KXV EHIRUH DGDSWLYH
,QUDGDUVHQVRUVWRGLVWLQJXLVKEHWZHHQVLJQDOVUHIOHFWHG WKUHVKROGLQJ ZH DSSOLHG D 07, PRYLQJ WDUJHW LQGLFDWLRQ
IURP SHGHVWULDQV DQG YHKLFOHV GLIIHUHQW IHDWXUHV RI WKH DOJRULWKP>@7KLVUHVXOWLVVKRZQLQ)LJXUHE
YHORFLW\SURILOHVDUHXVHG7KDWLVZKLOHSHGHVWULDQVKDYHDQ
H[WHQGHG'RSSOHUVSHFWUXPGXHWRWKHLUNLQHPDWLFSURSHUWLHV
D PRYLQJ YHKLFOH KDV D VLQJOH UDGLDO YHORFLW\ >@ 0RUHRYHU
DGGLWLRQDO PLFURSDWWHUQV H[LVW LQ WKH 'RSSOHU IUHTXHQF\
VSHFWUXP DSSHDULQJ DV VLGHEDQGV DURXQG WKH 'RSSOHU
IUHTXHQF\RIWKHEXONPRWLRQRIDPRYLQJKXPDQ>@7KLVLV
WKHPRVWLPSRUWDQWFOXHIRUXVHLQWKHSURSRVHGVHQVRUIXVLRQ
PHWKRG
7KXVLQWKHVHQVRUIXVLRQVWHSRIWKHSURSRVHGPHWKRG
WDUJHWFODVVLILFDWLRQLVFDUULHGRXWEDVHGRQWKHYDULRXVLPDJH
GDWDEDVHV WRJHWKHU ZLWK D 'RSSOHU GDWDEDVH ,Q WKH 'RSSOHU
GDWDEDVH WKH FRUUHVSRQGLQJ 'RSSOHU VSHFWUXP SDWWHUQV RI
YDULRXV REMHFWV DQG PRWLRQV DUH VDYHG 7KDW LV WKH WDUJHW
FODVVLILFDWLRQ SHUIRUPDQFH FDQ LQFUHDVH XVLQJ WZR REMHFW
FKDUDFWHULVWLFVRULHQWHGIURPWZRGLIIHUHQWVHQVRUV7RGRWKLV
D
WKH 'RSSOHU VSHFWUXP RI WKH 52, LV SURYLGHG WR WKH WDUJHW
FODVVLILHUYLDPLGOHYHOIXVLRQ
F
)LJXUH7KHPHDVXUHGUDQJHSURILOHVRIWKHUDGDUVHQVRU
DIHZFHOOVFDQEHVHOHFWHGDVWKHHIIHFWLYHVFDWWHULQJSRLQWVRI WKH KXPDQ DFFXUDWHO\ LI WKH FDPHUD VHQVRU LV RQO\ HPSOR\HG
REMHFWV+RZHYHULQ)LJXUHFZHRQO\PDUNHGRQHUDQJH IRU WDUJHW FODVVLILFDWLRQ +RZHYHU LQ WKLV SDSHU ZH PRQLWRU
'RSSOHUFHOOZLWKWKHPD[LPXPPDJQLWXGH7KLVLVLQGLFDWHG WKH 'RSSOHU VSHFWUXP IRU VHQVRUIXVLRQEDVHG WDUJHW
E\WKHUHGFLUFOH FODVVLILFDWLRQ )LJXUH VKRZV WKH 'RSSOHU VSHFWUXP RI WKH
PRYLQJ REMHFW VHOHFWHG LQ )LJXUH F 7KXV EDVHG RQ WKH
8VLQJ WKH GHWHFWHG LQIRUPDWLRQ EDVHG RQ WKH 52, WKH LPDJH IHDWXUH DQG 'RSSOHU SDWWHUQ ZH FDQ GHWHUPLQH WKH
YLVLRQVHQVRUFDQILQGWKHREMHFWVRILQWHUHVW)LJXUHVKRZV FODVVRIWKHREMHFWVHOHFWHGE\WKH52,RIWKHUDGDUVHQVRU
WKHVHOHFWHGKXPDQZLWKDZLQGRZER[LQGLFDWHGE\WKHZKLWH
OLQH DQG WKH FRRUGLQDWLQJ PDSSLQJ SRLQW GHQRWHG E\ WKH UHG &RQFOXVLRQV
FLUFOH'XULQJWKHPHDVXUHPHQWVWHSLQWKLVSDSHUEHFDXVHWKH
VLQJOH UHFHLYH UDGDU DQWHQQD LV XVHG ZH FDQQRW HVWLPDWH WKH ,Q WKLV SDSHU ZH SURSRVHG D PXOWLOHYHO IXVLRQ VFKHPH
DQJOHSRVLWLRQ7KXVLQWKHLPDJHSURFHVVLQJVWHSDOOSL[HOV XVLQJ UDGDU DQG FDPHUD VHQVRUV )LUVW ZH XVH WKH 52,
GLVSOD\HG LQ WKH VDPH UDQJH ZHUH VFDQQHG ,Q WKH IXWXUH DV GHWHFWHG E\ WKH UDGDU VHQVRU IRU LPDJH SURFHVVLQJ 6HFRQG
WUDQVFHLYHUVZLWKPXOWLSOHUHFHLYHDQWHQQDVDUHXVHGZHZLOO ZHGHWHUPLQHWKHWDUJHWFODVVXVLQJDQLPDJHIHDWXUH'%DQG
EH DEOH WR JHQHUDWH WKH 52, ZLWK WKH UDQJH DQG DQJOH D 'RSSOHU VSHFWUXP SDWWHUQ '% ,Q WKH IXWXUH WKH
FRRUGLQDWHV,QVXFKDFDVHWKHLPDJHSURFHVVLQJWLPHFDQEH FODVVLILFDWLRQ SUREDELOLW\ ZLOO EH HVWLPDWHG EDVHG RQ FDPHUD
UHGXFHG DQGUDGDUVHQVRUVXQGHUYDULRXVVFHQDULRV
5HIHUHQFHV
>@ (+\XQ<6-LQ-+/HH'HVLJQDQGGHYHORSPHQW
RI DXWRPRWLYH EOLQG VSRW GHWHFWLRQ UDGDU V\VWHP EDVHG RQ
52, SUHSURFHVVLQJ VFKHPH ,-$7 9RO 1R SS
)HE
>@ (+\XQ-+/HH7HFKQRORJ\7UHQGVRI$XWRPRWLYH
5DGDU´ $XWR-RXUQDO 9RO 1R SS $XJ
>@ (8521&$3 ³ 5RDG 0DS ± (XURSHDQ QHZ FDU
DVVHVVPHQWSURJUDPPH´-XQH
>@ 6KXQJXDQJ :X 6WHSKHQ 'HFNHU 3HQJ &KDQJ
7KHRGRUH &DPXV -D\DQ (OHGDWK ³&ROOLVLRQ 6HQVLQJ E\
6WHUHR9LVLRQDQG5DGDU6HQVRU)XVLRQ´,(((7UDQVDFWLRQ
)LJXUH 7KUHH WDUJHWV VHOHFWHG IURP DQ LPDJH FDSWXUHG E\ RQ ,QWHOOLJHQW7UDQVSRUWDWLRQ 6\VWHPV 9RO 1R SS
WKH FDPHUD VHQVRU +HUH WKH UHG FLUFOH GHQRWHV WKH SRLQWV
H[WUDFWHGWKURXJKFRRUGLQDWHPDSSLQJWRJHWKHUZLWKWKHUDGDU >@ 5XGL /LQGO /HRQKDUG :DOFKVKDXV ³7KUHH/HYHO (DUO\
VHQVRU )XVLRQIRU 5RDG 8VHU'HWHFWLRQ´ %0: JURXSUHVHDUFKD
WHFKQRORJ\UHSRUW
>@ 6HWIIHQ +HXHO +HUPDQQ 5RKOLQJ ³3HGHVWULDQ
5HFRJQLWLRQLQ$XWRPRWLYH5DGDU6HQVRU´,(((,56
>@ 'RPLQLN .HOOQHU 0LFKDHO %DUMHQEUXFK -HQV
.ODSSVWHLQ\ -XUJHQ 'LFNPDQQ\ DQG .ODXV 'LHWPD\HU
³:KHHO ([WUDFWLRQ EDVHG RQ 0LFUR 'RSSOHU 'LVWULEXWLRQ
XVLQJ+LJK5HVROXWLRQ5DGDU´,(((,&0,0SS
$SULO
>@ (XJLQ+\XQ<RXQJ6HRN-LQ-RQJ+XQ/HH³0RYLQJ
DQG 6WDWLRQDU\ 7DUJHW 'HWHFWLRQ 6FKHPH XVLQJ &RKHUHQW
,QWHJUDWLRQDQG6XEWUDFWLRQIRU$XWRPRWLYH)0&:5DGDU
6\VWHPV´ ,((( ,QWHUQDWLRQDO 5DGDU &RQIHUHQFH
0D\
>@ +\XQ ( -LQ <6 /HH -+ ³$ PHRWKRG IRU PXOWLWDUJHW
UDQJHDQGYHORFLW\GHWHFWLRQLQDXWRPRWLYH)0&:UDGDU´
F 0'3,VHQVRUV9RO1R-DQ
)LJXUH 'RSSOHUSURILOH RI WKH GHWHFWHG KXPDQ H[WUDFWHG
$FNQRZOHGJHPHQW7KLVUHVHDUFKZDVVXSSRUWHGE\WKH7HFKQRORJ\
IURPWKHUDGDUVHQVRU
7UDQVIHU DQG &RPPHUFLDOL]DWLRQ 3URJUDP WKURXJK ,11232/,6
)RXQGDWLRQ'*DQGE\WKH'*,675 '3URJUDP
,Q)LJXUHWKHKXPDQLVSDUWLDOO\RFFOXGHGE\REMHFW +566 7KH ERWK SURJUDPV ZHUH IXQGHG E\ WKH 0LQLVWU\ RI
ORFDWHG DW FORVH UDQJH 7KXV ZH PD\ QRW EH DEOH WR FODVVLI\ 6FLHQFH,&7DQG)XWXUH3ODQQLQJ.RUHD