Ant

ipodal
lyI
nvar
iantMet
ri
csf
orFastRegr
essi
on-
BasedSuper
-Resol
uti
on

OBJECTI
VE

Todet
ectt
heant
ipodal
l
yinv
ari
antmet
ri
csf
orf
astr
egr
essi
on-
based
super
-r
esol
uti
on.

ABSTRACT

Av
eryf
astr
egr
essi
on-
basedal
gor
it
hm,whi
chbui
l
dsont
hedensel
y
popul
atedanchor
ednei
ghbour
hoodsandsubl
i
nearsear
chst
ruct
ures.
Ev
ent
hough,t
hebenefit
sofusi
ngant
ipodal
l
yar
eval
i
dat
edi
nvar
iant
met
ri
cs,
mostoft
hebi
nar
yspl
i
tsuseEucl
i
deandi
stance,
whi
chdoes
nothandl
eant
ipodesopt
imal
l
y.

SYSTEM ANALYSI
S
Exi
sti
ngSy
stem

Li
nearRegr
essi
oni
sthemostsi
mpl
eregr
essi
onscheme,
i.
e.f
oreach
out
putv
ari
abl
eitper
for
ms a l
i
nearwei
ght
ed sum oft
he i
nput
v
ari
abl
es.

SRal
gor
it
hmsar
eusual
l
yper
for
medi
naf
eat
urespaceot
hert
han
t
hatoft
her
awl
umi
nancepi
xel
val
ues.

pr
oposedadi
ff
erentappr
oachwhent
rai
ningl
i
nearr
egr
essor
sforSR:
Usi
ngspar
ser
epr
esent
ati
onsasanchorpoi
ntst
othemani
fol
d,but
f
ormi
ng t
he nei
ghbour
hoods wi
thr
aw mani
fol
d sampl
es (
e.g.
f
eat
ures,
pat
ches)
.

PROPOSEDSYSTEM

Wi
tht
heai
m ofcapt
uri
ngal
lthosef
eat
urest
hatar
erel
evantf
ort
he
char
act
eri
zat
ion,we pr
opose a PCA al
gor
it
hm r
epl
aced wi
tht
he
KPCAal
gor
it
hm.

KPCAAl
gor
it
hm

ker
nelpr
inci
palcomponentanal
ysi
s(ker
nelPCA)
 i
san ext
ensi
on
of
 pr
inci
palcomponentanal
ysi
s(PCA)usi
ng t
echni
quesof
 ker
nel
met
hods.

Usi
ngaker
nel
,theor
igi
nal
l
yli
nearoper
ati
onsofPCA ar
edonei
n
a 
repr
oduci
ngker
nel
Hil
ber
tspacewi
thanon-
li
nearmappi
ng.

KPCA per
for
msacl
assi
call
i
nearPCA ont
heker
nelf
unct
ion.I
n
cont
rastt
oli
nearPCA,KPCA i
scapabl
eofcapt
uri
ngpar
toft
he
hi
gher
-or
derst
ati
sti
cswhi
char
epar
ti
cul
arl
yimpor
tantf
orencodi
ng
 i
magest
ruct
ure

Appl
icat
ions

pl
ater
ecogni
ti
onsy
stemsi
ntegr
ated 
appl
i
cat
ions

compr
essi
on

f
usi
on

Adv
ant
ages

KPCAal
gor
it
hm f
ori
magecompr
essi
onar
efoundef
fect
ive

SOFTWARE

Mat
lab(
R2014-
a)
I
NTRODUCTI
ON

SUPER-
RESOLUTI
ON (
SR)t
echni
quesai
mtoext
endt
her
esol
uti
onofa
si
gnalsur
passi
ngt
hel
i
mit
soft
heor
igi
nalcapt
uredev
ice.I
ncr
easi
ngt
he
r
esol
uti
onofani
magewi
thoutf
urt
heri
nfor
mat
ioni
sadeepl
yil
l
-posed
pr
obl
em,and t
her
efor
e,i
tneedst
o beaddr
essed wi
th a cer
tai
n pr
ior
knowl
edge.Al
thoughSRi
sar
elat
ivel
yyoungr
esear
chf
iel
d,manyoft
hose
pr
ior
shav
ebeenpr
oposed.Someoft
hesi
mpl
estandear
li
esti
mageSR
met
hodswer
ebasedonpi
ecewi
sel
i
nearandsmoot
hpr
ior
s(i
.e.bi
l
inear
andbi
cubi
cint
erpol
ati
on,
respect
ivel
y),
resul
ti
ngi
nfasti
nter
pol
ati
on-
based
al
gor
it
hms.TsaiandHuang[
1]showedt
hati
twaspossi
blet
oreconst
ruct
hi
gher
-r
esol
uti
oni
magesbyr
egi
ster
ingandf
usi
ngmul
ti
plei
mages,t
hus
pi
oneer
ingav
astamountofappr
oachesonmul
ti
-i
mageSR,
oft
encal
l
ed

Thepr
oposedAI
S(Ant
ipodal
l
yInv
ari
antSuper
Resol
uti
on)achi
evesbot
h
t
hebestqual
i
ty(
PSNR)andt
hef
ast
estspeed(
s)i
ncompar
isonwi
thmost
r
ecentst
ate-
of-
thear
tmet
hods.Al
lal
gor
it
hms r
un Set
14 wi
th a ×2
upscal
i
ngf
act
or,seeTabl
eII
Iformor
einf
ormat
ion.r
econst
ruct
ion-
based
SR.Thi
sideawasf
urt
herr
efi
ned,amongot
her
s,wi
tht
hei
ntr
oduct
ionof
i
ter
ati
veback-
proj
ect
ionf
ori
mpr
ovedr
egi
str
ati
onbyI
raniandPel
eg[
2],
al
thoughf
urt
heranal
ysi
sbyBakerandKanade[
3]andLi
nandShum [
4]
showed f
undament
all
i
mit
son t
hist
ypeofSR,mai
nlycondi
ti
oned by
r
egi
str
ati
onaccur
acy
.Lear
ning-
basedSR,al
soknownasexampl
e-based,
ov
ercame some of t
he af
orement
ioned l
i
mit
ati
ons by av
oidi
ng t
he
necessi
tyofar
egi
str
ati
onpr
ocessandbybui
l
dingt
hepr
ior
sfr
om i
mage
st
ati
sti
cs.Theor
igi
nalwor
kbyFr
eemanetal
.[5]ai
mst
olear
nfr
om
pat
chorf
eat
ure-
basedexampl
est
opr
oduceef
fect
ivemagni
fi
cat
ionwel
l
bey
ond t
he pr
act
icall
i
mit
s of mul
ti
-i
mage SR. Exampl
e-based SR
appr
oaches usi
ng di
cti
onar
ies ar
e usual
l
ydi
vi
ded i
ntot
wo cat
egor
ies:
i
nter
nalandext
ernaldi
cti
onar
y-based SR.Thef
ir
stexpl
oit
sthest
rong
sel
fsi
mil
ari
typr
ior
.Thi
spr
iori
slear
ntdi
rect
lyf
rom t
her
elat
ionshi
pof
i
magepat
chesacr
ossdi
ff
erentscal
esoft
hei
nputi
mage.Theopeni
ng
wor
kont
hissubcat
egor
ywasi
ntr
oducedbyGl
asneretal
.[6]
,pr
esent
inga
power
fulf
ramewor
kforf
usi
ngr
econst
ruct
ion-
basedandexampl
e-based
SR.Fur
therr
esear
ch on t
his cat
egor
y by Fr
eedman and Fat
tal[
7]
i
ntr
oducedamechani
sm f
orhi
gh-
fr
equencyt
ransf
erbasedonexampl
es
f
rom asmal
lar
eaar
oundeachpat
ch,
thusbet
terl
ocal
i
zingt
hecr
oss-
scal
e
sel
f-
simi
l
ari
typr
iort
othespat
ialnei
ghbor
hood.Ther
ecentwor
kofYanget
al
.[8]f
urt
herdev
elopst
hei
deaofl
ocal
i
zingt
hecr
oss-
scal
esel
f-
simi
l
ari
ty
pr
iorar
ri
vi
ngt
othei
n-pl
acepr
ior
,i.
e.t
hebestmat
chacr
ossscal
esi
s
l
ocat
edexact
lyi
nthesameposi
ti
oni
fthescal
eissi
mil
arenough.Ext
ernal
di
cti
onar
y-based SR uses ot
heri
mages t
o bui
l
dthei
rdi
cti
onar
ies.A
r
epr
esent
ati
ve wi
del
y used appr
oach i
sthe one based on spar
se
decomposi
ti
on.Themai
nideabehi
ndt
hisappr
oachi
sthedecomposi
ti
on
ofeachpat
chi
nthei
nputi
magei
ntoacombi
nat
ionofaspar
sesubsetof
ent
ri
esi
nacompactdi
cti
onar
y.Thewor
kofYangetal
.[9]usesanext
ernal
dat
abasecomposedofr
elat
edl
ow andhi
gh-
resol
uti
onpat
chest
ojoi
ntl
y
l
ear
n a compactdi
cti
onar
ypai
r.Dur
ing t
est
ing,each i
mage pat
ch i
s
decomposedi
ntoaspar
sel
i
nearcombi
nat
ionoft
heent
ri
esi
nthel
ow-
r
esol
uti
on(
LR)di
cti
onar
yandt
hesamewei
ght
sar
eusedt
ogener
atet
he
hi
gh-
resol
uti
on(
HR)pat
chasal
i
nearcombi
nat
ionoft
heHRent
ri
es.Bot
h
t
hedi
cti
onar
ytr
aini
ngandt
est
ingar
ecost
lyduet
otheL1r
egul
ari
zat
ion
t
erm enf
orci
ngspar
sit
y.Thewor
kofZey
deetal
.[10]ext
endsspar
seSRby
pr
oposi
ngsev
eralal
gor
it
hmi
cspeed-
upswhi
chal
soi
mpr
oveper
for
mance.
Howev
er,t
hebot
tl
eneckofspar
sit
ymet
hodsst
il
lremai
nsi
nthespar
se
decomposi
ti
on.I
nthenei
ghborembeddi
ngal
gor
it
hm ofGaoetal
.[11]t
hey
pr
oposeamet
hodt
hatbui
l
dsnei
ghbor
hoodsf
orl
i
nearembeddi
ngi
na
uni
fi
edf
eat
uresubspacespannedbyLR–HRi
magepat
chesr
athert
hani
n
t
heor
igi
nal
LRf
eat
urespaceal
one.Mor
erecent
ly,
regr
essi
on-
basedSRhas
r
ecei
vedagr
eatdealofat
tent
ionbyt
her
esear
chcommuni
ty.I
nthi
scase,
t
hegoali
stol
ear
nacer
tai
nmappi
ngf
rom t
hemani
fol
doft
heLRpat
ches
t
othatofHRpat
ches,f
oll
owi
ngt
hemani
fol
dassumpt
ional
readyusedi
n
t
heear
li
erwor
kofChangetal
.[12]
.Themappi
ngoft
hemani
fol
dis
assumedt
obel
ocal
l
yli
nearandt
her
efor
esev
erall
i
nearr
egr
essor
sar
e
usedandanchor
edt
othemani
fol
dasapi
ecewi
sel
i
near
izat
ion[
13]
.The
key obser
vat
ion of per
for
ming t
he nei
ghbor embeddi
ng j
ust f
or a
pr
edef
inedsetofanchorpoi
ntsal
l
owedt
opr
epr
ocesst
hem dur
ingt
rai
ning
t
ime,t
husl
esseni
ngsubst
ant
ial
l
ythet
est
ingt
imecompl
exi
ty.Ourr
ecent
wor
k[14]put
sli
ghti
nhow t
opr
oper
lycr
eat
ethenei
ghbor
hoodsi
nsuch
met
hods,obt
aini
ngsi
zabl
ebenef
it
sint
ermsofr
econst
ruct
ionqual
i
ty.I
n
addi
ti
on t
othat
,we al
so expl
ored subl
i
nearsear
ch st
ruct
uresf
ort
he
r
egr
essornear
estnei
ghborsear
ch,ast
hist
akesasi
gni
fi
cantquot
aoft
he
r
unni
ngt
imef
rom wi
thi
nthepr
ocessi
ngoft
hewhol
eSRpi
pel
i
ne.

PROPOSEDSYSTEM

I
nthi
spaperwef
oll
ow t
hesamer
esear
chdi
rect
ionandwei
ntr
oducet
he
f
oll
owi
ngcont
ri
but
ions:

1)West
udyandpr
ovi
dei
nsi
ghtaboutt
hebehav
iorofdi
stancemet
ri
cs
useddur
ingt
her
egr
essi
onpr
ocess.Wedet
ectt
hei
mpor
tanceofant
ipodal
i
nvar
iance.

2)Wepr
ovi
deasui
tabl
eandef
fi
cientspher
icalhashi
ngf
ramewor
kto
expl
oita f
astr
egr
essorsear
ch.I
n or
dert
o benef
itf
rom ant
ipodal
i
nvar
iance,
wei
ntr
oduceanewsi
mpl
eyetef
fect
ivet
ransf
orm whi
chmakes
Eucl
i
deandi
stance(
andal
sospher
ical
hashi
ng)ant
ipodal
l
yinv
ari
ant
.

3)Wepr
esentanov
elf
eat
uret
ransf
orm basedongr
adi
ent
sandI
ter
ati
ve
BackPr
oject
ionwhi
chi
mpr
ovest
hequal
i
tyoft
heupscal
edi
maget
hanks
t
oabet
terf
ir
stcoar
seappr
oxi
mat
ion

Theconf
luenceofal
lthement
ionedcont
ri
but
ionsy
iel
dsst
ate-
of-
the-
art
speedandqual
i
tyr
esul
ts.Wi
thourpr
oposedappr
oxi
mat
eregr
essorsear
ch
wear
ethef
ast
est(
about×4−11t
imesf
ast
er)andwegr
eat
lyi
mpr
ovei
n
qual
i
ty(
about0.
2dBhi
gher
)compar
edt
oourpr
edecessorA+.I
naddi
ti
on,
whenper
for
mingexhaust
ivesear
chweobt
ainev
enbet
terPSNRr
esul
ts(
up
t
o0.
32dBhi
gher
).

Pr
obl
em st
atementSuper
-Resol
uti
onai
mst
oupscal
eimageswhi
chhav
e
an unsat
isf
act
ory pi
xelr
esol
uti
on whi
l
e pr
eser
ving t
he same v
isual
shar
pness,
mor
efor
mal
l
y

wher
eYi
sthei
nputi
mage,Xi
stheout
putupscal
edi
mage,↑(
·)i
san
upsampl
i
ngoper
atorandcal
l
igr
aphi
cfontdenot
est
hespect
rum ofan
i
mage.I
nthe l
i
ter
atur
ethi
str
ansf
ormat
ion hasusual
l
ybeen model
ed
backwar
dsast
her
est
orat
ionofanor
igi
nali
maget
hathassuf
fer
eda
cer
tai
ndegr
adat
ion[
9]

wher
eB(
·)i
sabl
urr
ingf
il
terand↓(
·)i
sadownsampl
i
ngoper
ator
.The
pr
obl
em i
susual
l
yaddr
essedatapat
chl
evel
,denot
edwi
thl
owercase(
e.g.
y
,x)andext
ract
edf
rom t
her
espect
iveupper
casei
mage(
e.g.Y,X)
.The
exampl
e-basedSRf
ami
l
ytackl
est
hesuper
-r
esol
uti
onpr
obl
em byf
indi
ng
meani
ngf
ulexampl
es f
rom whi
ch a HR count
erpar
tis al
readyknown,
namel
ythecoupl
eofdi
cti
onar
iesDlandDh.Thel
owr
esol
uti
onpat
chyi
s
decomposedasal
i
nearcombi
nat
ionoft
heat
omsi
ntheLRdi
cti
onar
yDl

wher
eNli
stheLRnei
ghbor
hoodchosent
osol
vet
hepr
obl
em andy
Fisa
f
eat
ureext
ract
edf
rom aLRpat
ch.Theal
gebr
aicsol
uti
oni
s

andcanbecomput
edof
fl
ine,t
her
efor
emov
ingt
hemi
nimi
zat
ionpr
obl
em
f
rom t
est
ingt
otr
aini
ngt
ime.Theypr
oposet
ousespar
sedi
cti
onar
iesofds
at
oms si
ze,t
rai
ned wi
tht
he K-
SVD al
gor
it
hm [
17]
.A r
egr
essorRji
s
anchor
edt
oeachat
om dji
nDl,
andt
henei
ghbor
hoodNli
nequat
ion(
8)i
s
sel
ect
edf
rom ak-
NNsubsetofDl

TheSRpr
obl
em canbeaddr
essedbyf
indi
ngt
heNNat
om djofev
eryi
nput
pat
chf
eat
urey
Fiandappl
yi
ngt
heassoci
atedRj
toi
t.I
nthespeci
fi
ccaseof
anei
ghbor
hoodsi
zek=ds,onl
yonegener
alr
egr
essori
sobt
ainedwhose
nei
ghbor
hoodcompr
isesal
ltheat
omsoft
hedi
cti
onar
yandconsequent
ly
doesnotr
equi
reaNNsear
ch.Thi
scasei
sref
err
edi
ntheor
igi
nalpaperas
Gl
obalRegr
essi
on(
GR)
.Ot
heradv
anceswi
thi
nthi
sfami
l
yofal
gor
it
hms
i
ncl
udet
heNai
veBay
esSRFor
est[
18]
,thatusesanensembl
eofbi
modal
t
rees i
n or
dert
o benef
itf
rom ant
ipodali
nvar
iance.I
tis al
so hi
ghl
y
compet
it
ivei
nspeedt
hankst
othenai
veBay
essel
ect
ionpr
ocedur
ewhi
ch
appl
i
es a si
ngl
eregr
essor per pat
ch,di
ff
erent
lyf
rom ot
her f
orest
appr
oaches,e.
g.SR For
estofSchul
teretal
.[19]
,thatappl
yasmany
r
egr
essor
sperpat
chast
henumberoft
reesi
ntheensembl
e.Anot
her
r
ecentf
oll
ow-
upofANR i
stheMul
ti
pleLi
nearMappi
ngs(
MLM)SR of
Zhangetal
.[20]
,whi
cht
rai
nseachr
egr
essorwi
thasetofor
thonor
mal
basi
srepr
esent
ati
veoft
hecor
respondentsubspace(
i.
e.asubdi
cti
onar
y)
and l
aterper
for
msapost
processi
ng st
agebased onnon-
localmeans
r
egul
ari
zat
ion.

PROPOSEDALGORI
THM

I
nthi
ssect
ionwepr
esentoural
gor
it
hm f
orf
astandef
fi
cientr
egr
essi
on-
basedSR,di
vi
dedi
nfoursubsect
ionst
hatdescr
ibet
hemai
npi
l
lar
sofour
cont
ri
but
ions.Weshowanov
erv
iewoft
hecompl
eteal
gor
it
hm i
nFi
gur
e2

A.Li
nearRegr
essi
onFr
amewor
kAsseeni
nSect
ionI
I,sel
ect
ingnon-
l
i
nearr
egr
essi
onschemes(
e.g.KRRof[
15]
)resul
tsi
npr
ohi
bit
ive
t
rai
ningcompl
exi
ty,whi
chi
susual
l
ymi
ti
gat
edbyr
educi
ngt
he
t
rai
ningset
sundercer
tai
nassumpt
ions.I
ntest
ingt
ime,al
though
 nonl
i
nearr
egr
essi
onSR met
hodsar
ereasonabl
yfast
,theyst
il
l
compar
emodest
lywi
thf
astst
ateoft
hear
tmet
hodssuchas[
13]
,
[
14]
,[
21]
.Li
nearRegr
essi
oni
sthemostsi
mpl
eregr
essi
onscheme,
i
.e.f
oreachout
putv
ari
abl
eitper
for
msal
i
nearwei
ght
edsum

B.

:Pi
pel
i
neofourSRal
gor
it
hm.Fi
rst
ly,weext
ractf
eat
urev
ect
ors
f
rom t
hegr
adi
ent
sofani
mageupscal
edt
hroughI
BP.Secondl
y,
wef
indt
hebest
-f
it
tedr
egr
essorRt
hroughspher
ical
hashi
ngusi
ng
nor
mal
i
zedf
eat
uresandourpr
oposedAI
T.Wet
henappl
ythe
r
espect
iver
egr
essort
oeachf
eat
urev
ect
orandr
econst
ructt
he
f
inal
imagewi
thov
erl
appi
ngpat
ches

oft
hei
nputv
ari
abl
es.Thi
sisanov
ersi
mpl
i
ficat
ionoft
heupscal
i
ng
pr
obl
em i
fweonl
yconsi
deronel
i
nearr
egr
essor
.Inst
ead,sev
eral
l
i
nearr
egr
essor
sar
eanchor
edt
odi
ff
erentpoi
ntsoft
hemani
fol
d,
obt
aini
ng a f
iner pi
ecewi
se l
i
near r
egr
essi
on model
.In such
st
rat
egi
es,dat
ahav
etobespl
i
tint
rai
ningt
imeanddur
ingt
est
ing
t
imet
hepr
operr
egr
essorhast
obesel
ect
ed.Fort
heSRpr
obl
em,
the
r
egr
essi
oni
sappl
i
edt
othei
nputf
eat
uresandai
mst
orecov
ercer
tai
n
component
soft
hepat
ch,
e.g.mi
ssi
nghi
ghf
requency
.Wemodelt
he
l
i
nearr
egr
essi
onf
ramewor
kinagener
alwayas
 Feat
ure Space and coar
se appr
oxi
mat
ion SR al
gor
it
hms ar
e
usual
l
yper
for
medi
naf
eat
urespaceot
hert
hant
hatoft
her
aw
l
umi
nancepi
xelv
alues.I
nthel
i
ter
atur
e,acommonr
ulef
ort
his
f
eat
uret
ransf
ormat
ioni
stoenf
orcemi
dandhi
ghf
requenci
esof
LRpat
ches,undert
heobser
vat
iont
hatsi
mil
ari
tybet
weenLRand
HR pat
ch st
ruct
ures i
s somehow i
mpr
oved and t
her
efor
ethe
pr
edi
cti
oni
seasi
er.Asf
ort
heHRf
eat
urespace(
i.
e.t
heout
put
f
eat
urespace)
,thesamepr
inci
pleofenf
orci
nghi
ghf
requenci
es
al
so appl
i
es,i
nthi
scaseundert
heassumpt
ion t
hatt
hehi
gh
f
requency bands ar
e condi
ti
onal
l
yindependentoft
he l
ower
f
requencybands,
andt
hussuppr
essi
ngl
ow-
fr
equencybandsf
rom
t
heHRf
eat
urespacecol
l
apsest
het
rai
ningdat
aforal
lpossi
ble
l
ow-
fr
equencyv
aluesi
ntoonev
alue[
5].Di
ff
erent
lyf
rom t
hei
nput
LRf
eat
urespace,i
ntheHRf
eat
urespaceweneedt
obeabl
eto
r
ever
set
hef
eat
uresi
ntopi
xel
-basedv
aluesf
ort
hef
inali
mage
r
econst
ruct
ion.Sev
eralf
eat
ureshav
ebeenpr
oposed:Theear
ly
wor
kofFr
eemanetal
.[5]al
readyusedasi
mpl
ehi
gh-
passf
il
ter
whi
chconsi
stedi
nthesubt
ract
ionofal
ow-
passf
il
ter
.Int
hesame

di
rect
ion,[
12]and [
9]used concat
enat
ed f
ir
st-and secondor
der
gr
adi
ent
s,asani
nexpensi
vesol
uti
ont
othesamehi
ghpassf
il
ter
appr
oxi
mat
ion.Thi
sty
peoff
eat
urewasf
urt
herr
efi
nedbyZey
deetal
.
[
10]byi
ntr
oduci
ngPCAcompr
essi
oni
nor
dert
oreducet
hef
eat
ure
di
mensi
onal
i
tyandmemor
yusage.I
tisi
mpor
tantt
oremar
kthat
mostf
eat
uret
ransf
ormat
ions ar
e comput
ed f
rom a f
ir
stcoar
se
appr
oxi
mat
ion,i
.e.t
heupscal
edi
mageX˜
.Weobser
vedt
hatt
he
ef
fectoft
hisf
ir
stappr
oxi
mat
ionhasbeenunnot
icedi
nthel
i
ter
atur
e,
i
nwhi
chusi
ngbi
cubi
cint
erpol
ati
onort
hepat
ch-
meanv
aluei
sa
commonpr
act
ice,bot
hinLRandHRf
eat
urespace.I
nthi
spaperwe
pr
oposeanew f
eat
uret
ransf
orm whi
cht
akesadv
ant
ageofabet
ter
coar
se appr
oxi
mat
ion t
o obt
aint
he LR i
nputf
eat
ures,whi
ch we
denot
ewi
thy
F.Themai
nideai
stoobt
ainani
mageappr
oxi
mat
ion
X˜bet
tert
hant
hatobt
ainedwi
thbi
cubi
cint
erpol
ati
onbutwhi
chi
s
st
il
lwi
thi
ncer
tai
nlow-
compl
exi
tyboundar
ies.Wepr
esentaf
eat
ure
t
ransf
orm based on a si
mpl
i
fied I
ter
ati
ve Back Pr
oject
ion (
IBP)
al
gor
it
hm of[
2],t
oget
herwi
thuni
dimensi
onal1-
stand2-
ndor
der
gr
adi
ent
s.Wer
efert
othi
snov
elf
eat
uret
ransf
orm asGr
adi
entI
BP
(
GIBP)
.St
art
ingwi
thani
nit
ialguessX˜
(0)oft
heHR i
mage,I
BP
si
mul
ates t
he i
magi
ng pr
ocess t
o obt
ain a LR i
mage Y˜ (
0)
cor
respondi
ngt
otheobser
vedi
nputi
mageY.Thedi
ff
erencei
mage
E(
0)=↑(
Y−Y˜(
0))i
scomput
edandusedt
oimpr
ovet
hei
nit
ialguess
byback-
proj
ect
ingeacher
rorv
alueont
othecor
respondi
ngf
iel
din
X˜
(0)
,namel
yX˜
(1)=X˜
(0)
+E(
0).Thi
spr
ocessi
srepeat
edi
ter
ati
vel
y:

I
ntheor
igi
nalwor
kofI
raniandPel
eg[
2],t
hei
magi
ngpr
ocesswas
model
edt
hroughaconv
olut
ionwi
thagi
venpoi
ntspr
eadf
unct
ion
andaback-
proj
ect
ionker
nel
.Inourl
ow-
compl
exi
tyappr
oach,we
model bot
h f
unct
ions wi
th t
he si
mpl
e and ef
fect
ive bi
cubi
c
downscal
i
ngandupscal
i
ngker
nel
.Wi
thasf
ewasn=2i
ter
ati
onst
he
coar
seappr
oxi
mat
ioni
mpr
ovesgr
eat
lywhencompar
edt
obi
cubi
c.
We f
il
ter t
his upscal
ed i
mage X˜ wi
th 1-
st and 2-
nd or
der
uni
dimensi
onalgr
adi
entf
il
ter
s(t
wov
ert
icalandt
wohor
izont
al)
.At
t
his poi
ntov
erl
appi
ng pat
ches ar
e ext
ract
ed f
rom each oft
he
gr
adi
enti
mages,andal
lthe4gr
adi
entpat
chescor
respondi
ngt
othe
samepat
chposi
ti
onar
econcat
enat
edt
oget
heri
naf
eat
urev
ect
ory
F.
Not
ethatdi
mensi
onal
i
tyoft
hisf
eat
urei
sfour

.3:A nor
mal
i
zed degr
ee 3 pol
ynomi
almani
fol
dil
l
ust
rat
ing t
he
pr
oposedappr
oachcompar
edt
otheonei
n[13]
.(a)Bi
dimensi
onal
mani
fol
d sampl
es. (
b) The mani
fol
d (
blue) and t
he spar
se
r
epr
esent
ati
onobt
ainedwi
thK-
SVDal
gor
it
hm (
green)of8at
oms.(
c)
Li
nearr
egr
essor
s(r
ed)t
rai
ned wi
tht
he nei
ghbor
hoods (
k = 1)
obt
ainedwi
thi
nthespar
sedi
cti
onar
y,asi
n[13]
.(d)Li
nearr
egr
essor
s
(
red)obt
ainedusi
ngourpr
oposedappr
oach:Thenei
ghbor
hoodsar
e
obt
ainedwi
thi
nthesampl
esf
rom t
hemani
fol
d(k=10)

t
imest
hepat
chsi
ze.I
fthi
sev
ent
ual
l
ybecomesamemor
ypr
obl
em,
a
PCAcompr
essi
oncanbeappl
i
edwi
thbar
elynoi
nfor
mat
ionl
oss[
10]
.
Sect
ion I
V-A assess t
he per
for
mance ofourpr
oposed f
eat
ure
t
ransf
orm.Asf
ort
heHR f
eat
urespace,weconsi
stent
lyuseI
BP,
wi
thoutt
he non-
rev
ersi
ble gr
adi
entst
ep and PCA compr
essi
on.
Dur
ingt
rai
ning,
wef
orm ourHRf
eat
uressi
mpl
ybyt
hesubt
ract
ionof
t
het
hef
ir
stcoar
seappr
oxi
mat
iont
othegr
ound-
tr
uthpat
chx−x
˜(2)
,
sot
hatourr
egr
essi
onst
agei
sspeci
ali
zedi
ncor
rect
ingt
heer
ror
s
t
hatI
BP i
sint
roduci
ng.Dur
ingt
est
ing,t
hisHR f
eat
uret
ransf
orm
r
equi
ressubst
it
uti
ngx˜byx˜(
2)i
nEquat
ion10

Cl
ust
eri
ngandt
rai
ningAswehav
epr
evi
ousl
yseen,
recentr
egr
essi
on
-
based SR use l
i
near r
egr
essor
s because t
hey can be easi
l
y
comput
ed i
ncl
osed f
orm and appl
i
ed asamat
ri
xmul
ti
pli
cat
ion.
Howev
er,t
hemappi
ngΨ i
shi
ghl
ycompl
exandnon-
li
near[
22]
.To
modelt
he non-
li
near
it
ynat
ure oft
he mappi
ng,an ensembl
e of
r
egr
essor
s {
Rk} i
s t
rai
ned, r
epr
esent
ing a l
ocal
l
y l
i
near
par
amet
ri
zat
ionofΨ,undert
heassumpt
iont
hatbot
hmani
fol
dsM
andN hav
easi
mil
arl
ocalgeomet
ry.Weanal
yzet
heef
fectont
he
di
str
ibut
ionoft
hoser
egr
essor
sint
hemani
fol
d(i
.e.t
heanchorpoi
nts)
and t
hei
mpor
tanceofpr
oper
lychoosi
ng t
heNli
n equat
ion (
8),
concl
udi
ngonanewt
rai
ningappr
oach.I
nthewor
kofTi
mof
teetal
.
[
13]
,anov
ercompl
etespar
ser
epr
esent
ati
oni
sobt
ainedf
rom t
he
i
nit
ialLRt
rai
ningpat
chesusi
ngK-
SVD[
17]
.Theat
omsi
nthi
snew
r
educeddi
cti
onar
yDlar
eusedbot
hasanchorpoi
ntst
othemani
fol
d
and dat
apoi
ntsf
ort
her
egr
essi
on t
rai
ning.I
nthei
rGR,a uni
que
r
egr
essorRGi
str
ainedwi
thal
lel
ement
soft
hedi
cti
onar
y,t
her
efor
e
accept
inghi
gherr
egr
essi
oner
ror
sduet
othesi
ngl
eli
near
izat
ionof
t
hemani
fol
d.I
nor
dert
oachi
eveamor
eaccur
ater
egr
essi
ont
hey
al
sopr
oposet
heAnchor
edNei
ghbor
hoodRegr
essi
on(
ANR)
.InANR
t
heyusesev
eral
regr
essor
sanchor
edt
oeachspar
sedi
cti
onar
yat
om.
They bui
l
dforeach oft
hose at
oms a nei
ghbor
hood ofk-
NN
composedbyot
herat
omswi
thi
nthesamespar
sedi
cti
onar
yDl,and
f
inal
l
ytr
ainar
idger
egr
essorwi
tht
hecor
respondi
ngnei
ghbor
hood.

Per
for
mingaspar
sedecomposi
ti
onofahi
ghnumberofpat
ches
ef
fi
cient
lycompr
essesdat
ainamuchsmal
l
erdi
cti
onar
y,y
iel
ding
at
omswhi
char
erepr
esent
ati
veoft
hewhol
etr
aini
ngdat
aset
,i.
e.t
he
whol
emani
fol
d.Fort
hisr
easont
heyar
esui
tabl
eto beusedas
anchorpoi
nts,
butal
sosub-
opt
imalf
ort
henei
ghbor
hoodembeddi
ng.
Theyar
esub-
opt
imalsi
ncet
henecessar
ylocalcondi
ti
onf
ort
he
l
i
near
it
yassumpt
ioni
sli
kel
ytobev
iol
ated.Theat
omsofaspar
se
di
cti
onar
yar
easetofcent
roi
dst
hatr
epr
esentt
hewhol
etr
aini
ngset
and,asaconsequence,t
heyar
enotl
i
kel
ytobesi
mil
aramongt
hem.
I
mpor
tantv
ari
ati
ons wi
thi
n spar
se at
oms ar
eto be expect
ed,
t
her
efor
e bei
ng unappr
opr
iat
ed f
or l
ocal
l
y l
i
near embeddi
ng.
Fol
l
owi
ngt
hisobser
vat
ion,i
n[14]wepr
oposedadi
ff
erentappr
oach
whent
rai
ningl
i
nearr
egr
essor
sforSR:Usi
ngspar
ser
epr
esent
ati
ons
asanchorpoi
ntst
othemani
fol
d,butf
ormi
ngt
henei
ghbor
hoodswi
th
r
aw mani
fol
dsampl
es(
e.g.f
eat
ures,pat
ches)
.Bydoi
ngso,wef
ind
cl
osernear
estnei
ghbor
s and,t
her
efor
e,f
ulf
il
lbet
tert
he l
ocal
condi
ti
on.Addi
ti
onal
l
y,a hi
gher number of l
ocali
ndependent
measur
ement
sisav
ail
abl
e(e.
g.meandi
stancef
or1000nei
ghbor
sin
t
her
aw-
pat
chappr
oachi
scompar
abl
etoa40at
om nei
ghbor
hoodi
n
t
he spar
se appr
oach)and we can cont
rolt
he numberofk-
NN
sel
ect
ed,i
.e.i
tisnotupper
-boundedbyt
hedi
cti
onar
ysi
ze.Weshow
al
ow-
dimensi
onal
exampl
eofourpr
oposedt
rai
ningschemei
nFi
gur
e
3.Bui
l
dingdenseandcompactcl
ust
erst
hrought
hismet
hodol
ogyi
s
akeycont
ri
but
ionasi
tboost
sper
for
manceatnocompl
exi
tycost
.
Par
all
eli
nti
me,Ti
mof
teetal
.pr
esent
edasi
mil
ari
deai
nthei
rA+
al
gor
it
hm [
23]
.Int
hepr
esentwor
kwef
urt
heropt
imi
zet
het
rai
ning
st
age byr
evi
ewi
ng t
he near
estnei
ghborsear
ch st
rat
egy
,whi
ch
f
oll
owsonSect
ionI
II
-D,wher
ewepr
oposet
heusageofdi
ff
erent
met
ri
cst
hatal
l
owev
enbet
terl
ocal
i
tycondi
ti
ons.

C.Sear
chst
rat
egy
:Ant
ipodal
l
yInv
ari
antSpher
icalHashi
ngI
nthi
s
sect
ionwest
udyt
hemostappr
opr
iat
emet
ri
csf
orev
aluat
ingt
he
si
mil
ari
tybet
weenf
eat
uresandanchorpoi
nts,pr
esent
ingsome
i
nsi
ght about t
hei
r behav
ior wi
th ant
ipodalpoi
nts and t
he
i
mpor
tanceofpr
oper
lydeal
i
ngwi
tht
hem.Weal
sopr
esentanov
el
sear
ch st
rat
egy t
hatspeeds up t
he r
egr
essorsel
ect
ion and
i
ntr
oduceamodi
fi
cat
ionoft
heSpher
ical

4:Runni
ngt
imesmeasur
edwhencomput
ing6-
bithashcodes(
6
hy
per
spher
es)f
ori
ncr
easi
ngnumberofquer
ies(
inl
ogar
it
hmi
c
axi
sandwi
thoutr
e-r
anki
ng)f
orsi
ngl
e-t
hreaded(
CPU)andpar
all
el
(
CPUandGPU)i
mpl
ement
ati
ons.Hashi
ngal
gor
it
hm [
24]i
nor
der
t
o benef
itf
rom ant
ipodali
nvar
iance.SR al
gor
it
hms r
equi
re
per
for
mingcompar
isonsbet
weenv
ect
orsi
nor
dert
ogui
det
he
al
gor
it
hm i
nit
sdeci
sions,e.
g.f
eat
ures,anchorpoi
nts,di
cti
onar
y
at
oms.Thecompar
isoni
sper
for
medf
oll
owi
ngagi
venmet
ri
c
f
unct
ionδ(
a,b)
a,b∈ Rm whi
cht
ypi
cal
l
yhasbeendesi
gnedt
o
quant
if
ythesi
mil
ari
tyordi
ssi
mil
ari
tyoft
het
wocompar
edv
ect
ors.
I
tisr
ecur
renti
nli
ter
atur
etheusageofEucl
i
deandi
stancef
ort
his
pur
pose.Weshowt
hatf
ort
hechar
act
eri
sti
csoft
hef
eat
uresused
i
nourpr
oposedal
gor
it
hm (
whi
char
eal
soshar
edi
not
herSR
al
gor
it
hms[
10]
,[13]
,[23]
,[14]
),Eucl
i
deandi
stancei
ssubopt
imal
asi
tfai
l
sto manageant
ipodalpoi
nts(
i.
e.t
wo poi
ntson t
he
sur
face ofa spher
e ar
e ant
ipodali
fthey ar
e di
amet
ri
cal
l
y
opposi
te)
.Wef
indf
urt
heri
nsi
ghtoft
heambi
guousv
ari
ati
onst
hat
ourdef
inedmet
ri
cshoul
dignor
ewhenl
ooki
ngatscal
armat
ri
x
mul
ti
pli
cat
ioni
nli
nearr
egr
essi
on,
i.
e.f
oragi
venscal
arλourl
i
near
r
egr
essi
onr
eadsλx=R(
λy)
.Ther
egr
essorRandt
heassoci
ated
l
i
nearoper
ati
onsar
enotmodi
fi
edbyt
hisscal
i
ngoper
ati
on,
asi
tis
pr
esentatbot
hsi
desoft
heequal
i
ty.Nor
mal
i
zingf
eat
urev
ect
ors
whencompar
ingt
hem i
ssomet
imesnecessar
yasmanydi
cti
onar
y
opt
imi
zat
ion al
gor
it
hms (
as i
tist
he case ofK-
SVD)enf
orce
nor
mal
i
zat
iont
oav
oida0sol
uti
on.Ev
enwheni
tisnotnecessar
y,
nor
mal
i
zat
ioni
sagoodpr
act
iceasi
tcol
l
apsest
rai
ningexampl
es
wi
thsi
mil
arst
ruct
uresbutdi
ff
erentscal
arnor
ms.Dur
ingt
est
ing,
t
henor
mal
i
zedf
eat
uresar
eusedt
ofi
ndt
hebest
-sui
tedr
egr
essor
,
whi
l
ethe non-
nor
mal
i
zed f
eat
ures ar
e necessar
yfort
he f
inal
mat
ri
x mul
ti
pli
cat
ion.Howev
er,v
ect
ornor
mal
i
zat
ion does not
handl
ethecaseswher
eλi
snegat
ive,ast
henor
msar
est
ri
ctl
y
posi
ti
veandcannotcompensat
eforλ∈R−.Thi
sist
hecaseof
ant
ipodalpoi
nts(
i.
e.t
her
eis a si
gn change)
.Tr
aini
ng and
assi
gni
ngdi
ff
erentr
egr
essor
sfort
woant
ipodalpoi
ntsdoesnot
i
ncr
easet
heper
for
mancebyabet
terspeci
ali
zat
ion,ast
hesi
gn
changei
sinbot
hsi
desoft
heequal
i
tyandt
her
egr
essorandt
he
associ
atedl
i
nearoper
ati
onsar
eident
icalf
ort
woant
ipodalpoi
nts
(
i.
e.x=R(
y)and−x=R(
−y)
).Eachr
egr
essori
sassoci
atedwi
than
anchorpoi
nt,whi
chdescr
ibesacer
tai
nmodei
nthest
ruct
ureof
pat
ches,r
egar
dlessoft
hisst
ruct
urebei
ngaposi
ti
veornegat
ive
change(
e.g.posi
ti
veornegat
ivechangei
nthegr
adi
ent
),whi
chi
s
descr
ibedbyt
hesi
gnoft
henor
mal
i
zedv
ect
or.Themet
ri
cut
il
ized
f
orsel
ect
ing t
he bestr
egr
essorshoul
dther
efor
e be abl
eto
associ
atet
wo ant
ipodalpoi
ntst
othesameancho poi
nt,t
hus
hav
ingant
ipodali
nvar
iance.I
nthesameway
,whenbui
l
dingt
he
nei
ghbor
hoodsdur
ingt
rai
ning,t
hisobser
vat
ional
soappl
i
es.We
def
ineanant
ipodal
inv
ari
antmet
ri
cas:

We pr
opose t
he usage oft
he Absol
ute Val
ue oft
he Cosi
ne
Si
mil
ari
ty(
AVCS)as a bet
teral
ter
nat
ivet
o compar
efeat
ure
v
ect
orst
o anchorpoi
nts.Thi
s met
ri
c δ|
·|
(a,b)= |
a ·b|i
s
ant
ipodal
l
yinv
ari
ant(
respect
sEquat
ion(
12))
,andf
urt
her
mor
e,
r
equi
res l
ess oper
ati
ons t
han a common Eucl
i
dean di
stance
cal
cul
ati
on.Dur
ingt
rai
ning,wecanusei
tast
her
ear
enot
ime
const
rai
ns.Ref
ert
o Sect
ionI
Vto assesst
hei
mpr
ovementof
usi
ngant
ipodal
l
yinv
ari
antmet
ri
csbot
hfort
rai
ningandf
ort
est
ing.
I
ntest
ingt
ime,t
hebest
-f
it
tedr
egr
essorhast
obesel
ect
edf
rom
wi
thi
ntheensembl
eoft
rai
nedr
egr
essor
.Al
thoughst
ateof
-t
hear
t
r
egr
essi
on-
basedSRhasal
readypushedf
orwar
dthespeedwi
th
r
egar
dto ot
herdi
cti
onar
y-based SR [
10]
,[9]
,fi
ndi
ng t
he r
ight
r
egr
essorf
oreach pat
ch t
akes a consi
der
abl
e quot
a oft
he
pr
ocessi
ngt
ime.I
nthewor
kof[
13]
,mostoft
heencodi
ngt
ime(
i.
e.
t
ime l
eftaf
tersubt
ract
ing shar
ed pr
ocessi
ng t
ime,i
ncl
udi
ng
bi
cubi
cint
erpol
ati
ons,pat
chext
ract
ions,et
c.)i
sspenti
nthi
stask
(
i.
e.∼96%oft
het
ime)
.Inor
dert
oreducet
hesear
chcompl
exi
ty,
bi
nar
yspl
i
ts(
e.g.t
reesorhashi
ngf
unct
ions)ar
etr
ainedf
ormi
nga
hi
erar
chi
calst
ruct
urecomposed bysev
eralspl
i
tnodes.These
spl
i
tnodesdi
vi
det
hespacei
nsuchawayt
hatt
hechi
l
drenl
eav
es
i
mpr
ovet
hei
rinf
ormat
iongai
n.Wef
acesev
eralchal
l
engeswhen
adapt
ingbi
nar
ysear
chst
ruct
urest
ooural
gor
it
hm:Theout
come
ofourt
rai
ningst
agei
sasetofanchorpoi
ntsassoci
atedwi
tht
hei
r
r
espect
iver
egr
essor
s.Thi
sdi
cti
onar
yisobt
ainedi
ndependent
ly
and ahead f
rom t
he sear
ch st
ruct
ure,does not hav
e any
hi
erar
chi
calst
ruct
ure and t
he nei
ghbor
hoods,whi
ch can be
i
nter
pret
edascl
ust
ers,
shar
eel
ement
samongt
hem.Fur
ther
mor
e,
ant
ipodali
nvar
iancei
sani
mpor
tantaspectont
hesear
chst
rat
egy
asdi
scussedpr
evi
ousl
y.Thi
ssear
chst
ruct
urehast
obebui
l
t,
t
her
efor
e, adapt
ed t
o t
his scenar
io. We choose hashi
ng
t
echni
ques ov
ert
ree-
based met
hods f
ort
wo mai
nreasons:
Hashi
ng schemespr
ovi
del
ow memor
yusage (
the numberof
spl
i
tti
ngf
unct
ionsi
nhashi
ng-
basedst
ruct
uresi
sO(
log2(
n))whi
l
e
i
ntr
ee-
basedst
ruct
uresi
sO(
n),
wher
enr
epr
esent
sthenumberof
cl
ust
ers)andar
ehi
ghl
ypar
all
eli
zabl
e.Bi
nar
yhashi
ngt
echni
ques
ai
mtoembedhi
gh-
dimensi
onalpoi
ntsi
nbi
nar
ycodes,
prov
idi
nga
compactr
epr
esent
ati
onofhi
gh-
dimensi
onaldat
a.Among t
hei
r
v
astr
angeofappl
i
cat
ions,
theycanbeusedf
oref
fi
cientsi
mil
ari
ty
sear
ch,i
ncl
udi
ng appr
oxi
mat
enear
estnei
ghborr
etr
iev
al,si
nce
hashi
ng codes pr
eser
ver
elat
ive di
stances.Ther
e hasr
ecent
ly
been act
ive r
esear
ch i
n dat
a-dependent hashi
ng f
unct
ions
opposed t
o hashi
ng met
hods such as [
25]whi
ch ar
e dat
a-
i
ndependent
.Dat
adependentmet
hods i
ntend t
o bet
terf
itt
he
hashi
ngf
unct
iont
othedat
adi
str
ibut
ion[
26]
,[27]t
hroughanof
f-
l
i
net
rai
ningst
age.

Amongt
hedat
a-dependentst
ate-
of-
the-
art
smet
hods,wesel
ect
t
heSpher
ical
Hashi
ngal
gor
it
hm ofHeoetal

5:
Ant
ipodal
l
yInv
ari
antTr
ansf
orm.(
a)AandCar
eant
ipodal
poi
nts
(
C=−A)andBandDar
ecl
oset
obeant
ipodes(
D≈−B)
.Al
though
t
heyar
ever
ysi
mil
arf
eat
ures,Eucl
i
deandi
stancef
ail
stof
indt
he
pr
opernear
estnei
ghbor
,asi
tcannothandl
ethechar
act
eri
sti
csof
ant
ipodalpoi
nts. (
b) I
n our pr
oposed Ant
ipodal
l
yInv
ari
ant
Tr
ansf
ormat
ion(
AIT)
,wef
orbi
dagi
venhal
f-
spaceoft
hemani
fol
d
(
int
hiscaset
henegat
iveyaxi
s,i
.e.y
F·ey<0)andwepr
ojectt
he
f
eat
uresl
ayi
ngoni
ttoi
tscor
respondentant
ipodalpoi
nt.Af
ter
t
ransf
ormat
ion, Eucl
i
dean di
stance can pr
oper
ly deal wi
th
ant
ipodal
poi
nts.

whi
chi
sabl
etodef
inecl
osedr
egi
onsi
nRm wi
thasf
ewasone
spl
i
tti
ngf
unct
ion.Thi
shashi
ngf
ramewor
kisusef
ult
omodelt
he
i
nver
sesear
chschemeandenabl
est
obenef
itf
rom subst
ant
ial
speed-
upsbyr
educi
ngt
heNNsear
chi
ntoappl
yi
ngapr
ecomput
ed
f
unct
ion,whi
chconv
eni
ent
lyscal
eswi
thpar
all
eli
mpl
ement
ati
ons,
asshowni
nFi
gur
e4.Spher
icalhashi
ngdi
ff
ersf
rom pr
evi
ous
appr
oachesbyset
ti
nghy
per
spher
est
odef
inehashi
ngf
unct
ions
onbehal
foft
hepr
evi
ousl
yusedhy
per
planes.A gi
venhashi
ng
f
unct
ionH(
yF)=(
h1(
yF)
,...,
hc(
yF)
)mapspoi
ntsf
rom Rm t
oa
base2Nc,i
.e.{
0,1}c.Ev
eryhashi
ngf
unct
ionhk(
yF)i
ndi
cat
es
whet
hert
hepoi
nty
Fisi
nsi
dekt
hhy
per
spher
e,model
edf
ort
his
pur
poseasapi
votpk∈Rm andadi
stancet
hreshol
d(i
.e.r
adi
us
oft
hehy
per
spher
e)t
k∈R+a

appl
yi
ngt
hesamehashi
ngf
unct
ionst
otheanchorpoi
nts.Rel
ati
ng
ahashcodewi
thar
egr
essorcanbedonedur
ingt
rai
ningt
ime.The
sear
chr
etur
nsk-
NNf
oreachanchorpoi
nt,t
husnotensur
ingt
hat
al
lthei
nputi
magepat
cheshav
ear
elat
edr
egr
essor(
i.
e.whenev
er
t
hepat
chi
snotwi
thi
nthek-
NNofanyoft
heanchorpoi
nts)
.Two
sol
uti
ons ar
e pr
oposed:(
a)use a gener
alr
egr
essorf
ort
he
pat
ches whi
ch ar
e noti
nthe k-
NN ofany anchorpoi
nt,as
pr
oposedby[
13]or(
b)uset
her
egr
essoroft
hecl
osestl
abel
ed
hash code cal
cul
ated wi
tht
he spher
icalHammi
ng di
stance,
def
inedby[
24]asdSH(
a,b)=PP(
a⊕b)(
a∧b),wher
e⊕ i
sthe
XOR bi
toper
ati
on and ∧ i
stheAND bi
toper
ati
on.Not
ethat
al
thoughnotbei
ngguar
ant
eed,i
trar
elyhappenst
hatapat
chi
s
notwi
thi
n anyoft
he k-
NN r
egr
essor
s(e.
g.f
ort
he sel
ect
ed
par
amet
erof6hy
per
spher
esi
tnev
eroccur
s).Si
ncewehav
enot
obser
vedsi
gni
fi
cantdi
ff
erencesi
nper
for
mance,wesel
ect(
a)as
t
hel
owestcompl
exi
tysol
uti
on.I
nasi
mil
arway
,ani
nver
sesear
ch
mi
ghtal
soassi
gnt
woormor
eregr
essor
stoasi
ngl
epat
ch.I
tis
commoni
nthel
i
ter
atur
etodoar
e-r
anki
ngst
rat
egyt
odealwi
th
t
hisi
ssue[
28]
.Ant
ipodal
l
yInv
ari
antSpH.Dur
ingt
heopeni
ngof
t
hissect
ionwedi
scussedt
hegoodnessofpr
oper
lydeal
i
ngwi
th
t
he ant
ipodes. Howev
er,as many ot
her bi
nar
y spl
i
ts,t
he
f
unct
ioni
ngofspher
icalhashi
ngi
sunabl
etor
ecogni
zeant
ipodal
v
ect
orsandputt
hem t
oget
heri
nabi
n.Thi
shappensbycauseof
i
tst
hreshol
dingmechani
sm,whi
chi
sbasedonadi
rect
ionl
ess
Eucl
i
deandi
stancef
rom t
hecent
erofahy
per
spher
etot
hef
eat
ure
poi
nt.Asseenpr
evi
ousl
y,ant
ipodalpat
chesshoul
duset
hesame
r
egr
essorast
hesamel
i
nearoper
ati
onsar
eappl
i
edf
orbot
hof
t
hem,wi
tht
heuni
quedi
ff
erenceoft
hei
nputf
eat
ures’si
gn.I
n
or
dert
obenef
itf
rom t
hespeed-
upsofbi
nar
yspl
i
tsdesi
gnedt
o
useEucl
i
deandi
stance,wepr
oposeanov
elt
ransf
ormat
ionwhi
ch
enabl
est
hebenef
it
sofant
ipodal
inv
ari
ancei
ntheEucl
i
deanspace.

Asdescr
ibedbyt
heBor
suk-
Ulam t
heor
em [
29]
,anycont
inuous
f
unct
ionf
rom anm di
mensi
onal
uni
tspher
eSm t
oanEucl
i
deanm
-
spaceRm mapst
woant
ipodal
poi
ntsi
ntoasi
ngl
epoi
nt.Thegoal
ofourt
ransf
ormat
ioni
ssi
mil
ar:t
odesi
gnaf
unct
ionwhi
chmaps
t
woant
ipodalpoi
ntsi
ntoasi
ngl
epoi
nt.Howev
er,i
nthesi
tuat
ion
addr
essedi
nthi
spapert
her
eisami
smat
chbet
weent
hespher
e
space and t
he Eucl
i
dean space,as we dealwi
thaf
unct
ion
mappi
ngf
rom S m−1t
oR m.Ouri
deabui
l
dsondesi
gni
nga
cont
inuoushy
per
planet
hatcr
ossesSm−1passi
ngt
hrought
he
or
igi
nofcoor
dinat
es,
thusmappi
ngt
wopoi
ntsi
ntoasi
ngl
eone.I
n
or
der t
o do t
hat
, we enf
orce a f
orbi
dden space r
egi
on,
cor
respondi
ngt
othenegat
ivehal
f-
spaceoft
heqt
hdi
mensi
on,
i.
e.
t
hef
eat
uresmustbey
F·eq∈R+,wher
eeqi
stheqt
hst
andar
d
basi
sint
he Eucl
i
dean m space.I
n ourpr
oposed Ant
ipodal
l
y
I
nvar
iantTr
ansf
ormat
ion(
AIT)
,al
lthef
eat
uresl
yi
ngout
sidet
hat
spacear
epr
oject
edt
othei
rant
ipodes:

Al
ow-
dimensi
onali
l
lust
rat
iveexampl
eisshowni
nFi
gur
e5.The
di
mensi
onqshoul
dhav
ebal
ancedposi
ti
veand

:Spher
icalhashi
ngadapt
edt
other
egr
essorsear
chf
orSR.Cer
tai
n
hashi
ng f
unct
ions ar
e opt
imi
zed on f
eat
ure pat
ch st
ati
sti
cs
cr
eat
ing a setofhy
per
spher
es i
nter
sect
ions t
hatar
e di
rect
ly
l
abel
ed wi
th ahash code.I
ntr
aini
ng t
ime,r
egr
essor
sfi
l
lthi
s
 i
nter
sect
ions(
i.
e.bi
ns)andi
ntest
ingt
imet
hehashi
ngf
unct
ioni
s
appl
i
edt
oeachpat
ch,whi
chwi
l
ldi
rect
lymapi
ttoar
egr
essor
.
negat
ivev
aluessot
hatt
het
ransf
ormat
ioni
sef
fect
ive.Forour
t
rai
ning f
eat
ure v
ect
ors (
around 500K exampl
es)
, al
lthe
di
mensi
onswer
ehi
ghl
yandsi
mil
arl
ybal
anced,sowedeci
dedt
o
sel
ectt
hedi
mensi
onwi
thl
essv
ari
ati
on(
PCAanal
ysi
s).I
frat
her
t
hant
ransf
ormi
ngt
hepoi
ntswej
ustwantt
ouseanant
ipodal
l
y
i
nvar
iantmet
ri
c whi
ch can oper
atei
nthe Eucl
i
dean space,
Equat
ion (
14)i
s appl
i
ed i
n bot
hvect
ors dur
ing t
he Eucl
i
dean
di
stancecal
cul
ati
on:

andt
henusedi
nthedi
stancemet
ri
cδ(
pk,y
F)ofEquat
ion(
13)
.Thankst
o
t
hisweobt
ainanAnt
ipodal
l
yInv
ari
antSpher
icalHashi
ng(
AISpH)whi
chi
s
opt
imalf
ort
heSR r
egr
essi
onpr
obl
em andcanbeusedf
oranyot
her
pr
obl
em whi
chshar
est
hesamef
eat
urechar
act
eri
sti
cs.I
nFi
gur
e7we
showt
headv
ant
agesofusi
ngourpr
oposedAI
T.I
nFi
gur
e7(
a),
weconf
ir
m
t
hatnei
ghbor
hoods cr
eat
ed wi
th AI
Tfeat
ures and Eucl
i
dean di
stance
met
ri
chav
elowerav
eragedi
stancest
hanwi
thoutt
ransf
ormat
ion,hence
obt
aini
ngabet
terl
ocalcondi
ti
onandhav
ingahi
ghernumberofsampl
es
av
ail
abl
eforagi
venmaxi
mum di
stance.I
nFi
gur
e7(
b),weassest
he
r
esi
l
iencet
o ant
ipodalv
ari
anceofAI
T:Theav
erageangul
ardi
stances
obt
ainedwi
thAI
T nei
ghbor
hoods(
Equat
ion15)ar
eappr
oachi
ngt
hose
cr
eat
edwi
thapur
eant
ipodal
l
yinv
ari
antmet
ri
c(i
.e.AVCS)
.Thi
sisf
urt
her
v
ali
dat
edbyt
her
esul
tsshowni
nTabl
eII
,wher
eAI
TandAVCSobt
ain
si
mil
arPSNRper
for
mance
I
:Av
erageper
for
mancei
nter
msofPSNR(
dB)andt
ime(
s)f
orbi
cubi
c
gr
adi
ent
sfeat
uresandourGI
BPf
eat
ures,r
unonSet
14andSet
5ona×2
magni
fi
cat
ion f
act
or.LR r
efer
ence i
mage (
i.
e.t
he numberofpat
ches
ext
ract
edi
ntheupperscal
esi
sthesameasi
ntheLRr
efer
encescal
e,t
hus
t
hepat
chsi
zeandov
erl
apr
ati
oar
epr
opor
ti
onal
l
yadapt
ed)
.A.Feat
ure
SpaceWecompar
etheper
for
manceofourpr
oposedGI
BPf
eat
ureswi
th
t
hef
eat
urespr
oposedbyZey
deetal
.whi
char
ebasedont
hegr
adi
ent
sof
t
hebi
cubi
cint
erpol
ati
on.Wecompr
essbot
hfeat
ureswi
thPCAi
nor
dert
o
r
educet
hedi
mensi
onal
i
ty,
sot
hatnotonl
ythef
eat
uresar
emor
ecompact
,
butspeci
all
yther
egr
essor
s.I
nTabl
eIweshowhowbyusi
ngourpr
oposed
f
eat
uresweconsi
stent
lyi
mpr
ovei
nqual
i
ty(
i.
e.f
rom 0.
06dBt
o0.
10dB)
wi
thr
espectt
hepr
evi
ousl
yusedf
eat
ures.Weal
soassest
hat
,asexpect
ed,
t
hecomput
ati
ont
imeoft
hewhol
eSRal
gor
it
hm i
ncr
easesasGI
BPr
equi
res
t
hecomput
ati
onofmor
ebi
cubi
cint
erpol
ati
ons(
thr
eei
nter
pol
ati
onagai
nst
asi
ngl
eone)
.Nev
ert
hel
esst
hei
ncr
easei
nrunni
ngt
imehasl
owi
nci
dence
wi
thr
espectt
hewhol
eSRpi
pel
i
ne(
i.
e.about3%oft
het
otal
SRt
ime)
.

Sear
chst
rat
egyI
nor
dert
oconf
ir
mthesuper
ior
it
yofant
ipodal
l
yinv
ari
ant
met
ri
cs,we t
estEucl
i
dean di
stance agai
nstAVCS and ourpr
oposed
AI
T+Eucl
i
deandi
stance.Weonl
ywantt
oev
aluat
ethei
nci
denceoft
he
di
ff
erentmet
ri
cs,excl
udi
ngf
rom t
heexper
iment
alset
upt
heef
fectoft
he
appr
oxi
mat
esear
chi
ntr
oduced wi
tht
heAI
SpH (
ast
hiscompar
ison i
s
expl
i
cit
lyshowni
ntheResul
tssect
ion)
.Weper
for
m exper
iment
swi
tht
he
t
hreedi
ff
erentmet
ri
csusedf
orbot
htr
aini
ngandt
est
ingsepar
atel
y.The
r
esul
tscl
ear
lyshowt
headv
ant
ageofusi
nganabsol
utei
nvar
iantmet
ri
c,as
usi
ngt
hem i
mpr
ovesgr
eat
lyt
hePSNR(
+0.
18dB)
.

LI
TERATURESURVEY:

Resear
cher
s hav
e been st
udy
ing t
he SI
SR met
hods f
or pr
act
ical
appl
i
cat
ions.Especi
all
y wi
tht
he SR per
for
mance on sy
nthet
ic dat
a
becomi
ngbet
terandbet
ter
,mor
eandmor
eat
tent
ionhasbeenpai
dto
RSI
SR.Fi
g.2pr
esent
stheov
eral
ltaxonomyofexi
sti
ngRSI
SRt
echni
ques.
Not
ethatwef
ocusmor
eondeepl
ear
ning-
basedmet
hods.Accor
dingt
o
t
hepr
imar
ypr
inci
plesandchar
act
eri
sti
csofexi
sti
ngRSI
SRmet
hods,we
gr
oupt
hem i
ntof
ourcat
egor
ies,
i.
e.,
degr
adat
ionmodel
i
ng-
basedmet
hods
[
88]
–[98]
,imagepai
rsbasedmet
hods[
79]
–[87]
,domai
ntr
ansl
ati
on-
based
met
hods[
99]
–[110]
,andsel
f-
lear
ning-
basedmet
hods[
94]
,[
111]
–[115]
.Fi
g.
3andTabl
eII
Isummar
izeexi
sti
ngRSI
SRmet
hods.I
tiswor
thnot
ingt
hat
onemet
hodmaybel
ongt
odi
ff
erentcat
egor
ieswheni
tisv
iewedf
rom
di
ff
erentper
spect
ives.Thef
oll
owi
ngsect
ionsi
ntr
oducet
hesemet
hodsi
n
det
ail
.

Degr
adat
ionModel
ing-
basedMet
hods

Compar
edwi
tht
heSRofsy
nthet
icLRi
mages,
oneoft
hemai
nchal
l
enges
f
orRSI
SRi
sthatt
hedegr
adat
ionmodel(
i.
e.,
theD(
·)i
nEq.(
1))i
sunknown.
Gener
all
y,t
he degr
adat
ion par
amet
erseti
s necessar
yto desi
gn t
he
obj
ect
ivef
unct
ionf
orr
econst
ruct
ion-
basedSI
SRmet
hodsder
ivedf
rom t
he
Maxi
mum aPost
eri
ori(
MAP)est
imat
ion.Forl
ear
ning-
basedmet
hodst
hat
ai
mtoobt
aint
hemappi
ngf
rom LRi
magest
othei
rHRcount
erpar
ts,t
he
degr
adat
ion par
amet
er set i
s cr
uci
alf
or bui
l
ding t
rai
ning dat
aset
s.
Ther
efor
e,aspr
esent
ed i
n Fi
g.4,an i
ntui
ti
ve wayi
sto est
imat
ethe
degr
adat
ion par
amet
eroft
he LR i
nputpr
iort
o super
-r
esol
vi
ng ort
o
i
ter
ati
vel
yopt
imi
ze t
he degr
adat
ion par
amet
erand t
he super
-r
esol
ved
i
mage.Eq.(
3))showst
hecommonl
yuseddegr
adat
ionmodel
,i
nwhi
cht
he
bl
urker
nelBandnoi
senar
eunknowns.Theexpl
orat
ionbyEf
ratetal
.[141]
showst
hatanaccur
atebl
urmodeli
smor
eimpor
tantt
othesuccessofSR
t
hanasophi
sti
cat
edi
magepr
ior
,inpar
ti
cul
arf
orr
eal
-wor
ldi
mages.As
hi
ghl
i
ght
ed i
n[141]
,mostoft
he exi
sti
ng degr
adat
ion model
i
ng-
based
met
hodsf
ocusont
her
ecov
eryoft
hebl
urker
nel
.

Di
ff
erentf
rom t
heabov
enumer
icalopt
imi
zat
ion-
basedappr
oaches[
88]
,
[
89]
,thebl
urker
neland t
hesuper
-r
esol
ved i
magecan al
so bej
oint
ly
opt
imi
zedusi
ngdeepneur
alnet
wor
ks[
90]
–[92]
.Gener
all
y,t
hemi
smat
ch
ofbl
urker
nel
swoul
dresul
tinar
ti
fact
sint
hesuper
-r
esol
vedi
mage,e.
g.,
ov
ersmoot
hingorov
er-
shar
peni
ng.Basedont
hisobser
vat
ion,
Guetal
.[90]
andCor
nil
l
ereetal
.[91]pr
oposet
opr
ogr
essi
vel
ycor
rectt
hei
naccur
ate
ker
nelaccor
ding t
othe qual
i
tyoft
he super
-r
esol
ved i
mage.16 Mor
e
speci
fi
cal
l
y,t
heydev
elopdegr
adat
ion-
awar
eSRnet
wor
kst
opr
oduceHR
i
mages,i
nwhi
cht
hebl
urker
neli
sut
il
izedasauxi
l
iar
yinf
ormat
ionf
orSR.
Meanwhi
l
e,cor
respondi
ngdeepneur
alnet
wor
ksar
edesi
gnedt
ocor
rect
t
heker
nelwi
tht
hegui
danceoft
hei
nter
medi
ateSRr
esul
t.Unl
i
kepr
evi
ous
appr
oaches[
90]
,[91]t
hatcombi
net
woormor
enet
wor
ks,
Huangetal
.[92]
dev
elopadeepal
ter
nat
ingnet
wor
k(DAN)f
orRSI
SR,i
nwhi
cht
hei
ter
ati
ve
opt
imi
zat
ionpr
ocessbet
weent
hesuper
resol
vedi
mageandt
hebl
urker
nel
i
sunf
oldedt
oanendt
o-endt
rai
nabl
enet
wor
k.17DAN[
92]consi
stsofa
chai
n of al
ter
nat
ely st
acked r
est
orer
s and est
imat
ors, whi
ch ar
e
r
esponsi
blef
ort
her
est
orat
ionoft
heHRi
mageandt
heest
imat
ionoft
he
bl
urker
nel
,respect
ivel
y.Fort
heabov
emet
hods,t
hesuper
-r
esol
vedi
mage
andt
hebl
urker
nel
cor
respondi
ngt
otheLRi
nputar
eit
erat
ivel
yref
ined.

Pr
evi
ousst
udi
eshav
eshownt
hatnat
urali
magepr
ior
ssuchaspat
ch
r
ecur
rencepr
oper
tyar
eusef
ulf
ordegr
adat
ionmodel
i
ng.I
n[93]
,Mi
chael
iet
al
.poi
ntoutt
hatt
hePoi
ntSpr
eadFunct
ion(
PSF)i
snott
heopt
imalbl
ur
ker
nel
,andt
heyf
urt
herpr
oposet
oobt
aint
hepr
inci
pledMAPest
imat
eof
t
hebl
urker
nelv
iamaxi
miz
ingt
hesi
mil
ari
tyofr
ecur
ri
ngi
magepat
ches
acr
ossscal
esoft
heLRi
nput
.18Theest
imat
edbl
urker
nelcanbeusedt
o
degr
adet
heLRi
nputornat
uralHRi
magesar
ti
fi
cial
l
y.I
nthi
sway
,thebl
ur
ker
nelest
imat
ionappr
oachi
ssmoot
hlypl
uggedi
ntobot
hsel
f-
exampl
e-
basedandext
ernal
-exampl
e-basedSRappr
oaches[
47]
,[54]
.Ther
esul
tsi
n
[
93]showt
hatt
heaccur
acyenhancementofbl
urker
nel
est
imat
ionl
eadst
o
anobv
iousi
mpr
ovementofSRper
for
manceonsy
nthet
icaswel
lasr
eal
-
wor
ldi
mages.Degr
adat
ion model
i
ng i
sal
so v
ital
l
yimpor
tantf
ordeep
l
ear
ning-
basedSRappr
oaches.Deepconv
olut
ionalneur
alnet
wor
ks(
CNN)
-
basedSI
SRappr
oachesusual
l
yachi
evest
ate-
of-
the-
art(
SOTA)r
esul
tson
st
andar
d benchmar
ks.Nev
ert
hel
ess,t
hei
rper
for
mancei
sli
mit
ed when
appl
i
edt
oreal
-wor
ldi
mages.Themai
nreasoni
sthatt
heker
nel(
e.g.
,
“
bicubi
c”ker
nel
)usedt
ogener
atet
rai
ningdat
aissi
gni
fi
cant
lydi
ff
erent
f
rom t
hebl
uri
nar
ealscenar
io.Toaddr
esst
hispr
obl
em,somer
ecent
ly
pr
esent
eddeepl
ear
ningbasedRSI
SR met
hods[
94]
–[98]adoptt
hepr
e-
est
imat
eddegr
adat
ionpar
amet
erst
ogener
atesampl
esf
ormodelt
rai
ning.
Forexampl
e,i
nspi
redby[
93]
,Bel
l
-Kl
i
gleretal
.[94]dev
elopani
mage-
speci
fi
cint
ernal
-GAN(
i.
e.,Ker
nel
GAN)t
olear
nthei
nter
naldi
str
ibut
ionof
pat
ches.19TheKer
nel
GAN[
94]i
str
ainedsol
elyusi
ngt
heLRt
esti
mage,
maki
ngi
tsdi
scr
imi
nat
orunabl
etodi
ff
erent
iat
ethepat
chdi
str
ibut
ionoft
he
or
igi
nalLR i
nputf
rom t
hatoft
hedegr
aded v
ersi
on oft
he LR i
mage
pr
oducedbyt
hegener
ator
.Af
tert
hej
ointt
rai
ningwi
tht
hedi
scr
imi
nat
or,
t
hegener
atorcanwel
lchar
act
eri
zet
hedegr
adat
ionpr
ocesswi
thani
mage-
speci
fi
cker
nel
.Then,
theLRt
esti
mageandi
tsdegr
adedv
ersi
ongener
ated
byt
hegener
atorf
orm pai
reddat
aforSRmodelt
rai
ning.Bul
atetal
.[95]
t
rai
n a gener
ati
veadv
ersar
ialnet
wor
k(GAN)
-based degr
adat
ion model
f
rom unpai
redHRandLRf
acei
mages,
andt
henuset
hel
ear
nednet
wor
kto
gener
atedi
magepai
rsf
orSRnet
wor
ktr
aini
ng.20Zhouetal
.[96]pr
opose
t
oobt
ainagr
oupofr
eal
i
sti
cbl
urker
nel
sfr
om r
eal
-wor
ldphot
ogr
aphsand
aGANi
str
ainedont
hem t
oaugmentt
hepoolofr
eal
i
sti
cbl
urker
nel
s.21
Wi
tht
heaugment
edker
nelpool
,mor
ereal
i
sti
canddi
ver
seLR-
HRi
mage
pai
rscanbeconst
ruct
edt
otr
aint
heSRmodel
.Anal
ogousl
y,Xi
aoetal
.[97]
modelspat
ial
l
yvar
iantdegr
adat
ionv
ial
ear
ningasetofbasi
sbl
urker
nel
s
andcor
respondi
ngpi
xel
-wi
sewei
ght
sfr
om r
eal
-wor
ldi
magepai
rs.The
l
ear
ned r
eal
i
sti
c degr
adat
ion modeli
sthen used t
o gener
ate pseudo-
r
eal
i
sti
cLR-
HRi
magepai
rs.Mor
erecent
ly,Jietal
.[98]t
aket
hisi
deaone
st
epf
urt
heranddev
elopanef
fect
ivedegr
adat
ionf
ramewor
kusi
ngv
ari
ous
r
eal
i
sti
c bl
urker
nel
s and noi
se di
str
ibut
ions,wi
nni
ng t
he NTI
RE 2020
Chal
l
engeonReal
-Wor
ldI
mageSuper
-Resol
uti
on[
122]
.22Theout
standi
ng
per
for
manceofdegr
adat
ionmodel
i
ng-
basedRSI
SRmet
hodsdemonst
rat
es
t
hatdegr
adat
ionmodel
i
ngi
smeani
ngf
ulandt
hiski
ndofappr
oachi
sa
f
easi
blesol
uti
ont
otheSRofr
eal
-wor
ldi
mages.
I
magePai
rs-
basedMet
hodsAl
thoughpai
redLR-
HRt
rai
ningdat
acanbe
sy
nthesi
zedf
rom hi
gh-
qual
i
tyi
magesaccor
dingt
opr
e-def
ineddegr
adat
ion
model
s[96]
,[98]
,der
ivi
ngexpl
i
citr
eal
i
sti
cdegr
adat
ionmodel
sfr
om r
eal
-
wor
ldi
mages i
s chal
l
engi
ng.To dealwi
tht
his pr
obl
em,r
esear
cher
s
[
79]
–[85]pr
oposet
odi
rect
lycol
l
ectt
hei
magesoft
hesamescenewi
th
di
ff
erentr
esol
uti
ons,whi
char
eusedt
oconst
ructr
eal
i
sti
cLR-
HRi
mage
pai
rsf
orRSI
SRmodelt
rai
ning.Ov
eral
l
,asi
l
lust
rat
edi
nFi
g.5,cur
rent
ly
t
her
ear
ethr
eemai
nway
stocol
l
ectr
eal
-wor
ldi
magesf
ordat
asetbui
l
ding,
i
ncl
udi
ngt
hef
ocal
lengt
hadj
ust
ing-
basedappr
oach[
80]
–[84]
,thehar
dwar
e
bi
nni
ng-
basedappr
oach[
79]
,andt
hebeam spl
i
tter
-basedappr
oach[
85]
.
Ther
epr
esent
ati
ver
eal
i
sti
cdat
aset
sbui
l
twi
tht
heabov
eimagecol
l
ect
ion
appr
oachesar
edescr
ibedi
nSect
ionI
II
-A.Ther
efor
e,t
hissect
ionf
ocuses
ont
heRSI
SRmodel
sdev
elopedont
heser
eal
-wor
lddat
aset
s.I
ntui
ti
vel
y,
gi
venLR-
HRi
magepai
rs,near
lyal
lexi
sti
ngsuper
visedSRmet
hods(
e.g.
,
SRCNN [
67]
,VDSR [
68]
,EDSR [
70]
,RDN [
74]
,SRGAN [
69]
,et
c.)canbe
adopt
edt
olear
nthemappi
ngf
rom LRi
magest
othei
rHRcount
erpar
ts.
Themappi
ngl
ear
nedf
rom r
eal
i
sti
cdat
aset
sissupposedt
oappl
ytot
heSR
ofr
eal
-wor
ldi
mages.Howev
er,
inf
actchal
l
engesr
emai
n.

Fori
nst
ance,t
hedegr
adat
ionker
nel
sofr
eal
-wor
ldi
magesar
egener
all
y
non-
uni
for
m,v
ary
ingwi
tht
hedept
hinar
ealscene.Thus,t
rai
ninganSR
modelt
hatt
reat
sal
lpi
xel
sthesamewayasmostpr
evi
ousdeepCNN-
basedSRappr
oachesmaynotbet
heopt
imalsol
uti
on.Fort
hispr
obl
em,
Caietal
.[80]pr
oposet
heLP-
KPN,whi
chcombi
nest
heLapl
aci
anpy
rami
d
wi
tht
hepi
xel
-wi
seker
nelpr
edi
cti
onnet
wor
k(KPN)
,achi
evi
nggoodSR
per
for
manceandhi
ghef
fi
ciency
.23Anot
herchal
l
enget
hatcannotbe
i
gnor
ed i
sthe mi
sal
i
gnmentbet
ween LR and HR i
mage pai
rsi
nthe
col
l
ect
edr
eal
i
sti
cdat
aset
s.Al
thoughi
mager
egi
str
ati
oni
sper
for
med t
o
al
i
gnr
eal
i
sti
cimagepai
rs,
misal
i
gnmenti
sunav
oidabl
e.Asaconsequence
ofmi
sal
i
gnment
,bl
urr
ingar
ti
fact
smaybei
ntr
oducedi
nther
econst
ruct
ed
HRi
mageswhenusi
ngt
hesedat
aset
stot
rai
ntheSRmodelwi
thpi
xel
-t
o-
pi
xell
osses(
e.g.
,l1andl
2).I
nspi
redbyt
heCont
ext
ualLoss[
142]andt
he
edge-
preser
ving bi
l
ater
alf
il
ter [
143]
,Zhang etal
.[83]pr
opose t
he
Cont
ext
ualBi
l
ater
all
oss(
CoBi
)tor
esol
vet
hisi
ssue.24CoBii
ntegr
atest
he
pi
xel
-l
eveli
nfor
mat
ion and spat
ialpi
xelcoor
dinat
est
o measur
eimage
si
mil
ari
ty.Mor
eov
er,t
wospacesi
ncl
udi
ngRGBi
magepat
chesandpr
e-
t
rai
nedper
cept
ual
feat
ures(
e.g.
,VGG-
19[
144]
)ar
ejoi
ntl
yconsi
der
edi
n
CoBi
toi
mpr
oveper
for
mancef
urt
her
.Iti
sdemonst
rat
edt
hatCoBii
srobust
t
othemi
l
dmi
sal
i
gnmenti
nther
eal
i
sti
cimagepai
rsf
orsuper
visedSR
modelt
rai
ning.Consi
der
ing t
hatpi
xel
-wi
se l
osses ar
e gener
all
ymor
e
f
ocusedonsmoot
hingf
latr
egi
onsandshar
pingedgeswhi
l
enegl
ect
ingt
he
r
ecov
eryofr
eal
i
sti
cdet
ail
soft
ext
urest
osomeext
ent
,Weietal
.[81]
dev
elopaComponentDi
vi
de-
and-
Conquer(
CDC)SRmodelf
orr
eal
wor
ld
i
mages.25Mor
especi
fi
cal
l
y,t
hef
lat
,edge,andcor
nercomponent
sar
e
f
ir
stpr
edi
ctedbyt
hreecomponent
-at
tent
ivebl
ocksr
espect
ivel
yinCDCand
t
hent
heyar
eaggr
egat
edt
opr
oducet
hef
inalSR i
magebasedont
he
l
ear
ned component
att
ent
ive maps.To achi
evet
his goal
,a gr
adi
ent
-
wei
ght
edl
osst
hatcanadaptt
hemodelt
rai
ningt
other
econst
ruct
ion
di
ff
icul
ti
esofdi
ff
erenti
magecomponent
sisappl
i
ed.Ther
esul
tsonReal
SR
[
80]andDReal
SR[
81]pr
ovet
hesuper
ior
it
yandgener
ali
zat
ioncapabi
l
ityof
CDC [
81]
.Gi
venadat
asetcont
aini
ngr
aw andcol
ori
mages,l
ear
ninga
mappi
ng f
rom LR r
aw i
magest
o HR col
ori
magesusi
ng deep neur
al
net
wor
ksi
sani
ntui
ti
veappr
oacht
oexpl
oitr
aw i
magesf
orSR.Howev
er,
oner
awi
magecoul
dcor
respondt
oasetofcol
ori
magesbecausei
tdoes
nothav
ethei
nfor
mat
ionf
ort
hepr
ocesses(
e.g.
,col
orcor
rect
ion)wi
thi
n
t
hei
magesi
gnalpr
ocessi
ngsy
stem,maki
ngt
heabov
enai
vewaydonot
wor
kwel
l
.Toaddr
esst
hispr
obl
em,Xuetal
.[86]
,[87]desi
gnat
wo-
branch
CNN whi
chj
oint
lyexpl
oit
stheLRr
aw dat
aandcor
respondi
ngLRcol
or
i
maget
orecov
erf
inest
ruct
uresandhi
gh-
fi
del
i
tycol
orappear
ances.

I
naddi
ti
ont
otheabov
echal
l
engesshar
edbyal
mostal
lreal
-wor
ldi
mages,
t
heSRofcer
tai
nki
ndsofi
mages(
e.g.
,texti
mages,
remot
esensi
ngi
mages,
andmedi
cali
mages)usual
l
yhasapar
ti
cul
ari
ty.Ther
efor
e,speci
fi
cSR
model
sshoul
dbedesi
gnedf
ort
hesescenar
ios.Forexampl
e,t
heTSRN
dev
elopedbyWangetal
.[84]i
sanSRnet
wor
kforr
ealscenet
exti
mages.
26Tol
ever
aget
hest
rongsequent
ialchar
act
eri
sti
csoft
exti
mages,t
heBi
-
di
rect
ionalLSTM (
BLSTM)mechani
sm i
saddedt
obasi
cresi
dualbl
ocks.I
n
or
dert
o addr
esst
hemi
sal
i
gnmentpr
obl
em i
nther
eal
i
sti
ctexti
mage
dat
asetText
Zoom,
TSRN[
84]i
ntr
oducesaspat
ialt
ransf
orm net
wor
kbased
cent
ralal
i
gnmentmodul
eint
hef
rontoft
henet
wor
k.Fur
ther
mor
e,ai
ming
atenhanci
ngt
heshapeboundar
yofchar
act
ers,agr
adi
entpr
iorl
ossi
s
combi
nedwi
tht
hel
2losst
otr
ainTSRN.I
tisdemonst
rat
edt
hatt
heSRof
r
eal
wor
ldt
exti
magesusi
ngTSRNdoesi
ncr
easet
her
ecogni
ti
onaccur
acy
.
Pr
edi
ctabl
y,i
nsomecases,
SRcanal
sobenef
itot
hercomput
erv
isi
ont
asks,
e.
g.,
obj
ect
iondet
ect
ion[
5]andsemant
icsegment
ati
on[
4].

Domai
nTr
ansl
ati
on-
basedMet
hodsAsi
ntr
oducedi
nSect
ionsI
II
-AandI
V-B,
i
tishar
dtoobt
ainr
eal
-wor
lddat
aset
swi
thwel
l
-al
i
gnedLR-
HRi
magepai
rs.
Mor
eof
tent
hannot
,whatwehav
ear
eonl
yLRi
magesf
ormodelt
rai
ningi
n
pr
act
icalappl
i
cat
ions.Orbet
tery
et,asetofHRi
magesar
eav
ail
abl
efor
r
efer
ence besi
des LR t
rai
ning i
mages,but t
her
eis no one-
to-
one
cor
respondencebet
weenLRandHRi
mages.Thesuper
visedappr
oaches
nol
ongerappl
yint
hesecasesduet
othel
ackofpai
redsampl
es.Pr
evi
ous
st
udi
es[
99]
–[107]
,[107]
–[110]demonst
rat
ethatdomai
ntr
ansl
ati
oni
sa
f
easi
blesol
uti
ont
odealwi
tht
hispr
obl
em.Fort
hiski
ndofRSI
SRappr
oach,
r
eal
-wor
ldLRi
mages,
synt
het
icLRi
mages(
alsoknownascl
eanLRi
mages
ori
dealLRi
mages)
,andHRi
magesar
ethoughtt
obei
ndi
ff
erentdomai
ns.
Consequent
ly,
theSRofr
eal
-wor
ldi
magesconv
ert
stot
het
ransl
ati
onf
rom
t
her
eal
-wor
ldLRi
magedomai
n(RLRD)t
otheHRi
magedomai
n(HRD)
.
Ov
eral
l
,aspr
esent
edi
nFi
g.6,t
her
ear
etwomai
nway
stocr
ossRLRDt
o
HRD,i
.e.
,the t
wo-
stageand t
heone-
stageappr
oaches,and t
hemost
not
abl
edi
ff
erencel
i
esi
nwhet
hert
hesy
nthet
icLRi
magedomai
n(SLRD)i
s
usedasar
elayst
ati
on

Mostex
ist
ingSI
SRappr
oachesar
etr
ainedusi
ngsy
nthet
icdat
a,t
her
eby
achi
evi
ngexcel
l
entper
for
manceoncl
eanLRi
mages.Asi
sknownt
oal
l
,
t
her
eisanot
iceabl
edomai
ngapbet
weenSLRD andRLRD,causi
ngt
he
degr
adat
ion ofSR accur
acyon r
eal
-wor
ldi
mages.I
ntui
ti
vel
y,one can
mi
ti
gat
ethi
sper
for
mancedegr
adat
ionv
iar
educi
ngt
hedomai
ngap,whi
ch
i
showt
het
wo-
stagedomai
ntr
ansl
ati
on-
basedRSI
SRappr
oacheswor
k.As
showni
nFi
g.6,
thet
wo-
stageappr
oachesgener
all
yincl
udet
womai
nst
eps,
i
.e.
,domai
ntr
ansl
ati
onandSR.Usual
l
y,gener
icSRmet
hodsar
eadopt
edi
n
t
heSRr
econst
ruct
ionphase,soast
obenef
itf
rom pr
etr
ainedSRmodel
s
l
ear
ned f
rom l
arge-
scal
e dat
aset
s.The Cy
cle-
in-
Cycl
e GAN (
CinCGAN)
dev
elopedbyYuanetal
.[99]i
sar
epr
esent
ati
veoft
hiski
ndofRSI
SR
met
hod.Mor
especi
fi
cal
l
y,Ci
nCGAN [
99]f
ir
stusesadomai
ntr
ansl
ati
on
net
wor
ktomapr
eal
i
sti
cLRi
magesi
nRLRDi
ntot
heSLRD,andt
henapr
e-
t
rai
neddeepSRnet
wor
kwi
tht
hei
dealdegr
adat
ionassumpt
ioni
sst
acked
t
oupscal
ethet
ransl
ati
onr
esul
ttot
hedesi
redsi
ze.Fi
nal
l
y,t
hedomai
n
t
ransl
ati
on and SR net
wor
ks ar
e end-
to-
end f
ine-
tuned.I
nspi
red by
Cy
cleGAN[
145]
,thedomai
ntr
ansl
ati
onnet
wor
kinCi
nCGAN[
99]i
str
ained
t
o map r
eal
i
sti
c LR i
nput
sto sy
nthet
ic LR i
mages (
i.
e.,“
bicubi
c”-
downsampl
ed i
mages)usi
ng unpai
red t
rai
ning dat
a,whi
ch usual
l
ycan
suppr
esst
hear
ti
fact
ssuchasnoi
sei
nreal
wor
ldLRi
magest
omaket
hem
mor
esui
tabl
efort
hef
oll
owi
ngSRnet
wor
k.Resul
tsshowt
hatt
heCi
nCGAN
[
99]t
rai
nedwi
thunpai
reddat
aachi
evescompar
abl
eSRper
for
manceas
super
visedmet
hods.I
nMCi
nCGAN[
100]
,theCi
nCGAN[
99]i
simpr
ovedv
ia
i
ntr
oduci
ngapr
ogr
essi
vemul
ti
-cy
clef
ramewor
kforl
arge-
scal
eupsampl
i
ng
andanewconst
rai
ntt
osuppr
esst
hecol
orf
luct
uat
ioni
ntr
aini
ng.Formor
e
st
abl
emodelt
rai
ningandbet
terSRper
for
mance,
theDDGANdev
elopedby
Ki
m etal
.[101]f
urt
hercombi
nest
hepi
xel
wisel
oss,t
heVGGf
eat
urel
oss,
and t
he SSI
M l
oss t
o measur
e si
mil
ari
ty.27 Meanwhi
l
e,a domai
n
di
scr
imi
nat
ort
hatt
akest
henoi
se,t
ext
ure,andcol
ori
ntoconsi
der
ati
on
si
mul
taneousl
yispr
oposedt
omaket
hegener
atedi
magemor
econsi
stent
wi
tht
he t
argetdomai
n di
str
ibut
ion.Mor
erecent
ly,Maeda etal
.[102]
pr
oposeanend-
to-
endt
rai
nabl
efr
amewor
kUI
SRPSt
ojoi
ntl
yopt
imi
zet
he
domai
ntr
ansl
ati
onnet
wor
kandt
heSRnet
wor
k,achi
evi
ngexcel
l
entSR
r
esul
ts on r
eal
-wor
ldf
ace i
mages and aer
iali
mages.Benef
it
ing t
he
ar
chi
tect
ure,i
tisconv
eni
entt
oint
egr
ateexi
sti
ngSRnet
wor
ksandpi
xel
-
wi
sel
ossf
unct
ionsi
ntoUI
SRPS[
102]
.

Di
ff
erentf
rom t
het
wo-
stagemet
hodsment
ionedabov
e,t
heone-
stage
domai
ntr
ansl
ati
on-
based RSI
SR f
ramewor
ks ai
m t
o pr
oduce a super
-
r
esol
vedi
magedi
rect
lyf
rom t
her
eal
-wor
ldLRi
nput
,aspr
esent
edi
nFi
g.6.
Howt
olear
nthet
ransl
ati
onmappi
ngf
rom RLRDt
oHRDwi
thoutr
eal
LR-
HR
i
magepai
rsi
sthecr
uci
ali
ssue.Pr
ajapat
ietal
.[103]pr
oposet
otr
ainaGAN
-
basednet
wor
kUSI
SRNett
oupsampl
ereal
-wor
ldi
mages.28Si
nceonl
y
unpai
redLRandHRt
rai
ningi
magesar
eav
ail
abl
e,USI
SRNeti
sopt
imi
zed
t
hroughunsuper
visedl
ear
ning.Bey
ondt
hest
andar
dGAN l
osses,api
xel
-
wi
secont
entl
oss,
aTot
al-
Var
iat
ionl
oss,
andaqual
i
tyassessmentl
ossar
e
combi
nedt
oopt
imi
zeUSI
SRNet
.Thecont
entl
ossmakest
heSRr
esul
tbe
notf
arf
rom t
hebi
cubi
cal
l
yupsampl
edv
ersi
on,
thuspr
eser
vingt
hepr
imar
y
cont
entoft
heLRi
mage.TheTot
al-
Var
iat
ionl
ossi
sint
egr
atedt
osuppr
ess
t
henoi
seandar
ti
fact
s.Forbet
terper
cept
ualqual
i
tyoft
hesuper
-r
esol
ved
i
mage,t
hel
ear
nedMeanOpi
nionScor
eisusedt
oconst
ructt
hequal
i
ty
assessmentl
oss.Thankst
othecombi
nat
ionofmul
ti
plel
osses,USI
SRNet
[
103]achi
evesgoodgener
ali
zat
ioncapaci
ty.I
nspi
redbyCy
cleGAN [
145]
,
somer
esear
cher
spr
oposet
olear
nthedi
rectr
elat
ionshi
pbet
weenRLRDt
o
HRDusi
ngcy
cleconsi
stency
-const
rai
nedGANs[
104]
,[105]
.29Gi
venanLR
i
mage,t
heLR-
to-
HRgener
ator(
i.
e.,RLRDt
oHRD)i
str
ainedt
oreconst
ruct
av
ivi
dHRi
maget
hatcanber
etur
nedt
otheLRi
nputby

t
hecor
respondi
ngHR-
to-
LRgener
ator(
i.
e.,HRDt
oRLRD)
.Conv
ersel
y,an
HR i
mageshoul
dbewel
lrecov
eredbyt
heLRt
o-HR gener
atorf
rom i
ts
downsampl
edv
ersi
onpr
oducedbyt
heHR-
to-
LRgener
ator
.Af
tert
hej
oint
t
rai
ningoft
het
wogener
ator
sandcor
respondi
ngdi
scr
imi
nat
ors,
theLR-
to-
HRgener
atormodel
sthedi
rectmappi
ngf
rom RLRDt
oHRDandi
tisused
t
oreconst
ructHR i
mages f
rom r
eal
i
sti
c LR i
mages.Consi
der
ing t
hat
l
ear
ningt
heend-
to-
endt
ransl
ati
onbet
weenRLRDt
oHRDf
rom unpai
redLR
-
HRdat
aischal
l
engi
ng,par
toft
heone-
stagedomai
ntr
ansl
ati
on-
based
RSI
SRmet
hods[
106]
–[110]al
souset
hesy
nthet
icLRi
mageasabr
idgei
n
t
het
rai
ningphase.Gi
venunpai
redr
eal
i
sti
cLRandHRi
mages,
Fri
tscheetal
.
[
106]f
ir
stbi
cubi
cal
l
ydownsampl
eHRi
mages.Bi
cubi
cal
l
ydownsampl
ed
r
esul
tsar
ethent
ransl
atedi
ntot
her
eal
i
sti
cdomai
ntomaket
hem f
oll
ow
r
ealscenechar
act
eri
sti
cs,
usi
ngast
andar
dGAN-
baseddomai
ntr
ansl
ati
on
net
wor
ktr
ainedont
hebi
cubi
cal
l
ydownsampl
edi
magesandr
eal
i
sti
cLR
i
magesi
nanunsuper
visedf
ashi
on.Taki
ngt
hepseudo-
real
i
sti
cLRi
mages
andcor
respondi
ngHRi
magesast
rai
ningsampl
epai
rs,t
heESRGAN[
146]
i
str
ainedf
orupsampl
i
ngi
nasuper
visedmanner
.Inor
dert
ogener
ate
i
mageswel
lmat
chi
ngt
het
argetdi
str
ibut
ion,bot
hdomai
ntr
ansl
ati
onand
SRnet
wor
ksar
eopt
imi
zedwi
thf
requencysepar
ati
on-
basedl
ossf
unct
ions.
30Mor
especi
fi
cal
l
y,t
hecol
orl
oss,t
het
ext
urel
oss,andt
heper
cept
ual
l
ossar
eempl
oyedt
othel
ow-
fr
equencycomponent
,thehi
gh-
fr
equency
component
,andt
hewhol
eimage,
respect
ivel
y.Not
ethatonl
ytheSRmodel
i
sneededt
oupscal
ereal
-wor
ldi
magesi
nthet
est
ingphasebecausei
tis
t
rai
nedont
hei
magepai
rst
hatf
oll
owr
eal
wor
ldi
magedi
str
ibut
ion.Ont
his
basi
s,r
ecent
ly Umeretal
.[107]i
mpr
ovet
he GAN-
based SR model
f
oll
owi
ng t
he r
eal
wor
ldi
mage obser
vat
ion model
,thus expl
oit
ing t
he
power
fulr
egul
ari
zat
ionand opt
imi
zat
iont
echni
quessi
mul
taneousl
y.31
Radetal
.[108]conv
ertr
eal
i
sti
cLRi
magest
obi
cubi
clook-
ali
kei
mages
basedont
hei
rcopy
ingmechani
sm andbi
cubi
cper
cept
uall
oss.Di
ff
erent
f
rom t
heabov
ewor
ks[
106]
–[108]whi
chuseasi
ngl
edi
rect
iondomai
n
t
ransl
ati
onmodel
,Lugmay
retal
.[109]andChenetal
.[110]pr
oposet
o
t
rai
nabi
-di
rect
ionaldomai
ntr
ansl
ati
onmodelwi
tht
hecy
cleconsi
stency
const
rai
ntsf
orbet
terr
obust
ness.

Ov
eral
l
,domai
ntr
ansl
ati
oni
sanef
fect
ivewayt
oreducet
hedomai
ngap
bet
weensy
nthet
icandr
eal
i
sti
cdat
a,t
husi
mpr
ovi
ngt
hegener
ali
zat
ion
capabi
l
ityofSRmodel
sonev
erchangi
ngr
eal
-wor
ldi
mages.I
ncont
rast
,the
t
wo-
stage domai
ntr
ansl
ati
on-
based RSI
SR appr
oaches can i
ntegr
ate
sy
nthet
ic dat
a-t
rai
ned SR model
s mor
e el
egant
ly,whi
l
ethe one-
stage
met
hodsgener
all
yhav
elowercompl
exi
tyi
nthet
est
ingphase.

Sel
f-
Lear
ning-
basedMet
hodsMostexi
sti
ngRSI
SRmet
hodsuseext
ernal
dat
aset(
i.
e.,
pai
redorunpai
redt
rai
ningdat
a)t
otr
ainSRmodel
s.Ther
efor
e,
t
heSRper
for
mancei
sti
ght
lyboundt
otheconsi
stencybet
weent
est
ing
dat
aandt
rai
ningdat
a.Howev
er,
real
-wor
ldi
magesdonotal
way
sobeyt
he
char
act
eri
sti
cs oft
rai
ning dat
a.I
n or
dert
oreduce t
he i
mpactoft
he
t
rai
ning–t
est
ingdi
scr
epancyonSRper
for
mance,r
esear
cher
spr
oposet
o
expl
oitt
hei
nter
nali
nfor
mat
ionoft
heLRi
nputt
olear
nimage-
speci
fi
cSR
modelasshowni
nFi
g.7.The“
Zer
o-Shot
”SR(
ZSSR)dev
elopedbyShocher
etal
.[111]i
soneoft
her
epr
esent
ati
ves.32Thesel
f-
super
visedappr
oach
ZSSR[
111]i
sbasedont
hecr
oss-
scal
eint
ernalr
ecur
renceofi
nfor
mat
ion,
whi
chi
sacommonpr
oper
tyofnat
urali
mages.Mor
especi
fi
cal
l
y,anei
ght
-
l
ayerCNNi
str
ainedt
omodeli
mage-
speci
fi
cLR-
HRr
elat
ionsi
nthet
est
ing
phase,usi
ngt
heexampl
epai
rsext
ract
edf
rom t
heLRt
esti
mageandi
ts
degr
adedv
ersi
on.I
nconsi
der
ati
onoft
hei
nsuf
fi
ciencyoft
rai
ningdat
a(t
he
t
esti
mageonl
y),dat
aaugment
ati
oni
sadopt
edwhenext
ract
ingi
mage-
speci
fi
cLRHRpai
rs.Si
nceZSSR[
111]canadapti
tsel
ftodi
ff
erentt
est
ing
i
mages,i
tachi
evesexcel
l
entSRper
for
manceonr
eal
wor
ldi
mages,whose
degr
adat
ionpr
ocessi
snon-
idealandunknown.Agai
nbasedont
hecr
oss-
scal
erecur
rencepr
oper
ty,Bel
l
-Kl
i
gleretal
.[94]pr
oposet
otr
ainani
mage-
speci
fi
cGAN(
Ker
nel
GAN)t
omodelt
hedegr
adat
ionpr
ocess(
i.
e.,t
hebl
ur
ker
nel
)oft
hei
nput
.19 Ther
efor
e,a f
ull
ysel
f-
super
vised i
magespeci
fi
c
RSI
SRf
ramewor
kcanbeachi
evedwhent
hebl
urker
nelest
imat
ionmodul
e
Ker
nel
GAN[
94]i
spl
uggedi
ntot
her
econst
ruct
ionmodul
eZSSR[
111]
.To
j
oint
lyt
rai
nthei
mage-
speci
fi
cdegr
adat
ionandSRnet
wor
ks,
Kim etal
.[112]
desi
gnauni
fi
edi
nter
nall
ear
ning-
basedSRf
ramewor
kDBPI
,consi
sti
ngof
an SR net
wor
kand a downscal
i
ng net
wor
k.33 I
nthe sel
f-
super
vised
t
rai
ningphaseofDBPI
,theSRnet
wor
kisopt
imi
zedt
oreconst
ructt
heLR
i
nputi
magef
rom t
hei
tsdownscal
edv
ersi
onpr
oducedbyt
hedownscal
i
ng
net
wor
k.Meanwhi
l
e,t
hedownscal
i
ngnet
wor
kist
rai
nedt
orecov
ert
heLR
i
nputi
magef
rom i
tssuper
-r
esol
vedv
ersi
ongener
atedbyt
heSRnet
wor
k.
Si
mil
arl
y,Emadetal
.[113]pr
oposet
heDual
SRt
hatj
oint
lyopt
imi
zesan
i
mage-
speci
fi
c downsampl
er and cor
respondi
ng upsampl
er. Mor
e
speci
fi
cal
l
y,Dual
SR [
113]i
str
ainedwi
tht
hecy
cle-
consi
stencyl
oss,t
he
maskedi
nter
pol
ati
onl
oss,
andt
headv
ersar
iall
ossusi
ngt
hepat
chesf
rom
t
he t
esti
mage.Resul
tsi
n[112]
,[113]show t
hatt
he compl
ement
ary
t
rai
ningoft
hei
mage-
speci
fi
cdegr
adat
ionandSRnet
wor
ksi
sbenef
ici
alt
o
t
her
econst
ruct
ionper
for
mance.

Al
though sel
f-
lear
ning-
based RSI
SR appr
oaches such as ZSSR [
111]
,
Ker
nel
GAN[
94]
,andDBPI[
112]canbeeasi
l
yadapt
edt
oLRi
nputi
mages,
t
heygener
all
yhav
etwo mai
nshor
tcomi
ngsduet
othesel
f-
super
vised
t
rai
ningst
rat
egy
.Fi
rst
,theopt
imi
zat
ionofSR model
sonl
yut
il
izest
he
i
nter
nali
nfor
mat
ion oft
he LR i
nput
,whi
l
e a gr
eatdealofext
ernal
i
nfor
mat
ion i
s negl
ect
ed.Second,t
hese met
hods ar
e usual
l
yti
me-
consumi
ngi
nthet
est
ingphasebecauseofonl
i
net
rai
ning.Toov
ercome
t
hesedi
sadv
ant
ages,met
a-l
ear
ningi
sint
roducedi
ntor
ecentsel
f-
lear
ning-
basedSR met
hods[
114]
,[115]
.BasedonZSSR [
111]
,Sohetal
.[114]
pr
esentt
hemet
a-t
ransf
erl
ear
ningf
orzer
o-shotSR(
MZSR)
,whi
chconsi
sts
oft
hreest
eps,i
.e.
,lar
ge-
scal
etr
aini
ng,met
a-t
ransf
erl
ear
ning,andmet
a-
t
est
.34I
nor
dert
oeaset
het
rai
ningoft
heSRnet
wor
kandt
hemet
a-
l
ear
ning,t
hel
arge-
scal
etr
aini
ngst
epf
ir
stt
rai
nsanei
ght
-l
ayerSRnet
wor
k
wi
tht
hepi
xel
-wi
sel
1lossont
hel
arge-
scal
edat
asetDI
V2K[
128]
.Themet
a
-
tr
ansf
erl
ear
ningpr
ocessai
mst
ofi
ndagener
ici
nit
ialpoi
ntf
ori
nter
nal
l
ear
ning f
oll
owi
ng t
heModel
-Agnost
icMet
a-Lear
ning [
147]
,maki
ng t
he
modelcan be qui
ckl
yadapt
ed t
o new i
magecondi
ti
onswi
thi
naf
ew
gr
adi
entupdat
es.I
nthemet
a-t
estphase,t
hei
nputt
esti
magei
sfi
rst
degr
adedt
opr
oduceexampl
epai
rsf
ormodelpar
amet
erupdat
e,andt
hen
i
tisf
edi
ntot
heupdat
edmodelt
ogener
ateanSRr
esul
t.Thankst
othe
met
atr
ansf
er l
ear
ning st
rat
egy
, MZSR [
114] achi
eves compet
it
ive
per
for
mancei
nter
msofbot
hthequal
i
tyoft
hesuper
-r
esol
vedi
mageand
t
he r
unni
ng t
ime.I
n[115]
,Par
k etal
.al
so pr
opose t
oimpr
ovet
he
per
for
manceofSOTA SRnet
wor
kssuchasRCAN [
73]usi
ngt
hemet
a-
l
ear
ningst
rat
egy
,wi
thoutchangi
ngt
heor
igi
nalar
chi
tect
ures.35Ont
he
whol
e,met
a-l
ear
ningbasedSRappr
oacheshav
est
rengt
hsi
nreconst
ruct
ion
qual
i
ty,
gener
ali
zat
ioncapabi
l
ity
,andpr
ocessi
ngef
fi
ciency
.
SOFTWAREDESCRI
PTI
ON

MATLAB:

MATLAB 
(mat
ri
x 
labor
ator
y) i
s a mul
ti
-par
adi
gm 
numer
ical
comput
ing 
env
ironment and 
four
th-
gener
ati
on pr
ogr
ammi
ng l
anguage.
A 
propr
iet
arypr
ogr
ammi
ng l
anguage 
dev
eloped by
 Mat
hWor
ks,MATLAB
al
l
ows 
mat
ri
x 
mani
pul
ati
ons, pl
ott
ing of
 f
unct
ions 
and dat
a,
i
mpl
ement
ati
onof
 al
gor
it
hms,cr
eat
ionof
 useri
nter
faces,andi
nter
faci
ng
wi
th pr
ogr
ams wr
it
ten i
n ot
her l
anguages,
i
ncl
udi
ng 
C, 
C++,
 
Jav
a, 
For
tran 
and 
Pyt
hon.

Al
though MATLAB i
sint
ended pr
imar
il
yfornumer
icalcomput
ing,an
opt
ionalt
ool
box uses t
he 
MuPAD 
symbol
i
c engi
ne,al
l
owi
ng access
t
o sy
mbol
i
ccomput
ing 
capabi
l
iti
es.Anaddi
ti
onalpackage,
 
Simul
i
nk,adds
gr
aphi
cal mul
ti
-domai
n si
mul
ati
on and 
model
-based
desi
gn 
for
 dy
nami
c and 
embeddedsy
stems.

MATLAB i
s a hi
gh-
per
for
mance l
anguage f
ort
echni
calcomput
ing.I
t
i
ntegr
atescomput
ati
on,v
isual
i
zat
ion,andpr
ogr
ammi
ngi
naneasy
-to-
use
env
ironmentwher
e pr
obl
ems and sol
uti
ons ar
e expr
essed i
nfami
l
iar
mat
hemat
ical
not
ati
on.

MATLABi
sani
nter
act
ivesy
stem whosebasi
cdat
ael
ementi
sanar
rayt
hat
doesnotr
equi
redi
mensi
oni
ng.Thi
sal
l
owsy
out
osol
vemanyt
echni
cal
comput
ingpr
obl
ems,especi
all
ythosewi
thmat
ri
xandv
ect
orf
ormul
ati
ons,
i
n af
ract
ionoft
het
imei
twoul
dtaket
owr
it
eapr
ogr
am i
nascal
arnon
i
nter
act
ivel
anguagesuchasCorFor
tran.

ThenameMATLABst
andsf
ormat
ri
xlabor
ator
y.MATLABwas
or
igi
nal
l
ywr
it
tent
opr
ovi
deeasyaccesst
omat
ri
xsof
twar
edev
elopedby
t
heLI
NPACK andEI
SPACKpr
oject
s.Today
,MATLABengi
nesi
ncor
por
ate
t
heLAPACK andBLASl
i
brar
ies,
embeddi
ngt
hest
ateoft
hear
tinsof
twar
e
f
ormat
ri
xcomput
ati
on.MATLABhasev
olv
edov
eraper
iodofy
ear
swi
th
i
nputf
rom manyuser
s.I
n uni
ver
sit
yenv
ironment
s,i
tist
he st
andar
d
i
nst
ruct
ionalt
oolf
ori
ntr
oduct
oryandadv
ancedcour
sesi
nmat
hemat
ics,
engi
neer
ing,
andsci
ence.I
nindust
ry,
MATLABi
sthet
ool
ofchoi
cef
orhi
gh-
pr
oduct
ivi
tyr
esear
ch,dev
elopment
,and anal
ysi
s.MATLAB f
eat
ures a
f
ami
l
y ofadd-
on appl
i
cat
ion-
speci
fi
c sol
uti
ons cal
l
ed t
ool
boxes.Ver
y
i
mpor
tantt
omostuser
sofMATLAB,t
ool
boxesal
l
ow y
out
oLear
nand
appl
yspeci
ali
zedt
echnol
ogy
.Tool
boxesar
ecompr
ehensi
vecol
l
ect
ionsof
MATLABf
unct
ions(
M-f
il
es)t
hatext
endt
heMATLABenv
ironmentt
osol
ve
par
ti
cul
arcl
assesofpr
obl
ems.Ar
easi
nwhi
cht
ool
boxesar
eav
ail
abl
e
i
ncl
udesi
gnalpr
ocessi
ng,cont
rolsy
stems,neur
alnet
wor
ks,f
uzzyl
ogi
c,
wav
elet
s,si
mul
ati
on,
andmanyot
her
s.

FEATURES:

• Hi
gh-
lev
ell
anguage f
or numer
icalcomput
ati
on,v
isual
i
zat
ion,and
appl
i
cat
iondev
elopment

•I
nter
act
iveenv
ironmentf
ori
ter
ati
veexpl
orat
ion,desi
gn,and pr
obl
em
sol
vi
ng

•Mat
hemat
icalf
unct
ionsf
orl
i
nearal
gebr
a,st
ati
sti
cs,Four
ieranal
ysi
s,
f
il
ter
ing,

opt
imi
zat
ion,numer
icali
ntegr
ati
on,and sol
vi
ng or
dinar
y di
ff
erent
ial
equat
ions

•Bui
l
t-i
ngr
aphi
csf
orv
isual
i
zingdat
aandt
ool
sforcr
eat
ingcust
om pl
ots

•Dev
elopmentt
ool
sfori
mpr
ovi
ngcodequal
i
tyandmai
ntai
nabi
l
ityand
maxi
mizi
ngper
for
mance

•Tool
sforbui
l
dingappl
i
cat
ionswi
thcust
om gr
aphi
cal
int
erf
aces

•Funct
ions f
ori
ntegr
ati
ng MATLAB based al
gor
it
hms wi
th ext
ernal
appl
i
cat
ionsandl
anguagessuchasC,
Jav
a,.
NET,
andMi
crosof
tExcel
.

USESOFMATLAB:

Mat
handcomput
ati
on

Al
gor
it
hm dev
elopment

Model
l
ing,
simul
ati
on,
andpr
otot
ypi
ng
Dat
aanal
ysi
s,expl
orat
ion,
andv
isual
i
zat
ion

Sci
ent
if
icandengi
neer
inggr
aphi
cs

Appl
i
cat
iondev
elopment
,incl
udi
nggr
aphi
cal
useri
nter
facebui
l
ding

MATLABSYSTEM:

Fi
vemai
npar
tst
hatt
heMATLABsy
stem consi
stsof
,andwhi
char
e:

Dev
elopmentEnv
ironment

TheMATLABMat
hemat
ical
Funct
ionLi
brar
y

TheMATLABLanguage

Handl
eGr
aphi
cs

TheAppl
i
cat
ionPr
ogr
am I
nter
face(
API
)

MATLABBASI
CS:

MATLABnumber
sandnumer
icf
ormat
s

Al
lnumer
icalv
ari
abl
es ar
e st
ored i
n MATLAB i
n doubl
e
pr
eci
sionf
loat
ing-
poi
ntf
orm.(
Inf
acti
tispossi
blet
ofor
cesomev
ari
abl
es
t
obeofot
hert
ypesbutnoteasi
l
yandt
hisabi
l
ityi
snotneededher
e.)
Fl
oat
ing-
poi
ntr
epr
esent
ati
onofnumber
sisessent
ial
l
yequi
val
entt
othe
“
sci
ent
if
icnot
ati
on”ofy
ourcal
cul
ator
.Speci
fi
cal
l
ya(
real
)numberxi
s
st
oredi
nbi
nar
yfl
oat
ingpoi
ntf
orm as
X=f
*2E

wher
efi
scal
l
edt
hemant
issaandEt
heexponent
.

St
ri
ngsandpr
int
ing

Thi
siseasi
l
yachi
evedbyusi
ngst
ri
ngsandt
heMATLABpr
int
f
unct
ionf
pri
ntfi
naspeci
alf
il
ecal
l
edst
art
up.
m whi
chwi
l
lbeexecut
ed
aut
omat
ical
l
ywheny
oust
artMATLAB.St
ri
ngscanbedef
inedi
nMATLAB
bysi
mpl
yencl
osi
ngt
heappr
opr
iat
est
ri
ngofchar
act
ersi
nsi
ngl
equot
es
suchas

>>S=’
MyName’

r
esul
tsi
ntheout
put

S=

Myname

Mor
ecompl
i
cat
edst
ri
ngscanbepr
int
edusi
ngt
heMATLABf
unct
ionf
pri
ntf
.
Thi
sisessent
ial
l
yaCpr
ogr
ammi
ngcommandwhi
chcanbeusedt
oobt
ain
awi
de-
rangeofpr
int
ingspeci
fi
cat
ions.Ver
yli
tt
leofi
tsdet
aili
sneededat
t
hisst
age.Weusei
ther
eonl
ytoi
l
lust
rat
epr
int
ingst
ri
ngs.Theuseof
f
pri
ntft
opr
intt
heNameandCour
sei
nfor
mat
ioni
sil
l
ust
rat
edby

>>f
pri
ntf
(’MyName\
nCour
se\
n’)

MyName

Cour
se

>>
wher
ethe\
nist
henew-
li
necommand.Thef
inalonei
sincl
udedt
oensur
e
t
hatt
henextMATLAB pr
omptoccur
satt
hebegi
nni
ngoft
henex
tli
ne
r
athert
hani
mmedi
atel
yatt
heendoft
hepr
int
edst
ri
ng.

Edi
ti
ngi
ntheCommandWi
ndow

Al
thoughwehav
enoty
etdonev
erymuchi
ntheMATLAB
CommandWi
ndow,i
tiswor
thsummar
izi
ngsomeoft
hebasi
cedi
ti
ng
oper
ati
onsav
ail
abl
ether
e.Mostoft
heel
ement
aryedi
ti
ngoper
ati
onsar
e
compl
etel
yst
andar
d:

BackSpaceandDel
etewor
kjustasi
nanyot
herWi
ndows-
basedpr
ogr
am;

Homemov
est
hecur
sort
othebegi
nni
ngoft
hecur
rentl
i
ne;

Endmov
est
hecur
sort
otheendoft
hecur
rentl
i
ne;

→and←mov
ethecur
soronepl
acer
ightorl
eft
;

Ct
rl→and← ct
rlmov
eonewor
dri
ghtorl
eft
;

Ct
rl
-CandCt
rl
-Vhav
ethei
rusual
eff
ectofcopy
ingandpast
ing.

Ther
ear
esomenonst
andar
dedi
ti
ngshor
tcut
swhi
char
eusef
ul:

Escdel
etest
hewhol
eoft
hecur
rentl
i
ne;

Ct
rl
-Kdel
etesf
rom t
hecur
sort
otheendoft
hecur
rentl
i
ne.

MATLAB’
smat
hemat
ical
funct
ions

Al
loft
hest
andar
dmat
hemat
icalf
unct
ionsof
tencal
l
edt
he
el
ement
aryf
unct
ions t
haty
ouwi
l
lmeeti
nyourCal
cul
uscour
sesar
e
av
ail
abl
einMATLABusi
ngt
hei
rusualmat
hemat
icalnames.Manyot
her
f
unct
ionst
hespeci
alf
unct
ionsar
eal
soi
ncl
uded;
youwi
l
lmostl
i
kel
ycome
acr
oss some of t
hese i
nlat
er mat
hemat
ics and,mor
e especi
all
y,
engi
neer
ingcour
ses.Theel
ement
aryf
unct
ionsar
eli
stedi
nyourUser
’s
Gui
de.Thi
sli
sti
ngi
ncl
udessev
eralf
unct
ionswhi
chwi
l
lnotbef
ami
l
iart
o
y
ouy
et,andsev
eralt
hatweshal
lnotdealwi
thi
nthi
sbook.Thei
mpor
tant
f
unct
ionsf
orourpur
posesar
e:

abs(
x)Absol
utev
alue

sqr
t(x)Squar
eroot

si
n(x)Si
ne

cos(
x)Cosi
ne

t
an(
x)Tangent

l
og(
x)Nat
ural
logar
it
hm

exp(
x)Exponent
ial
funct
ion,
ex

at
an(
x)I
nver
set
angent
,orar
ctan

asi
n(x)I
nver
sesi
ne,
orar
csi
n

acos(
x)I
nver
secosi
ne,
orar
cos

cosh(
x)Hy
per
bol
i
ccosi
ne

si
nh(
x)Hy
per
bol
i
csi
ne

Vect
orsandMat
ri
ces

I
nMATLABt
hewor
dvect
orshoul
dreal
l
ybei
nter
pret
edsi
mpl
y
as“
li
stofnumber
s”.St
ri
ctl
yitcoul
dbeal
i
stofot
herobj
ect
sthannumber
s
but“
li
stofnumber
s”wi
l
lfi
tourneedsf
ornow.Thesecanbeusedt
o
r
epr
esentphy
sicalv
ect
orsbutar
emuchmor
ever
sat
il
ethant
hataswe
shal
lsee.Ther
ear
etwobasi
cki
ndsofMATLABv
ect
ors:RowandCol
umn
v
ect
ors.Ast
henamessuggest
,ar
owv
ect
orst
oresi
tsnumber
sinal
ong
“
hor
izont
all
i
st”suchas

1,
2,
3.4,
1.23,
-
10.
3.2.
1

whi
chi
sar
ow v
ect
orwi
th6component
s.A col
umnv
ect
orst
oresi
ts
number
sinav
ert
ical
li
stsuchas

3.4

1.23

-
10.3

2.1

whi
chi
sacol
umnv
ect
orwi
th(
thesame)6component
s.I
nmat
hemat
ical
not
ati
ont
hesear
ray
sar
eusual
l
yencl
osedi
nbr
acket
s[]

Amat
ri
xissi
mpl
yar
ect
angul
arar
rayofnumber
s(orot
herobj
ect
s).Thusa
mat
ri
xisar
ow ofcol
umnsi
nwhi
cheachent
ryi
sacol
umnv
ect
oroft
he
samel
engt
h,oracol
umnofr
owsi
nwhi
cheachent
ryi
sar
owoft
ehsame
l
engt
h.Theent
ri
esar
eindexedi
nasi
mil
arwayt
o4
vect
ors:
A(2,
3)r
efer
sto
t
heel
ementi
nRow2andCol
umn3.Thust
heel
ement
sofa3mat
ri
xar
e
l
abel
l
edi
nMATLABasf
oll
ows:
Mat
hemat
ical
l
ythi
ssamear
rayi
susual
l
ywr
it
tenas

Thesamemat
ri
xisar
owofcol
umns

oracol
umnofr
ows:

MATLABpr
ogr
am cont
rol

Thev
ari
ousway
sofcont
rol
l
ingt
hef
lowofapi
eceofMATLAB
code.Thesei
ncl
ude:

f
orl
oops— whi
chenabl
eust
ohav
eanoper
ati
onr
epeat
edaspeci
fi
ed
numberoft
imes.Thi
smayber
equi
redi
nsummi
ngt
ermsofaser
ies,or
speci
fyi
ngt
heel
ement
sofanonuni
for
mlyspacedv
ect
orsuchast
hef
ir
st
t
ermsofasequencedef
inedr
ecur
siv
ely
.

whi
l
eloops— whi
chal
l
ow t
henumberoft
imest
hel
oopoper
ati
oni
s
per
for
medt
obedet
ermi
nedbyt
her
esul
ts.Thi
sisof
tenusedi
nit
erat
ive
pr
ocessessuchasobt
aini
ngappr
oxi
mat
ionst
othesol
uti
onofanequat
ion.

I
fel
se—whi
chi
susedwhendi
ff
erentact
ionsar
etobet
akendependi
ngon
t
hev
alueofsomev
ari
abl
e.

MATLAB/
SIMULI
NKFORPOWERELECTRONI
C

Themodel
l
ingandsi
mul
ati
onofapowerel
ect
roni
cconv
ert
er
canbecar
ri
edoutusi
ngMATLAB/
SIMULI
NKsof
twar
e.

Thebl
ockswhi
char
eusedt
oachi
evet
hemodel
l
ingasf
oll
ow:

1.Repeat
ingSequence:t
hisbl
ockwi
l
lbeusedt
ogener
ateanumberof
pul
sesi
nter
msoft
ime.

2. Si
newav
e:i
s used t
o gener
ate a si
nusoi
dali
nputwi
th ampl
i
tude,
f
requencyandphase.

3.Swi
tchFunct
ion(
Thy
):i
susedt
oswi
tchbet
weent
hesi
newav
efor
m and
t
hef
ir
ingpul
sewhi
chi
sgener
atedf
rom t
her
epeat
ingsequence.

4.ORLogi
c:i
susedt
otur
nthet
hyr
ist
orof
fwhencur
rentr
eacheszer
o.

Thepowersemi
conduct
ordev
icescanbeoper
atedasswi
tchesbyappl
yi
ng
cont
rol si
gnal
s t
o t
he gat
e t
ermi
nal of t
hyr
ist
ors. The swi
tchi
ng
char
act
eri
sti
cisshowni
nDi
agr
am.Ther
iset
imeoft
heanodecur
rentofa
t
hyr
ist
ori
sdef
ineast
het
imet
akenf
ort
het
hyr
ist
orv
olt
aget
ofal
lfr
om t
he
90%t
othe10%l
evel
.Thesum oft
hedel
ayandr
iset
imesi
sthet
urnont
ime
oft
het
hyr
ist
or.Bot
hri
set
imeandt
hef
allt
imedet
ermi
net
hequal
i
tyofa
pul
sewav
efor
m.
CODE:
RESULT-SCREENSHOTS

CONCLUSI
ON

Weext
endourpr
evi
ouswor
konSRbasedondensel
ytr
ainedr
egr
essor
s
andspher
icalhashi
ngsear
ch[
14]
.Inouri
nit
ialwor
k,weachi
evehi
gh
qual
i
typer
for
manceandv
astspeed-
upscompar
edt
otheor
igi
nalwor
kof
[
13]andot
hercont
empor
arymet
hods.I
nthi
spaper
,wef
urt
heranal
yzet
he
f
eat
uresusedandt
hemet
ri
csi
nvol
veddur
ingt
her
egr
essi
onpr
ocess.The
cont
ri
but
ionsoft
hepaperar
ethr
eef
old:(
1)Wedet
ectand st
udyt
he
i
mpor
tanceofant
ipodali
nvar
iancei
noursear
chspace,pr
oposi
ngt
heuse
oft
he absol
utev
alue oft
he cosi
ne si
mil
ari
tyf
orexhaust
ive sear
ch
whenev
ert
imei
snotaconst
rai
n(i
.e.dur
ingt
rai
ning)
,(2)wepr
oposea
nov
elt
ransf
orm whi
chboost
stheant
ipodali
nvar
iancei
ntheEucl
i
dean
space,whi
chweembedi
ntheSpher
icalHashi
ngal
gor
it
hm ofHeoetal
.
[
24]
,thusobt
aini
nganAnt
ipodal
l
yInv
ari
antSpher
icalHashi
ng,and(
3)we
pr
esentaf
eat
uret
ransf
ormt
hatper
for
msbet
tert
hankst
oabet
tercoar
se
appr
oxi
mat
ionoft
heupscal
edgr
adi
ent
sobt
ainedbyI
BP.Ther
egr
essor
s
obt
ainedwi
thanant
ipodal
l
yinv
ari
antmet
ri
cshowaneatgai
ninPSNRov
er
t
hoseobt
ainedwi
thEucl
i
deandi
stanceand,
fur
ther
mor
e,AI
SHi
sopt
imal
l
y
adapt
edt
othesear
chast
hel
ossi
nqual
i
tycompar
edt
oanexhaust
ive
sear
chi
smi
nimal

REFERENCES

[
1]R.Tsai
andT.Huang,
“Mul
ti
plef
ramei
mager
est
orat
ionandr
egi
str
ati
on,
”
i
nPr
oc.Adv
ancesi
nComput
erVi
sionandI
magePr
ocessi
ng,
vol
.1,
1984.

[
2]M.I
raniand S.Pel
eg,“
Impr
ovi
ng r
esol
uti
onbyi
mager
egi
str
ati
on,
”
CVGI
P:Gr
aphi
cal
Model
sandI
magePr
ocessi
ng,
vol
.53,
no.3,
1991.

[
3]S.BakerandT.Kanade,“
Limi
tsonsuper
-r
esol
uti
onandhow t
obr
eak
t
hem,
”IEEETr
ans.onPat
ter
nAnal
ysi
sandMachi
neI
ntel
l
igence,
vol
.24,
no.
9,
pp.1167–1183,
2002.

[
4]Z.Li
nandH.
-Y.Shum,“
Fundament
all
i
mit
sofr
econst
ruct
ion-
based
super
resol
uti
onal
gor
it
hmsunderl
ocalt
ransl
ati
on,
”IEEETr
ans.onPat
ter
n
Anal
ysi
sandMachi
neI
ntel
l
igence,
vol
.26,
no.1,
2004.

[
5]W.Fr
eeman,
T.Jones,
andE.Paszt
or,
“Exampl
e-basedsuper
-r
esol
uti
on,
”
I
EEETr
ans.Comput
erGr
aphi
csandAppl
i
cat
ions,
vol
.22,
no.2,
2002.

[
6]D.Gl
asner
,S.Bagon,
andM.I
rani
,“Super
-r
esol
uti
onf
rom asi
ngl
eimage,
”
i
nPr
oc.I
EEEI
nter
nat
ional
Conf
.onComput
erVi
sion,
2009.

[
7]G.Fr
eedmanand R.Fat
tal
,“I
mageand v
ideo upscal
i
ng f
rom l
ocal
sel
fexampl
es,
”ACM Tr
ans.onGr
aphi
cs,
vol
.30,
no.2,
pp.12:
1–12:
11,
2011.

[
8]J.Yang,Z.Li
n,andS.Cohen,“
Fasti
magesuper
-r
esol
uti
onbasedoni
n-
pl
aceexampl
eregr
essi
on,
”inPr
oc.I
EEEConf
.onComput
erVi
sionand
Pat
ter
nRecogni
ti
on,
2013.

[
9]J.Yang,
J.Wr
ight
,H.T.
S.,
andY.Ma,
“Imagesuper
-r
esol
uti
onv
iaspar
se
r
epr
esent
ati
on,
”IEEE Tr
ans.on I
magePr
ocessi
ng,v
ol.19,no.11,pp.
2861–2873,
2010.

[
10]R.Zey
de,M.El
ad,andM.Pr
ott
er,“
Onsi
ngl
eimagescal
e-upusi
ng
spar
se-
repr
esent
ati
ons,
”in Pr
oc.I
nter
nat
ionalConf
.on Cur
ves and
Sur
faces,
2012.

[
11]X.Gao,K.Zhang,D.Tao,andX.Li
,“Joi
ntl
ear
ningf
orsi
ngl
eimage
super
-r
esol
uti
on v
ia a coupl
ed const
rai
nt,
”IEEE Tr
ans. on I
mage
Pr
ocessi
ng,
vol
.21,
no.2,
pp.469–480,
2012.

[
12]H.Chang,D.
-Y.Yeung,and Y.Xi
ong,“
Super
-r
esol
uti
on t
hrough
nei
ghborembeddi
ng,
”Pr
oc.I
EEEConf
.onComput
erVi
sionandPat
ter
n
Recogni
ti
on,
2004.

[
13]R.Ti
mof
te,V.D.Smet
,and L.V.Gool
,“Anchor
ed nei
ghbor
hood
r
egr
essi
on f
or f
ast exampl
e-based super
-r
esol
uti
on,
”in Pr
oc. I
EEE
I
nter
nat
ional
Conf
.onComput
erVi
sion,
2013.

[
14]E.Per
ez-
Pel
l
iter
o,J.Sal
vador
,I.Tor
res,
J.Rui
z-Hi
dal
go,
andB.Rosen-´
hahn,“
Fastsuper
-r
esol
uti
onv
iadensel
ocalt
rai
ningandi
nver
ser
egr
essor
sear
ch,
”Pr
oc.Asi
anConf
.onComput
erVi
sion,Lect
ureNot
esi
nComput
er
Sci
ence,
2014.

[
15]K.I
.Ki
m andY.Kwon,“
Singl
e-i
magesuper
-r
esol
uti
onusi
ngspar
se
r
egr
essi
on and nat
urali
mage pr
ior
,”I
EEE Tr
ans.Pat
ter
n Anal
ysi
sand
Machi
neI
ntel
l
igence,
vol
.32,
no.6,
2010.

[
16]M.Tappen,B.Russel
,and W.Fr
eeman,“
Expl
oit
ing t
he spar
se
der
ivat
ivepr
iorf
orsuper
-r
esol
uti
onandi
magedemosai
cing,
”2003.

[
17]M.Ahar
on,M.El
ad,and A.Br
uckst
ein,“
K-SVD:An al
gor
it
hm f
or
desi
gni
ngov
ercompl
etedi
cti
onar
iesf
orspar
ser
epr
esent
ati
on,
”IEEETr
ans.
onSi
gnal
Processi
ng,
vol
.54,
no.11,
2006.

[
18]J.Sal
vadorandE.Per
ez-
Pel
l
iter
o,“
Nai
veBay
esSuper
-Resol
uti
on´
For
est
,”i
nPr
oc.I
EEEI
nt.Conf
.onComput
erVi
sion,
2015,
pp.325–333.

[
19]S.Schul
ter
,C.Lei
stner
,andH.Bi
schof
,“Fastandaccur
atei
mage
upscal
i
ngwi
thsuper
-r
esol
uti
onf
orest
s,”i
nIEEEConf
,onComput
erVi
sion
andPat
ter
nRecogni
ti
on,
June2015.

[
20]K.Zhang,D.Tao,X.Gao,X.Li
,andZ.Xi
ong,“
Lear
ningmul
ti
plel
i
near
mappi
ngsf
oref
fi
cientsi
ngl
eimagesuper
-r
esol
uti
on,
”IEEETr
ans.onI
mage
Pr
ocessi
ng,
vol
.24,
no.3,
pp.846–861,
2015.

[
21]C.Dong,C.Loy
,K.He,andX.Tang,“
Lear
ningadeepconv
olut
ional
net
wor
kfori
magesuper
-r
esol
uti
on,
”inPr
oc.Eur
opeanConf
.onComput
er
Vi
sion,
2014.

[
22]G.Pey
re,
“Mani
fol
dmodel
sforsi
gnal
sandi
mages,
”´Comput
erVi
sion
andI
mageUnder
standi
ng,
vol
.113,
no.2,
2009.

[
23]R.Ti
mof
te,V.D.Smet
,and L.V.Gool
,“A+:Adj
ust
ed anchor
ed
nei
ghbor
hoodr
egr
essi
onf
orf
astsuper
-r
esol
uti
on,
”inPr
oc.Asi
anConf
.on
Comput
erVi
sion,
Lect
ureNot
esi
nComput
erSci
ence,
2014.

[
24]J.
-P.Heo,
Y.Lee,
J.He,
S.-
F.Chang,
andS.
-E.Yoon,
“Spher
ical
hashi
ng,
”
i
nPr
oc.I
EEEConf
.onComput
erVi
sionandPat
ter
nRecogni
ti
on,
2012.

[
25]P.I
ndy
kandR.Mot
wani
,“Appr
oxi
mat
enear
estnei
ghbor
s:Towar
ds
r
emov
ingt
hecur
seofdi
mensi
onal
i
ty,
”inPr
oc.oft
heThi
rt
iet
hAnnualACM
Sy
mposi
um onTheor
yofComput
ing,
ser
.STOC’
98,
1998.

[
26]Y.Wei
ss,
A.Tor
ral
ba,
andR.Fer
gus,
“Spect
ral
hashi
ng,
”2008.

[
27]J.Wang,S.Kumar
,andS.
-F.Chang,“
Semi
-super
visedhashi
ngf
or
scal
abl
eimager
etr
iev
al,
”2010.

[
28] K. He and J. Sun, “
Comput
ing near
est
-nei
ghbor f
iel
ds v
ia
pr
opagat
ionassi
stedkd-
tr
ees,
”inPr
oc.I
EEEConf
.onComput
erVi
sionand
Pat
ter
nRecogni
ti
on,
2012.

[
29]J.Mat
ousek,
Usi
ngt
heBor
suk-
Ulam Theor
em:
Lect
uresonTopol
ogi
cal
Met
hodsi
nCombi
nat
ori
csandGeomet
ry.Spr
ingerPubl
i
shi
ngCompany
,
I
ncor
por
ated,
2007.

[
30]C.Dong,
C.Loy
,K.He,
andX.Tang,
“Imagesuper
-r
esol
uti
onusi
ngdeep
conv
olut
ionalnet
wor
ks,
”IEEE Tr
ans.on Pat
ter
n Anal
ysi
sand Machi
ne
I
ntel
l
igence,
vol
.PP,
no.99,
2015.