EricYuan'sBlog

PerstandoetPraestando

Home

AboutMe

ContinuouslyAdaptiveMeanSHIFT
ByERIC |Published:SEPTEMBER18,2013

IntheMotionCapturecourselecturethisweek,teacherstalkedaboutcamShift
algorithm,Iheardaboutthisalgorithmsinceaboutthreeorfouryearsbefore,
butInevertriedit,andduringthatclassIthoughtImustdoitthistime.

Search
Tosearch,typeandhitenter

TagCloud

Algorithmalphabeta

activecontour

ArmadilloBilateralFilterBlur

pruning

CamShiftisatrackingalgorithm,whichisbasedonMeanShiftalgorithm,what
camShiftdoisnothingbutdomeanShiftineverysingleframeofavideo,and
recordtheresultswegotbymeanShift.
CamShiftalgorithmincludesthesethreeparts:
1.BackProjection
2.MeanShift
3.Track

C++camShiftClusterCNNComputer
VisionConvolutionalNeuralNetworks

DeepLearningdeepnetwork
DynamicProgrammingGradient

DescentimageblendingImageProcessing

LeetCode

imagepyramid

MachineLearning
Matlab
MNISTneural
networksNLPOctaveOpenCV
meanShift

PanoramaPCAPoissonblendingPooling
pyramidblendingRANSACRecurrentNeural

andIwillsimplyexplaineachofthesestepsinthisblog.

NetworksRNNsseamcarvingSIFTsnake

SoftmaxSparseAutoencoder
1.BackProjection.
Backprojectionisamethodwhichusingthehistogramofanimagetoshowup
theprobabilitiesofcolorsmayappearineachpixel.Letsseehowtogettheback
projectionofanimageusingOpenCV.
1
2
3
4
5
6
7

cvtColor(image,hsv,CV_BGR2HSV);
intch[]={0,0};
hue.create(hsv.size(),hsv.depth());
mixChannels(&hsv,1,&hue,1,ch,1);
calcHist(&hue,1,0,Mat(),hist,1,&hsize,&phranges);
normalize(hist,hist,0,255,CV_MINMAX);
calcBackProject(&hue,1,0,hist,backproj,&phranges,1,
true);

FirstwetransformthepicturespacetoHSVspace(oranyspacewhichincludea
Hchannelthatrepresentthehueofeachpixel,ofcourse,valueofhueisbetween
0to180,youcanseemoreinfoinwiki.)Secondly,wesplittheHchannelout,as
asinglegrayscaleimage,andgetitshistogram,andnormalizeit.Thirdly,use
calcBackProject()functiontocalculatethebackprojectionoftheimage.

UFLDLUnsupervisedLearning

track

ZCA

Pages
AboutMe

Categories
Algorithm
DynamicProgramming
Graphics
LeetCode
MachineLearning
Maths
NLP
OpenCV
Somethingelse
Twaddle
Uncategorized
Vision

Letmeuseanexampletoexplainhowwegetthebackprojection.
Archives
June2015
April2015
March2015
October2014
September2014
July2014

June2014
May2014
April2014
March2014
February2014
January2014
November2013
October2013
September2013

Friends
JekyCui'sblog
QiCai'sBlog

Ifthisisourinputimage,wecanseeitisacolorfulmosaicpicture.
Aswetalkedabove,transformthepictureintoHSVspaceandhereisthehue
channel.

Thisisitshistogramlookslike.

calcBackProject()functionactuallycalculatetheweightofeachcolorinthe
wholepictureusinghistogram,andchangethevalueofeachpixeltotheweight
ofitscolorinwholepicture.Forinstance,ifonepixelscoloris,sayyellow,and
thecoloryellowsweightinthispictureis20%,thatis,thereare20%ofpixels
colorinthewholepictureisthiskindofyellow,wechangethispixelsvaluefrom
yellowto0.2(or0.2*255ifusinginteger),bydoingthismethodtoallpixels,we
getthebackprojectionpicture.

2.MeanShift
WhatismeanShift?MeanShiftisnothingbutanalgorithmwhichfindingmodes
inasetofdatasamplesrepresentinganunderlyingprobabilitydensityfunction
(PDF)inR^N.Itisanonparametricclusteringtechniquewhichdoesnotrequire
priorknowledgeofthenumberofclusters,anddoesnotconstraintheshapeof
theclusters.Iwillnotshowyouthoseannoyingformulae,Illshowyouhowit
worksbygraphandwords.
Imagineweareinaspace,andthedimensionofthisspaceisd,(ofcourse,d
maybebiggerthan2)andtherearealotofpointsinthisspace,whatweare
goingtodoisclusteringthesepoints.Wenowmakeaspherewhichcenterisany
ofthepoints,andradiusis,say,h.Becauseweareinahighdimensionalspace,
sowhatwejustmadeisahighdimensionalsphere.Bynow,everysinglepoint
insidethisspacecanbeseenasavector(directedline),andthesum
(normalized)ofthesevectorsiswhatwecalledmeanshift.Bythismeanshift
vector,wecangetthecurrentmasscenter.

Inc++,thecurrentmasscentercanbecalculatedlikethis:
01
02
03
04
05
06
07
08
09
10
11
12

M00=0.0;
M10=0.0;
M01=0.0;
for(inti=0;i<probmap.height;i++){
for(intj=0;j<probmap.width;j++){
M00+=probmap.at(i,j);
M10+=i*probmap.at(i,j);
M01+=j*probmap.at(i,j);
}
}
Center_x=M01/M00;
Center_y=M10/M00;

AboveisthecoreofmeanShiftalgorithm,sothewholealgorithmis:
a.Initializethesphere,includingthecenterandradius.

b.Calculatethecurrentmasscenter.
c.Movethespherescentertomasscenter.

d.repeatstepbandc,untilconverge,thatis,currentmasscenteraftercalculate,
isthesamepointwithcenterofsphere.

InOpenCV,meanShiftfunctionissomethinglikethis:
1 CV_IMPLint
2 cvMeanShift(constvoid*imgProb,CvRectwindowIn,
3 CvTermCriteriacriteria,CvConnectedComp*
comp);
Inwhich,imgProbis2Dobjectprobabilitydistribution,whichistheresultofback
projectionabovewindowInisCvRectofwindowinitialsizenumItersmeans
thatifthealgorithmiteratesthismanytimes,stop,thatpreventsinsomecases
thefunctionjustrepeatsandrepeatswindowOutisthelocation,heightand
widthofconvergedwindow.
3.Track
Thelaststepistracking,ifwehaveavideo,orframescapturedbyourweb
camera,whatweneedtodoisjustusemeanShiftalgorithmtoeverysingleframe,
andtheinitialwindowofeachframeisjusttheoutputwindowoftheprior
frame.
ByusingOpenCVcamshift()function,wecangetaRotatedRect,whichis
definedinOpenCVlike:
01
02
03
04
05
06

classRotatedRect
{
public:
//constructors
RotatedRect();
RotatedRect(constPoint2f&_center,constSize2f&_siz
e,float_angle);
07 RotatedRect(constCvBox2D&box);
08 //returnsminimaluprightrectanglethatcontainsth
erotatedrectangle

09
10
11
12
13
14
15
16
17
18

RectboundingRect()const;
//backwardconversiontoCvBox2D
operatorCvBox2D()const;
//masscenteroftherectangle
Point2fcenter;
//size
Size2fsize;
//rotationangleindegrees
floatangle;
};

Interesting,exceptcenter,size,wecanalsogetanangleoftherectangle,which
meanswecantracktheorientationofourtarget,thisisaveryusefulfeature.
Testing
First,ItriedtotrackmyKendamaballs,itworkswell.

Moreover,see,Imdrawingwithmyface!

ThisentrywaspostedinAlgorithm,OpenCVandtaggedcamShift,meanShift,OpenCV.
Bookmarkthepermalink.Postacommentorleaveatrackback:TrackbackURL.

CanwemakeaTransparentscreen?

SeamCarving

16Comments

sara
PostedApril20,2014at11:10am|Permalink

hiiwanttoaskaquestioncaniusethistofindgoalpostinsoccervideo???
Reply

Eric
PostedApril20,2014at1:46pm|Permalink

HeySara,
Idontthinkitsagoodwaytofindgoalpostusingmeanshift.Itsnicefor
trackingobjectwhichyoualreadyrecognized,butcanhardlyfind
somethingnew.Foryoursituation,maybeyoushouldextractfeaturesof
goalpost,assoonasyoufindit,usemeanshiftfortrackingit.

Reply

LakshayGarg

PostedNovember5,2014at5:06pm|Permalink

TryreadingabouttheHoughLinetransform
Reply

hassan
PostedDecember18,2014at6:40am|Permalink

youmustsearchofmorphologicaloperationstofindtheskeleton(basic
shape)oftheobject.Inyourcaseitisagoalpostsoyoushallhavea
skeletonofthatandperformmorphologicaloperationsforthat.Youhave
toworkalittle.
Regards
Reply

Faraz
PostedMay11,2014at1:12pm|Permalink

Iamusingopencvcodeformeanshiftcalculationbutiftheobjectisnotinthescreen
theprogramcrashes.
howtosolvethis?
Reply

George
PostedAugust26,2014at3:59pm|Permalink

Inyourcode,line6includesI(i,j)whichisneverdefined.Bradskicallsitthepixel
(probability)value.Howdoyoucalculateit?
Thankyou.
Reply

Eric
PostedAugust27,2014at11:27pm|Permalink

Ieditedthatcodeformoreclear,yesitisactuallyaprobabilitymap,we
cangetitbydoingBackProjection.Check
OpenCV/samples/cpp/camshiftdemo.cpp
Reply

mohammad
PostedDecember15,2014at2:02pm|Permalink

dohaveanyideaaboutspeedtrackingofobjectineveryframe.
thankyou.
Reply

Eric
PostedDecember15,2014at4:48pm|Permalink

Hi,forspeedcalculating,weshouldknowtheworldcoordination,means
thecamerashouldbecalibrated,thenwecanfigureoutthespeedof
objectsbyprojectthedistanceinimageintorealworldcoordination(we

shouldalsoknowthetimebetweenthetwoimageshavebeentaken,say,
40ms).MaybeLKmethodcandothedistancecalculationpartwell.
Reply

Shiloh
PostedJanuary18,2015at9:09am|Permalink

Hi,isitpossibletousecamshiftformultipleobjects?Canyouexplainalittlebitwhat
shouldIdotoachievethat?
Reply

Peerapong
PostedFebruary10,2015at10:16pm|Permalink

Iwanttoaskyousomequestion.Incaseofwehaveonlygrayscaleimage,This
algorithmcanbeprocessedorcant?thankyou
Reply

Eric
PostedMarch10,2015at2:37pm|Permalink

HiPeerapong,
Ithinkso,becausefor3channelimages,wejustusetheHchannel(hue),
becauseforRGBrepresentation,thecorrelationamongthethreechannels
ishighhowever,byusinggrayscaleimage,thecorrelationthingisnot
necessarilytobeconcernedanymore,andwecandirectlygetthe
histogramoftheimage,anddobackprojection.
Reply

Siraj
PostedFebruary16,2015at12:59am|Permalink

incaseofcameshifalgorithm,youhavecolculatedonlythebackprojectionbuthow
tousetheCamShiftalgorithm.ihavecapturevediofrommycameandwanttotracthe
faceinvedio.howicandothis..???
Thanks
Reply

Eric
PostedMarch10,2015at2:40pm|Permalink

HeySiraj,
IfyouareusingOpenCV,youmaywanttocheck
opencv/samples/cpp/camshiftdemo.cpp
Also,thereispythonversionofthecodeinsidepythonfolders
Reply

Bruce
PostedMarch18,2015at5:00am|Permalink

HiEric,
HaveyouconsideredtheperformancedifferencebetweentheMeanShiftandthe
Camshift?
Frommyobservation,bothoftheirefficiencyisnotacceptable.Itriedthemonmyi7,
8GRAMnotebook,thatsveryslowfortracking.Anyway,Iamconsideringto
implementitonGPU.Thanks~
Reply

eren
PostedMay11,2015at3:56am|Permalink

HiEric,
Doyouhavetheimplementationofthetutorialcodeshared?Iwouldliketotryitonmy
own,ifitworksornot.
Thanks.
Reply

PostaComment
Youremailisneverpublishednorshared.Requiredfieldsaremarked*
Name*

Email*

Website

Comment

YoumayusetheseHTMLtagsandattributes<ahref=""title=""><abbrtitle=""><acronym
title=""><b><blockquotecite=""><cite><code><deldatetime=""><em><i><qcite="">
<s><strike><strong>

PostComment

PoweredbyWordPress.BuiltontheThematicThemeFramework.