EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

UNIT IV
3D RECONSTRUCTION
ShapefromX-Activerangefinding-Surfacerepresentations-
Point-basedrepresentationsVolumetricrepresentations-Model-based reconstruction
- Recovering texture maps and albedosos.

1. Shape from X:

"Shape from X" refers to a category of computer vision and computer graphics
techniquesthataimtorecoverthethree-dimensional(3D)shapeorstructureofobjects or scenes from
different types of information or cues, represented by the variable "X".
The"X"canstandforvarioussourcesormodalitiesthatprovideinformationaboutthe scene. Some
common examples include:

ShapefromShading(SfS):Thistechniqueinvolvesrecovering3Dshape
informationfromvariationsinbrightnessandshadingin2Dimages.Itassumes that the shading
patterns in an image are influenced by the underlying 3D
geometry.

ShapefromStereo(SfS):Thismethodutilizesthedisparityorparallax
informationbetweentwoormoreimagesofascenetakenfromdifferent

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

viewpoints.Bytriangulatingcorrespondingpoints,the3Dstructureofthescene can be
reconstructed.

ShapefromMotion(SfM):SfMaimstorecoverthe3Dstructureofasceneby
analyzingthemotionofobjectsorthecameraitself.Thisisoftenachievedby tracking
features across multiple frames of a video sequence.

ShapefromTexture(SfT):SfTreliesonthevariationsintexturepatternsacross surfaces to infer

their 3D structure. The assumption is that different surface orientations result in distinct
texture variations.

ShapefromFocus(SfF):InSfF,thedepthinformationisinferredfromthe
variationinimagesharpnessorfocus.Byanalyzingthefocusinformationat different depths,
the 3D shape can be estimated.
ShapefromDefocus(SfD):SimilartoSfF,SfDleveragestheeffectsofdefocusing
inimagestoestimatedepthinformation.Objectsatdifferentdistancesfromthe camera will
exhibit different degrees of blur.

Shape from Light (SfL): This technique involves using information about the
lightingconditionsinascenetoinfer3Dshape.Theinteractionbetweenlightand surfaces
provides cues about the geometry.

Theseapproachesdemonstratethediversityofmethodsusedincomputervisionto
recover3Dshapeinformationfromdifferenttypesofvisualcues.Thechoiceofthe specific "X"
(shading, stereo, motion, etc.) depends on the available data and the
characteristicsofthescenebeingreconstructed.

2. Activerangefinding:
Active range finding is a technique used in computer vision and remote sensing to
determinethedistancetoobjectsinasceneactively.Unlikepassivemethodsthatrely
onexistingambientillumination,activerangefindinginvolvesemittingasignalorprobe
towardsthetargetandmeasuringthetimeittakesforthesignaltoreturn.Thisprocess
isoftenbasedontheprinciplesoftime-of-flightorphase-shiftmeasurement.Thegoal is to obtain
accurate depth or distance information about the surfaces in the scene.

Hereareafewcommonmethodsofactiverangefinding:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

LaserRangeFinding:Thismethodinvolvesemittinglaserbeamstowardsthe
targetandmeasuringthetimeittakesforthelaserpulsestotraveltotheobject and back. By
knowing the speed of light, the distance to the object can be
calculated.

StructuredLight:Instructuredlightrangefinding,aknownlightpattern,oftena grid or a set of

stripes, is projected onto the scene. Cameras capture the
deformedpatternonsurfaces,andthedistortionhelpscalculatedepth information based on the
known geometry of the projected pattern.

Time-of-Flight(ToF)Cameras:ToFcamerasemitmodulatedlightsignals(often infrared) and

measure the time it takes for the light to travel to the object and
return.Thephaseshiftofthemodulatedsignalisusedtodeterminethedistance to the object.

UltrasoundRangeFinding:Ultrasoundwavesareemitted,andthetimeittakes
forthewavestobouncebacktoasensorismeasured.Thismethodiscommonly used in
environments where optical methods may be less effective, such as in low-light
conditions.

Active range finding has various applications, including robotics, 3D scanning,

autonomousvehicles,augmentedreality,andindustrialinspection.Theabilitytoactively
measuredistancesisvaluableinscenarioswhereambientlightingconditionsmayvary or when accurate
depth information is essential for understanding the environment.

3. Surfacerepresentations:

Surfacerepresentationsincomputervisionrefertothewaysinwhichthegeometryor shape of surfaces

in a three-dimensional (3D) scene is represented. These
representationsarecrucialfortaskssuchas3Dreconstruction,computergraphics,and virtual reality.
Different methods exist for representing surfaces, and the choice often
dependsontheapplication'srequirementsandthecharacteristicsofthedata.Hereare some common
surface representations:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

PolygonalMeshes:
● Description:Meshesarecomposedofvertices,edges,andfacesthat
definethesurfacegeometry.Triangularandquadrilateralmeshesare most
common.
● Application:Widelyusedincomputergraphics,gaming,and3Dmodeling. Point
Clouds:
● Description:Asetof3Dpointsinspace,eachrepresentingasampleon the surface of
an object.
● Application:Generatedby3Dscanners,LiDAR,ordepthsensors;usedin
applications like autonomous vehicles, robotics, and environmental
mapping.
Implicit Surfaces:
● Description:Representsurfacesasthezerolevelsetofascalarfunction. Points inside
the surface have negative values, points outside have positive values, and points
on the surface have values close to zero.
● Application:Usedinphysics-basedsimulations,medicalimaging,and shape
modeling.
NURBS(Non-UniformRationalB-Splines):
● Description:Mathematicalrepresentationsusingcontrolpointsandbasis functions to
define smooth surfaces.
● Application:Commonlyusedincomputer-aideddesign(CAD),automotive design,
and industrial design.
VoxelGrids:
● Description:3Dgridswhereeachvoxel(volumetricpixel)representsa small
volume in space, and the surface is defined by the boundary
betweenoccupiedandunoccupiedvoxels.
● Application:Usedinmedicalimaging,volumetricdataanalysis,and
computational fluid dynamics.
LevelSetMethods:
● Description:Representsurfacesasthezerolevelsetofa
higher-dimensionalfunction.Theevolutionofthisfunctionovertime captures the
motion of the surface.
● Application:Usedinimagesegmentation,shapeoptimization,andfluid dynamics
simulations.
Octrees:
● Description:Hierarchicaltreestructuresthatrecursivelydividespaceinto
octants.Eachleafnodecontainsinformationaboutthegeometrywithin that region.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Application:Usedinreal-timerendering,collisiondetection,andefficient storage of
3D data.

Thechoiceofsurfacerepresentationdependsonfactorssuchasthenatureofthe scene, the desired level

of detail, computational efficiency, and the specific
requirementsoftheapplication.

4. Point-basedrepresentations:
Point-basedrepresentationsincomputervisionandcomputergraphicsreferto methods that
represent surfaces or objects using a set of individual points in
three-dimensional(3D)space.Insteadofexplicitlydefiningtheconnectivitybetween
pointsasinpolygonalmeshes,point-basedrepresentationsfocusonthespatial
distributionofpointstodescribethesurfacegeometry.Herearesomecommon point-based
representations:

PointClouds:
● Description:Acollectionof3Dpointsinspace,eachrepresentingasample on the
surface of an object or a scene.
● Application:Pointcloudsaregeneratedby3Dscanners,LiDAR,depth sensors, or
photogrammetry. They find applications in robotics, autonomous vehicles,
environmental mapping, and 3D modeling.
DensePointClouds:
● Description:Similartopointcloudsbutwithahighdensityofpoints, providing
more detailed surface information.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Application:Usedinapplicationsrequiringdetailed3Dreconstructions, such as
cultural heritage preservation, archaeological studies, and
industrialinspections.
SparsePointSets:
● Description: Representations where only a subset of points is used to
describethesurface,resultinginasparserdatasetcomparedtoadense point cloud.
● Application:Sparsepointsetsareusefulinscenarioswhere
computationalefficiencyiscrucial,suchasreal-timeapplicationsand large-scale
environments.
PointSplats:
● Description:Representeachpointasadiscorasplatin3Dspace.The size and
orientation of the splats can convey additional information.
● Application:Commonlyusedinpoint-basedrenderingandvisualizationto represent
dense point clouds efficiently.
PointFeatures:
● Description:Representsurfacesusingdistinctivepointsorkeypoints,
eachassociatedwithlocalfeaturessuchasnormals,color,ortexture information.
● Application:Widelyusedinfeature-basedregistration,objectrecognition, and 3D
reconstruction.
PointSetSurfaces:
● Description:Representsurfacesasasetofunorganizedpointswithout connectivity
information. Surface properties can be interpolated from neighboring points.
● Application:Usedinsurfacereconstructionfrompointcloudsand point-
based rendering.
RadialBasisFunction(RBF)Representations:
● Description:Useradialbasisfunctionstointerpolatesurfaceproperties between
points. These functions define a smooth surface that passes through the given
points.
● Application:Commonlyusedinshapemodeling,surfacereconstruction, and
computer-aided design.

Point-basedrepresentationsareparticularlyusefulwhendealingwithunstructuredor
irregularlysampleddata.Theyprovideflexibilityinrepresentingsurfaceswithvarying
levelsofdetailandarewell-suitedforcapturingcomplexandintricatestructures.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

5. Volumetricrepresentations:

Volumetricrepresentationsincomputervisionandcomputergraphicsaremethods
usedtodescribeandmodelthree-dimensional(3D)spaceinavolumetricmanner.
Unlikesurfacerepresentations,whichfocusondefiningthesurfacegeometryexplicitly, volumetric
representations capture information about the entire volume, including the
interiorofobjects.Herearesomecommonvolumetricrepresentations:

VoxelGrids:
● Description:Aregulargridofsmallvolumeelements,calledvoxels,where each voxel
represents a small unit of 3D space.
● Application:Usedinmedicalimaging,computer-aideddesign(CAD),
computationalfluiddynamics,androbotics.Voxelgridsareeffectivefor representing
both the exterior and interior of objects.
Octrees:
● Description:Ahierarchicaldatastructurethatrecursivelydivides3Dspace into octants.
Each leaf node in the octree contains information about the occupied or
unoccupied status of the corresponding volume.
● Application:Octreesareemployedforefficientstorageandrepresentation of
volumetric data, particularly in real-time rendering, collision detection,
andadaptiveresolution.
Signed Distance Fields (SDF):
● Description:Representthedistancefromeachpointinspacetothe
nearestsurfaceofanobject,withpositivevaluesinsidetheobjectand negative values
outside.
● Application: Used in shape modeling, surface reconstruction, and
physics-basedsimulations.SDFsprovideacompactrepresentationof geometry and are
often used in conjunction with implicit surfaces.
3DTextureMaps:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Description:Extendtheconceptof2Dtexturemappingto3Dspace,
associatingcolororotherpropertieswithvoxelsinavolumetricgrid.
● Application:Employed in computer graphics, simulations, and
visualizationtorepresentcomplexvolumetricdetailssuchassmoke, clouds, or other
phenomena.
PointCloudswithOccupancyInformation:
● Description:Combinetheideaofpointcloudswithadditionalinformation about the
occupancy of each point in space.
● Application:Usefulinscenarioswherecapturingboththesurfaceand interior
details of objects is necessary, such as in robotics and 3D
reconstruction.
TensorFields:
● Description:Representthelocalstructureofavolumetricregionusing tensors.
Tensor fields capture directional information, making them suitable for
anisotropic materials and shapes.
● Application:Commonlyusedinmaterialsscience,biomechanics,and
simulations where capturing anisotropic properties is important.
ShellMaps:
● Description:Representthesurfacesofobjectsasacollectionofshellsor layers, each
encapsulating the object's geometry.
● Application:Usedincomputergraphicsandsimulationtoefficiently
representcomplexobjectsandenabledynamiclevel-of-detailrendering.

Volumetricrepresentationsarevaluableinvariousapplicationswhereacomprehensive understanding
of the 3D space is required, and they offer flexibility in capturing both
surfaceandinteriordetailsofobjects.Thechoiceofrepresentationoftendependson
thespecificrequirementsofthetaskathandandthecharacteristicsofthedatabeing modeled.

6. Model-basedreconstruction:
Model-basedreconstructionincomputervisionreferstoacategoryoftechniquesthat involve
creating a 3D model of a scene or object based on predefined models or
templates.Thesemethodsleveragepriorknowledgeaboutthegeometry,appearance, or structure of
the objects being reconstructed. Model-based reconstruction is often
usedinscenarioswhereaknownmodelcanbefittedtotheobserveddata,providinga structured and
systematic approach to understanding the scene. Here are some key aspects and applications of
model-based reconstruction:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

PriorModelRepresentation:
● Description:Inmodel-basedreconstruction,amathematical
representationorageometricmodeloftheobjectorsceneisassumedor known in
advance.
● Application: Commonly used in computer-aided design (CAD), medical
imaging,andindustrialinspection,whereknownshapesorstructurescan be explicitly
represented.
ModelFitting:
● Description:Thereconstructionprocessinvolvesadjustingtheparameters
ofthemodeltobestfittheobserveddata,typicallyobtainedfromimages or sensor
measurements.
● Application: Used in applications such as object recognition, pose
estimation,and3Dreconstructionbyaligningthemodelwiththeobserved features.
GeometricConstraints:
● Description:Constraintsonthegeometryofthescene,suchasthe
relationshipsbetweendifferentcomponentsortheexpectedshape
characteristics, are incorporated into the reconstruction process.
● Application:Appliedinrobotics,augmentedreality,andcomputervision tasks
where geometric relationships play a crucial role.
DeformableModels:
● Description:Modelsthatcanadaptanddeformtofittheobserveddata, allowing for
more flexible and realistic representations.
● Application:Commonlyusedinmedicalimagingfororgansegmentation and shape
analysis, as well as in computer graphics for character
animation.
StereoVisionwithModelConstraints:
● Description:Stereovisiontechniquesthatincorporateknownmodelsto improve
depth estimation and 3D reconstruction.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Application:Usedinstereomatchingalgorithmsand3Dreconstruction pipelines to
enhance accuracy by considering geometric priors.
ParametricSurfaces:
● Description: Representing surfaces using parametric equations or
functions,allowingforefficientadjustmentofparametersduringthe
reconstruction process.
● Application:Appliedincomputergraphics,virtualreality,andindustrial design
where surfaces can be described mathematically.
Multi-ViewReconstructionwithKnownModels:
● Description:Leveragingmultipleviewsorimagesofascenetoreconstruct a 3D model
while incorporating information from known models.
● Application: Common in photogrammetry and structure-from-motion
applications where multiple perspectives contribute to accurate 3D
reconstruction.

Model-basedreconstructionisvaluablewhenthereispriorknowledgeabouttheobjects or scenes being

reconstructed, as it allows for more efficient and accurate
reconstructioncomparedtopurelydata-drivenapproaches.Thisapproachis
particularlyusefulinfieldswhereawell-definedunderstandingoftheunderlying geometry is available.

7. Recoveringtexturemapsand albedos:
Recovering texture maps and albedos in computer vision and computer graphics
involvesestimatingthesurfaceappearance,color,andreflectancepropertiesofobjects
inascene.Theseprocessesareintegraltocreatingrealisticanddetailed3Dmodelsfor applications like
virtual reality, computer games, and simulations. Here's a brief
overviewoftheseconcepts:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

TextureMaps:
● Description:Texturemappinginvolvesapplyinga2Dimage,knownasa texture
map, onto a 3D model's surface to simulate surface details, patterns, or color
variations.
● Recovery Process: Texture maps can be recovered through various
methods,includingimage-basedtechniques,photogrammetry,orusing specialized
3D scanners. These methods capture color information
associatedwiththesurfacegeometry.
● Application: Used in computer graphics, gaming, and virtual reality to
enhancethevisualappearanceof3Dmodelsbyaddingrealisticsurface details.
Albedo:
● Description:Albedorepresentstheintrinsiccolororreflectanceofa
surface,independentoflightingconditions.Itisameasureofhowmuch light a surface
reflects.
● Recovery Process: Albedo can be estimated by decoupling surface
reflectancefromlightingeffects.Photometricstereo,shape-from-shading, or using
multi-view images are common methods to recover albedo
information.
● Application:Albedoinformationiscrucialincomputervisionapplications, such as
material recognition, object tracking, and realistic rendering in
computergraphics.

RecoveringTextureMapsandAlbedosofteninvolvesthefollowingtechniques: Photometric Stereo:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Description:Atechniquethatusesmultipleimagesofanobject
illuminatedfromdifferentdirectionstorecoversurfacenormalsand, subsequently,
albedo information.
● Application:Usedinscenarioswheredetailedsurfacepropertiesare needed,
such as facial recognition, material analysis, and industrial
inspection.
Shape-from-Shading:
● Description: Inferring the shape of a surface based on variations in
brightnessorshadinginimages.Bydecouplingshadingfromgeometry, albedo
information can be estimated.
● Application:Appliedincomputervisionforshaperecovery,aswellasin computer
graphics to enhance the realism of rendered images.
Multi-ViewStereo (MVS):
● Description: In the context of 3D reconstruction, MVS involves capturing
imagesofascenefrommultipleviewpointsandrecoveringbothgeometry and texture
information.
● Application:Commonlyusedin3Dmodeling,virtualreality,andcultural heritage
preservation to create detailed and textured 3D models.
ReflectanceTransformationImaging(RTI):
● Description:Atechniquethatcapturesaseriesofimageswithcontrolled
lightingconditionstorevealsurfacedetails,includingalbedovariations.
● Application:Widelyusedinculturalheritagepreservationandart
restoration for capturing fine details on surfaces.

Recoveringtexturemapsandalbedosiscrucialforcreatingvisuallyappealingand
realistic3Dmodels.Thesetechniquesbridgethegapbetweenthegeometryofthe
objectsandtheirappearance,contributingtotheoverallfidelityofvirtualoraugmented environments.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

B.Tech [AIML/DS]