EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

UNITIII
FEATURE-BASEDALIGNMENT&MOTIONESTIMATION
2Dand3Dfeature-basedalignment-Poseestimation-Geometricintrinsic calibration-
Triangulation-Two-framestructurefrommotion-Factorization
-Bundleadjustment-Constrainedstructureandmotion-Translational alignment -
Parametric motion - Spline-based motion - Optical flow - Layered motion.

1. 2Dand3Dfeature-basedalignment:

2D and 3D Feature-Based Alignment

Feature-basedalignmentisatechniqueusedincomputervisionandimageprocessing
toalignormatchcorrespondingfeaturesindifferentimagesorscenes.Thealignment can be
performed in either 2D or 3D space, depending on the nature of the data.

2DFeature-BasedAlignment:
● Definition:In2Dfeature-basedalignment,thegoalistoalignandmatch features in
two or more 2D images.
● Features:Featurescanincludepoints,corners,edges,orotherdistinctive patterns.
● Applications:Commonlyusedinimagestitching,panoramacreation, object
recognition, and image registration.
3DFeature-BasedAlignment:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Definition:In3Dfeature-basedalignment,thegoalistoalignandmatch features in
three-dimensional space, typically in the context of 3D
reconstructionorscene understanding.
● Features:Featurescanincludekeypoints,landmarks,orotherdistinctive 3D points.
● Applications: Used in 3D reconstruction, simultaneous localization and
mapping(SLAM),objectrecognitionin3Dscenes,andaugmentedreality.
Techniquesfor2Dand3DFeature-BasedAlignment:
● Correspondence Matching: Identifying corresponding features in different images
or 3D point clouds.
● RANSAC(RandomSampleConsensus):Robustestimationtechniqueto find the
best-fitting model despite the presence of outliers.
● TransformationModels:Applyingtransformationmodels(affine,
homography for 2D; rigid body, affine for 3D) to align features.
● IterativeOptimization:Refiningthealignmentthroughiterative
optimization methods such as Levenberg-Marquardt.
Challenges:
● NoiseandOutliers:Real-worlddataoftencontainsnoiseandoutliers, requiring
robust techniques for feature matching.
● ScaleandViewpointChanges:Featuresmayundergochangesinscaleor viewpoint,
requiring methods that are invariant to such variations.
Applications:
● ImageStitching:Aligningandstitchingtogethermultipleimagestocreate panoramic
views.
● RoboticsandSLAM:Aligningconsecutiveframesinthecontextofrobotic navigation
and simultaneous localization and mapping.
● MedicalImaging:Aligning2Dslicesor3Dvolumesforaccuratemedical image
analysis.
Evaluation:
● AccuracyandRobustness:Theaccuracyandrobustnessoffeature-based alignment
methods are crucial for their successful application in various domains.

Feature-basedalignmentisafundamentaltaskincomputervision,enablingthe
integrationofinformationfrommultipleviewsormodalitiesforimprovedanalysisand understanding
of the visual world.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

2. Poseestimation:

Poseestimationisacomputervisiontaskthatinvolvesdeterminingthepositionand
orientationofanobjectorcamerarelativetoacoordinatesystem.Itisacrucialaspect
ofunderstandingthespatialrelationshipsbetweenobjectsinascene.Poseestimation can be applied to
both 2D and 3D scenarios, and it finds applications in various fields,
includingrobotics,augmentedreality,autonomousvehicles,andhuman-computer
interaction.

2DPoseEstimation:
● Definition:In2Dposeestimation,thegoalistoestimatetheposition
(translation)andorientation(rotation)ofanobjectinatwo-dimensional image.
● Methods: Techniques include keypoint-based approaches, where
distinctivepoints(suchascornersorjoints)aredetectedandusedto
estimatepose.CommonmethodsincludePnP(Perspective-n-Point) algorithms.
3DPoseEstimation:
● Definition:In3Dposeestimation,thegoalistoestimatethepositionand orientation of
an object in three-dimensional space.
● Methods:Ofteninvolvesassociating2Dkeypointswithcorresponding3D points. PnP
algorithms can be extended to 3D, and there are other methods like Iterative
Closest Point (ICP) for aligning a 3D model with a point cloud.
Applications:
● Robotics:Poseestimationiscrucialforroboticsystemstonavigateand interact with
the environment.
● AugmentedReality:Enablesthealignmentofvirtualobjectswiththe real-world
environment.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● AutonomousVehicles:Usedforunderstandingthepositionand
orientation of the vehicle in its surroundings.
● HumanPoseEstimation:Estimatingtheposeofaperson,oftenusedin applications
like gesture recognition and action recognition.
CameraPoseEstimation:
● Definition:Estimatingtheposeofacamera,whichinvolvesdeterminingits position
and orientation in the scene.
● Methods:Cameraposecanbeestimatedusingvisualodometry,SLAM
(Simultaneous Localization and Mapping), or using known reference points
in the environment.
Challenges:
● Ambiguity:Limitedinformationorsimilarappearanceofdifferentposes can
introduce ambiguity.
● Occlusion:Partiallyorfullyoccludedobjectscanmakeposeestimation challenging.
● Real-timeRequirements:Manyapplications,especiallyinroboticsand augmented
reality, require real-time pose estimation.
EvaluationMetrics:
● Commonmetricsincludetranslationandrotationerrors,whichmeasure the
accuracy of the estimated pose compared to ground truth.
DeepLearningApproaches:
● Recent advances in deep learning have led to the development of neural
network-basedmethodsforposeestimation,leveragingarchitectureslike
convolutional neural networks (CNNs) for feature extraction.

Poseestimationisafundamentaltaskincomputervisionwithwidespreadapplications.
Itplaysacrucialroleinenablingmachinestounderstandthespatialrelationships between objects and
the environment.

3. Geometricintrinsiccalibration:

Geometricintrinsiccalibrationisaprocessincomputervisionandcameracalibration that involves

determining the intrinsic parameters of a camera. Intrinsic parameters describe the internal
characteristics of a camera, such as its focal length, principal
point,andlensdistortioncoefficients.Accuratecalibrationisessentialforapplications

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

like3Dreconstruction,objecttracking,andaugmentedreality,whereknowingthe intrinsic properties of

the camera is crucial for accurate scene interpretation.

Herearekeypointsrelatedtogeometricintrinsiccalibration: Intrinsic
Parameters:
● FocalLength(f):Representsthedistancefromthecamera'sopticalcenter to the image
plane. It is a critical parameter for determining the scale of objects in the scene.
● PrincipalPoint(c):Denotesthecoordinatesoftheimagecenter.It
representstheoffsetfromthetop-leftcorneroftheimagetothecenterof the image
plane.
● LensDistortionCoefficients:Describeimperfectionsinthelens,suchas radial and
tangential distortions, that affect the mapping between 3D
worldpointsand2Dimagepoints.
CameraModel:
● Thecameramodel,oftenusedforintrinsiccalibration,isthepinhole
cameramodel.Thismodelassumesthatlightentersthecamerathrough a single point
(pinhole) and projects onto the image plane.
Calibration Patterns:
● Intrinsiccalibrationistypicallyperformedusingcalibrationpatternswith
knowngeometricfeatures,suchaschessboardpatterns.Thesepatterns allow for the
extraction of corresponding points in both 3D world
coordinatesand2Dimagecoordinates.
Calibration Process:
● ImageCapture:Multipleimagesofthecalibrationpatternarecaptured from
different viewpoints.
● FeatureExtraction:Detectedfeatures(corners,intersections)inthe
calibrationpatternareidentifiedinbothimageandworldcoordinates.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Parameter Estimation: Theintrinsic parameters areestimated using

mathematicaloptimizationtechniques,suchasnonlinearleastsquares optimization.
● Evaluation:Theaccuracyofcalibrationisoftenassessedbyreprojecting 3D points
onto the images and comparing with the detected 2D points.
RadialandTangentialDistortions:
● RadialDistortion:Deviationfromaperfectpinholecameramodeldueto radial
symmetry. Corrected using distortion coefficients.
● TangentialDistortion:Causedbythelensnotbeingperfectlyparalleltothe image plane.
Corrected using tangential distortion coefficients.
MultipleViews:
● Calibrationisoftenperformedusingmultipleviewstoimproveaccuracy and handle
lens distortions more effectively.
Applications:
● Intrinsiccalibrationisessentialforvariouscomputervisionapplications,
including3Dreconstruction,cameraposeestimation,andstereovision.

Accurategeometricintrinsiccalibrationisacriticalstepinensuringthatthecamera
modelaccuratelyrepresentsthemappingbetweenthe3Dworldandthe2Dimage, facilitating precise
computer vision tasks

4. Triangulation:

Triangulation, in the context of computer vision and 3D computer graphics, is a

techniqueusedtodeterminethe3Dcoordinatesofapointinspacebycomputingits position relative to
multiple camera viewpoints. The process involves finding the
intersectionpointoflinesorraysoriginatingfromcorresponding2Dimagepointsin different camera
views.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

Herearekeypointsrelatedtotriangulation:

BasicConcept:
● Triangulationisbasedontheprincipleoffindingthe3Dlocationofapoint in space by
measuring its projection onto two or more image planes.
CameraSetup:
● Triangulationrequiresatleasttwocameras(stereovision)ormoreto
capturethesamescenefromdifferentviewpoints.Eachcameraprovides a 2D
projection of the 3D point.
MathematicalRepresentation:

EpipolarGeometry:
● Epipolargeometryisutilizedtorelatethe2Dprojectionsofapointin
differentcameraviews.Itdefinesthegeometricrelationshipbetweenthe
twocameraviewsandhelpsestablishcorrespondencesbetweenpoints.
TriangulationMethods:
● DirectLinearTransform(DLT):Analgorithmicapproachthatinvolves solving a
system of linear equations to find the 3D coordinates.
● IterativeMethods:AlgorithmsliketheGauss-Newtonalgorithmorthe
Levenberg-Marquardtalgorithmcanbeusedforrefining theinitial estimate
obtained through DLT.
AccuracyandPrecision:
● Theaccuracyoftriangulationisinfluencedbyfactorssuchasthe
calibrationaccuracyofthecameras,thequalityoffeaturematching,and the level of
noise in the image data.
BundleAdjustment:
● Triangulation is often used in conjunction with bundle adjustment, a
techniquethatoptimizestheparametersofthecamerasandthe3Dpoints simultaneously
to minimize the reprojection error.
Applications:
● 3DReconstruction:Triangulationisfundamentaltocreating3Dmodelsof scenes or
objects from multiple camera views.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● StructurefromMotion(SfM):UsedinSfMpipelinestoestimatethe3D structure of
a scene from a sequence of images.
● StereoVision:Essentialfordepthestimationinstereovisionsystems. Challenges:
● Ambiguity: Ambiguities may arise when triangulating points from two
viewsiftheviewsarenotwell-separatedorifthepointisnearthebaseline connecting the
cameras.
● NoiseandErrors:Triangulationresultscanbesensitivetonoiseanderrors in feature
matching and camera calibration.

Triangulationisacoretechniqueincomputervisionthatenablesthereconstructionof
3Dgeometryfrommultiple2Dimages.Itplaysacrucialroleinapplicationssuchas3D
modeling,augmentedreality,andstructure-from-motionpipelines.

5. Two-framestructurefrommotion:

Two-Frame Structure from Motion

StructurefromMotion(SfM)isacomputervisiontechniquethataimstoreconstructthe three-
dimensionalstructureofascenefromasequenceoftwo-dimensionalimages.
Two-frameStructurefromMotionspecificallyreferstothereconstructionofscene geometry using
information from only two images (frames) taken from different
viewpoints.Thisprocessinvolvesestimatingboththe3Dstructureofthesceneandthe camera motion
between the two frames.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

HerearekeypointsrelatedtoTwo-FrameStructurefromMotion:

BasicConcept:
● Two-frameStructurefromMotionreconstructsthe3Dstructureofascene by analyzing
the information from just two images taken from different perspectives.
CorrespondenceMatching:
● Establishing correspondences between points or features in the two
imagesisacrucialstep.Thisisoftendonebyidentifyingkeyfeatures
(suchaskeypoints)inbothimagesandfindingtheircorrespondences.
EpipolarGeometry:
● Epipolar geometry describes the relationship between corresponding
pointsintwoimagestakenbydifferentcameras.Ithelpsconstrainthe possible 3D
structures and camera motions.
EssentialMatrix:
● Theessentialmatrixisafundamentalmatrixinepipolargeometrythat
encapsulatestheessentialinformationabouttherelativeposeoftwo calibrated
cameras.
CameraPoseEstimation:
● Thecameraposes(positionsandorientations)areestimatedforboth
frames.Thisinvolvessolvingfortherotationandtranslationbetweenthe two camera
viewpoints.
Triangulation:
● Triangulationisappliedtofindthe3Dcoordinatesofpointsinthescene. By knowing
the camera poses and corresponding points, the depth of scene points can be
estimated.
BundleAdjustment:
● Bundleadjustmentisoftenusedtorefinetheestimatesofcameraposes and 3D points.
It is an optimization process that minimizes the error
betweenobservedandpredictedimagepoints.
DepthAmbiguity:
● Two-frameSfMissusceptibletodepthambiguity,meaningthatthe
reconstructedscenecouldbescaledormirroredwithoutaffectingthe projections onto
the images.
Applications:
● Robotics:Two-frameSfMisusedinroboticsforenvironmentmappingand navigation.
● AugmentedReality:Reconstructionofthe3Dstructureforoverlaying virtual
objects onto the real-world scene.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● ComputerVisionResearch:StudyingtheprinciplesofSfMandepipolar geometry.
Challenges:
● NoiseandOutliers:Theaccuracyofthereconstructioncanbeaffectedbynoise and outliers in
the correspondence matching process.
● LimitedBaseline:Withonlytwoframes,thebaseline(distancebetweencamera viewpoints)
may be limited, leading to potential depth ambiguities.

Two-frameStructurefromMotionisafundamentalconceptincomputervision,
providingafoundationforunderstanding3Dscenestructurefromapairofimages.Itis
oftenextendedtomulti-frameSfMformorerobustreconstructionsinscenarioswhere more images are
available.

6. Factorization:

Factorizationinthecontextofcomputervisiontypicallyreferstothefactorizationof
matricesortensorsrepresentingdatainvariouscomputervisiontasks.Onecommon
applicationisinthefieldofstructurefrommotion(SfM)andmultiple-viewgeometry. Here are key
points related to factorization in computer vision:

MatrixFactorizationinSfM:
● ProblemStatement:Instructurefrommotion,thegoalistoreconstruct the 3D
structure of a scene from a sequence of 2D images taken from different
viewpoints.
● MatrixRepresentation: The correspondencematrix, alsoknown as the
measurementmatrix,isconstructedbystackingtheimagecoordinatesof
corresponding points from multiple views.
● Matrix Factorization: Factorizing the correspondence matrix into two
matricesrepresentingcameraparametersand3Dstructureisacommon approach.
This factorization is often achieved through techniques like
SingularValueDecomposition(SVD).
SingularValueDecomposition(SVD):
● Application:SVDisfrequentlyusedinmatrixfactorizationproblemsin computer
vision.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

Applications:
● StructurefromMotion(SfM):Factorizationisusedtorecovercamera poses and
3D scene structure from 2D image correspondences.
● BackgroundSubtraction:Matrixfactorizationtechniquesareemployedin background
subtraction methods for video analysis.
● FaceRecognition:EigenfaceandFisherfacemethodsinvolvefactorizing
covariance matrices for facial feature representation.
Non-NegativeMatrixFactorization(NMF):
● Application:NMFisavariantofmatrixfactorizationwherethefactorsare constrained
to be non-negative.
● UseCases:Itisappliedinareassuchastopicmodeling,image
segmentation, and feature extraction.
TensorFactorization:
● ExtensiontoHigherDimensions:Insomecases,dataisrepresentedas tensors, and
factorization techniques are extended to tensors for applications like multi-
way data analysis.
● Example:CanonicalPolyadicDecomposition(CPD)isatensor
factorization technique.
RobustFactorization:
● Challenges:Noiseandoutliersinthedatacanaffecttheaccuracyof factorization.
● RobustMethods:Robustfactorizationtechniquesaredesignedtohandle noisy data
and outliers, providing more reliable results.
DeepLearningApproaches:
● AutoencodersandNeuralNetworks:Deeplearningmodels,including
autoencoders,canbeconsideredasaformofnonlinearfactorization.
FactorizationMachine(FM):
● Application:FactorizationMachinesareusedincollaborativefilteringand
recommendation systems to model interactions between features.

Factorizationplaysacrucialroleinvariouscomputervisionandmachinelearningtasks, providing a
mathematical framework for extracting meaningful representations from

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

dataandsolvingcomplexproblemslike3Dreconstructionanddimensionality reduction.

7. Bundle adjustment:

Bundle Adjustment is a crucial optimization technique in computer vision and

photogrammetry.Itisusedtorefinetheparametersofa3Dscene,suchascamera
posesand3Dpoints,byminimizingthereprojectionerrorbetweentheobservedimage points and their
corresponding projections from the 3D scene. Bundle Adjustment is
commonlyemployedinthecontextofstructurefrommotion(SfM),simultaneous localization and
mapping (SLAM), and 3D reconstruction.

HerearekeypointsrelatedtoBundleAdjustment:
Optimization Objective:
● Minimization of Reprojection Error: Bundle Adjustment aims to find the
optimalsetofparameters(cameraposes,3Dpoints)thatminimizesthe difference
between the observed 2D image points and their projections onto the image
planes based on the estimated 3D scene.
ParameterstoOptimize:
● Camera Parameters: Intrinsic parameters (focal length, principal point) and
extrinsic parameters (camera poses - rotation and translation).
● 3DSceneStructure:Coordinatesof3Dpointsinthescene.
Reprojection Error:
● Definition: Thereprojectionerroristhedifferencebetweentheobserved
2Dimagepointsandtheprojectionsofthecorresponding3Dpointsonto the image
planes.
● SumofSquaredDifferences:Theobjectiveistominimizethesumof squared
differences between observed and projected points.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

BundleAdjustmentProcess:
● Initialization:Startwithinitialestimatesofcameraposesand3Dpoints.
● ObjectiveFunction:Defineanobjectivefunctionthatmeasuresthe
reprojection error.
● Optimization:Useoptimizationalgorithms(suchasLevenberg-Marquardt, Gauss-
Newton,orothers)toiterativelyrefinetheparameters,minimizing the reprojection
error.
SparseandDenseBundleAdjustment:
● SparseBA:Considersasubsetof3Dpointsandimagepoints,makingit
computationally more efficient.
● DenseBA:Involvesall3Dpointsandimagepoints,providinghigher accuracy
but requiring more computational resources.
SequentialandGlobalBundleAdjustment:
● SequentialBA:Optimizescameraposesand3Dpointssequentially, typically
in a sliding window fashion.
● GlobalBA:Optimizesallcameraposesand3Dpointssimultaneously.
Providesamoreaccuratesolutionbutiscomputationallymore demanding.
Applications:
● StructurefromMotion(SfM):Refinesthereconstructionof3Dscenes from a
sequence of images.
● SimultaneousLocalizationandMapping(SLAM):Improvestheaccuracy of camera
pose estimation and map reconstruction in real-time
environments.
● 3DReconstruction:Enhancestheaccuracyofreconstructed3Dmodels from
images.
Challenges:
● LocalMinima:Theoptimizationproblemmayhavemultiplelocalminima, making it
essential to use robust optimization methods.
● OutliersandNoise:BundleAdjustmentneedstoberobusttooutliersand noise in the
input data.
IntegrationwithOtherTechniques:
● Feature Matching: Often used in conjunction with feature matching
techniquestoestablishcorrespondencesbetween2Dand3Dpoints.
● CameraCalibration:BundleAdjustmentmaybeprecededbyorintegrated with
camera calibration to refine intrinsic parameters.

BundleAdjustmentisafundamentaloptimizationtechniquethatsignificantlyimproves the accuracy of

3D reconstructions and camera pose estimations in computer vision

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

applications.Ithasbecomeacornerstoneinmanysystemsdealingwith3Dscene understanding and

reconstruction.

8. Constrainedstructureandmotion:

ConstrainedStructureandMotion

ConstrainedStructureandMotionreferstoasetoftechniquesandmethodsin
computervisionandphotogrammetrythatincorporateadditionalconstraintsinto the
structurefrommotion(SfM)process.Thegoalistoimprovetheaccuracyandreliability of 3D
reconstruction by imposing constraints on the estimated camera poses and 3D scene points.
These constraints may come from prior knowledge about the scene,
sensorcharacteristics,oradditionalinformation.

HerearekeypointsrelatedtoConstrainedStructureandMotion: Introduction
of Constraints:
● Prior Information: Constraints can be introduced based on prior
knowledgeaboutthescene,suchasknowndistances,planarstructures, or object
shapes.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● SensorConstraints:Informationaboutthecamerasystem,suchasfocal length or
aspect ratio, can be incorporated as constraints.
TypesofConstraints:
● Geometric Constraints: Constraints that enforce geometric relationships, such as
parallel lines, perpendicularity, or known distances between
points.
● SemanticConstraints:Incorporatingsemanticinformationaboutthe
scene,suchastheknowledgethatcertainpointsbelongtoaspecific object or
structure.
BundleAdjustmentwithConstraints:
● ObjectiveFunction:Thebundleadjustmentproblemisformulatedwithan
objectivefunctionthatincludesthereprojectionerror,aswellasadditional terms
representing the constraints.
● Optimization:Optimizationtechniques,suchasLevenberg-Marquardtor Gauss-
Newton, are used to minimize the combined cost function.
Advantages:
● ImprovedAccuracy:Incorporatingconstraintscanleadtomoreaccurate
andreliablereconstructions,especiallyinscenarioswithlimitedornoisy data.
● HandlingAmbiguities:Constraintshelpinresolvingambiguitiesthatmay arise in
typical SfM scenarios.
CommonTypesofConstraints:
● PlanarConstraints:Assumingthatcertainstructuresinthescenelieon planes,
which can be enforced during reconstruction.
● ScaleConstraints:Fixingorconstrainingthescaleofthescenetoprevent scale
ambiguity in the reconstruction.
● ObjectConstraints:Incorporatingconstraintsrelatedtospecificobjectsor entities in the
scene.
Applications:
● ArchitecturalReconstruction:Constrainingthereconstructionbasedon known
architectural elements or planar surfaces.
● RoboticsandAutonomousSystems:Utilizingconstraintstoenhancethe accuracy of
pose estimation and mapping in robotic navigation.
● AugmentedReality:Incorporatingsemanticconstraintsformoreaccurate alignment of
virtual objects with the real world.
Challenges:
● CorrectnessofConstraints:Theaccuracyofthereconstructiondepends on the
correctness of the imposed constraints.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● ComputationalComplexity:Someconstrainttypesmayincreasethe
computational complexity of the optimization problem.
IntegrationwithSemanticTechnologies:
● Semantic3DReconstruction:Integratingsemanticinformationintothe
reconstruction process to improve the understanding of the scene.

ConstrainedStructureandMotionprovidesawaytoincorporateadditionalinformation
anddomainknowledgeintothereconstructionprocess,makingitavaluableapproach for scenarios
where such information is available and reliable. It contributes to more
accurateandmeaningful3Dreconstructionsincomputervisionapplications.

9. Translationalalignment

Translationalalignment,inthecontextofcomputervisionandimageprocessing,refers to the process

of aligning two or more images based on translational transformations.
Translationalalignmentinvolvesadjustingthepositionofimagesalongthexandyaxes
tobringcorrespondingfeaturesorpointsintoalignment.Thistypeofalignmentisoften a fundamental
step in various computer vision tasks, such as image registration,
panoramastitching,andmotioncorrection.
Herearekeypointsrelatedtotranslationalalignment:
Objective:
● The primary goal of translational alignment is to align images by minimizing
the translation difference between corresponding points or features in the
images.
TranslationModel:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

CorrespondenceMatching:
● Correspondencematchinginvolvesidentifyingcorrespondingfeaturesor points in
the images that can be used as reference for alignment.
Commontechniquesincludekeypointdetectionandmatching.
AlignmentProcess:
● Thetranslationalalignmentprocesstypicallyinvolvesthefollowingsteps:

Applications:
● ImageStitching:Inpanoramacreation,translationalalignmentisusedto align images
before merging them into a seamless panorama.
● MotionCorrection:Invideoprocessing,translationalalignmentcorrects for
translational motion between consecutive frames.
● RegistrationinMedicalImaging:Aligningmedicalimagesacquiredfrom different
modalities or at different time points.
Evaluation:
● Thesuccessoftranslationalalignmentisoftenevaluatedbymeasuring
theaccuracyofthealignment,typicallyintermsofthedistancebetween
corresponding points before and after alignment.
Robustness:
● Translational alignment is relatively straightforward and computationally
efficient.However,itmaybesensitivetonoiseandoutliers,particularlyin the
presence of large rotations or distortions.
IntegrationwithOtherTransformations:
● Translational alignment is frequently used as an initial step in more
complexalignmentprocessesthatinvolveadditionaltransformations, such as
rotational alignment or affine transformations.
AutomatedAlignment:
● Inmanyapplications,algorithmsfortranslationalalignmentaredesigned to operate
automatically without requiring manual intervention.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

Translational alignment serves as a foundational step in various computer vision

applications,providingasimpleandeffectivemeanstoalignimagesbeforefurther processing or
analysis.

10. Parametricmotion

Parametricmotionreferstothemodelingandrepresentationofmotionincomputer
visionandcomputergraphicsusingparametricfunctionsormodels.Insteadofdirectly
capturingthemotionwithasetofdiscreteframes,parametricmotionmodelsdescribe how the motion
evolves over time using a set of parameters. These models are often
employedinvariousapplications,suchasvideoanalysis,animation,andtracking. Here are key
points related to parametric motion:
ParametricFunctions:
● Parametricmotionmodelsusemathematicalfunctionswithparameters
torepresentthemotionofobjectsorscenesovertime.Thesefunctions could be
simple mathematical equations or more complex models.
TypesofParametricMotionModels:
● LinearModels:Simplestformofparametricmotion,wheremotionis
representedbylinearequations.Forexample,linearinterpolationbetween keyframes.
● PolynomialModels:Higher-orderpolynomialfunctionscanbeusedto model
more complex motion. Cubic splines are commonly used for smooth motion
interpolation.
● TrigonometricModels:Sinusoidalfunctionscanbeemployedtorepresent periodic
motion, such as oscillations or repetitive patterns.
● ExponentialModels:Capturebehaviorsthatexhibitexponentialgrowthor decay,
suitable for certain types of motion.
KeyframeAnimation:
● Inparametricmotion,keyframesarespecifiedatcertainpointsintime,
andthemotionbetweenkeyframesisdefinedbytheparametricmotion
model.Interpolationisthenusedtogenerateframesbetweenkeyframes.
ControlPointsandHandles:
● Parametricmodelsofteninvolvecontrolpointsandhandlesthatinfluence the shape
and behavior of the motion curve. Adjusting these parameters allows for creative
control over the motion.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

Applications:
● ComputerAnimation:Usedforanimatingcharacters,objects,orcamera movements
in 3D computer graphics and animation.
● VideoCompression:Parametricmotionmodelscanbeusedtodescribe the motion
between video frames, facilitating efficient compression
techniques.
● VideoSynthesis:Generatingrealisticvideosorpredictingfutureframesin a video
sequence based on learned parametric models.
● MotionTracking:Trackingthemovementofobjectsinavideobyfitting parametric
motion models to observed trajectories.
SmoothnessandContinuity:
● Oneadvantageofparametricmotionmodelsistheirabilitytoprovide smooth and
continuous motion, especially when using interpolation techniques between
keyframes.
ConstraintsandConstraints-BasedMotion:
● Parametricmodelscanbeextendedtoincludeconstraints,ensuringthat
themotionadherestospecificrulesorconditions.Forexample,enforcing constant
velocity or maintaining specific orientations.
MachineLearningIntegration:
● Parametricmotionmodelscanbelearnedfromdatausingmachine learning
techniques. Machine learning algorithms can learn the
parametersofthemotionmodelfromobservedexamples.
Challenges:
● Designingappropriateparametricmodelsthataccuratelycapturethe
desiredmotioncanbechallenging,especiallyforcomplexornon-linear motions.
● Ensuringthatthemotionremainsphysicallyplausibleandvisually appealing
is crucial in animation and simulation.

Parametricmotionprovidesaflexibleframeworkforrepresentingandcontrolling
motioninvariousvisualcomputingapplications.Thechoiceofparametricmodel
dependsonthespecificcharacteristicsofthemotiontoberepresentedandthedesired level of control
and realism.

11. Spline-basedmotion

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

Spline-basedmotionreferstotheuseofsplinecurvestomodelandinterpolatemotion
incomputergraphics,computer-aideddesign,andanimation.Splinesaremathematical curves that
provide a smooth and flexible way to represent motion paths and
trajectories.Theyarewidelyusedin3Dcomputergraphicsandanimationforcreating
naturalandvisuallypleasingmotion,particularlyinscenarioswherecontinuousand smooth paths are
desired.
Herearekeypointsrelatedtospline-basedmotion: Spline
Definition:
● SplineCurve:Asplineisapiecewise-definedpolynomialcurve.Itconsists
ofseveralpolynomialsegments(typicallylow-degree)thataresmoothly connected at
specific points called knots or control points.
● TypesofSplines:CommontypesofsplinesincludeB-splines,cubic splines, and
Bezier splines.
SplineInterpolation:
● Splinecurvesareoftenusedtointerpolatekeyframesorcontrolpointsin
animation.Thismeansthecurvepassesthroughorfollowsthespecified keyframes,
creating a smooth motion trajectory.
B-spline(BasisSpline):
● B-splinesarewidelyusedforspline-basedmotion.Theyaredefinedbya set of control
points, and their shape is influenced by a set of basis
functions.
● LocalControl:Modifyingthepositionofacontrolpointaffectsonlyalocal portion of
the curve, making B-splines versatile for animation.
CubicSplines:
● Cubicsplinesareaspecifictypeofsplinewhereeachpolynomialsegment is a cubic
(degree-3) polynomial.
● NaturalMotion:Cubicsplinesareoftenusedforcreatingnaturalmotion paths due to
their smoothness and continuity.
BezierSplines:
● Beziersplinesareatypeofsplinethatisdefinedbyasetofcontrolpoints. They have
intuitive control handles that influence the shape of the curve.
● BezierCurves:CubicBeziercurves,inparticular,arefrequentlyusedfor creating
motion paths in animation.
SplineTangentsandCurvature:
● Spline-basedmotionallowscontroloverthetangentsatcontrolpoints,
influencingthedirectionofmotion.Curvaturecontinuityensuressmooth transitions
between segments.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

Applications:
● Computer Animation: Spline-based motion is extensively used for
animatingcharacters,cameramovements,andobjectsin3Dscenes.
● PathGeneration:Designingsmoothandvisuallyappealingpathsfor objects to
follow in simulations or virtual environments.
● MotionGraphics:Creatingdynamicandaestheticallypleasingvisual effects in
motion graphics projects.
ParametricRepresentation:
● Spline-basedmotionisparametric,meaningthepositionofapointonthe spline is
determined by a parameter. This allows for easy manipulation
andcontroloverthemotion.
InterpolationTechniques:
● KeyframeInterpolation:Splinecurvesinterpolatesmoothlybetween
keyframes, providing fluid motion transitions.
● Hermite Interpolation: Splines can be constructed using Hermite
interpolation,wherebothpositionandtangentinformationatcontrol points are
considered.
Challenges:
● Overfitting:Insomecases,splinecurvescanbeoverlyflexibleandleadto overfitting if
not properly controlled.
● ControlPointPlacement:Choosingtherightplacementforcontrolpoints is crucial
for achieving the desired motion characteristics.

Spline-basedmotionprovidesanimatorsanddesignerswithaversatiletoolforcreating
smoothandcontrolledmotionpathsincomputer-generatedimagery.Theabilityto adjust the shape of
the spline through control points and handles makes it a popular choice for a wide range of
animation and graphics applications.

12. Opticalflow

Optical flow is a computer vision technique that involves estimating the motion of
objectsorsurfacesinavisualscenebasedontheobservedchangesinbrightnessor
intensityovertime.Itisafundamentalconceptusedinvariousapplications,including motion analysis,
video processing, object tracking, and scene understanding.

Herearekeypointsrelatedtoopticalflow:

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

MotionEstimation:
● Objective:Theprimarygoalofopticalflowistoestimatethevelocity
vector(opticalflowvector)foreachpixelinanimage,indicatingthe apparent
motion of that pixel in the scene.
● Pixel-levelMotion:Opticalflowprovidesadenserepresentationofmotion at the pixel
level.
BrightnessConstancyAssumption:
● Assumption:Opticalflowisbasedontheassumptionofbrightness
constancy,whichstatesthatthebrightnessofapointinthescene remains
constant over time.

OpticalFlowEquation:
● Derivation:Theopticalflowequationisderivedfromthebrightness
constancyassumptionusingpartialderivativeswithrespecttospatial coordinates and
time.

DenseandSparseOpticalFlow:
● DenseOpticalFlow:Estimatingopticalflowforeverypixelintheimage, providing
a complete motion field.
● SparseOpticalFlow:Estimatingopticalflowonlyforselectedkeypointsor features in
the image.
ComputationalMethods:
● Correlation-basedMethods:Matchimagepatchesorwindowsbetween consecutive
frames to estimate motion.
● Gradient-basedMethods:Utilizeimagegradientstocomputeopticalflow.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● VariationalMethods:Formulateenergyminimizationproblemstoestimate optical
flow.
Lucas-KanadeMethod:
● Awell-knowndifferentialmethodforestimatingopticalflow,particularly suited for
small motion and local analysis.
Horn-SchunckMethod:
● Avariationalmethodthatminimizesaglobalenergyfunction,takinginto account
smoothness constraints in addition to brightness constancy.
Applications:
● VideoCompression:Opticalflowisusedinvideocompressionalgorithms to predict
motion between frames.
● ObjectTracking:Trackingmovingobjectsinavideosequence.
● Robotics:Providingvisualfeedbackfornavigationandobstacle
avoidance.
● AugmentedReality:Aligningvirtualobjectswiththereal-worldscene.
Challenges:
● IlluminationChanges:Opticalflowmaybesensitivetochangesin
illumination.
● Occlusions:Occlusionsandcomplexmotionpatternscanposechallenges for accurate
optical flow estimation.
● LargeDisplacements:Traditionalmethodsmaystrugglewithhandling large
displacements.
DeepLearningforOpticalFlow:
● Recent advances in deep learning have led to the development of neural
network-basedmethodsforopticalflowestimation,suchasFlowNetand PWC-Net.

Opticalflowisavaluabletoolforunderstandingandanalyzingmotioninvisualdata.
Whiletraditionalmethodshavebeenwidelyused,theintegrationofdeeplearninghas brought new
perspectives and improved performance in optical flow estimation.

13. Layeredmotion

Layeredmotion,inthecontextofcomputervisionandmotionanalysis,referstothe representation and

analysis of a scene where different objects or layers move
independentlyofeachother.Itassumesthatthemotioninascenecanbedecomposed into multiple
layers, each associated with a distinct object or surface. Layered motion

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

modelsareemployedtobettercapturecomplexsceneswithmultiplemovingentities, handling
occlusions and interactions between objects.
Herearekeypointsrelatedtolayeredmotion: Layered
Motion Models:
● Objective:Thegoaloflayeredmotionmodelsistorepresentthemotionof
distinctobjectsorsurfacesinasceneindependently,allowingforamore accurate
description of complex motion scenarios.
● Assumption:Itassumesthattheobservedmotioninascenecanbe decomposed
into the motion of different layers.
KeyConcepts:
● Independence:Layersareassumedtomoveindependentlyofeachother, simplifying
the modeling of complex scenes.
● Occlusions: Layered motion models can handle occlusions more
effectively,aseachlayerrepresentsaseparateentityinthescene.
MotionLayerSegmentation:
● SegmentationProcess:The processofidentifying andseparating the
differentmotionlayersinavideosequenceisreferredtoasmotionlayer segmentation.
● ForegroundandBackground:Layersmightrepresenttheforegroundand background
elements in a scene.
ChallengesinLayeredMotion:
● InteractionHandling:Representingtheinteractionbetweenlayers,suchas occlusions or
overlapping motions.
● DynamicSceneChanges:Adaptingtochangesinthescene,includingthe appearance
or disappearance of objects.
OpticalFlowforLayeredMotion:
● Opticalflowtechniquescanbeextendedtoestimatethemotionof individual
layers in a scene.
● Layer-SpecificOpticalFlow:Applyingopticalflowindependentlyto different
layers.
MultipleObjectTracking:
● Layeredmotionmodelsarecloselyrelatedtomultipleobjecttracking,as each layer
can correspond to a tracked object.
Applications:
● SurveillanceandSecurity:Trackingandanalyzingthemotionofmultiple objects in
surveillance videos.

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

● Robotics:Layeredmotionmodelscanaidrobotsinunderstandingand navigating
dynamic environments.
● AugmentedReality:Aligningvirtualobjectswiththereal-worldsceneby
understanding the layered motion.
RepresentationFormats:
● Layerscanberepresentedinvariousformats,suchasdepthmaps, segmentation
masks, or explicit motion models for each layer.
IntegrationwithSceneUnderstanding:
● Layeredmotionmodelscanbeintegratedwithhigher-levelsceneund

B.Tech [AIML/DS]
EAIDS254 – COMPUTER VISION JEPPIAAR UNIVERSITY

B.Tech [AIML/DS]