Camera Models and Parameters

We will discuss camera geometry in more detail.

Particularly, we will outline what parameters are important
within the model. These parameters are important to several
key computer vision tasks and must be computed
(calibrated) using approaches we will discuss in later
lectures.
 Important Definitions
Frame of reference: a measurements are made with respect to a
particular coordinate system called the frame of reference.
World Frame: a fixed coordinate system for representing objects
(points, lines, surfaces, etc.) in the world.
Camera Frame: coordinate system that uses the camera center as its
origin (and the optic axis as the Z-axis)
Image or retinal plane: plane on which the image is formed, note that
the image plane is measured in camera frame coordinates (mm)
Image Frame: coordinate system that measures pixel locations in the
image plane.
Intrinsic Parameters: Camera parameters that are internal and fixed to
a particular camera/digitization setup
Extrinsic Parameters: Camera parameters that are external to the
camera and may change with respect to the world frame.
 Camera Models Overview
Extrinsic Parameters: define the location and orientation of the camera
with respect to the world frame.
Intrinsic Parameters: allow a mapping between camera coordinates
and pixel coordinates in the image frame.
Camera model in general is a mapping from world to image
coordinates.
This is a 3D to 2D transform and is dependent upon a number of
independent parameters.
 Pinhole Model Revisited
Select a coordinate system (,x,y,z) for the three-dimensional space to
be imaged
Let (u,v) be the retinal plane
Then, the two are related by:
f u v
= =
z x y
Which is written linearly in homogeneous coordinates as:

x
U f 0 0 0
V = 0 f 0 0 y
z
S 0 0 1 0
1
 The Retinal Plane

x
M y
z

F
x

y
u v
m u
v

f
 Camera orientation in the world
The position of the camera in the world must be recovered
rotational component
translation component
Describes absolute position of the focal plane in the world coordinate
system
This is a Euclidean transform from one coordinate system to another
-translation, rotation

F
z

x
 Translation
Frames A and B are related through
a pure translation

yA
rA rA = rB + tA


yB
xA rB
where ta represents
A
B the pure
tA
xB translation from
frame A to frame
B written in frame
A coordinates
 Rotations
Frames A and B are related through
a pure rotation

A position vector r, in

yB
yA
r frame B, can be
expressed in the A

xB coordinate frame by
employing the 3 X 3

xA transformation matrix
ARB . . .
 Rotation Matrix
v v
rA = A RB rB
) ) ) ) ) )
i A i B i A j B i A k B
rx A rx
ry = )j i) ) ) ) ) B

A A B j A j B j A k B ry B

rz A ) ) ) ) ) ) rz B
k A i B k A j B k A k B

This projection of frame B onto frame A clearly converts
a position vector, rB , written in frame B, into the

corresponding coordinates in frame A, rA .
 Interpreting the Rotation Matrix
To interpret the rotation matrix for this
transformation:
the rows of ARB represent the projection of
the basis vectors for frame A onto the basis
vectors of frame B
the columns of ARB represent the basis
vectors of frame B projected onto the basis
vectors of frame A
Rotations

One way to specify the rotation matrix ARB is

to write the base vectors
) ) )
(i , j,k ) B

in frame A coordinates and to enter the result

into the columns of ARB
Rotations
)A
If xB is a column vector representing the x
axis of frame B written in frame A
coordinates, then

cos( ) -sin( ) 0
)A )A )A
ARB = x y z = sin( ) cos( ) 0
B B B
0 0 1
Rotations: x
For completeness, we will look at the rotation
matrix for rotations about all three axes:

1 0 0
)
rot ( x , ) = 0 cos( ) sin( )

0 sin( ) cos( )
Rotations: y
For completeness, we will look at the rotation
matrix for rotations about all three axes:

cos( ) 0 sin( )
)
rot ( y, ) = 0 1 0

sin( ) 0 cos( )
Rotations: z
For completeness, we will look at the rotation
matrix for rotations about all three axes:

cos( ) sin( ) 0
)
rot ( z , ) = sin( ) cos( ) 0

0 0 1
 Extrinsic Parameters
Recall the fundamental equations of perspective projection
assumed the orientation of the camera and world frame known
this is actually a difficult problem known as extrinsic pose problem
using only image information recover the relative position and
orientation of the camera and world frames
This transformation is typically defined by:
3-D translation vector T=[x,y,z]T
defines relative positions of each frame
3x3 rotation matrix, R
rotates corresponding axes of each frame into each other
R is orthogonal: (RTR = RRT =I)
 Extrinsic Parameters
XC
RT
ZW
ZC

YC
YW

( )
XW r11 r12 r13
Note: we write R= r21 r22 r23
r31 r32 r33

How to write both rotation and translation as a single,

composed transform?
 Homogeneous Transformations
))
y0y
0
v
r 0v v
r0 ) r2
) y1
y2
) )
0 x0 x2
v 2
t0 )
1 x1

We consider the general case where frame 0

and frame 2 are related to one another
through both rotation and translation
Homogeneous Transformations
)
y0
v
r0 v
) ) r2
y2 y1

) )
0 x0 x2
v 2
t0 )
1 x1

The homogeneous transform is a

mechanism for expressing this form of
compound transformation
 Homogeneous Transforms
Expand the dimensionality of the domain space
Same transformation now can be expressed in a linear
fashion
Linear transforms can be easily composed and written as a
single matrix multiply
Vectors, in homoeneous space take on a new parameter r.
This is the scale of the vector along the new axis and is
arbitrary: [x y z r]
Normalization, after the transform has been applied is
accomplished simply by dividing each vector component
by r [x y z 1] = [x/r y/r z/r r/r]
Homogeneous Transformations

Let 0T2 be the compound

transformation consisting of a
translation from 0 to 1, followed by
a rotation from 1 to 2
Homogeneous Transformations
In vector notation, this homogeneous
transformation and corresponding
homogeneous position vectors are written:

r rx
T
0 2 = [ 1

0 0 0
R2 t0

1
] r
r y
r2 =
rz
r
Then, r0 = 0T2 r2
r r 1 2
= 1 R 2 r2 + t0
 Composing Transformations

The homogeneous transform

provides a convenient means of
constructing compound
transformations
Composing Transformations

Example: Suppose

T = 0T1 1T2 2T3 3T4

0 4

where:
)
0T1 = translation( x0 , 1.0)
)
1T 2 = translation( y1 , 1.0)
)
2T 3 = translation( z2 , 1.0)
)
3T 4 = rotation ( y3 , / 4)
Composing Transformations
Example: 0T4 = 0T1 1T2 2T3 3T4
)
) z3 )
z4 x4
) )
4 y3, y4
/4

)
3 x3

)
z2
) ) )
z0 z1 y2
)
2
) y1 )
y0 x2
0 1
) )
x0 x1
Composing Transformations

The resulting compound transformation is

0.707 0 0.707 1

0 1 0 1
0T4 =
0.707 0 0.707 1

0 0 0 1
 Extrinsic Parameters
XC
RT
ZW
ZC

YC
YW

pc = P p w
( )
XW r11 r12 r13
R= r21 r22 r23
r31 r32 r33
R T
P=
0 0 0 1

[
T = Tx Ty Tz ]
T
 Intrinsic Parameters
Characterize the optical, geometric, and digital
characteristics of the camera
Defined by:
perspective projection: focal length f
transformation between camera frame and pixel coordinates
geometric distortion introduced by the lens
Transform between camera frame and pixels:

x = -(xim - ox)sx
y = -(yim - oy)sy

(ox,oy) image center (principle point)

(sx,sy) effective size of pixels in mm in horizontal and
vertical directions
 Camera Lens Distortion
Optical system itself a source of distortions
evident at the image periphery
worsened by large field of view
Modeled accurately as radial distortion

x = xd (1 + k1r2 + k2r4)
y = yd (1 + k1r2 + k2r4)

(xd,yd) distorted points, and r2= x2d + y2d

note: this is a radial displacement of the image points
because k2 << k1, k2 is often ignored.
 Camera models (again)
Using the equations from previous slides, we are able to transform
pixel coordinates to world points this is our camera model.

X Y
Recall x= f
Z
y= f
Z

Linear Perspective Projection Equations:

RT1(Pw - T)
-(xim - ox)sx = f
RT3(Pw - T)

RT2(Pw - T)
-(xim - ox)sx = f
RT3(Pw - T)

Ri, i=1,2,3 is a 3D vector formed by the I-th row of R

 Linear Matrix Representation
If we neglect radial distortion, we can rewrite the linear
perspective transform as a matrix product.
A matrix is used for intrinsic and extrinsic parameters.

Mint =
(-f/sx
0
0
0
-f/sy
0
ox
oy
1
)
Mext=
(r11
r21
r31
r12
r22
r32
r13
r23
r33
-RT1T
-RT1T
-RT1T
)
 Using the Perspective Equations
3x3 Mint only depends on the intrinsic parameters
3x4 Mext depends on extrinsic parameters
We make use of these by introducing homogeneous coordinates to
point vectors in the world.
Pw must be expressed in homogenous coordinates to allow direct
multiplication to Mint and Mext

x1 Xw
x2 = Mint Mext Yw
Zw
x3
1
[x1,x2,x3]T is the projected point, using the vector we compute image
coordinates:
x1/x3 = xim
x2/x3 = yim
 The Perspective Transform

Mext : from world to camera frame

Mint : from camera to image
Can be viewed, formally, as a relation between a 3D point
and its perspective projection on the image plane.
Maps points in projective space, space of vectors [xw,yw,zw]T to the
projective plane, space of vectors [x1,y1,z1]T.
defined up to a
11 independent parameters
Camera models from the Projective Equations

Various models can be derived by setting appropriate constraints on

the projection equations
The Perspective Model
ox = oy = 0
sx = sy = 1.0
The Weak-Perspective Model
note that image point p of world point P is given by:
Xw f RT1(T - P)
Yw =
p=M Zw
f RT2(T - P)
1 f RT3(T - P)
 Weak-Perspective Continued

RT3(P - T) is the distance of P from the perspective

center along the optical axis.
Therefore:
RT3(Pi - P)
T << 1
R 3(P - T)

Is the weak-perspective approximation. Using this

approximation the perspective matrix can be written
to eliminate negligible terms.