Linear calibration of a rotating and zooming camera ∗

Lourdes de Agapito, Richard I. Hartley and Eric Hayman

Department of Engineering,
Oxford University
and
G.E. Corporate Research and Development
1 Research Circle,
Niskayuna, NY 12309

Abtract apply in case of a minimal assumption of zero-skew.

A linear self-calibration method is given for com- These methods applied to moving cameras, and were
puting the calibration of a stationary but rotating iterative1 .
camera. The internal parameters of the camera are al- Recent papers ([1, 9]) have extended calibration ca-
lowed to vary from image to image, allowing for zoom- pability by giving algorithms for rotating cameras, al-
ing (change of focal length) and possible variation of lowing changing internal parameters, similar in intent
the principal point of the camera. In order for calibra- to the algorithms of [5, 6, 8]. The particular scenario
tion to be possible some constraints must be placed on of a rotating and zooming camera is common in prac-
the calibration of each image. The method works un- tice. For instance a camera at a sports arena under-
der the minimal assumption of zero-skew (rectangular goes this sort of motion as it rotates and zooms to
pixels), or the more restrictive but reasonable condi- follow a game. Unlike the algorithm of Hartley ([4]),
tions of square pixels, known pixel aspect ratio, and these methods use interation, based on the Levenberg-
known principal point. Being linear, the algorithm is Marquardt method to minimize a non-linear cost func-
extremely rapid, and avoids the convergence problems tion. The linear methods of ([4]) do not appear to be
characteristic of iterative algorithms. immediately extendable to the case of varying internal
parameters.
1 Introduction In this paper however it is shown that a simple trick,
The subject of self-calibration of a camera has re- first used in a different context in [11] transfers the ba-
ceived considerable attention, following the ground- sic calibration equation ((3) below) to one for which
breaking paper of Maybank and Faugeras [7]. This a simple linear method applies. The resulting linear
idea opens the possibility of calibration of a camera method is extremely simple involving a least-squares
in the field, without the aid of jigs, or knowledge of solution of a set of homogeneous equations in 6 un-
the position of world-points. A subsequent paper of knowns. Each image in the sequence leads to one to
Hartley ([4]) gave a method for the self-calibration of four equations, depending on the amount of assumed
a rotating, but stationary camera, for which the the- knowledge of the camera. It turns out that the linear
ory of [7] does not apply. The algorithm of [4] had the method, apart from being quicker than the iterative
advantage of being linear and hence very simple and methods gives results of comparable quality.
rapid, unlike the Maybank-Faugeras method, which
was somewhat complex. 2 Rotating cameras
The methods of these two papers required the cal- We consider a set of images taken with cameras
ibration of the camera to be fixed over a sequence all located at the same point in space, which will be
of images – no zooming was allowed. Subsequently taken to be the coordinate origin. As has been shown
interest in zooming cameras led to a method for self- in (for instance) [4] one may analyse this situation by
calibration of cameras with changing internal param- representing each of the cameras as a 3×3 matrix Pi . A
eters ([5]). These results were strengthened in [6, 8] to
1 It may be noted that these papers were all predated by
∗ This work was sponsored by DARPA contract F33615-94- the paper [3] that gave a non-iterative algorithm for two-view
C-1549 calibration in the case of known aspect ratio and principal point.

1
point in the i-th image, represented by a homogeneous 2. The square-pixel constraint : For each cam-
3-vector xi corresponds to a ray in space consisting era, s = 0 and αx = αy .
of points of the form λP−1i xi . Points on this ray are
mapped into the j-th image to a point xj = Pj P−1 i xi .
3. The known principal point constraint : For
Denoting the transformation each camera, (x0 , y0 ) = (0, 0).

If the pixels have a known aspect ratio other than

Hij = Pj P−1
i (1)
1, or the principal point is at a different know point
one sees that the i-th and j-th images are related by other than the origin, then a simple change of image
a projective transformation Hij . In practice, one may coordinates converts to one of the cases above.
compute these projective transformations between im- Now, a constraint such as the zero-skew constraint
ages by finding matching points in the two images and is not reflected in any simple way in the entries of
computing the projective transformation that relates ω∗ = KK . There seems to be no easy way to use
the points. At least four matched points are necessary (3) to develop an algorithm to compute the camera
for computing the projective transformation between calibration, enforcing constraints of this type. There-
two images. Practical methods for computing these fore, in [1, 9], iterative algorithms are proposed, us-
transformations are given in [4, 2]. ing Levenberg-Marquardt iteration to find a non-linear
The projective transformation may be related to least squares solution, parametrized directly by the en-
the calibration matrices of each of the cameras, as tries of the Ki . This may potentially cause problems
follows. Each camera matrix Pi may be decomposed with lack of convergence, or convergence to local min-
into an upper-triangular calibration matrix Ki and a ima, not to mention the greater complexity of coding.
rotation matrix Ri representing the orientation of the A simple observation, however, leads to a linear al-
camera : Pi = Ki Ri . Substituting in (1) one obtains gorithm. Taking the inverse of (3) gives

Hij = Kj Rj R−1 −1
= Kj Rij K−1 . (2) Hij − ω i H−1
ij = ω j (5)
i Ki i

Since Rij is a rotation, Rij Rij  = I. Straight-forward where ω i = Ki − K−1

i . Now, one may verify that with
computation then shows that Hij Ki Ki  Hij  = Kj Kj  . K of the form (4), with s = 0,
This formula is often written as
ω = K− K−1
 
Hij ω ∗i Hij  = ω∗j (3) 1/α2x 0 −x0 /α2x
=  0 1/α2y −y0 /α2y 
where ω ∗i = Ki Ki  is the dual image of the absolute −x0 /α2x −y0 /α2y 1 + x20 /α2x + y02 /α2y
conic ([4]) in the i-th image. This formula then repre- (6)
sents the transformation of a conic under a projective
transformation. In the case where Ki = Kj , this for- so ω represents a conic of the form
mula may be used to generate a set of linear equations
in the entries of ω ∗i which may be used to solve for (x − x0 )2 /α2x + (y − y0 )2 /α2y + 1 = 0 ,
ω ∗i , and subsequently for Ki by Choleski factorization.
This is briefly the calibration method described in [4]. the image of the absolute conic. The important points
The difficulty with extending this linear solution to to note here are
the case of a camera with varying intrinsic parame-
ters under minimal assumptions such as zero skew or Proposition 2.1. 1. If s = K12 = 0, then ω 12 = 0.
known aspect ratio is that such constraints are not eas- 2. If s = 0 and αx = αy , then ω 11 = ω 22 .
ily related to the entries of ω ∗i . Consider a calibration
matrix   3. If s = 0 and x0 = 0, then ω 13 = 0. Similarly if
αx s x0 y0 = 0 then ω 23 = 0.
K= αy y0  (4)
1 3 Generation and solution of equations
Three constraints are possible, of which the first two Select one image as being a reference image, and
were tested in detail in this paper. let H0j be the homography relating the reference im-
age to the j-th image. For j = 0, one has H00 = I.
1. The zero-skew constraint : For each camera, Let the image of the absolute conic in the refer-
s = 0. ence image be ω0 . Since it is symmetric, it may be
parametrized by its six diagonal and above-diagonal camera motion contains some component of rotation
entries, which may be denoted in some designated or- about the pricipal axis (Z-axis) of the camera. If some
der as (a1 , a2 , . . . , a6 ). Since the homographies H−1 0j Z axis motion is included, however, then results are
are known, the entries of ω j may be expressed lin- good. Thus, to obtain good results, one must either
early in terms of the entries ai of ω0 . Now each of have some Z-rotations, or else include square-pixel
the equation types in Proposition 2.1 gives one linear constraints. It should be realized that this failure
equation in the entries of ω j , hence a linear equation mode does not represent a weaknesses in the linear
in the entries ai of ω 0 . For each image each condition algorithm of this paper, but rather arises from a situ-
gives one equation, and the set of all equations may ation in which self-calibration is intrinsically unstable.
be written as Ea = 0, where a = (a1 , a2 , . . . , a6 ) ,
and each row of E represents one equation. Given at X-axis rotation. Consider the case where the
rotation is about the X axis of the camera and the
least five equations one can find a solution up to (non-
cameras have zero skew. In addition, suppose for the
essential) scale. With more than 5 equations, one finds
a least-squares solution in the usual way ([4]), that is present that the coordinate origin is at the principal
point, so that the principal point (x0 , y0 ) = (0, 0). In
by finding a that minimizes ||Ea|| subject to ||a|| = 1.
this case the calibration matrix of each camera is di-
In forming these equations, one should include
agonal, K = diag(αx , αy , 1), and the transformation
equations for the reference image. Consider the zero-
matrix H−1 
0j = K0 RX Kj is of the form
skew condition. With H00 = I, the corresponding
equation becomes simply a2 = 0, assuming that a2  
1 0 0
reprents the (1, 2) entry of ω 0 . This equation should
H−1
0j = 0 x x 
be included with the equations arising from the other
0 x x
images in the matrix E of all equations. A (possibly in-
ferior) alternative would be to parametrize ω in terms where the x values represent non-zero entries, not all
of only 5 entries, setting the (1, 2) entry to zero. Then the same. Now (5) may be written as
each image other than the reference image gives one
equation. The difference is that in this latter case, the ωj = H0j − ω 0 H−1
0j (7)
computed skew will be exactly zero in the reference
   
image but non-zero (because the equations will not be 1 0 0 a1 a2 a3 1 0 0
satisfied exactly) in all other images. This approach =  0 x x   a2 a4 a5   0 x x  .
would unreasonably single out the reference image for 0 x x a3 a5 a6 0 x x
special treatment. The same remarks apply in the (8)
square-pixel and known-principal point cases as well.
Since one requires at least 5 equations in the 6 in- From this one easily verifies that the expression for
dependent homogeneous entries of ω0 , one requires a the (1, 2) entry of ωj involves only the entries a2 and
total of 5 images to solve in the zero-skew case, 3 im- a3 of ω 0 . Assuming that a2 = 0 (the skew is zero) and
ages in the square pixel case, and only 2 images if one referring to the expression (6) for ω 0 , one deduces that
also knows the principal point. the zero-skew constraint imposes a constraint only on
By solving for the vector (a0 , a1 , . . . , a6 ) one deter- x0 , allowing one to deduce that x0 = 0. There is no
mines the value of ω 0 . The other ωj may be computed constraint on y0 , αx or αy , which may vary freely.
from (5). From this one may compute the individual Thus, although y0 = 0 is assumed in this analysis,
calibration matrices Kj by inversion and Choleski fac- there is no way this may be deduced using the zero-
torization of ω j , or else directly from ωj using (6). skew constraint.
The algorithm will fail if the computed value of α2x One may verify that the choice of coordinate sys-
or α2y is negative, which is equivalent to ω j not being tems used to compute the homographies H0j does not
positive definite. This rarely occurs in practice except materially alter the constraints imposed on the cali-
in unstable situations in which calibration is intrinsi- bration by a rotation about the X-axis, as analyzed
cally infeasible, or with inaccurate or incorrect input above.
data.
Y -axis rotation. In a similar way, if the rotation
4 Unstable configurations is about the Y axis, then the expression for (ω j )12
In the experimental evaluation of these algorithms, involves only a2 and a5 , as may be deduced in a similar
it was observed that the results were quite unstable manner. Thus, x0 , αx and αy may vary freely. If all
if only the zero-skew constraint is used, unless the rotations R0j are about the X or Y axis, then there is
no way to determine the parameters αx and αy of the shows the results of this experiment. They show that
camera. If on the other hand, one includes a square- for typical image noise levels of σ smaller than 0.5 pix-
pixel constraints, or rotations about the Z-axis, then els the algorithm performs well. The plot for the prin-
it turns out that the stability problems disappear. cipal point shows a large variation when σ = 1 pixel.
However, it is well known that the principal point is
Pan-tilt motion. A further important situation a poorly constrained parameter which easily fits its
that leads to failure is that of a camera mounted on value to the noise. Note also that the small devia-
an alt-azimuth mounting, allowing panning about a tion of the camera (5◦ ) represents a very demanding
vertical axis, and tilting about a horizontal elevation test for the algorithm. With wider excursions of the
axis. In this case, the rotation of the camera with re- camera, better results would be achieved ([4]).
spect to a horizonal reference position may be written In figure 1 we also show the performance of the
in the form Rx Ry , (pan followed by tilt). One easily linear algorithms versus the iterative non-linear algo-
verifies that this rotation matrix has the distinguish- rithm described in [1]. The non-linear algorithm was
ing property that the (1, 2) element is zero. Mimicking run imposing equivalent constraints of zero skew and
the analysis for X-axis rotation above, and mutiplying square pixels on the intrinsic parameters to minimize a
out as in (7) reveals that the (1, 2) element of ωj does non-linear cost function using a Levenberg-Marquardt
not depend on the element a4 = (ω 0 )22 . According to method. For clarity we have only shown the result
(6), this implies that αy is not determined. Further- when the image noise level of σ = 0.5 pixels which
more, although element a5 of ω 0 may be determined, shows that the results obtained with the linear algo-
it represents the quotient y0 /α2y . Hence y0 depends on rithm are comparable with those from the non-linear
αy , and so can not be determined either. iterative method.
5 Experiments 5.2 Real image sequences
In this section we present experimental results ob- The image sequences used in our experiments were
tained using the linear algorithm on both synthetic taken using a camera with a zoom lens mounted on
and real image sequences. a Yorick stereo head/eye platform [10]. The camera
5.1 Synthetic data was rotated using one of the two independent vergence
Experiments were first carried out with synthetic axes to pan the camera, and the common elevation
data to evaluate the performance of the linear algo- axis to tilt it. Note therefore that the motion selected
rithm using the zero-skew and the square-pixel con- for these experiments falls in the case of the pan-tilt
straints described in section 2. The data was created degeneracy described in section 4. The mechanics of
to simulate a camera with a zoom lense providing a our head do not permit rotations about the Z axis.
total focal length range of 12.5 mm to 35 mm. A cloud This experiment will therefore illustrate how the skew
of 250 points was randomly generated within a con- zero constraint does not resolve this ambiguity and the
fined spheric space of diameter 2 m lying in front of the square pixels constraint must be imposed.
rotating camera at a distance of 5 m. The points were Two image sequences were taken. In the first se-
then projected onto each of the image planes arising quence, the focal length of the camera remained fixed,
from the different orientations of the camera and the while the pan and the tilt of the camera were varied to
location of each image point was then perturbed in a perform a circular trajectory. In the second sequence,
random direction by a distance governed by a Gaus- the focal length of the camera was set to increase lin-
sian distribution with zero mean and standard devia- early, using the controlled zoom lens, while the camera
tion σ measured in pixels. The size of the image planes performed a similar circular motion. The encoders of
was 384×288 pixels. The skew of the image axes was the head/eye platform provided ground truth values
taken to be zero, the aspect ratio of the image pix- for the pan and tilt angles of the camera which are
els equal to one and the principal point was asumed accurate to 0.01 of a degree. The servo control of the
to be located at the centre of the image. The camera zoom lens provided ground truth values of the position
motion was such that the principal ray described a cir- of the zoom lens for each frame in the image sequence.
cular trajectory of radius θ = 5◦ measured from the The camera was then calibrated, using an accurately
positive Z axis. To avoid the degenerate configuration machined calibration grid and classical calibration al-
described at the end of section 4 a small amount of cy- gorithm, to obtain ground truth values for the inter-
clorotation about the principal axis was then added to nal parameters at each of the different positions of the
each camera position. The focal length of the camera zoom lens. Radial lens distortion was modelled using
increased linearly throughout the sequence. Figure 1 a one parameter model and the images were appropri-
 FOCAL LENGTH - var focal length FOCAL LENGTH - var focal length
5000 5000
4500 sigma 0.1 4500 sigma 0.1
sigma 0.25 sigma 0.25
Focal length (pixels)

Focal length (pixels)

4000 sigma 0.5 4000 sigma 0.5
3500 LM sigma 0.5 3500 LM sigma 0.5
sigma 0.75 sigma 0.75
3000 sigma 1.0 3000 sigma 1.0
2500 2500
2000 2000
1500 1500
1000 1000
0 5 10 15 20 25 30 0 5 10 15 20 25 30
Frame Frame
PRINCIPAL POINT - var focal length PRINCIPAL POINT - var focal length

250 sigma 0.1 250 sigma 0.1

sigma 0.25 sigma 0.25
sigma 0.5 sigma 0.5
200 LM sigma 0.5 200 LM sigma 0.5
v_o (pixels)

v_o (pixels)

sigma 0.75 sigma 0.75

150 sigma 1.0 150 sigma 1.0

100 100

50 50

0 0
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
u_o (pixels) u_o (pixels)

ASPECT RATIO- var focal length

1.3
sigma 0.1
1.2 sigma 0.25
sigma 0.5
LM sigma 0.5
Aspect ratio

1.1 sigma 0.75

sigma 1.0
1

0.9

0.8
0 5 10 15 20 25 30
Frame

Figure 1: Calibration results with synthetic data in presence of various degrees of image noise, with one run at
each noise level. Computed values for the focal length (top), the location of the principal point (middle) and
the aspect ratio (bottom) imposing the zero skew (left) and square-pixels (right) constraints. Results obtained
using the non-linear iterative method of [1] imposing the zero skew and the square-pixels constraint are shown for
σ = 0.5. The aspect ratio was set to one by the algorithm when the square-pixels constraint was used
Figure 2: Mosaics constructed from the two bookshelf sequences during which the camera panned and tilted while
the focal length remained fixed (left) and was varied (right).

ately warped to correct for this factor. case of critical rotation sequences for which the cali-
The homographies that relate corresponding points bration problem is inherently unstable. This serves as
between views were computed in two stages. First, the a warning that the data used does not support a use-
inter-image homographies were computed from cor- ful estimate of the cameras’ calibration parameters. It
responding corners and second, they were refined by is probable that in these cases a calibration estimate
minimizing the global reprojection image error using given by any other algorithm, such as the previous
a bundle-adjustment technique [4, 2]. Figure 2 shows iterative algorithms, would be virtually useless.
the mosaics constructed by registering both image se- The linear methods do not apply in some cases for
quences. which the iterative algorithms may be used, such as
When the zero skew constraint was imposed very fixed, but unknown aspect ratio, or fixed, but un-
poor results were obtained for the calibration param- known principal point. The common cases of zero
eters. The fundamental ambiguity described in sec- skew and known aspect ratio are covered, however. In
tion 4 was confirmed by observing that the two small- practice, skew is almost always zero, and the aspect
est singular values of the equation matrix E were very ratio is usually known to be one, or is available from
close to zero, implying that there is a one parameter a spec-sheet for the camera. In any case the aspect
family of possible solutions to the calibration. Impos- ratio is essentially invariant, and could be determined
ing the zero skew constraint also failed to provide a off-line. Further experiments (not explained in detail
solution when used in the non-linear iterative mini- here) show that the linear algorithm can be used in a
mization. linear search over a range of feasible aspect ratios to
Figure 3 shows the results obtained with the lin- determine the aspect ratio that gives the best fit to
ear algorithm imposing the square-pixels constraint. the data.
The results confirm the good performance of the lin-
ear method, which in these particular experiments give References
better estimates than the iterative algorithm.
[1] L. de Agapito, E. Hayman, and I. D. Reid Self-
6 Conclusion calibration of a rotating camera with varying intrinsic
parameters. In Proc. British Machine Vision Confer-
The key idea of the paper is to focus on the image of ence, pages 105–114, 1998.
the absolute conic rather than its dual. This leads to a
linear algorithm for the constrained calibration prob- [2] D. Capel and A. Zisserman. Automated mosaicing
lem, rather than the iterative algorithms previously with super-resolution zoom. In Proceedings of the
reported. The linear algorithm is extremely simple IEEE Conference on Computer Vision and Pattern
to implement and performs very well compared with Recognition, pages 885–891, June 1998.
iterative algorithms, ofter giving better results. The [3] R. I. Hartley Estimation of relative camera positions
method fails in the case where the computed image of for uncalibrated cameras. In Proc. European Confer-
the absolute conic is not positive-definite. However, ence on Computer Vision, LNCS 588, pages 579–587.
this did not occur in our experiments, except in the Springer-Verlag, 1992.
 FOCAL LENGTH - fixed focal length FOCAL LENGTH - var focal length
950
ground truth 2400 ground truth
900 Linear Linear
2200
Focal length (pixels)

Focal length (pixels)

850 L M fixed pp L M fixed pp
L M var pp 2000 L M var pp
800 1800
750 1600
700 1400
650 1200

600 1000
0 5 15 1020 25 30 0 5 10
15 20 25 30
Frame Frame
PRINCIPAL POINT - fixed focal length PRINCIPAL POINT - var focal length
170
ground truth 250 ground truth
165 Linear Linear
160 L M fixed pp L M fixed pp
L M var pp 200 L M var pp
v_o (pixels)

155 v_o (pixels)

150
150
100
145
140 50

135 0
160 170 180 190 200 210 0 50 100 150 200 250 300 350
u_o (pixels) u_o (pixels)
MOTION - fixed focal length MOTION - var focal length
24 22
ground truth 21 ground truth
22 linear linear
LM fix pp 20 LM fix pp
20 LM var pp LM var pp
19
pan (deg)

tilt (deg)

18 18
17
16
16
14
15
12 14
-10 -8 -6 -4 -2 0 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1
tilt (deg) pan (deg)

Figure 3: Ground truth and computed values for the focal length (top), the location of the principal point
(middle) and the motion of the camera (bottom) for the fixed focal length (left) and the variable focal length
(right) bookshelf sequences. Results are shown for (i) the linear algorithm imposing the square-pixels constraint
(ii) the Levenberg-Marquardt algorithm imposing the square-pixels constraint and (iii) the Levenberg-Marquardt
algorithm imposing both the square-pixels and the fixed principal point constraints. For visualization purposes,
the motion was represented by plotting pan versus elevation angles. Also for clarity, only the central section of
the image is shown in the plot depicting the principal point.
Note that the value of the focal length obtained from the linear algorithm is more accurate than the LM algorithm
run using the same calibration assumptions (LM-var-pp). However, in both sequences, focal length estimation is
improved by using the linear estimate as a starting estimate for iteration fixing the principal point (LM-fixed-pp).
The principal point (in reality invariant during the fixed focal length sequence, and varying only slightly during
the variable focal length sequence) appears to be better estimated by the linear algorithm.
 [4] R. I. Hartley. Self-calibration of Stationary Cameras.
International Journal of Computer Vision, volume 22,
number 1, pages 5–23, February, 1997.
[5] A. Heyden and K. Åström. Euclidean reconstruction
from image sequences with varying and unknown fo-
cal length and principal point. In Proceedings of the
IEEE Conference on Computer Vision and Pattern
Recognition, 1997.
[6] A. Heyden and K. Åström. Minimal conditions on
intrinsic parameters for euclidean reconstruction. In
Proc. Asian Conference on Computer Vision, Hong
Kong, 1998.
[7] S. Maybank and O. Faugeras. A theory of self-
calibration of a moving camera. International Journal
of Computer Vision, 8(2):123–151, 1992.
[8] M. Pollefeys, R. Koch, and L. Van Gool. Self cali-
bration and metric reconstruction in spite of varying
and unknown internal camera parameters. In Proc.
6th International Conf. on Computer Vision, Bom-
bay, pages 90–96, 1998.
[9] Yongduek Seo and Ki Sang Hong, Auto-calibration of
a Rotating and Zooming Camera, In Proc. of IAPR
workshop on Machine Vision Applications (MVA’98),
Makuhari, Chiba, Japan, Nov. 17 – 19, 1998.
[10] P. M. Sharkey, D. W. Murray, S. Vandevelde, I. D.
Reid, and P. F. McLauchlan. A modular head/eye
platform for real-time reactive vision. Mechatronics,
3(4):517–535, 1993.
[11] A. Zisserman, D. Liebowitz, and M. Armstrong. Re-
solving ambiguities in auto-calibration. Philosophi-
cal Transactions of the Royal Society of London. Vol
356(1740):1193–1211, 1998.