Geometrical Analysis of Polynomial Lens Distortion Models

Abstract

Polynomial functions are a usual choice to model the nonlinearity of lenses. Typically, these models are obtained through physical analysis of the lens system or on purely empirical grounds. The aim of this work is to facilitate an alternative approach to the selection or design of these models based on establishing a priori the desired geometrical properties of the distortion functions. With this purpose we obtain all the possible isotropic linear models and also those that are formed by functions with symmetry with respect to some axis. In this way, the classical models (decentering, thin prism distortion) are found to be particular instances of the family of models found by geometric considerations. These results allow to find generalizations of the most usually employed models while preserving the desired geometrical properties. Our results also provide a better understanding of the geometric properties of the models employed in the most usual computer vision software libraries.

Appendix: Proofs of Theorems

1.1 Lemma to Prove Theorem 1

Lemma 1

Let V be a complex vector space with a basis $\left\{ u_{1},\ldots ,u_{p},v_{1},\ldots ,v_{q}\right\}$ and a complex endomorphism $f:V\rightarrow V$ given by

$$\begin{aligned} f(u_{i})&=\lambda u_{i},\,i=1,\ldots ,p\\ f(v_{j})&=\bar{\lambda }v_{j},\,j=1,\ldots ,q\\ \lambda&=\lambda _{1}+i\lambda _{2}\in \mathbb {C}{\setminus }\mathbb {R}. \end{aligned}$$

Then, the irreducible invariant subspaces of f with respect to the realification $V_{\mathbb {R}}$ of V (i.e., the consideration of V as a real vector space by restricting the scalars to the real numbers) are of the form

$$\begin{aligned} \mathcal{S}_{(\alpha :\beta )}=\left\{ \gamma \sum _{i=1}^{p}\alpha _{i}u_{i}+\bar{\gamma } \sum _{j=1}^{q}\bar{\beta }_{j}v_{j}:\gamma \in \mathbb {C}\right\} , \end{aligned}$$

where $(\alpha :\beta )$ is an abbreviation for

$$\begin{aligned} (\alpha _{1}:\ldots :\alpha _{p}:\beta _{1}:\ldots :\beta _{q})\in \mathbb {P}^{p+q-1}. \end{aligned}$$

Besides, if $\left( \alpha :\beta \right) \not =\left( \alpha ':\beta '\right)$ then

$$\begin{aligned} \mathcal {S}_{\left( \alpha :\beta \right) }\cap \mathcal {S}_ {\left( \alpha ':\beta '\right) }=\left\{ 0\right\} . \end{aligned}$$

Proof

A basis for $V_{\mathbb {R}}$ is given by

$$\begin{aligned} \left\{ u_{1},iu_{1},\ldots ,u_{p},iu_{p},v_{1},iv_{1},\ldots ,v_{q},iv_{q}\right\} , \end{aligned}$$

so we can identify $V\approx \mathbb {C}^{p+q}$ and $V_{\mathbb {R}}\approx \mathbb {R}^{2(p+q)}.$ With this identification, the matrix $\mathtt {M}$ of f as an endomorphism of $\mathbb {R}^{2(p+q)}$ is block-diagonal with p blocks

$$\begin{aligned} \mathtt {B}=\begin{pmatrix}\lambda _{1} &{}\quad -\lambda _{2}\\ \lambda _{2} &{}\quad \lambda _{1} \end{pmatrix} \end{aligned}$$

and q blocks $\mathtt {B}^{\top }$. From the diagonalization

$$\begin{aligned} \mathbf {\mathtt {B}}&=\mathtt {U}\begin{pmatrix}\bar{\lambda } &{} 0\\ 0 &{} \lambda \end{pmatrix}\bar{\mathtt {U}}^{\top },\text { where}\\ \mathtt {U}&=\frac{\sqrt{2}}{2}\begin{pmatrix}1 &{} 1\\ i &{}\quad -i \end{pmatrix}, \end{aligned}$$

we easily obtain a diagonalization of $\mathtt {M}$ and from it we see that the eigenvectors of this matrix associated with the eigenvalue ${\lambda }$ are of the form

$$\begin{aligned} \mathbf {w}&=\left( \alpha _{1},-i\alpha _{1},\ldots , \alpha _{p},-i\alpha _{p},\beta _{1},i\beta _{1}, \ldots ,\beta _{q},i\beta _{q}\right) ^{\top }\nonumber \\&\alpha _{i},\beta _{j}\in \mathbb {C}, \end{aligned}$$

(41)

and those associated with the eigenvalue $\bar{\lambda }$ are their conjugates. Given a non null vector $\mathbf{w}=\mathbf {w}_{1}+i\mathbf {w}_{2}$ of this form, $\mathbf {w}$ and $\bar{\mathbf {w}}$ span an invariant subspace of $\mathtt {M}$ whose realification admits the basis $\left\{ \mathbf {w}_{1},\mathbf {w}_{2}\right\}$. Denoting $\alpha _{i}=a_{i}+ib_{i}$, $\beta _{j}=c_{j}+id_{j}$, we have

$$\begin{aligned} \mathbf {w}_{1}&=\left( a_{1},b_{1},\ldots ,a_{p},b_{p},c_{1},-d_{1}, \ldots ,c_{q},-d_{q}\right) ^{\top }\\ \mathbf {w}_{2}&=\left( b_{1},-a_{1},\ldots ,b_{p},-a_ {p},d_{1},c_{1},\ldots ,d_{q},c{}_{q}\right) ^{\top }. \end{aligned}$$

The elements of this subspace have coordinates of the form

$$\begin{aligned} r_{1}\mathbf {w}_{1}+r_{2}\mathbf {w}_{2},\,r_{1},r_{2}\in \mathbb {R}, \end{aligned}$$

that correspond to the elements of V

$$\begin{aligned} r_{1}\left( \sum _{i=1}^{p}\underbrace{\left( a_{i}+ib_{i}\right) }_ {\alpha _{i}}u_{i}+\sum _{j=1}^{q}\underbrace{\left( c_{j}-id_{j}\right) } _{\bar{\beta }_{j}}v_{j}\right) \\ +r_{2}\left( \sum _{i=1}^{p}\underbrace{\left( b_{i}-ia_{i}\right) }_ {-i\alpha _{i}}u_{i}+\sum _{j=1}^{q}\underbrace{\left( d_{j}+ic_{j}\right) } _{i\bar{\beta }_{i}}v_{j}\right) \\ =\underbrace{(r_{1}-ir_{2})}_{\gamma }\sum _{i=1}^{p}\alpha _{i}u_{i} +\underbrace{(r_{1}+ir_{2})}_{\bar{\gamma }}\sum _{j=1}^{q}\bar{\beta }_{j}v_{j}, \end{aligned}$$

and so the subspace generated by $\mathbf {w}_{1}$ and $\mathbf {w}_{2}$ is of the form $\mathcal{S}_{(\alpha :\beta )}$, as required. Finally, let us see that all the irreducible subspaces are of this form. Since $\mathtt {M}$ is real and without real eigenvectors, its irreducible invariant subspaces are bidimensional. Therefore, let us consider an invariant bidimensional real subspace $W\subset \mathbb {R}^{2(p+q)}\subset \mathbb {C}^{2(p+q)}$. Let $W^{\mathbb {C}}=W\oplus iW$ be the associated complex vector subspace. The eigenvalues of the restriction to $W^{\mathbb {C}}$ of the endomorphism given by $\mathtt {M}$ must be complex conjugated and so they are $\left\{ \lambda ,\bar{\lambda }\right\}$. The eigenvector $\mathbf {x}=\mathbf {x}_{1}+i\mathbf {x}_{2}$ associated with the first eigenvalue must be of the form (41). The endomorphism being real, the conjugate vector $\bar{\mathbf {x}}$ must belong to the invariant subspace $W^{\mathbb {C}}$ and so the real vectors $\mathbf {x}_{1},\mathbf {x}_{2}\in W$, and therefore, W is of the form $\mathcal{S}_{(\alpha :\beta )}$ as required.

As for the last assertion, just observe that if

$$\begin{aligned} \gamma \sum _{i=1}^{m}\alpha _{i}\mathbf {u}_{i}+ \bar{\gamma }\sum _{j=1}^{n}\bar{\beta }_{j}\mathbf {v}_{j} =\gamma '\sum _{i=1}^{m}\alpha '_{i}\mathbf {u}_ {i}+\bar{\gamma '}\sum _{j=1}^{n}\bar{\beta }'_{j}\mathbf {v}{}_{j} \end{aligned}$$

then, the vectors being a base, we have that $\gamma \alpha _{i}=\gamma '\alpha '_{i}$ and $\bar{\gamma }\bar{\beta }_{j}=\bar{\gamma }'\bar{\beta }_{j}'$ and so $(\alpha _{1}:\ldots :\alpha _{m}:\beta _{1}: \ldots :\beta _{n})=(\alpha '_{1}:\ldots :\alpha '_{m}:\beta '_{1}:\ldots :\beta '_{n})$. $\square$

1.2 Proof of Theorem 2

If $\mathcal{S}$ is a subspace of $\mathcal {P}^{(n)}$ generated by some set of monomials and $f\in \mathcal {P}^{(n)}$, we define the projection$P_{\mathcal {S}}(f)$ as the polynomial obtained by keeping in f only the monomials in $\mathcal{S}$. Therefore, we have a linear mapping

$$\begin{aligned} P_{\mathcal {S}}:\mathcal {P}^{(n)}\longrightarrow \mathcal {S}. \end{aligned}$$

Now we can proceed to the proof of theorem 2.

Proof

We consider displacement functions expressed as complex polynomials in the variables z and $\bar{z}$,

$$\begin{aligned} f(z,\overline{z})=\sum _{(k,l)\in G^{(n)}}\gamma _{kl}z^{k}\bar{z}^{l}\in \mathcal{P}^{(n)} \end{aligned}$$

with reflection symmetry with respect to some axis. Therefore, the coefficients can be obtained through the parameterization (26).

Let us suppose that we have a real vector space L of functions of this form which, at the same time, is invariant under the action of the unitary group SO(2) according to (20), i.e.,

$$\begin{aligned} \gamma _{kl}\mapsto e^{i\theta m}\gamma _{kl}. \end{aligned}$$

Given an element f of L there must exist an element $f_{0}$ of its orbit under the action of SO(2) with reflection symmetry with respect to the horizontal axis, i.e., with real coefficients $\gamma _{kl}=a_{kl}\in \mathbb {R}$. Therefore, L is determined by its subset $L_{\mathbb {R}}$ of its elements with real coefficients.

Denoting $m=k+l-1$ and $m'=k'+l'-1$, let us consider two pairs (k, l) and $(k',l')$ such that

$$\begin{aligned} mm'\not =0,\,\,\text {and }|m|\not =|m'|. \end{aligned}$$

Let us see that $L_{\mathbb {R}}$ cannot contain a polynomial with both coefficients $a_{kl}\ne 0$ and $a_{k'l'}\ne 0$. We denote by $\mathcal{S}$ the set of polynomials only with monomials $z^{k}\bar{z}^{l},\,z^{k'}\bar{z}^{l'}$. Since L is a linear subspace, so is its image by the linear mapping $P_{\mathcal{S}}$, that cancels all monomials but $z^{k}\bar{z}^{l}$ and $z^{k'}\bar{z}^{l'}$. If such a polynomial existed, both

$$\begin{aligned} c\left( a_{kl}z^{k}\bar{z}^{l}+a_{k'l'}z^{k'}\bar{z}^{l'}\right) \end{aligned}$$

and

$$\begin{aligned} a_{kl}e^{i\theta m}z^{k}\bar{z}^{l}+a_{k'l'}e^{i\theta m'}z^{k'}\bar{z}^{l'} \end{aligned}$$

would belong to this image for any $c,\theta \in \mathbb {R}$, so that its sum

$$\begin{aligned} a_{kl}\left( c+e^{i\theta m}\right) z^{k}\bar{z}^{l}+a_{k'l'}\left( c+e^{i\theta m'}\right) z^{k'}\bar{z}^{l'} \end{aligned}$$

must also be in the image, and therefore satisfy (29), so that

$$\begin{aligned} \left( \frac{c+e^{i\theta m}}{c+e^{-i\theta m}}\right) ^{2m'}=\left( \frac{c+e^{i\theta m'}}{c+e^{-i\theta m'}}\right) ^{2m} \end{aligned}$$

for any $c,\theta \in \mathbb {R}$. If this were true we would have that

$$\begin{aligned} F(z)=\left( \frac{c+z^{m}}{c+z^{-m}}\right) ^{2m'}= \left( \frac{c+z^{m'}}{c+z^{-m'}}\right) ^{2m}=G(z), \end{aligned}$$

(42)

but

$$\begin{aligned} \left( \frac{d^{3}F}{\mathrm{d}z^{3}}-\frac{d^{3}G}{\mathrm{d}z^{3}}\right) (1)= -\frac{4\,c{\left( c-1\right) }}{{\left( c+1\right) } ^{3}}{\left( m'^{2}-m^{2}\right) }mm'\not =0 \end{aligned}$$

unless $|m|=|m'|$ or $mm'=0$, and therefore, we have found a contradiction.

Let us see now that the image of $L_{\mathbb {R}}$ by the mapping $P_{\mathcal{W}}$, that only keeps the non-invariant monomials of each polynomial cannot be of dimension larger than one. It is easy to check that a vector space is of dimension larger than one if and only if some projection onto a coordinate plane has dimension larger than one. In our case, this means that there are two different monomials $z^{k}\bar{z}^{n-k},\,z^{k'}\bar{z}^{n'-k'}$ such that $L_{\mathbb {R}}$ contains polynomials

$$\begin{aligned} \ldots +1z^{k}\bar{z}^{l}+0z^{k'}\bar{z}^{l'}+\ldots \end{aligned}$$

and

$$\begin{aligned} \ldots +0z^{k}\bar{z}^{l}+1z^{k'}\bar{z}^{l'}+\ldots \end{aligned}$$

with $m=k+l-1$, $m'=k+l-1$, $mm'\ne 0$, and using first the isotropy of L and then its linearity, we see that L must contain a polynomial

$$\begin{aligned} \ldots +e^{i\theta m}z^{k}\bar{z}^{l}+e^{i\theta 'm'}z^{k'}\bar{z}^{l'}+\ldots \end{aligned}$$

for any $\theta ,\theta '$. And applying (29) to the coefficients of these monomials we would have for all $\theta ,\theta '\in \mathbb {R}$,

$$\begin{aligned} \left( \frac{e^{i\theta m}}{e^{-i\theta m}}\right) ^{2m'}&=\left( \frac{e^{i\theta 'm'}}{e^{-i\theta 'm'}}\right) ^{2m}\\ \Leftrightarrow e^{4i\theta mm'}&=e^{4i\theta 'mm'}, \end{aligned}$$

which is not true unless $mm'=0$.

Therefore, if $P_{\mathcal{W}}\left( L_{\mathbb {R}}\right)$ contains polynomials with some monomial $z^{k}\bar{z}^{l}$ with $m=k-l-1\ne 0$, $L_{\mathbb {R}}$ must be one-dimensional and, since it can only contain polynomials with monomials with $k-l-1\in \left\{ -m,m\right\}$, it must be of the form

$$\begin{aligned} P_{\mathcal{W}}\left( L_{\mathbb {R}}\right)&=\left\{ \alpha \left( f+g\right) :\alpha \in \mathbb {R}\right\} , \end{aligned}$$

where $f\in \mathcal{P}_{m}^{(n)}$, $g\in \mathcal{P}_{-m}^{(n)}$ are polynomials with real coefficients, so that the projection of L onto the space of non-invariant monomials is

$$\begin{aligned} P_{\mathcal{W}}\left( L\right)&=\left\{ \alpha \left( e^{im\theta }f+e^{-im\theta }g\right) e^ {im\theta }:\alpha \in \mathbb {R}\right\} , \end{aligned}$$

(43)

that corresponds to $\mathcal {M}_{m}^{(n)}\left[ f,g\right]$ in (23) with $\gamma =\alpha e^{im\theta }$.

So we have the following possibilities:

• (a) If L does not contain polynomials with invariant monomials, it must of the form (43),

• (b) If L only contains polynomials with invariant monomials, L can be any linear subspace of invariant polynomials with real coefficients.

• (c) Finally, if L contains polynomials with invariant monomials and polynomials with non-invariant monomials, since L is an invariant subspace it must contain an irreducible subspace of non-invariant monomials that must be of the form (43), and only one. Therefore, L must also contain its projection onto the space of invariant polynomials, and consequently, L is the direct sum of a space of the form (43) and a linear space of invariant polynomials with real coefficients. $\square$