1. Introduction

Specular freeform surfaces are recently more widely used in fields, such as optics, aerospace and MEMS/NEMS. Interferometry is an accurate non-contact method for measuring this type of surface. However, interferometric methods are complicated and expensive to operate [1–3]. Recently, deflectometry [1, 4] is considered to be a serious competitor to interferometry [5]. Deflectometry measures local slope of specular surfaces through utilization of the deformation of a sample pattern after reflection from a tested surface. By integrating [6–8] the calculated slope data, the 3D topography can be reconstructed.

In a deflectometry measurement, slope value calculation and the integration step are known to be sensitive to system errors [5, 9]. Therefore an accurate calibration for deflectometry is necessary and important. For achieving this, some research efforts on the calibration of phase measuring deflectometry (PMD) or fringe-reflection 3D measurement have been made [10–12]. In the method of PMD [1], a stepwise calibration was proposed, which contains two steps. The first step is to calibrate the internal parameters of a CCD camera. The second step is to compute the geometric relation between CCD and light source. For this combined method, recently, Lei Huang [13] proposed an optimization method to calibrate a CCD camera by using an active phase target, which can allow the root mean square of reprojection error to be reduced to less than 1/1000 a pixel. Additionally, Y. Xiao [12] describes a system of geometrical calibration to optimize the relation between the CCD camera and LCD by using a markerless flat mirror. These two steps can make each procedure work flexibly and accurately, while this stepwise method has a shortage of error propagation, because each step has to individually deal with the imperfect data. In order to reduce this affection, E. Olesch [10] proposed a self-calibration method, which can deal with the inaccurate data and reduce the global accuracy to below 1 µm for a planar mirror on an $80mm\times 80mm$ field. Although these calibration methods can work well in the method for PMD or fringe reflection 3D measurement, a systematic accurate calibration method for stereo deflectometry has not been discussed.

In this paper, an iterative optimization calibration method for parameters of stereo deflectometry is proposed by combining a stepwise method and self-calibration method. A stepwise method is used to calculate the initial parameters. The self-calibration method is used to optimize the final result. A detailed description is given and initial test results are discussed.

2. Principle

For presenting the calibration process, a projection model of a stereo deflectometry system is utilized, as shown in Fig. 1, where fringe patterns are projected on the LCD and then are captured by the two CCD cameras via the flat reference mirror. When taking the calibration, the flat mirror will be placed at three or more different positions, where both of the two CCD cameras can capture the patterns via this mirror. When taking the measurement, the flat mirror will be replaced by the detected specular surface. For simplicity, the frames of the LCD screen and CCD cameras are defined as {L}, {C1} and {C2}, respectively. {VL} is a virtual LCD screen, which is the image of the LCD.

 

Fig. 1 Projection model of stereo deflectometry

Download Full Size | PDF

The whole calibration process is carried out based on the following steps: initial calculation of each CCD camera internal parameters, initial calculation of the geometrical relation between the LCD and the CCD, and overall optimization for all these parameters. At each step, the iteration process is operated to improve the accuracy of the values obtained.

2.1 Initial calculation

During the calibration, the reference mirror is placed at three or more different positions. At each position, sinusoidal fringe patterns displayed on the LCD screen are simultaneously reflected into the two CCD cameras. For calculating the internal parameters of CCD cameras, {VL} is defined as the right-hand frame though it is the image of the real LCD screen. Therefore when the flat mirror is located at different positions, it can be assumed that it is the virtual control points $p_1\text{'}$,$p_2\text{'}$ that are virtually mapped into the camera {C1} and {C2}. Then a traditional camera calibration method can be used. Here assuming the imaging model of CCD cameras can be expressed by a nonlinear mathematical function [14] $f$, then the image points in camera imaging plane can be expressed as

Where $p\text{'}$ represents the virtual control points in the virtual LCD frame {VL}, and $m$ are the image points in the camera frame. Based on global pose estimation algorithm [14], camera internal parameters $F_C$, external parameters $R_{VLC}$,$T_{VLC}$between the virtual LCD screen and camera {C1} and {C2} can be estimated respectively by assessing the reprojection of the control points in frame {VL} onto the CCD image plane, as shown in Eq. (2)

Where $\Vert m-f(p\text{'})\Vert$means the sum of the squared residuals. For achieving accurate initial CCD parameters, Eq. (2) is iterated until it reaches a minimum or the iteration is finished. At each iteration, image points satisfying the following condition will be identified to be set invalid.

Where $abs$ means the absolute value, and $RMS$ represents the root mean square value of the reprojection errors of the control points.

In fact, the fringe patterns captured by cameras are from the real LCD screen {L}, therefore it is the real image $p_1,p_2$ that is transformed into the CCD frame {C1} and {C2}. According to the geometric relation, the relation between $R_{VLC}$,$T_{VLC}$and $R_{LC}$,$T_{LC}$can be described using the following expression

Where $n_C$is the mirror normal vector expressed in the camera frame, $d$ is the distance between the flat mirror and the camera along $n_C$ vector, the rotation matrix $R_{LC}$ and translation vector $T_{LC}$ are used to describe the geometrical relation from frame {L} to frame {C}. Matrix $I_3$ is the $3\times 3$ identity matrix, and $e_3={\left[\begin{array}{ccc}0& 0& 1\end{array}\right]}^T$. According to the P3P algorithm [11, 15] and Mahalanobis [15] distance criterion, transformation matrix $R_{L{C}_1}$,$T_{L{C}_1}$ between LCD and CCD 1, $R_{L{C}_2}$,$T_{L{C}_2}$ between LCD and CCD 2 can be obtained respectively.

Here the initial CCD internal parameters $F_C$, and initial geometric relation between LCD and CCD are obtained separately. For obtaining accurate estimation, the iteration operation is applied to optimize all the initials via one cost function, which is shown in Eq. (5).

Where, $f$ represents the nonlinear mathematical function of the CCD imaging model. $R_{VLC},T_{VLC}$ is the camera external matrix corresponding to the virtual LCD system {VL} but is calculated on the basis of the real LCD system {L}, which can be calculated via the expression of Eq. (4). $F_C{}^{*}$ are the optimized CCD parameters and $R_{LC}{}^{*},T_{LC}{}^{*}$ are the optimized geometric relation. Equation (5) is assessed by evaluating the reprojection of control points $p$. These control points are located at the real LCD plane {L} but not from {VL}. To make Eq. (5) converge and to obtain accurate estimation, image points satisfying the condition of Eq. (6) will be identified and will be set invalid.

Until now, initial CCD parameters and geometrical relation from LCD to each CCD have been estimated, while the parameters in frame {C1} are independent of those in frame {C2}.

Actually, when the reference mirror locates at one position, fringe patterns shown in the LCD are simultaneously reflected into camera {C1} and {C2}, which means the normal vector of the reference mirror is expressed in frame {C1} should be the same as the value from frame {C2} if the two vectors are expressed in the same coordinate system. Based on this, the calibration result can be more accurate if all the parameters are considered in one optimization function.

2.2 Optimization function

We define $d_p{n}_L$ as an intermediate variable, where $n_L$ is the mirror normal vector expressed in the LCD frame, $d_p$ is the distance between the flat mirror and the LCD along its normal vector. On the basis of Eq. (4) and geometric relation of the whole system, $n_L$ and $d_p$ can be averaged as the following formula

Where $n_{C1}$ is the mirror normal vector expressed in camera {C1} frame, $d_1$ is the distance between the reference mirror and camera {C1} along $n_{C1}$ vector. $R_{LC1}$,$T_{LC1}$ are the transformation matrix described from frame {L} to frame {C1}. Similar parameters are expressed in camera {C2} frame.

Merging Eq. (7) and Eq. (5), an optimization cost function can be constructed as

Where $m_1$ and $m_2$ are the image coordinates at frame {C1} and {C2}, and $f^{*}$ represents the transformation of the control points from frame {L} to each CCD image plane. The other parameters have been stated above, and can be calculated by Eq. (9).

The cost function of Eq. (8) is determined by simultaneously assessing the reprojection of the control points onto the {C1} and {C2} image planes, and can be resolved with use of the Levenberg-Marquardt algorithm. Still, for obtaining accurate calibration results and to reduce the affection of outlier points, this cost function is iterated by eliminating some invalid points at each iteration. And the whole calibration workflow is shown in Fig. 2.

 

Fig. 2 Calibration flowchart

Download Full Size | PDF

3. Experiments

3.1 Calibration work

To verify the methods proposed above, a practical system calibration is carried out. The two cameras to be calibrated are both CCD sensors (imaging source with a resolution of $1280\times 960$ and pixel pitch of 3.5 µm) with a 12 mm lens. An LCD monitor (Dell E151Fpp with a resolution of $1024\times 768$ pixels and pixel pitch of 0.297 mm) is used as the active target. The whole system is shown Fig. 3 (a).

 

Fig. 3 (a) Stereo deflectometry system. (b) One captured fringe pattern by camera 1. (c) Same pattern captured by camera 2.

Download Full Size | PDF

In the calibration procedure, a reference mirror with 4 inch diameter (flatness $<\lambda /20$) is placed at 9 different positions. At each position, sinusoidal patterns with fringe numbers of 132, 143 and 144 in the x and y direction are projected, and the finest fringe period in either x or y direction is 8 pixels. Figure 3(b) and 3(c) shows one fringe pattern captured by camera 1 and camera 2. For reducing the LCD and CCD nonlinear influence and obtaining precise phase values, an eight-frame phase shifting technique is used for each frequency. Figure 4 shows one unwrapped phase result. After using eight-frame shifting and heterodyne phase unwrapping algorithm, the RMS (root mean square) value of the phase error is reduced to about 0.006 radians.

 

Fig. 4 One unwrapped phase and its corresponding RMSE (root mean square error) in CCD 1. (a) Along x direction. (b) Along y direction

Download Full Size | PDF

To improve the calibration accuracy, control points for each image are chosen with a period of 20 pixels. This means approximately $50\times 40$ points are selected for each captured pattern. The physical coordinates for these control points on the LCD frame {L} can be calculated by the relations of $x={\phi }_x{p}_x/2\pi$ and $y={\phi }_y{p}_y/2\pi$, where $p_x$ and $p_y$ are the finest fringe period, ${\phi }_x$ and ${\phi }_y$ are the unwrapped phase.

Based on the chosen control points, calibration work is done by following the Fig. 2 flowchart. Figure 5 shows how the RMS value of the reprojection error varies during the calibration process. Step 1 and step 2 represent the process of the initial calculation, and step 3 is the final optimization operation. As Fig. 5 shows, in different steps, the reprojection error are decreased progressively until reaching a stable value while iteration is ongoing. When an optimization algorithm is applied, the RMS value of the reprojection error in frame {C1} and {C2} are both reduced to 0.06 pixels, which greatly improve the final calibration accuracy. If we do not use the last optimization algorithm and simply combine the results from step2, Fig. 6 displays the reprojection errors distribution in frame {C1} and {C2}, and the RMS value in each frame are (0.8839, 1.1395) and (0.6781, 1.411) pixels. Therefore, considering all the parameters in one function can significantly improve the calibration accuracy. On the basis of the optimization calibration results, Fig. 7 shows where the reference mirror is located within the system during the calibration process.

 

Fig. 5 RMS of the reprojection errors. (a) On CCD 1 plane. (b) On CCD 2 plane

Download Full Size | PDF

 

Fig. 6 Reprojection errors before optimization. (a) On CCD 1 plane. (b) On CCD 2 plane

Download Full Size | PDF

 

Fig. 7 Reference mirror poses in the system

Download Full Size | PDF

3.2 Surface measurement

To verify that the described calibration algorithm can result in accurate surface measurement, a 2 inch flat mirror with flatness $<\lambda /10$ and 2 inch concave surface with ${200}_{-0.2}^{+0.0}$mm diameter are measured. Figure 8 shows the calculated slope data when measuring the 2 inch flat mirror, and resulting error with a maximum slope error measured is approximately $4\times {10}^{-4}$ radians. By integrating the slope data with radial basis function [8], a 3D surface is reconstructed as shown in Fig. 9. Figure 9(b) shows the reconstruction error, where the less accurate parts of reconstruction error are around 300 nm and are located at the margin and stitching area. Choosing one row of the reconstruction error shown in Fig. 8(c), error is below 100 nm.

 

Fig. 8 Calculated slope data and resulting error. (a) Slope data along x direction. (b) Slope data error along x direction. (c) Slope data along y direction. (d) Slope data error along y direction

Download Full Size | PDF

 

Fig. 9 (a) Reconstruction flat surface. (b) Resulting error. (c) Resulting error of one row

Download Full Size | PDF

The same measurement procedure is applied for measuring the 2 inch concave surface. Figure 10(a) shows the reconstructed concave 3D shape, and its diameter is 199.825 mm. By fitting the reconstructed surface to the calculated diameter, Fig. 10(b) displays the residuals between reconstructed surface concave and the fitted surface, where the maximum deviation is around 150 nm. Through these two measurement, it can be seen that after using the proposed iterative optimization calibration method, surface measurement by stereo deflectometry can be improved.

 

Fig. 10 (a) Reconstructed concave surface. (b) Residual between reconstructed surface and fitted sphere

Download Full Size | PDF

4 Conclusion

This paper proposes an accurate calibration method for a stereo deflectometry system. Stepwise calibration is used to calculate the original parameters and an optimization algorithm is employed to optimize all the parameters in one cost function. At each calibration process, an iterative algorithm is used to improve the calibration accuracy. Results show use of the proposed algorithm provides RMS reprojection error in two cameras as low as 0.06 pixels. And on the basis of this calibration work, reconstruction error of a 2 inch flat mirror with flatness $<\lambda /10$ can be below 100 nm, and maximum deviation between the reconstructed concave surface and its fitted sphere is around 150 nm.