A Minimal Set of Parameters Based Depth-Dependent Distortion Model and Its Calibration Method for Stereo Vision Systems
Xin Ma, Puchen Zhu, Xiao Li, Xiaoyin Zheng, Jianshu Zhou, Xuchen Wang, Kwok Wai Samuel Au
Abstract
Depth position highly affects lens distortion, especially in close-range photography, which limits the measurement accuracy of existing stereo vision systems. Moreover, traditional depth-dependent distortion models and their calibration methods have remained complicated. In this work, we propose a minimal set of parameters based depth-dependent distortion model (MDM), which considers the radial and decentering distortions of the lens to improve the accuracy of stereo vision systems and simplify their calibration process. In addition, we present an easy and flexible calibration method for the MDM of stereo vision systems with a commonly used planar pattern, which requires cameras to observe the planar pattern in different orientations. The proposed technique is easy to use and flexible compared with classical calibration techniques for depth-dependent distortion models in which the lens must be perpendicular to the planar pattern. The experimental validation of the MDM and its calibration method showed that the MDM improved the calibration accuracy by 56.55% and 74.15% compared with the Li’s distortion model and traditional Brown’s distortion model. Besides, an iteration-based reconstruction method is proposed to iteratively estimate the depth information in the MDM during three-dimensional reconstruction. The results showed that the accuracy of the iteration-based reconstruction method was improved by 9.08% compared with that of the non-iteration reconstruction method.
Index Terms:
Camera calibration, distortion model, stereo vision, three-dimensional reconstruction.
I Introduction
Lens distortion existing in optic measurement, which is usually caused by refraction and focusing of light rays through camera lenses, will result in changes in the shape and size of an image, subsequently affecting the image’s quality and accuracy. The lens distortion is mainly classified as a radial distortion and a decentering (or tangential[1]) distortion[2]. Radial distortion occurs when straight lines in the image appear curved or bulging at edges, causing a distortion in the proportion between the center and edges of the image. On the other hand, the decentering distortion refers to bending or warping of straight lines in the image along the horizontal or vertical direction, resulting in an overall skewed or distorted shape. The occurrence of lens distortion is due to the construction of the lens and the propagation of light rays within it. In the design and manufacturing process of lenses, certain unavoidable flaws exist, leading to inaccurate refraction and focusing of light rays, thereby causing distortions.
Accurate lens distortion calibration is the basis of precise vision measurement, especially in stereo vision systems [3]. Conventional lens distortion models, which consider radial and decentering lens distortions [4], [5] usually take the spatial constraints of the calibration target as the target and employ the nonlinear least-square optimization method to conduct the calibration (Zhang’s [6] method). However, these models neglect the effect of depth information on lens distortion [7], [8]. When measuring a distant target, the performance of these models is satisfactory. However, for large depths in close-range photogrammetry, depth information significantly impacts lens distortion; thus, adopting these models [9]-[11] can result in the low accuracy of the vision measurement system.
Magill [12] explained the relationship between radial distortion and magnification for the first time. However, the unknown parameters (distortion function for inverted infinite focus) in the distortion model could not be calibrated. Then, Brown presented a more applicable depth-dependent radial distortion model [13]. Consequently, Fryer proposed a depth-dependent decentering distortion model [14]. In these depth-dependent distortion models, no distortion function exists for inverted infinite focus, making the calibration of the lens distortion model possible. However, distortion functions at different focused depths must be calibrated, which entails adjusting the lens to focus on planes at different depths during the calibration process. To reduce the complexity of the calibration process, Alvarez derived a new radial distortion model by combining the Brown and Fraser [15] models. During the calibration, the lens focus does not need to be adjusted, increasing the stability of the calibration [16]. However, the decentering and in-plane affine distortion parameters are not calibrated. Consequently, Sun [17] and Li [18] proposed two depth-dependent distortion models, which consider both decentering and radial lens distortions.
However, these depth-dependent lens distortion models have more than 16 unknown parameters, which makes the depth-dependent lens distortion models complicated. For example, the depth-dependent radial lens distortion model and decentering lens distortion model in [18] have 9 unknown parameters and 8 unknown parameters respectively, which need to be calibrated separately in 9 steps in total. The above-mentioned lens distortion models are listed in Table I. During the calibration procedures, especially when estimating the radial and decentering lens distortion parameters in the defocused plane in the depth-dependent lens distortion model, the checkerboard orientations need to be adjusted by mounting the checkerboard on an electric control platform and the camera needs to be adjusted to be perpendicular to the checkerboard. Thus, it will take plenty of time to adjust the checkerboard orientations during the calibration procedures and the additional equipment such as the electric control platform contribute to an increase in costs, which makes the calibration procedures inefficient. Besides, no effective three-dimensional (3D) reconstruction method is proposed in the reference for handling the input-output coupling (depth information) issue.
TABLE I: Comparison among different lens distortion models.
Method
NumberofParameters
ConsideringDepth-dependent
ConsideringRadialDistortion
ConsideringDecenteringDistortion
Advantages
Disadvantages
Brown’s
Model [13]
6
✓
✓
✓
The foundation of
other distortion models
No distrotion functions exists
for inverted infinite focus
RFDM [7]
23
✗
✓
✓
Accurate and flexible in3D shape measurement
Depth information is notconsidered compared with [9]
DRDM [9]
20
✓
✓
✓
Accurate and flexibleunder large-field-of-view
More parameters to becalibrated compared with [16]
ADDM [16]
16
✓
✓
✗
Valid in any scenario
Decentering distortion is notconsidered compared with [18]
Li’s Model [18]
16
✓
✓
✓
Avoiding manuallyadjusting of the focus
More parameters to becalibrated compared with MDM
MDM
8
✓
✓
✓
The camera is not requiredto be perpendicular to the pattern
Hard to find accurateoptimized initial values
In this paper, we propose a minimal set of parameters based depth-dependent lens distortion model (MDM), which considers the radial and decentering distortions of the lens to improve the accuracy of stereo vision systems and simplify their calibration process. Compared with [18], the minimal set of parameters in the MDM has a total of 8 unknown parameters of both radial and decentering lens distortion models, which makes the model more simplified. Besides, we present an easy and flexible calibration method for the MDM of stereo vision systems with a commonly used planar pattern, which requires cameras to observe the planar pattern at different orientations. Compared with some general calibration methods for the depth-dependent distortion models, such as the model proposed by Li and Sun, the proposed technique does not need the camera to be adjusted to be perpendicular to the planar pattern during the calibration, which is easy to use and flexible. Several experiments have been conducted to validate the MDM and its calibration method. The results showed the MDM improves the calibration accuracy by 56.55% and 74.15% compared with the Li’s distortion model and conventional Brown’s distortion model. Moreover, an iteration-based reconstruction method is proposed to iteratively estimate the depth information in the MDM during 3D reconstruction for handling the input-output coupling (depth information) issue. The results showed that the iteration-based reconstruction method improved the calibration accuracy by 9.08% compared with the non-iteration reconstruction method.
This paper is organized as follows: Section II introduces the minimal set of parameters based depth-dependent lens distortion model. Section III introduces the calibration method for the MDM and elaborates on the 3D reconstruction method for the MDM. Section IV provides the experimental details, and Section V outlines the conclusions of the paper.
II Minimal Set of Parameters Based Depth-Dependent Lens Distortion Model
In this section, the minimal set of parameters based depth-dependent radial lens distortion model (MDM-R) and the minimal set of parameters based depth-dependent decentering lens distortion model (MDM-D) are proposed.
II-AMinimal Set of Parameters Based Depth-Dependent Radial Lens Distortion Model
In this section, we simplified the classical depth-dependent radial lens distortion model based on the Magill model.
The Magill model was first simplified by Brown in 1971 [13], which overcame the limited practicality. The simplified model is shown in (1):
(1)
where denotes the focal length. When the lens is adjusted to focus on the distances and , the radial distortions in the focal planes are defined as and . Then, the distortion function for the lens adjusted to focus on distance can be computed. The -th radial distortion coefficients for the focused object plane at distance are described by:
(2)
where , and and represent the -th radial distortion coefficients when the lens is adjusted to focus on the distances and , respectively. However, an empirical model was proposed by Fraser and Shortis for the distortion of any defocused plane [15]:
(3)
where represents the radial distortion coefficient at defocused distance for the lens focused at distance ; is a constant correction factor; and denote the radial distortion coefficients in the focal planes at distances and , respectively. According to (3), the radial distortion function at can be obtained when the lens is adjusted to focus at distance :
(4)
Thus, by adopting the results to the distortion of a point in the defocused plane at distance , the relationship among , , and can be described as:
(5)
Then (6) can be obtained by eliminating the focus distance, and the distortion in the focal plane:
(6)
According to (6), can be written as
(7)
When the lens is adjusted to focus at distance , the radial distortion coefficients in any defocused plane with the depth of can be obtained via two distortions corresponding to object distances and , respectively [18]:
(8)
According to (8), , , , , and are known since two object planes are intentionally set. Thus, in (8) depends on only, and (8) can be further simplified as follows:
(9)
Here and are the -th simplified radial lens distortion parameters in the short-distance defocused plane with the depth of ; and are the radial distortion functions in the and directions, respectively; is the uncorrected corner point image coordinate in the checkerboard image; is the principal point coordinate; is the distortion radius of the image point. Then, the MDM-R can be expressed as shown in (9).
II-BMinimal Set of Parameters Based Depth-Dependent Decentering Lens Distortion Model
In this section, we simplified the classical decentering lens distortion model based on the Brown’s distortion model. The decentering lens distortion model proposed by Brown in 1966 [19] is described as follows:
(10)
where denotes the compensation coefficient; and represent the components of the decentering distortion in the and directions, respectively; and are the object distances corresponding to the two focal planes, respectively.
Based on (10), a depth-dependent decentering distortion model is proposed in [18], which is described as follows:
(11)
where , , and represent the -th decentering distortion parameters in the defocused planes at the object distances , , and when the lens is adjusted to focus at distance , respectively. According to (13), the decentering distortion parameters in any defocused plane depend on only the object distance since , , and are known. However, (13) also has 6 decentering lens distortion parameters to be calibrated separately, which makes the decentering lens distortion model complicated. Similarly, the checkerboard orientations need to be adjusted by mounting the checkerboard on an electric control platform, and the camera also needs to be adjusted to be perpendicular to the checkerboard to estimate and during the calibration procedures, which makes the calibration procedures inefficient.
Thus, a minimal set of decentering distortion parameters is described, and the decentering distortion model can be further simplified as follows:
(12)
Here and are the -th decentering lens distortion parameters in the short-distance defocused plane with the depth of ; and are the decentering distortion functions in the and directions, respectively. The MDM-D is described as shown in (14), which has a minimal set of 4 dencentering lens distortion parameters.
III Calibration and 3D Reconstruction Methods for the MDM
In this section, a flexible three-step calibration method for the MDM is proposed. First, the intrinsic and extrinsic camera parameters of the stereo vision system are calculated by the Zhang’s calibration method. Then, the MDM is estimated using linear constraints. Finally, the intrinsic and extrinsic camera parameters of the stereo vision system and the parameters in the MDM are optimized based on the distance constraints between adjacent points on the checkerboard. Besides, an iteration-based reconstruction method is proposed.
III-ACalibration of Intrinsic and Extrinsic Camera Parameters
As shown in Fig. 1, stereo vision systems consist of two cameras and two lenses. The MDM calibration method employs the most commonly used checkerboard. horizontal lines and vertical lines are arranged on the checkerboard. The two lines intersect to form a corner point. The distance between adjacent corner points is precisely known. In image acquisition, the lenses and cameras are first fixed on the optical platform, and the checkerboard is randomly placed in multiple orientations within a certain depth range. For each orientation, the two cameras capture the images of the checkerboard at the same time. A total of images of the checkerboard are acquired.
For the corrected -th row and -th column corner point on the checkerboard plane in the world coordinate system , the mapping relationship between its 3D reconstructed information and the left image coordinate and that between its 3D reconstructed information and the right image coordinate are introduced as follows:
(13)
Here, and are the rotation matrix and the translation matrix, expressing the transformation between the left image coordinate system and the world coordinate system; and are the rotation matrix and the translation matrix, expressing the transformation between the right image coordinate system and the world coordinate system; and are the intrinsic parameter matrices of the left and right cameras, respectively; and are the scaling factors; and are the values of the -th row and -th column in the left and right camera projection matrices, respectively; and are the corrected image coordinates in the -th row and -th column in the checkerboard images under the left and right image coordinate systems, respectively. The image coordinates of corner points in the checkerboard images are extracted, and the initial values of the intrinsic and extrinsic parameters of the cameras are calculated using the Zhang’s calibration method [4].
III-BCalibration of MDM Parameters
In this section, the radial and decentering lens distortion parameters in the MDM are roughly estimated using linear constraints. For any corrected -th row and -th column corner point on the checkerboard plane,
its left image coordinate and right image coordinate are obtained by (16),
where , , , , , , , and are the radial lens distortion parameters; , ,, , , , , and are the decentering lens distortion parameters; and are the uncorrected corner point image coordinates in the -th row and -th column in the checkerboard images under the left and right image coordinate systems, respectively. and are principal point coordinates under the left and right image coordinate systems, respectively; and are distortion radii of image points under the left and right image coordinate systems, respectively; and are the depth information of the corner points in the -th row and -th column in the checkerboard images under the left and right image coordinate systems, respectively.
(14)
As shown in Fig. 2, when the corner points on the same straight lines in the left and right checkerboard images are known, the line equations determined by the corner points under the left and right image coordinate systems are described as:
(15)
where and are the line parameters of the -th line in the left checkerboard image; and are the line parameters of the -th line in the right checkerboard image; Then, (18) is obtained by substituting and into and in (17), respectively.
(16)
and are the first and second-order parameters of the radial distortion under the left image coordinate system, respectively; and are the first and second-order parameters of the decentering distortion under the left image coordinate system, respectively; and are the first and second-order parameters of the radial distortion under the right image coordinate system, respectively; and are the first and second-order parameters of the decentering distortion under the right image coordinate system, respectively.
If there are lines in an image (left or right image) and observation points are extracted from each line, we obtain equations. In these equations, there are variables ( line parameters and 8 distortion parameters). If , the optimal solution of the distortion parameters is obtained.
is the 4-by-3 projection matrix, expressing the transformation relationship of the left camera coordinate system relative to the right camera coordinate system, where and are the rotation and translation matrices of the left camera coordinate system relative to the right camera coordinate system, respectively; is the 4-by-3 projection matrix, expressing the transformation relationship of the right camera coordinate system relative to the left camera coordinate system, where and are the rotation and translation matrices of the right camera coordinate system relative to the left camera coordinate system, respectively; is the element in the -th row and -th column of ; is the element in the -th row and -th column of ; and are the focal lengths of the left and right cameras, respectively; and are roughly estimated by substituting the calibrated intrinsic and extrinsic camera parameters, and into the equations.
III-COverall Optimization of Intrinsic and Extrinsic Camera Parameters and MDM Parameters
In this section, we optimize the intrinsic and extrinsic camera parameters along with the MDM parameters based on the known distances between adjacent points on the checkerboard. First, according to (15), by eliminating and , we obtain (19):
(17)
To solve , (19) is rewritten as follows:
(18)
Here, is the transposition of matrix ; is the inverse of matrix . Then, the objective function is described by (21):
(19)
Here, presents the 3D information of corner points in the -th row and -th column in the -th checkerboard image related to the intrinsic and extrinsic camera parameters and the MDM parameters; and represent the actual length and width of the rectangle on the checkerboard plane, respectively. Finally, the Levenberg-Marquardt optimization method is employed to optimize the intrinsic and extrinsic camera parameters and the MDM parameters. The optimized initial values are the intrinsic and extrinsic camera parameters calibrated by using the Zhang’s calibration method and the MDM parameters calibrated using linear constraints. Based on the optimization variables, optimized initial values, and optimization objective function, the optimized values of the intrinsic and extrinsic camera parameters and the MDM parameters are obtained.
III-DIteration-Based 3D Information Reconstruction Method
For the reconstruction of the 3D information of the target, the depth information of the target in the MDM or MDM-R is unknown, which can be roughly estimated using the linear pinhole model. However, the accuracy of the estimated depth information of the target is limited, because the linear pinhole model does not consider the lens distortion. Thus, in this section, an iteration-based 3D information reconstruction method is proposed.
Using the proposed iteration-based 3D information reconstruction method, the depth information of the target in the MDM or MDM-R is updated iteratively, which improves the overall reconstruction accuracy. The detailed implementation is shown in Algorithm 1. In the above iteration-based 3D information reconstruction method, the inputs are the intrinsic and extrinsic stereo camera parameters, the MDM/MDM-R parameters, and a sufficiently small . First, the initial depth information of the target is estimated using the pinhole model; consequently, the iteration algorithm runs, and the depth information of the target updates accordingly. When , the process of iteration is finished, and more accurate 3D information of the target is reconstructed.
IV Experiments and Results
IV-AExperimental Setup
As shown in Fig. 3, the setup of the stereo vision system comprised two cameras (Nano-M2420, DALSA, Canada), two lenses (GX-5M-1216, GXTECH, China), and a computer. The resolution of the camera was pixels, and the pixel size was . The focal length of each lens was 12 mm. The images were transmitted from the camera to the computer using the TCP/IP protocol, and the frequency of each camera was 20 Hz. The cameras were mounted on a beam-like holder with a quick-release buckle, and the holder was mounted on the optical platform.
As shown in Fig. 4, the checkerboard plane contains 12×9 (150×105 mm) squares. There are 8 horizontal and 11 vertical lines distributed on the checkerboard, and the lines intersect to form a total of 88 corner points. The distances between adjacent corner points are 15 mm with a positioning accuracy of 0.005 mm.
IV-BRelationship Between the Number of Calibration Image Pairs and the Calibration Accuracy of the MDM
We conducted several experiments to demonstrate the relationship between the number of calibration image pairs and the calibration accuracy of the MDM. The stereo vision system was used to acquire 8, 9, 10, 11, 12, 13, and 14 pairs of calibration images with different checkerboard orientations, respectively. Then, the proposed method was used to calibrate the MDM-R and the MDM using different pairs of calibration images. Finally, another 5 pairs of images with different checkerboard orientations were acquired to validate the accuracy of the MDM-R and the MDM, respectively.
The calibration error was defined as:
(20)
where ; ; represents different calibration images. Fig. 5(a) shows the relationship between the number of calibration image pairs and for a total of 5 tested calibration image pairs of the MDM-R. From Fig. 5(a), the of the MDM-R tended to decrease as the number of calibration image pairs increased. In particular, the decreased rapidly when the number of calibration image pairs ranged from 8 to 12. However, it decreased slowly when the number of calibration image pairs exceeded 13. Fig. 5(b) shows the relationship between the number of calibration image pairs and the of the MDM. The results indicated that decreased slowly when the number of calibration image pairs ranged from 8 to 14, but the maximum error decreased rapidly. Based on the results shown in Fig. 5, the calibration accuracy was satisfied for the MDM-R and the MDM with 13 pairs of calibration images.
TABLE II: calibration errors and calibration time with different lens distortion models.
Calibration Errors with Different Distortion Models (m)
Calibration Time (s)
Brown’s radialdistortion model
Li’s radialdistortion model
MDM-R
Brown’s model
Li’s model
MDM
Brown’s model
Li’s model
MDM
167.5
79.0
35.3
73.1
43.5
18.9
342
1328
351
IV-CCalibration Accuracy Comparison
In this section, the calibration accuracy of the MDM-R and MDM was verified. Besides, the efficiency of the proposed calibration method was confirmed.
In the experiment, firstly, the checkerboard was placed within a depth range of 500 mm to 900 mm from the cameras. Secondly, according to Section II.B, 13 pairs of calibration images with different checkerboard orientations were acquired. Thirdly, our proposed calibration method was used to calibrate the MDM-R and MDM, while Zhang’s calibration method was adopted for calibrating Brown’s radial distortion model and Brown’s distortion model. In addition, according to Li’s calibration method, Li’s radial distortion model and Li’s distortion model were calibrated. During the calibration of these lens distortion models, the calibration time was recorded simultaneously. The calibration errors and corresponding calibration time with different distortion models are shown in Table II.
From the results in Table II, the calibration error of the MDM-R decreased by 55.32% and 78.93% compared to that of Li’s radial distortion model and Brown’s radial distortion model, respectively. Additionally, the calibration error of the MDM decreased by 56.55% and 74.15% compared to that of Li’s distortion model and Brown’s distortion model, respectively.
Without a specific requirement of the checkerboard’s orientation compared with Li’s calibration method, the calibration time of the MDM and Brown’s distortion model was roughly four times shorter than that of Li’s distortion model. This demonstrated that our proposed calibration method had higher calibration efficiency. Moreover, depth information was considered in the MDM and Li’s distortion model, resulting in higher calibration accuracy compared to Brown’s distortion model. Due to more parameters needing to be calibrated in Li’s distortion model compared with the MDM, the calibration accuracy of Li’s distortion model was lower than that of the MDM.
Figure 6: The one-dimensional (1D) target
IV-DMeasurement Accuracy Comparison
In this section, the measurement accuracy of the MDM and MDM-R was verified using a 1D target, and the proposed iteration-based reconstruction method was also validated. As shown in Fig. 6, the distance between the corner points and on the 1D target was precisely known as 150 mm. The positioning accuracy of the 1D target was 0.005 mm.
In the experiment, the 1D target was placed within the depth range of 680 mm to 800 mm sequentially, and the image pairs of the 1D target were acquired accordingly for measurement accuracy verification. Then, the distance between the corner points and was measured at different depths using the stereo vision system with different lens distortion models and 3D reconstruction methods.
Brown’s radial distortion model and Li’s radial distortion model were calibrated using Zhang’s calibration method and Li’s calibration method, respectively. Brown’s and Li’s distortion models were calibrated using Zhang’s calibration method and Li’s calibration method, respectively. The MDM-R and MDM were calibrated by the proposed calibration method.
The non-iteration reconstruction method was employed for Brown’s radial distortion model, as well as Brown’s distortion model. For conducting 3D reconstruction, the depth information in Li’s radial distortion model and Li’s distortion model was roughly estimated. However, the iteration-based reconstruction method was adopted to address the same problem in MDM-R and MDM. The initial depth information was obtained by pin-hole model, and Algorithm 1 in Section III.B was used to iteratively calculate more accurate depth information. The 3D reconstruction error using the iteration-based reconstruction method was compared with that using the non-iteration reconstruction method.
The measurement errors with these lens distortion models and 3D reconstruction methods at different depths are shown in Table III.
From the results in Table III, it was obvious that the measurement errors using Li’s radial distortion model and Li’s distortion model were larger than those using the MDM-R and MDM. Besides, the measurement errors using Brown’s radial distortion model and Brown’s distortion model were larger than those using Li’s radial distortion model and Li’s distortion model. Due to more accurate depth information calculated with each iteration, the measurement accuracy by using the iteration-based reconstruction method was higher.
Compared with the non-iteration reconstruction method for the MDM, the measurement errors using the iteration-based reconstruction method were decreased by 9.08% (on average). The results in Table III not only verified the measurement accuracy of our proposed distortion model calibrated by the proposed calibration method but also demonstrated the validity of the iteration-based reconstruction method.
TABLE III: Measurement errors with different distortion models at different depths.
with differentdepths (mm)
Measurement Errors with Different Distortion Models (m)
Brown’s radialdistortion model
Li’s radialdistortion model
MDM-R
Brown’smodel
Li’smodel
MDM
Non-iterationreconstruction
Iteration-basedreconstruction
Non-iterationreconstruction
Iteration-basedreconstruction
680
384.2
97.1
41.2
39.1
96.0
56.9
24.8
22.9
700
397.9
99.5
46.6
44.1
99.5
58.4
26.6
24.1
720
414.1
102.3
51.1
48.2
103.0
60.6
27.5
25.3
740
422.8
105.6
56.7
52.9
105.6
61.9
29.3
26.4
760
459.1
110.2
61.0
58.7
110.0
64.5
32.5
29.6
780
474.4
115.1
67.2
63.6
113.8
67.3
34.8
31.9
800
497.2
119.6
71.4
67.3
115.4
69.9
38.2
34.1
TABLE IV: Calibration results of the kinematic parameters of the robotic arm.
Link
Zhang’s
method
MDM
Li’s
method
Nominal
value
Zhang’s
method
MDM
Li’s
method
Nominal
value
Zhang’s
method
MDM
Li’s
method
Nominal
value
Zhang’s
method
MDM
Li’s
method
Nominal
value
1
0.80
0.35
0.81
0.00
0.87
1.42
0.24
0.00
365.88
366.29
365.19
365.00
0.00
0.00
0.00
0.00
2
1.06
1.82
0.46
0.00
91.07
91.53
90.31
90.00
65.51
66.39
65.18
65.00
0.00
0.00
0.00
0.00
3
0.84
0.54
0.67
0.00
-91.27
-90.54
-90.56
-90.00
396.49
395.19
396.13
395.00
0.00
0.00
0.00
0.00
4
21.46
20.38
20.52
220.00
91.03
90.59
90.16
90.00
-53.98
-53.52
-54.24
-55.00
0.00
0.00
0.00
0.00
5
-18.94
-19.13
-19.90
-20.00
-90.24
-91.92
-91.17
-90.00
385.27
386.95
385.71
385.00
0.00
0.00
0.00
0.00
6
1.45
1.82
0.31
0.00
90.21
90.43
91.23
90.00
100.35
101.51
100.19
100.00
91.46
90.73
91.37
90.00
7
111.35
110.38
110.16
110.00
90.52
90.76
90.64
90.00
135.10
134.41
135.11
135.00
0.00
0.00
0.00
0.00
IV-ERobotic Arm Kinematic Parameters Calibration
In this section, we applied our proposed stereo vision system to the kinematic parameters calibration of a robotic arm.
As shown in Fig. 7, the experimental setup comprised a seven-degree-of-freedom robotic arm (Flexiv Robotics, Rizon 4), a stereo vision system, and a spherical marker. The stereo vision system consisted of two cameras, two lenses, and two LED lights. The marker had a diameter of 20 mm and was attached to the end-effector of the robotic arm.
Figure 7: Experimental setup for kinematic parameters calibration of the robotic arm
In the experiment, firstly, Brown’s distortion model, Li’s distortion model, and MDM were used to calibrate the stereo vision system. From the results in Table III, the measurement accuracy of the stereo vision system calibrated by MDM was higher than calibrated by Brown’s distortion model and Li’s distortion model. Secondly, the stereo vision system with different distortion models was used to calibrate the kinematic parameters of the robotic arm. The method in [20] was employed to calibrate the kinematic parameters of the robotic arm. Thirdly, to validate the performance of the kinematic parameters calibration, the end-effector of the robotic arm was moved to the 50 desired positions. The position of the end-effector of the robotic arm was measured by the stereo vision system.
Figure 8: Robotic arm end-effector position errors by using stereo vision system calibrated by different distortion models
The results are shown in Fig. 8. The position errors were calculated by the distance between the desired position and the measurement position of the end-effector of the robotic arm. The RMS position error before the kinematic parameters calibration of the robotic arm was 2833.6 m. By using the stereo vision system with Zhang’s distortion model, MDM, and Li’s distortion model for calibrating the robotic arm, the RMS position errors were 1456.7 m, 780.6 m, and 1020.9 m, respectively. The above results demonstrated that the stereo vision system with higher measurement accuracy for the kinematic parameters calibration of the robotic arm will result in higher motion accuracy of the robotic arm, which will improve the performance of the robotic arm with some tasks that require high accuracy.
V Conclusion
In this paper, we proposed an MDM, which considers both the radial and decentering lens distortions. Besides, a flexible three-step calibration method for the MDM was presented. The proposed flexible calibration method requires the cameras to observe a planar pattern in different orientations. Unlike the existing calibration methods for depth-dependent distortion models, the lenses are not required to be perpendicular to the planar pattern during our calibration procedures. In the experimental section, we demonstrated that the MDM improved the calibration accuracy by 56.55% and 74.15% compared with Li’s distortion model and the traditional Brown’s distortion model, respectively. Besides, the proposed calibration method was more efficient compared with Li’s calibration method. In addition, the results demonstrated that the measurement accuracy using the iteration-based reconstruction method was improved by 9.08%, on average, compared with using the non-iteration reconstruction method. In the future, we will apply our proposed stereo vision system to various scenarios, such as registration between different systems.
References
[1]
C. Ricolfe-Viala, A.J. Sanchez-Salmeron, “Lens distortion models evaluation,” Appl. Optics, vol. 49, no. 30, pp. 5914–5928, Oct. 2010.
[2]
P. B. Xiong, S. K. Wang, W. B. Wang, Q. X. Ye, and S. J. Ye, “Model-independent lens distortion correction based on sub-pixel phase encoding,” Sensors, vol. 49, no. 30, pp. 5914–5928, Oct. 2010.
[3]
H. Shang, C.Y. Liu, and R.J. Wang, “Measurement methods of 3D shape of large-scale complex surfaces based on computer vision: A review,” Measurement, vol. 197, Jun. 2022, Art. no. 111302.
[4]
R. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses,” IEEE Rob. Autom. Mag., vol. 3, no. 4, pp. 323–324, Aug. 1987.
[5]
J. Weng, P. Cohen, and M. Herniou, “Camera calibration with distortion models and accuracy evaluation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 10, pp. 965–980, Oct. 1992.
[6]
Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 11, pp. 1330–1334, Nov. 2000.
[7]
L.B. Bu, R. Wang, X. S. Wang, Z. W. Hou, Y. C. Zhou, Y. F. Wang, and F. L. Bu, “Calibration method for fringe projection profilometry based on rational function lens distortion model,” Measurement, vol. 217, Aug. 2023, Art. no. 112996.
[8]
Z. R. Gao, Y. Gao, Y. Su, Y. Liu, Z. Fang, Y. R. Wang, and Q. C. Zhang,“Stereo camera calibration for large field of view digital image correlation using zoom lens,” Measurement, vol. 185, Nov. 2021, Art. no. 109999.
[9]
X. Wang, Y. Gao, and Z.Z. Wei, “Calibration method for large-field-of-view stereo vision system based on distance-related distortion model,” Opt. Express, vol. 31, no. 13, pp. 21816–21833, Jun. 2023.
[10]
Z.Y. Xia, R.B. Xia, and J.B. Zhao, “3D spatial distortion model based on the Lagrange difference in a binocular visual system,” Appl. Optics, vol. 62, no. 8, pp. 1952–1960, Mar. 2023.
[11]
Y.C. Hou, C.Y. Liu, H. Wang, Y.Y. Fan, B.W. An, and Y. Liu, “High-precision visual imaging model and calibration method for multi-depth-of-field targets,” Optik, vol. 268, Oct. 2022, Art. no. 169788.
[12]
A.A. Magill, “Variation in distortion with magnification,” J. Photogramm. Eng., vol. 54, no. 3, pp. 148–149, Mar. 1955.
[13]
D.C. Brown, “Close-range camera calibration,” J. Res. Natl. Bur. Stand., vol. 37, no. 8, pp. 855–866, Jan. 1971.
[15]
C.S. Fraser, M.R. Shortis, “Variation of distortion within the photographic field,” Photogramm. Eng. Remote Sens., vol. 58, no. 6, pp. 851–855, Jun. 1992.
[16]
L. Alvarez, L. Gómez, and J.R. Sendra, “Accurate depth dependent lens distortion models: An application to planar view scenarios,” J. Math. Imaging Vis., vol. 39, no. 10, pp. 75–85, Jan 2011.
[17]
P. Sun, N. Lu, and M. Dong, “Modelling and calibration of depth-dependent distortion for large depth visual measurement cameras,” Opt. Express, vol. 25, no. 9, pp. 9834–9847, May 2017.
[18]
X. Li, W. Li, X. Yuan, X. Yin, and X. Ma, “DoF-dependent and equal-partition based lens distortion modeling and calibration method for close-range photogrammetry,” Sensors, vol. 20, no. 20, p. 5934, Oct. 2020.
[19]
D.C. Brown, “Decentering distortion of lenses,” Photogramm. Eng., vol. 32, no. 3, pp. 444–462, Jan. 1966.
[20]
S. Hayati, K. Tso, and G. Roston, “Robot geometry calibration,” in Proc. IEEE Int. Conf. Robot. Automat., vol. 2, no. 3, pp. 947–951, Jan. 1988.