Robust and computationally efficient online image stabilisation framework based on adaptive dual motion vector integration
Authors: 
Adeel Yousaf, Muhammad Shehzad Hanif, Muhammad Jaleed Khan, Mahboob Iqbal, Khurram KhurshidAuthors Info & Claims
Adeel Yousaf, Muhammad Shehzad Hanif, Muhammad Jaleed Khan, Mahboob Iqbal, Khurram KhurshidAuthors Info & ClaimsAbstract
Image stabilisation aims to compensate and smoothen the effects of undesired trembling motion of cameras mounted on non‐static platforms. It becomes quite a challenging task in the case of moving platforms, such as ground vehicles, unmanned aerial vehicles, and handheld devices. Many satisfactory solutions to the image stabilisation problem are proposed in the recent literature, but most of these methods are not adaptable for handling a wide range of intentional motions with minimum lag, especially in real‐time scenarios. In this study, the authors propose an online two‐dimensional image stabilisation technique based on dual motion vector integration, which is a novel adaptive motion smoothing technique that employs an average length of motion vectors to estimate the intentional motion. The overall computational cost of the proposed system is significantly reduced by employing frame‐shaking judgment that only allows processing of jittering frames. Promising experimental results have been obtained on challenging videos obtained from hand‐held and vehicle‐mounted cameras which demonstrate the robustness and effectiveness of the proposed technique against feature point mismatching and presence of moving objects within the scene at a frame rate of 30 frames per second.
7 References
[1]
Hu, W.C., Chen, C.H., Chen, T.Y., et al: ‘Moving object detection and tracking from video captured by moving camera’, J. Vis. Commun. Image Represent., 2015, 30, pp. 164–180
[2]
Choi, W., Pantofaru, C., Savarese, S.: ‘A general framework for tracking multiple people from a moving camera’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (7), pp. 1577–1591
[3]
Hu, W.‐C.: ‘Real‐time on‐line video object segmentation based on motion detection without background construction’, Int. J. Innov. Comput. Inf. Control, 2011, 7, (4), pp. 1845–1860
[4]
Hu, W.‐C., Yang, C.‐Y., Huang, D.‐Y.: ‘Robust real‐time ship detection and tracking for visual surveillance of cage aquaculture’, J. Vis. Commun. Image Represent., 2011, 22, (6), pp. 543–555
[5]
Hu, W.C., Chen, C.H., Chen, C.M., et al: ‘Effective moving object detection from videos captured by a moving camera’, Intell. Data Anal. Appl., 2014, 1, pp. 343–353
[6]
Karpenko, A., Jacobs, D., Baek, J., et al: ‘Digital video stabilization and rolling shutter correction using gyroscopes’. CSTR 1, 2011
[7]
Yousaf, A., Khurshid, K., Khan, M.J., et al: ‘A survey on real time video stabilization methods for UAV's in infrared range’. IEEE Int. Conf. on Aerospace Science & Engineering, Islamabad, Pakistan, 2017, pp. 1–9
[8]
Cardani, B.: ‘Optical image stabilization for digital cameras’, IEEE Control Syst., 2006, 26, (2), pp. 21–22
[9]
Chang, H.C., Lai, S.H., Lu, K.R.: ‘A robust and efficient video stabilization algorithm’. IEEE Int. Conf. on Multimedia and Expo, Taipei, June 2004, pp. 29–321
[10]
Strzodka, R., Garbe, C.: ‘Real‐time motion estimation and visualization on graphics cards’. Proc. of the Conf. on Visualization, Austin, Texas, USA, 2004, pp. 545–552
[11]
Lee, K.Y., Chuang, Y.Y., Chen, B.Y., et al: ‘Video stabilization using robust feature trajectories’. IEEE 12th Int. Conf. on Computer Vision, Kyoto, 2009, pp. 1397–1404
[12]
Yang, J., Schonfeld, D., Mohamed, M.: ‘Robust video stabilization based on particle filter tracking of projected camera motion’, IEEE Trans. Circuits Syst. Video Technol., 2009, 19, (7), pp. 945–954
[13]
Wang, C., Kim, J.H., Byun, K.Y., et al: ‘Robust digital image stabilization using the Kalman filter’, IEEE Trans. Consum. Electron., 2009, 55, (1), pp. 6–14
[14]
Juan, L., Gwun, O.: ‘A comparison of SIFT, PCA‐SIFT and SURF’, Int. J. Image Process., 2009, 3, (4), pp. 143–152
[15]
Lowe, D.G.: ‘Object recognition from local scale‐invariant features’. The Proc. of the Seventh IEEE Int. Conf. on Computer Vision, Kerkyra, Greece, 1999, pp. 1150–11572
[16]
Alahi, A., Ortiz, R., Vandergheynst, P.: ‘Freak: fast retina keypoint’. IEEE Conf. on Computer Vision and Pattern Recognition, Providence, Rhode Island, 2012, pp. 510–517
[17]
Bay, H., Ess, A., Tuytelaars, T., et al: ‘Speeded‐up robust features (SURF)’, Comput. Vis. Image Underst., 2008, 110, (3), pp. 346–359
[18]
Leutenegger, S., Chli, M., Siegwart, R.Y.: ‘BRISK: binary robust invariant scalable key‐points’. IEEE Int. Conf. on Computer Vision (ICCV), Barcelona, 2011, pp. 2548–2555
[19]
Harris, C., Stephens, M.: ‘A combined corner and edge detector’. Alvey Vision Conf., Manchester, UK, 1988, vol. 15, no. 50, pp. 10–5244
[20]
Fischler, M.A., Bolles, R.C.: ‘Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography’, Commun. ACM, 1981, 24, (6), pp. 381–395
[21]
Wang, Y., Hou, Z., Leman, K., et al: ‘Real‐time video stabilization for unmanned aerial vehicles’. MVA IAPR Conf. on Machine Vision Applications, Nara, 2011, pp. 336–339
[22]
Wang, Y., Chang, R., Chua, T.W., et al: ‘Video stabilization based on high degree b‐spline smoothing’. Proc. of the 21st Int. Conf. on Pattern Recognition, Tsukuba, Japan, 2012, pp. 3152–3155
[23]
Ryu, Y.G., Roh, H.C., Chung, M.J.: ‘Long‐time video stabilization using point‐feature trajectory smoothing’. IEEE Int. Conf. on Consumer Electronics, Las Vegas, 2011, pp. 189–190
[24]
Dong, J., Xia, Y.: ‘Real‐time video stabilization based on smoothing feature trajectories’, Appl. Mech. Mater., 2014, 519, pp. 640–643
[25]
Erturk, S.: ‘Image sequence stabilisation: motion vector integration (MVI) versus frame position smoothing (FPS)’. Proc. of the 2nd Int. Symp. Image and Signal Processing and Analysis, Pula, Croatia, 2001, pp. 266–271
[26]
Rawat, P., Singhai, J.: ‘Adaptive motion smoothening for video stabilization’, Int. J. Comput. Appl., 2013, 72, (20), pp. 14–20
[27]
Wu, S., Zhang, D.C., Zhang, Y., et al: ‘Adaptive smoothing in real‐time image stabilization’. Visual Information Processing XXI, Baltimore, 2012, p. 83990
[28]
Liu, F., Gleicher, M., Jin, H., et al: ‘Content‐preserving warps for 3D video stabilization’, ACM Trans. Graph., 2009, 28, (3), p. 44
[29]
Zhang, G., Hua, W., Qin, X., et al: ‘Video stabilization based on a 3D perspective camera model’, Vis. Comput., 2009, 25, (11), pp. 997–1008
[30]
Buehler, C., Bosse, M., McMillan, L.: ‘Non‐metric image‐based rendering for video stabilization’. Computer Vision and Pattern Recognition, Kauai, Hawaii, USA, 2001, pp. 609–614
[31]
Goldstein, A., Fattal, R.: ‘Video stabilization using epipolar geometry’, ACM Trans. Graph., 2012, 31, (5), pp. 1–10
[32]
Liu, F., Gleicher, M., Wang, J., et al: ‘Subspace video stabilization’, ACM Trans. Graph., 2011, 30, (1), p. 4
[33]
Wang, Y.S., Liu, F., Hsu, P.S., et al: ‘Spatially and temporally optimized video stabilization’, IEEE Trans. Vis. Comput. Graphics, 2013, 17, pp. 1354–1361
[34]
Morimoto, C., Chellappa, R.: ‘Evaluation of image stabilization algorithms’. Proc. of Int. Conf. on Acoustics, Speech and Signal Processing, Seattle, WA, USA, 1998, pp. 2789–2792
[35]
Matsushita, Y., Ofek, E., Ge, W., et al: ‘Full‐frame video stabilization with motion in painting’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (7), pp. 1150–1163
[36]
Erturk, S., Dennis, T.J.: ‘Image sequence stabilisation based on DFT filtering’, IEE Proc., Vis. Image Signal Process., 2000, 147, (2), pp. 95–102
[37]
Yang, J., Schonfeld, D., Chen, C., et al: ‘Online video stabilization based on particle filters’. IEEE Int. Conf. on Image Processing, Atlanta, GA, USA, 2006, pp. 1545–1548
[38]
Chen, C.H., Chen, T.Y., Hu, W.C., et al: ‘Video stabilization for fast moving camera based on feature point classification’. Robot, Vision and Signal Processing, Third Int. Conf., Kaohsiung, Taiwan, 2015, pp. 10–13
[39]
Litvin, A., Konrad, J., Karl, W.C.: ‘Probabilistic video stabilization using Kalman filtering and mosaicking’, Image Video Commun. Process., 2003, 5022, pp. 663–675
[40]
Erturk, S.: ‘Real‐time digital image stabilization using Kalman filters’, Real‐Time Imaging, 2002, 8, (4), pp. 317–328
[41]
Dong, J., Xia, Y., Yu, Q., et al: ‘Instantaneous video stabilization for unmanned aerial vehicles’, J. Electron. Imaging, 2014, 23, (1), p. 13002
[42]
Corsini, G., Diani, M., Masini, A.: ‘Video sequence stabilization for real‐time remote sensing applications’. Geoscience and Remote Sensing Symp., Denver, CO, USA, 2006, pp. 988–991
[43]
Hu, W.C., Chen, C.H., Chen, T.Y., et al: ‘Real‐time video stabilization for fast‐moving vehicle cameras’, Multimedia Tools Appl., 2018, 77, (1), pp. 1237–1260
[44]
Kim, S.K., Kang, S.J., Wang, T.S., et al: ‘Feature point classification based global motion estimation for video stabilization’, IEEE Trans. Consum. Electron., 2013, 59, (1), pp. 267–272
[45]
Liang, Y.M., Tyan, H.R., Chang, S.L., et al: ‘Video stabilization for a camcorder mounted on a moving vehicle’, IEEE Trans. Veh. Technol., 2004, 53, (6), pp. 1636–1648
[46]
Zhang, Y., Xie, M., Tang, D.: ‘A central sub‐image based global motion estimation method for in‐car video stabilization’. Knowledge Discovery and Data Mining, Phuket, Thailand, 2010, pp. 204–207
[47]
Aguilar, W.G., Angulo, C.: ‘Real‐time video stabilization without phantom movements for micro aerial vehicles’, EURASIP J. Image Video Process., 2014, 46, (1), p. 46
[48]
Liu, S., Yuan, L., Tan, P., et al: ‘Bundled camera paths for video stabilization’, ACM Trans. Graph., 2013, 32, (4), p. 78
[49]
Chen, C.H., Hsu, C.W., Chen, C.H.: ‘Video stabilization by feature‐block motion estimation’. IEEE 8th Int. Conf. on Intelligent Systems Design and Applications, Kaohsiung, Taiwan, 2008, pp. 557–560
[50]
‘Deshaker method’. Available at http://www.guthspot.se/video/deshaker.htm
[51]
Shen, H., Pan, Q., Cheng, Y., et al: ‘Fast video stabilization algorithm for UAV’. Intelligent Computing and Intelligent Systems, Shanghai, China, 2009, pp. 542–546
[52]
Dong, J., Liu, H.: ‘Video stabilization for strict real‐time applications’, IEEE Trans. Circuits Syst. Video Technol., 2017, 27, (4), pp. 716–724
[53]
Vincent, O.R., Folorunso, O.: ‘A descriptive algorithm for Sobel image edge detection’. Proc. of Informing Science & IT Education Conf., Macon, United States, 2009, pp. 97–107
[54]
Kalal, Z., Mikolajczyk, K., Matas, J.: ‘Forward‐backward error: automatic detection of tracking failures’. IEEE Int. Conf. on Pattern Recognition, Istanbul, Turkey, 2010, pp. 2756–2759
[55]
Ester, M., Kriegel, H.P., Sander, J., et al: ‘A density‐based algorithm for discovering clusters in large spatial databases with noise’. Second Int. Conf. on Knowledge Discovery and Data Mining, Portland, Oregon, 1996, pp. 226–231
[56]
‘Video Database’. Available at http://www.liushuaicheng.org/SIGGRAPH2013/index.htm
Index Terms
- Robust and computationally efficient online image stabilisation framework based on adaptive dual motion vector integration
Index terms have been assigned to the content through auto-classification.
Recommendations
Lossy encoding of motion vectors using entropy-constrained vector quantization
ICIP '95: Proceedings of the 1995 International Conference on Image Processing (Vol. 3)-Volume 3 - Volume 3Many well-known video coding algorithms employ block motion estimation and compensation. But not much attention has been given to the problem of rate allocation for encoding the motion vectors and motion compensated difference frame. We propose entropy-...
Global rigidity constraints in image displacement fields
ICCV '95: Proceedings of the Fifth International Conference on Computer VisionImage displacement fields-optical flow fields, stereo disparity fields, normal flow fields-due to rigid motion possess a global geometric structure which is independent of the scene in view. Motion vectors of certain lengths and directions are ...
Perceptual quality evaluation for motion deblurring
Motion deblurring has been widely studied. However, the relevant quality evaluation of motion deblurred images remains an open problem. The motion deblurred images are usually contaminated by noise, ringing and residual blur (NRRB) simultaneously. ...
Comments
Information & Contributors
Information
Published In
© 2019 The Institution of Engineering and Technology.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 28 June 2019
Author Tags
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Aug 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in