Abstract
In this paper, we introduce a nonlinear robust fractional order variational framework for motion estimation from image sequences (video). Particularly, the presented model provides a generalization of integer order derivative based variational functionals and offers an enhanced robustness against outliers while preserving the discontinuity in the dense flow field. The motion is estimated in the form of optical flow. For this purpose, a level set segmentation based fractional order variational functional composed of a non-quadratic Charbonnier norm and a regularization term is propounded. The non-quadratic Charbonnier norm introduces a noise robust character in the model. The fractional order derivative demonstrates non-locality that makes it competent to deal with discontinuous information about edges and texture. The level set segmentation is carried on the flow field instead of images, which is a union of disjoint and independently moving regions such that each motion region contains objects of equal flow velocity. The resulting fractional order partial differential equations are numerically discretized using Grünwald–Letnikov fractional derivative. The nonlinear formulation is transformed into a linear system which is solved with the help of an efficient numerical technique. The results are evaluated by conducting experiments over a variety of datasets. The accuracy and efficiency of the propounded model is also depicted against recently published works.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
http://visual.cs.ucl.ac.uk/pubs/flowconfidence/supp/index.html (2011)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011). https://doi.org/10.1007/s11263-010-0390-2
Ballester, C., Garrido, L., Lazcano, V., Caselles, V.: A TV-L1 optical flow method with occlusion detection. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 31–40. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32717-9_4
Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)
Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: Conference on Computer Vision and Pattern Recognition, pp. 41–48 (2009)
Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vis. 61(3), 211–231 (2005). https://doi.org/10.1023/B:VISI.0000045324.43199.43
Caputo, M.: Linear models of dissipation whose Q is almost frequency independent-2. Geophys. J. Int. 13(5), 529–539 (1967)
Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004). https://doi.org/10.1023/B:JMIV.0000011325.36760.1e
Chen, D., Sheng, H., Chen, Y., Xue, D.: Fractional-order variational optical flow model for motion estimation. Philos. Trans. Roy. Soc. A Math. Phys. Eng. Sci. 371(1990), 20120148 (2013)
Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72(2), 195–215 (2007). https://doi.org/10.1007/s11263-006-8711-1
Dong, C.Z., Celik, O., Catbas, F.N., OŠBrien, E.J., Taylor, S.: Structural displacement monitoring using deep learning-based full field optical flow methods. Struct. Infrastruct. Eng. 16(1), 51–71 (2020)
Fang, N., Zhan, Z.: High-resolution optical flow and frame-recurrent network for video super-resolution and deblurring. Neurocomputing 489, 128–138 (2022)
Ferrari, F.: Weyl and Marchaud derivatives: a forgotten history. Mathematics 6(1), 6 (2018)
Galvin, B., McCane, B., Novins, K., Mason, D., Mills, S., et al.: Recovering motion fields: an evaluation of eight optical flow algorithms. In: BMVC, vol. 98, pp. 195–204 (1998)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 4th edn (2018)
Guan, L., et al.: Study on displacement estimation in low illumination environment through polarized contrast-enhanced optical flow method for polarization navigation applications. Optik 210, 164513 (2020)
Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)
Huang, H., et al.: Cloud motion estimation for short term solar irradiation prediction. In: International Conference on Smart Grid Communications, pp. 696–701 (2013)
Huang, Z., Pan, A.: Non-local weighted regularization for optical flow estimation. Optik 208, 164069 (2020)
Jeon, M., Choi, H.S., Lee, J., Kang, M.: Multi-scale prediction for fire detection using convolutional neural network. Fire Technol. 57(5), 2533–2551 (2021). https://doi.org/10.1007/s10694-021-01132-y
Khan, M., Kumar, P.: A nonlinear modeling of fractional order based variational model in optical flow estimation. Optik 261, 169136 (2022)
Kumar, P., Khan, M., Gupta, S.: Development of an IR video surveillance system based on fractional order TV-model. In: 2021 International Conference on Control, Automation, Power and Signal Processing (CAPS), pp. 1–7. IEEE (2021)
Kumar, P., Kumar, S.: A modified variational functional for estimating dense and discontinuity preserving optical flow in various spectrum. AEU-Int. J. Electron. Commun. 70(3), 289–300 (2016)
Kumar, P., Kumar, S., Balasubramanian, R.: A fractional order total variation model for the estimation of optical flow. In: Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, pp. 1–4 (2015)
Kumar, P., Kumar, S., Balasubramanian, R.: A vision based motion estimation in underwater images. In: International Conference on Advances in Computing, Communications and Informatics, pp. 1179–1184 (2015)
Kumar, P., Raman, B.: Motion estimation from image sequences: a fractional order total variation model. In: Raman, B., Kumar, S., Roy, P.P., Sen, D. (eds.) Proceedings of International Conference on Computer Vision and Image Processing. AISC, vol. 460, pp. 297–307. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-2107-7_27
Lu, J., Yang, H., Zhang, Q., Yin, Z.: A field-segmentation-based variational optical flow method for PIV measurements of nonuniform flows. Exp. Fluids 60(9), 1–17 (2019). https://doi.org/10.1007/s00348-019-2787-1
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: 7th International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)
Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Dover Publications (2012)
Marchaud, A.: Sur les dérivées et sur les différences des fonctions de variables réelles. J. de Mathématiques Pures et Appliquées 6, 337–426 (1927)
Miller, K.S.: Derivatives of noninteger order. Math. Mag. 68(3), 183–192 (1995)
Miller, K.S., Ross, B.: An Introduction to the Fractional Calculus and Fractional Differential Equations. Wiley (1993)
Nagel, H.H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 5, 565–593 (1986)
Oldham, K., Spanier, J.: The Fractional Calculus Theory and Applications of Differentiation and Integration to Arbitrary Order. Elsevier (1974)
Otte, M., Nagel, H.-H.: Optical flow estimation: advances and comparisons. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 800, pp. 49–60. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-57956-7_5
Rao, S., Wang, H., Kashif, R., Rao, F.: Robust optical flow estimation to enhance behavioral research on ants. Digit. Sig. Process. 120, 103284 (2022)
Rinsurongkawong, S., Ekpanyapong, M., Dailey, M.N.: Fire detection for early fire alarm based on optical flow video processing. In: 9th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, pp. 1–4 (2012)
Schnorr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: 12th International Conference on Pattern Recognition, vol. 1, pp. 661–663 (1994)
Senst, T., Eiselein, V., Sikora, T.: Robust local optical flow for feature tracking. IEEE Trans. Circ. Syst. Video Technol. 22(9), 1377–1387 (2012)
Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2432–2439 (2010)
Tu, Z., Poppe, R., Veltkamp, R.: Estimating accurate optical flow in the presence of motion blur. J. Electron. Imaging 24(5), 053018 (2015)
Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002). https://doi.org/10.1023/A:1020874308076
Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 optical flow. In: BMVC, vol. 1, p. 3 (2009)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74936-3_22
Zimmer, H., Bruhn, A., Weickert, J.: Optic flow in harmony. Int. J. Comput. Vis. 93(3), 368–388 (2011). https://doi.org/10.1007/s11263-011-0422-6
Zimmer, H., et al.: Complementary optic flow. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 207–220. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03641-5_16
Acknowledgements
The authors Pushpendra Kumar and Nitish Kumar Mahala acknowledge the support of NBHM, Mumbai for grant no. 02011/24/2021 NBHM (R.P)/ R &D- II/8669 and the author Muzammil Khan expresses gratitude to MHRD, New Delhi, Government of India.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kumar, P., Khan, M., Mahala, N.K. (2023). A Segmentation Based Robust Fractional Variational Model for Motion Estimation. In: Gupta, D., Bhurchandi, K., Murala, S., Raman, B., Kumar, S. (eds) Computer Vision and Image Processing. CVIP 2022. Communications in Computer and Information Science, vol 1776. Springer, Cham. https://doi.org/10.1007/978-3-031-31407-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-31407-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31406-3
Online ISBN: 978-3-031-31407-0
eBook Packages: Computer ScienceComputer Science (R0)