Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

HMPMR strategy for real-time tracking in aerial images, using direct methods

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The vast majority of approaches make use of features to track objects. In this paper, we address the tracking problem with a tracking-by-registration strategy based on direct methods. We propose a hierarchical strategy in terms of image resolution and number of parameters estimated in each resolution, that allows direct methods to be applied in demanding real-time visual-tracking applications. We have called this strategy the Hierarchical Multi-Parametric and Multi-Resolution strategy (HMPMR). The Inverse Composition Image Alignment Algorithm (ICIA) is used as an image registration technique and is extended to an HMPMR-ICIA. The proposed strategy is tested with different datasets and also with image data from real flight tests using an Unmanned Aerial Vehicle, where the requirements of direct methods are easily unsatisfied (e.g. vehicle vibrations). Results show that using an HMPMR approach, it is possible to cope with the efficiency problem and with the small motion constraint of direct methods, conducting the tracking task at real-time frame rates and obtaining a performance that is comparable to, or even better than, the one obtained with the other algorithms that were analyzed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. Anderson, C.H., Bergen, J.R., Burt, P.J., Ogden, J.M.: Pyramid methods in image processing. RCA Eng. 29(6), 33–41 (1984)

    Google Scholar 

  2. Baker, S., Datta, A., Kanade, T.: Parameterizing Homographies. Tech. Rep. CMU-RI-TR-06-11, Robotics Institute, Pittsburgh (2006)

  3. Baker, S., Matthews, I.: Equivalence and efficiency of image alignment algorithms. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition 1, pp. 1090–1097 (2001)

  4. Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(1), 221–255 (2004)

    Article  Google Scholar 

  5. Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: ECCV ’92: Proceedings of the Second European Conference on Computer Vision, pp. 237–252 (1992)

  6. Bouguet, J.Y.: Pyramidal implementation of the Lucas Kanade feature tracker: description of the algorithm. Technical report, OpenCV Document, Intel Microprocessor Research Labs (2002)

  7. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly (2008)

  8. Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983). doi:10.1109/TCOM.1983.1095851

    Article  Google Scholar 

  9. Can, A., Stewart, C., Roysam, B., Tanenbaum, H.: A feature-based, robust, hierarchical algorithm for registering pairs of images of the curved human retina. IEEE Transactions on Pattern Analysis and Machine Intelligence (2002)

  10. Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking in airborne videos by multi-motion layer analysis. Mach. Vis. Appl. 23, 921–935 (2012)

    Article  Google Scholar 

  11. Corral, E.M.: Efficient model-based 3d tracking by using direct image registration. Ph.D. thesis, Facultad de Informática. Universidad Politécnica de Madrid, Spain (2012)

  12. Dufaux, F., Konrad, J.: Efficient, robust, and fast global motion estimation for video coding. IEEE Trans. Image Process. 9(3), 497–501 (2000)

    Article  Google Scholar 

  13. Dupac, J., Matas, J., Naiser, F.: Ultra-fast tracking based on zero-shift points. Image Vis. Comput. 30(12), 1016–1031 (2012)

    Article  Google Scholar 

  14. García Carrillo, L., Rondon, E., Sanchez, A., Dzul, A., Lozano, R.: Stabilization and trajectory tracking of a quad-rotor using vision. J. Intell. Robot. Syst. 61, 103–118 (2011)

    Article  Google Scholar 

  15. Hager, G., Belhumeur, P.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998). doi:10.1109/34.722606

    Article  Google Scholar 

  16. Hanna, K., Okamoto, N.: Combining stereo and motion analysis for direct estimation of scene structure. In: Proceedings of Fourth International Conference on Computer Vision, pp. 357–365 (1993). doi:10.1109/ICCV.1993.378192

  17. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, New York (2003)

    Google Scholar 

  18. Hess, R.: SIFT feature detector implementation in C. http://www.web.engr.oregonstate.edu/~hess/index.html (2007)

  19. Hess, R.: An open-source SIFT library. In: Proceedings of the International Conference on Multimedia, pp. 1493–1496 (2010). http://portal.acm.org/citation.cfm?id=1874256

  20. Holzer, S., Ilic, S., Navab, N.: Multilayer adaptive linear predictors for real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 105–117 (2013). doi:10.1109/TPAMI.2012.86

  21. Hwangbo, M., Kim, J.S., Kanade, T.: Inertial-aided klt feature tracking for a moving camera. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009. IROS 2009, pp. 1909–1916 (2009). doi:10.1109/IROS.2009.5354093

  22. Irani, M., Anandan, P.: About direct methods. In: Vision Algorithms: Theory and Practice, Lecture Notes in Computer Science, vol. 1883, pp. 267–277. Springer, Berlin (2000)

  23. Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994). doi:10.1007/BF01420982

    Article  Google Scholar 

  24. Jurie, F., Dhome, M.: Real time robust template matching. In: Rosin, P.L., Marshall, A.D. (eds.) British Machine Vision Conference, BMVC 2002, September, 2002, pp. 123–132. British Machine Vision Association, Cardiff (2002)

    Google Scholar 

  25. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012). doi:10.1109/TPAMI.2011.239

    Article  Google Scholar 

  26. Kumar, R., Sawhney, H., Samarasekera, S., Hsu, S., Tao, H., Guo, Y., Hanna, K., Pope, A., Wildes, R., Hirvonen, D., Hansen, M., Burt, P.: Aerial video surveillance and exploitation. Proc. IEEE 89(10), 1518–1539 (2001)

    Article  Google Scholar 

  27. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  28. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)

  29. Malis, E., Benhimane, S.: A unified approach to visual tracking and servoing. Robot. Auton. Syst. 52(1), 39–52 (2005). doi:10.1016/j.robot.2005.03.014. (Advances in Robot Vision)

    Article  Google Scholar 

  30. Martinez, C., Mejias, L., Campoy, P.: A multi-resolution image alignment technique based on direct methods for pose estimation of aerial vehicles. In: Proceedings of the International Conference on Digital Image Computing Techniques and Applications (DICTA), pp. 542–548 (2011). doi:10.1109/DICTA.2011.97

  31. Martínez, C., Richardson, T., Thomas, P., du Bois, J.L., Campoy, P.: A vision-based strategy for autonomous aerial refueling tasks. Robot. Auton. Syst. 61(8), 876–895 (2013). doi:10.1016/j.robot.2013.02.006

    Article  Google Scholar 

  32. Mejias, L., Saripalli, S., Campoy, P., Sukhatme, G.: Visual servoing approach for tracking features in urban areas using an autonomous helicopter. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 2503–2508, Orlando (2006)

  33. Mondragon, I.F., Campoy, P., Correa, J., Mejias, L.: Visual model feature tracking for UAV control. In: IEEE International Symposium on Intelligent Signal Processing, 2007. WISP 2007, pp. 1–6 (2007). doi:10.1109/WISP.2007.4447629

  34. Mondragón, I.F., Campoy, P., Martinez, C., Olivares-Mendez, M.: 3D pose estimation based on planar object tracking for UAVs control. In: Proceedings of IEEE International Conference on Robotics and Automation 2010 ICRA2010, Anchorage (2010)

  35. Rao, C., Guo, Y., Sawhney, H., Kumar, R.: A heterogeneous feature-based image alignment method. In: 18th International Conference on Pattern Recognition, 2006. ICPR 2006 (2006)

  36. Sawhney, H., Kumar, R.: True multi-image alignment and its application to mosaicing and lens distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 21(3), 235–243 (1999). doi:10.1109/34.754589

    Article  Google Scholar 

  37. Sawhney, H.S., Hsu, S., Kumar, R.: Robust video mosaicing through topology inference and local to global alignment. In: Proc. European Conference on Computer Vision, pp. 103–119 (1998)

  38. Sheikh, Y., Khan, S., Shah, M.: Feature-based georegistration of aerial images. In: International Conference on Geosensor Networks (2004)

  39. Shi, J., Tomasi, C.: Good features to track. In: 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’94), pp. 593–600 (1994)

  40. Shum, H.Y., Szeliski, R.: Panoramic vision. chap. Construction of Panoramic Image Mosaics with Global and Local Alignment, pp. 227–268. Springer, Secaucus (2001). http://portal.acm.org/citation.cfm?id=378207.378241

  41. Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vis. 2(1), 1–104 (2006). doi:10.1561/0600000009

    Article  Google Scholar 

  42. Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’97, pp. 251–258. ACM Press/Addison-Wesley Publishing Co., New York (1997). doi:10.1145/258734.258861

  43. Teuliere, C., Eck, L., Marchand, E.: Chasing a moving target from a flying uav. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4929–4934 (2011). doi:10.1109/IROS.2011.6094404

  44. Torr, P.H.S., Zisserman, A.: Feature based methods for structure and motion estimation. In: Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, ICCV ’99, pp. 278–294. Springer, London (2000)

  45. Tsaig, Y., Averbuch, A.: Automatic segmentation of moving objects in video sequences: a region labeling approach. IEEE Trans. Circuits Syst. Video Techol. 12(7), 597–612 (2002)

    Article  Google Scholar 

  46. Turcajova, R., Kautsky, J.: A hierarchical multiresolution technique for image registration. In: Proceedings of SPIE Mathematical Imaging: Wavelet Applications in Signal and Image Processing (1996)

  47. Ye, G.: Image Registration and Super-resolution Mosaicing. Ph.D. thesis, The University of New South Wales (2005)

  48. Zhang, H., Yuan, F.: Vehicle tracking based on image alignment in aerial videos. In: EMMCVPR’07: Proceedings of the 6th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 295–302. Springer, Berlin (2007)

  49. Zimmermann, K., Matas, J., Svoboda, T.: Tracking by an optimal sequence of linear predictors. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 677–692 (2009)

    Article  Google Scholar 

  50. Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Article  Google Scholar 

Download references

Acknowledgments

The work described in this paper is the result of several research stages conducted at the Computer Vision Group of the Universidad Politécnica de Madrid. The authors would like to thank the Universidad Politécnica de Madrid, the Consejería de Educación de la Comunidad de Madrid, and the Fondo Social Europeo (FSE) for the Ph.D. Scholarships of some of the Authors. This work has been supported by the Spanish Ministry of Science under grant MICYT DPI2010-20751-C02-01.

Author information

Authors and Affiliations

  1. Computer Vision Group, Universidad Politécnica de Madrid, Centro de Automática y Robótica (CAR) UPM-CSIC, C/José Gutiérrez Abascal 2, 28006 , Madrid, Spain

    Carol Martínez, Pascual Campoy, José Luis Sánchez-Lopez & Miguel A. Olivares-Méndez

  2. Department of Industrial Engineering, Faculty of Engineering, Pontificia Universidad Javeriana, Bogotá D.C., Colombia

    Iván F. Mondragón

  3. Automation Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg, Luxembourg

    Miguel A. Olivares-Méndez

Authors
  1. Carol Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pascual Campoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Iván F. Mondragón
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Luis Sánchez-Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Miguel A. Olivares-Méndez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carol Martínez.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 48222 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martínez, C., Campoy, P., Mondragón, I.F. et al. HMPMR strategy for real-time tracking in aerial images, using direct methods. Machine Vision and Applications 25, 1283–1308 (2014). https://doi.org/10.1007/s00138-014-0617-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0617-2

Keywords

Search

Navigation