Cited By
View all- Chauhan THéjja-Brichard YCottereau B(2020)Modelling binocular disparity processing from statistics in natural scenesVision Research10.1016/j.visres.2020.07.009176(27-39)Online publication date: Nov-2020
A unified approach to learning depth and motion features
Article No.: 78, Pages 1 - 7
Abstract
We present a model for the joint estimation of disparity and motion. The model is based on learning about the interrelations between images from multiple cameras, multiple frames in a video, or the combination of both. We show that learning depth and motion cues, as well as their combinations, from data is possible within a single type of architecture and a single type of learning algorithm, by using biologically inspired "complex cell" like units, which encode correlations between the pixels across image pairs. Our experimental results show that the learning of depth and motion makes it possible to achieve competitive performance in 3-D activity analysis, and to outperform existing hand-engineered 3-D motion features by a very large margin.
References
[1]
E. H. Adelson and J. R. Bergen. Spatiotemporal energy models for the perception of motion. J. OPT. SOC. AM. A, 2(2):284–299, 1985.
[2]
C. F. Cadieu and B. A. Olshausen. Learning Intermediate-Level Representations of Form and Motion from Natural Movies. Neural Computation, 24(4):827–866, Dec. 2011.
[3]
D. Fleet, H. Wagner, and D. Heeger. Neural encoding of binocular disparity: Energy models, position shifts and phase shifts. Vision Research, 36(12):1839–1857, June 1996.
[4]
D. J. Fleet, A. D. Jepson, and M. R. Jenkin. Phase-based disparity measurement. CVGIP: Image understanding, 53(2):198–210, 1991.
[5]
A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.
[6]
S. Hadfield and R. Bowden. Hollywood 3d: Recognizing actions in 3d natural scenes. In CVPR, Portland, Oregon, June 23-28 2013.
[7]
S. Hadfield, K. Lebeda, and R. Bowden. Natural action recognition using invariant 3d motion encoding. In Computer Vision–ECCV 2014, pages 758–771. Springer, 2014.
[8]
R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.
[9]
C. Kanan and G. Cottrell. Robust classification of objects, faces, and flowers using natural image statistics. In CVPR, pages 2472–2479, 2010.
[10]
K. Konda and R. Memisevic. Learning to combine depth and motion. arXiv preprint arXiv:1312.3429, 2013.
[11]
K. R. Konda, R. Memisevic, and V. Michalski. Learning to encode motion using spatio-temporal synchrony. In Proceedings of ICLR, April 2014.
[12]
Q. Le, W. Zou, S. Yeung, and A. Ng. Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In CVPR, 2011.
[13]
R. Memisevic. Gradient-based learning of higher-order image features. In ICCV, pages 1591–1598. IEEE, 2011.
[14]
R. Memisevic. On multi-view feature learning. In Proceedings of ICML, pages 161–168, 2012.
[15]
I. Ohzawa, G. C. Deangelis, and R. D. Freeman. Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. Science, 249(4972):1037–1041, 1990.
[16]
N. Qian. Computing stereo disparity and motion with known binocular cell properties. Neural Computation, 6(3):390–404, 1994.
[17]
S. Rifai, P. Vincent, X. Muller, X. Glorot, and Y. Bengio. Contractive Auto-Encoders: Explicit Invariance During Feature Extraction. In ICML, 2011.
[18]
D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV, 47(1-3):7–42, 2002.
[19]
G. W. Taylor, R. Fergus, Y. LeCun, and C. Bregler. Convolutional learning of spatio-temporal features. In ECCV, 2010.
Index Terms
- A unified approach to learning depth and motion features
Recommendations
Extraction and temporal segmentation of multiple motion trajectories in human motion
A new method for extraction and temporal segmentation of multiple motion trajectories in human motion is presented. The proposed method extracts motion trajectories generated by body parts without any initialization or any assumption on color ...
Descriptive temporal template features for visual motion recognition
In this paper, a human action recognition system is proposed. The system is based on new, descriptive 'temporal template' features in order to achieve high-speed recognition in real-time, embedded applications. The limitations of the well-known 'Motion ...
Accurate motion deblurring using camera motion tracking and scene depth
WACV '13: Proceedings of the 2013 IEEE Workshop on Applications of Computer Vision (WACV)In this paper, we propose an estimation algorithm for spatially-variant blur due to camera motion. To estimate the most accurate latent image, we integrated depth sensor (Microsoft Kinect) and IMU sensor with the camera. The joint analysis of the blurry ...
Comments
Information & Contributors
Information
Published In
December 2014
692 pages
ISBN:9781450330619
DOI:10.1145/2683483
- General Chairs:
- A. G. Ramakrishnan,
- Jitendra Malik,
- Program Chairs:
- Alex Efros,
- C. V. Jawahar,
- Manik Varma
Copyright © 2014 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 December 2014
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICVGIP '14
ICVGIP '14: Indian Conference on Computer Vision Graphics and Image Processing
December 14 - 18, 2014
Bangalore, India
Acceptance Rates
Overall Acceptance Rate 95 of 286 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 100Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Aug 2024
Other Metrics
Citations
Cited By
View all- Chauhan THéjja-Brichard YCottereau B(2020)Modelling binocular disparity processing from statistics in natural scenesVision Research10.1016/j.visres.2020.07.009176(27-39)Online publication date: Nov-2020
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in