Article

Camera phone based motion sensing: interaction techniques, applications and performance study

Authors:

John CannyAuthors Info & Claims

UIST '06: Proceedings of the 19th annual ACM symposium on User interface software and technology

Pages 101 - 110

https://doi.org/10.1145/1166253.1166270

Published: 15 October 2006 Publication History

Abstract

This paper presents TinyMotion, a pure software approach for detecting a mobile phone user's hand movement in real time by analyzing image sequences captured by the built-in camera. We present the design and implementation of TinyMotion and several interactive applications based on TinyMotion. Through both an informal evaluation and a formal 17-participant user study, we found that 1. TinyMotion can detect camera movement reliably under most background and illumination conditions. 2. Target acquisition tasks based on TinyMotion follow Fitts' law and Fitts law parameters can be used for TinyMotion based pointing performance measurement. 3. The users can use Vision TiltText, a TinyMotion enabled input method, to enter sentences faster than MultiTap with a few minutes of practicing. 4. Using camera phone as a handwriting capture device and performing large vocabulary, multilingual real time handwriting recognition on the cell phone are feasible. 5. TinyMotion based gaming is enjoyable and immediately available for the current generation camera phones. We also report user experiences and problems with TinyMotion based interaction as resources for future design and development of mobile interfaces.

Supplementary Material

JPG File (1166270.jpg)

Download
18.02 KB

index.html (index.html)

Slides from the presentation

Download
.96 KB

ZIP File (p101-slides.zip)

Supplemental material for Camera phone based motion sensing: interaction techniques, applications and performance study

Download
9.05 MB

Audio only (1166270.mp3)

Download
6.42 MB

Video (1166270.mp4)

Download
155.56 MB

References

[1]

Ballagas, R., Rohs, M. et al, Sweep and Point & Shoot: Phonecam-Based Interactions for Large Public Displays. In Ext. Abstracts of CHI '05, April 2005

Digital Library

[2]

Drab, S., Artner, N., Motion Detection as Interaction Technique for Games & Applications on Mobile Devices, In Ext. Abstracts of PERVASIVE '05: (PERMID), Munich, Germany.

[3]

EyeMobile, http://www.eyemobile.com

[4]

Fitts, P. M. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381--391, 1954.

[5]

Fitzmaurice, G. W., Zhai, S., Chignell, M., Virtual Reality for Palmtop Computers. In ACM TOIS, pp. 197--218. July 1993.

Digital Library

[6]

Forsyth, D., and Ponce, J. Computer Vision: A Modern Approach. Prentice Hall, Upper Saddle River, NJ, 2003.

Digital Library

[7]

Freeman, W. T., et. al., Computer Vision for Interactive Computer Graphics. In IEEE Computer Graphics and Applications, May-June, pp. 42--53, 1998

Digital Library

[8]

Furht, B., Greenberg, J., Westwater, R., Motion Estimation Algorithm for Video Compression, Kluwer Academic Publishers, Boston/Dordrecht/London, 1997

Digital Library

[9]

Hannuksela, J., Sangi, P., and Heikkila J., A Vision-Based Approach for Controlling User Interfaces of Mobile Devices, In Proc. of IEEE Workshop on Vision for Human-Computer Interaction (V4HCI), 2005

Digital Library

[10]

Hansen, T., et al., Mixed Interaction Space - Designing for Camera Based Interaction with Mobile Devices, In Ext. Abstracts of CHI '05, April 2005

Digital Library

[11]

Harrison, B. L., Fishkin, K., A. et al, Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces. In Proc. of CHI'98, pp. 17--24, April 1998.

Digital Library

[12]

Hinckley, K., Pierce, J., Sinclair, M., Horvitz, E., Sensing Techniques for Mobile Interaction. In Proc. of UIST 2000, pp. 91--100.

Digital Library

[13]

Horn, B., and Schunck, B. Determining optical flow. AI Memo 572. Massachusetts Institue of Technology, 1980.

Digital Library

[14]

Kuhn, P., Algorithms, Complexity Analysis and VLSI Architectures for MPEG-4 Motion Estimation, Kluwer Academic Publishers.

Digital Library

[15]

Kurtenbach, G., and Buxton, W. User learning and performance with marking menus. In Proc. of CHI'94, pp. 258--264, April 1994

Digital Library

[16]

Moehring, M., Lessig C., and Bimber O. Optical tracking and video see-through AR on consumer cell phones. In Proc. of Workshop on Virtual and Augmented Reality of the GI-Fachgruppe AR/VR, pp. 193--204. 2004

[17]

Moeslund, T. B., and Granum, E. A survey of computer vision-based human motion capture. Computer Vision and Image Understanding 18, pp. 231--268, 2001.

Digital Library

[18]

Rekimoto, J., Tilting Operations for Small Screen Interfaces. In Proceedings of. UIST 1996, pp. 167--168.

Digital Library

[19]

Rekimoto, J., Ayatsuka, Y., CyberCode: designing augmented reality environments with visual tags, In Proc. of DARE 2000, pp. 1--10.

Digital Library

[20]

Rohs, M., Zweifel, P., A Conceptual Framework for Camera Phone-based Interaction Techniques, In Proc. of PERVASIVE 2005, Munich, Germany, May 2005

Digital Library

[21]

The Development of Camera Phone Module Industry, 2005-2006, http://www.okokok.com.cn/Abroad/Abroad_show.asp?ArticleID=1034

[22]

Wang, L., Hu, W., and Tan, T., Recent developments in human motion analysis, Pattern. Recognition, 36 pp. 585--601, 2003

[23]

Want, R. TiltType: Accelerometer-Supported Text Entry for Very Small Devices, In Proc. of UIST 2002.

Digital Library

[24]

Wigdor, D., Balakrishnan, R., TiltText: Using tilt for text input to mobile phones. In Proc. of UIST 2003.

Digital Library

[25]

Yee, K. P., Peephole Displays: Pen Interaction on Spatially Aware Handheld Computers. In Proc. of CHI 2003, pp. 1--8, 2003.

Digital Library

Cited By

Myers B(2024)Pick, Click, Flick!undefinedOnline publication date: 14-Mar-2024
Khudhair ABen Ahmed A(2023)Utilizing Circumcoronene and BN Circumcoronene for the delivery and adsorption of the anticancer drug floxuridineComputational and Theoretical Chemistry10.1016/j.comptc.2023.1140751222(114075)Online publication date: Apr-2023
Babic TReiterer HHaller M(2022)Understanding and Creating Spatial Interactions with Distant Displays Enabled by Unmodified Off-The-Shelf SmartphonesMultimodal Technologies and Interaction10.3390/mti61000946:10(94)Online publication date: 19-Oct-2022
Show More Cited By

Index Terms

Camera phone based motion sensing: interaction techniques, applications and performance study
1. Human-centered computing
  1. Human computer interaction (HCI)
  2. Interaction design
    1. Interaction design theory, concepts and paradigms

Recommendations

TinyMotion: camera phone based interaction methods
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

This paper presents TinyMotion, a pure software approach that detects the movements of cell phones in real time by analyzing image sequences captured by the built-in camera. Typical movements that TinyMotion detects include - horizontal and vertical ...
Sensing-based interaction for information navigation on handheld displays
MobileHCI '07: Proceedings of the 9th international conference on Human computer interaction with mobile devices and services

Information navigation on handheld displays is characterized by the small display dimensions and limited input capabilities of today's mobile devices. Special strategies are required to help users navigate to off-screen content and develop awareness of ...
SHRIMP: solving collision and out of vocabulary problems in mobile predictive input with motion gesture
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Dictionary-based disambiguation (DBD) is a very popular solution for text entry on mobile phone keypads but suffers from two problems: 1. the resolution of encoding collision (two or more words sharing the same numeric key sequence) and 2. entering out-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '06: Proceedings of the 19th annual ACM symposium on User interface software and technology

October 2006

354 pages

ISBN:1595933131

DOI:10.1145/1166253

General Chair:
Pierre Wellner
IDIAP Research Institute, Switzerland
,
Program Chair:
Ken Hinckley
Microsoft Research, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

UIST06

Sponsor:

UIST06: The 19th Annual ACM Symposium on User Interface Software and Technology

October 15 - 18, 2006

Montreux, Switzerland

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

98
Total Citations
View Citations
3,193
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)7

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Myers B(2024)Pick, Click, Flick!undefinedOnline publication date: 14-Mar-2024
Khudhair ABen Ahmed A(2023)Utilizing Circumcoronene and BN Circumcoronene for the delivery and adsorption of the anticancer drug floxuridineComputational and Theoretical Chemistry10.1016/j.comptc.2023.1140751222(114075)Online publication date: Apr-2023
Babic TReiterer HHaller M(2022)Understanding and Creating Spatial Interactions with Distant Displays Enabled by Unmodified Off-The-Shelf SmartphonesMultimodal Technologies and Interaction10.3390/mti61000946:10(94)Online publication date: 19-Oct-2022
CHEN LCHEN D(2021)A Comparison Study on Camera-Based Pointing Techniques for Handheld DisplaysIEICE Transactions on Electronics10.1587/transele.2020DIP0003E104.C:2(73-80)Online publication date: 1-Feb-2021
Kim DPark KLee GIqbal SMacLean KChevalier FMueller S(2020)OddEyeCamProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379337.3415889(85-97)Online publication date: 20-Oct-2020
Chen LChen DTan Z(2020)Fingertip-Specific Mobile Interaction Through Camera-Based Fingertip IdentificationJournal of Physics: Conference Series10.1088/1742-6596/1673/1/0120461673(012046)Online publication date: 24-Nov-2020
Liu CWu NZhang JSu W(2019)Tilt Space: A Systematic Exploration of Mobile Tilt for Design PurposeHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29387-1_29(497-517)Online publication date: 2-Sep-2019
Liu CLiu CMao HSu W(2019)Tilt-Scrolling: A Comparative Study of Scrolling Techniques for Mobile DevicesIntelligent Computing Methodologies10.1007/978-3-030-26766-7_18(189-200)Online publication date: 24-Jul-2019
Babic TReiterer HHaller M(2018)Pocket6Proceedings of the 2018 ACM Symposium on Spatial User Interaction10.1145/3267782.3267785(2-10)Online publication date: 13-Oct-2018
Chen LChen DEmrouznejad AQian Z(2018)CaMapProceedings of the 2nd International Conference on Computer Science and Application Engineering10.1145/3207677.3278067(1-5)Online publication date: 22-Oct-2018
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents