research-article

Multitoe: high-precision interaction with back-projected floors based on high-resolution multi-touch input

Authors:

Thomas Augsten,

Konstantin Kaefer,

Caroline Fetzer,

Torsten Becker,

Christian Holz,

Patrick BaudischAuthors Info & Claims

UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technology

Pages 209 - 218

https://doi.org/10.1145/1866029.1866064

Published: 03 October 2010 Publication History

Abstract

Tabletop applications cannot display more than a few dozen on-screen objects. The reason is their limited size: tables cannot become larger than arm's length without giving up direct touch. We propose creating direct touch surfaces that are orders of magnitude larger. We approach this challenge by integrating high-resolution multitouch input into a back-projected floor. As the same time, we maintain the purpose and interaction concepts of tabletop computers, namely direct manipulation.

We base our hardware design on frustrated total internal reflection. Its ability to sense per-pixel pressure allows the floor to locate and analyze users' soles. We demonstrate how this allows the floor to recognize foot postures and identify users. These two functions form the basis of our system. They allow the floor to ignore users unless they interact explicitly, identify and track users based on their shoes, enable high-precision interaction, invoke menus, track heads, and allow users to control high-degree of freedom interactions using their feet. While we base our designs on a series of simple user studies, the primary contribution on this paper is in the engineering domain.

Supplementary Material

JPG File (189i-baudisch.jpg)

Download
6.68 KB

MOV File (189i-baudisch.mov)

Download
71.33 MB

References

[1]

}}Bay, H., Tuytelaars, T., and Van Gool, L. Surf: Speeded up robust features. Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, 346--359, 2008.

Digital Library

[2]

}}Brumitt, B., Meyers, B., Krumm, J., Kern, A., and Shafer, S. A. EasyLiving: Technologies for Intelligent Environments. In Proc. Ubicomp 2000, 12--29.

Digital Library

[3]

}}Cao, X., Wilson A., Balakrishnan, R., Hinckley, K., and Hudson, S. ShapeTouch: Leveraging Contact Shape on Interactive Surfaces. In Proc. TABLETOP'08, 129--136.

[4]

}}Choi, I., Ricci, C. Foot-mounted gesture detection and its application in virtual environments. IEEE International Conference on Systems, Man and Cybernetics. '97, 4248--4253.

[5]

}}Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. Surround-Screen Projection-based Virtual Reality: The Design and Implementation of the CAVE. In Proc. SIGGRAPH'93, 135--142.

Digital Library

[6]

}}Davidson, P., Han, J. Extending 2D object arrangement with pressure-sensitive layering cues. In Proc. UIST'08, 87--90.

Digital Library

[7]

}}Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. In Proc. UIST'01, 219--226.

Digital Library

[8]

}}Greenberg, S. and Buxton, B. Usability evaluation considered harmful (some of the time). In Proc. CHI '08, 111--120.

Digital Library

[9]

}}Grønbæk, K., Iversen, O. S., Kortbek, K. J., Nielsen, K. R., and Aagaard, L. IGameFloor: a platform for co-located collaborative games. In Proc. ACE '07, 64--71.

Digital Library

[10]

}}Han, J. Y. Low-cost multi-touch sensing through frustrated total internal reflection. In Proc. UIST '05, 115--118.

Digital Library

[11]

}}Hartmann, Björn, Morris, M. R., Benko, H., and Wilson, A. Augmenting Interactive Tables with Mice & Keyboards. In Proc. UIST'09, 149--152.

Digital Library

[12]

}}Holz, C., and Baudisch, P. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints. In Proc. CHI'10, 10 pages.

Digital Library

[13]

}}Krogh, P. G., Ludvigsen, M., Lykke-Olesen, A. Help me pull that cursor. A Collaborative Interactive Floor Enhancing Community Interaction. In Proc. OZCH'04, 22--24.

[14]

}}LaViola, J. J., Feliz, D. A., Keefe, D. F., and Zeleznik, R. C. Hands-free multi-scale navigation in virtual environments. In Proc. I3D '01, pp. 9--15.

Digital Library

[15]

}}Microsoft Surface http://www.microsoft.com/surfac.

[16]

}}Morris, M. R. Designing Tabletop Groupware. In UIST'05 Doctoral Symposium.

[17]

}}Mota, S. and Picard, R. Automated posture analysis for detecting learner's interest level. In Proc. CVPR'03, 49--56.

[18]

}}OpenCV http://opencv.willowgarage.co.

[19]

}}Orr, R. J. and Abowd, G. D. The smart floor: a mechanism for natural user identification and tracking. In CHI'00 Extended Abstracts, 275--276.

Digital Library

[20]

}}Paradiso, J. A., Hu, E. Expressive Footwear for Computer-Augmented Dance Performance. In Proc. ISWC '97, 165--166.

Digital Library

[21]

}}Paradiso, J., Abler, C., Hsiao, K., Reynolds, M. The Magic Carpet - Physical Sensing for Immersive Environments. In Proc. CHI' 97, 277--278.

Digital Library

[22]

}}Paradiso, J., Hu, E., Hsiao, K. Y. Instrumented Footwear for Interactive Dance. In Proc. IDAT '99, 89--92.

[23]

}}Pavlou, M. and Allinson, N. M. Automatic extraction and classification of footwear patterns. In Proc. IDEAL'06, 721--728.

Digital Library

[24]

}}Pavlou, M. and Allinson, N. M. Footwear Recognition. In Proc. Encyclopedia of Biometrics'09, 557--562.

[25]

}}Pier, K. and Landay, J. Issues for Location-Independent Interfaces. Tech. Rep. ISTL92-4, Xerox PARC, Dec 1992.

[26]

}}Piper, A., O'Brien, Ringel Morris, M., and Winograd, T. SIDES: a cooperative tabletop computer game for social skills development. In Proc. CSCW'06, 1--10.

Digital Library

[27]

}}Potter, R. L., Weldon, L. J., Shneiderman, B. Improving the accuracy of touch screens: an experimental evaluation of three strategies. In Proc. CHI '88, 27--32.

Digital Library

[28]

}}Richardson, B., Leydon, K., Fernstrom, M., and Paradiso, J. A. Z-Tiles: building blocks for modular, pressure-sensing floorspaces. In CHI '04 Extended Abstracts, 1529--1532.

Digital Library

[29]

}}Rosenberg, I. and Perlin, K. The UnMousePad: an interpolating multi-touch force-sensing input pad. In Proc. SIGGRAPH '09/ACM Trans. Graph. (Vol. 28, No. 3, Article 65), 1--9.

Digital Library

[30]

}}Shirai, A., Sato, M., Kume, Y., and Kusahara, M. Foot Interface: Fantastic PhantomSlipper. Report ER14, Tokyo Institute of Polytechnics, 1998; see also SIGGRAPH 98Conference abstracts and applications, 114.

Digital Library

[31]

}}Smart Table http://smarttech.co.

[32]

}}Sugiura, A. and Koseki, Y. A user interface using fingerprint recognition: holding commands and data objects on fingers. In Proc. UIST '98, 71--79.

Digital Library

[33]

}}Visell, Y., Law, A., and Cooperstock, J. R. Touch Is Everywhere: Floor Surfaces as Ambient Haptic Interfaces. EEE Trans. Haptics 2, 3 (Jul. 2009), 148--15.

Digital Library

[34]

}}Vogel D. and Balakrishnan R. Occlusion-Aware Interfaces. In Proc. CHI '10, 10 pages.

Digital Library

[35]

}}Vogel, D. and Baudisch, P. Shift: a technique for operating pen-based interfaces using touch. In Proc. CHI '07, 657--666.

Digital Library

[36]

}}Wang, F., Cao, X., Ren, X., Irani, P. Detecting and leveraging finger orientation for interaction with direct-touch surfaces. In Proc. UIST '09, 23--32.

Digital Library

[37]

}}Ware, C., Arthur, K., and Booth, K. S. Fish tank Virtual Reality. In Proc. INTERCHI '93, 37--42.

Digital Library

[38]

}}Matsushita, N., and Rekimoto, J. HoloWall: Designing a Finger, Hand, Body, and Object Sensitive Wall. In Proc UIST'97, 209--210.

Digital Library

Cited By

Tsai CYen RKim DVogel D(2024)Gait Gestures: Examining Stride and Foot Strike Variation as an Input Method While WalkingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676342(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676342
Chan LMi THsueh ZHuang YHsu M(2024)Seated-WIP: Enabling Walking-in-Place Locomotion for Stationary Chairs in Confined SpacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642395(1-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642395
Sawada NYamamoto TSugiura Y(2024)Converting Tatamis into Touch Sensors by Measuring Capacitance2024 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII58957.2024.10417676(554-558)Online publication date: 8-Jan-2024
https://doi.org/10.1109/SII58957.2024.10417676
Show More Cited By

Index Terms

Multitoe: high-precision interaction with back-projected floors based on high-resolution multi-touch input
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens
    2. Interaction paradigms

Recommendations

GravitySpace: tracking users and their poses in a smart room using a pressure-sensing floor
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

We explore how to track people and furniture based on a high-resolution pressure-sensitive floor. Gravity pushes people and objects against the floor, causing them to leave imprints of pressure distributions across the surface. While the sensor is ...
Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces

Selecting and moving digital content on interactive tabletops often involves accessing the workspace beyond arm's reach. We present Pointable, an in-air, bimanual perspective-based interaction technique that augments touch input on a tabletop for ...
Enabling beyond-surface interactions for interactive surface with an invisible projection
UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technology

This paper presents a programmable infrared (IR) technique that utilizes invisible, programmable markers to support interaction beyond the surface of a diffused-illumination (DI) multi-touch system. We combine an IR projector and a standard color ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technology

October 2010

476 pages

ISBN:9781450302715

DOI:10.1145/1866029

General Chair:
Ken Perlin
New York University
,
Program Chairs:
Mary Czerwinski
Microsoft Research
,
Rob Miller
MIT CSAIL

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '10

Sponsor:

UIST '10: The 23nd Annual ACM Symposium on User Interface Software and Technology

October 3 - 6, 2010

New York, New York, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

86
Total Citations
View Citations
1,750
Total Downloads

Downloads (Last 12 months)51
Downloads (Last 6 weeks)5

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tsai CYen RKim DVogel D(2024)Gait Gestures: Examining Stride and Foot Strike Variation as an Input Method While WalkingProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676342(1-16)Online publication date: 13-Oct-2024
Chan LMi THsueh ZHuang YHsu M(2024)Seated-WIP: Enabling Walking-in-Place Locomotion for Stationary Chairs in Confined SpacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642395(1-13)Online publication date: 11-May-2024
Sawada NYamamoto TSugiura Y(2024)Converting Tatamis into Touch Sensors by Measuring Capacitance2024 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII58957.2024.10417676(554-558)Online publication date: 8-Jan-2024
Liu JEns BProuzeau ASmiley JNixon IGoodwin SDwyer T(2023)DataDancing: An Exploration of the Design Space For Visualisation View Management for 3D Surfaces and SpacesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580827(1-17)Online publication date: 19-Apr-2023
Li XMatthies D(2022)ShoeTect: Detecting Body Posture, Ambulation Activity, Gait Abnormalities, and Terrain with Multisensory Smart FootwearProceedings of the 7th International Workshop on Sensor-based Activity Recognition and Artificial Intelligence10.1145/3558884.3558904(1-10)Online publication date: 19-Sep-2022
Yan ZZhou JWu YLiu GLuo DZhou ZMi HSun LChen XTao YZhang YWang G(2022)Shoes++Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35346206:2(1-29)Online publication date: 7-Jul-2022
Yoshida TOkazaki NTakaki KHirose MKitagawa SInami M(2022)Flexel: A Modular Floor Interface for Room-Scale Tactile SensingProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545699(1-12)Online publication date: 29-Oct-2022
Jacobsen Rvan Berkel NSkov MJohansen SKjeldskov J(2022)Do You See What I Hear? — Peripheral Absolute and Relational Visualisation Techniques for Sound ZonesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501938(1-13)Online publication date: 29-Apr-2022
Umezu NYuming B(2022)Floor Projection with QR code-based Interactions and Visitor Drawings2022 IEEE 11th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE56475.2022.10014058(318-320)Online publication date: 18-Oct-2022
Ullmer BShaer OMazalek AHummels C(2022)Weaving Fire into FormundefinedOnline publication date: 20-Jul-2022
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents