research-article

HACHIStack: dual-layer photo touch sensing for haptic and auditory tapping interaction

Authors:

Hiroyuki KajimotoAuthors Info & Claims

CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1411 - 1420

https://doi.org/10.1145/2470654.2466187

Published: 27 April 2013 Publication History

Abstract

We present a novel photo touch sensing architecture, HACHIStack. It can measure the approaching velocity of an object and predict its contact time with the touch screen using two optical sensing layers above the surface. The photo sensing layers form three unique capabilities: high-speed sampling, velocity acquisition, and contact time prediction. This work quantitatively examines these capabilities through two laboratory experiments, and confirms that the capabilities of HACHIStack are sufficient for multimodal interaction, in particular, touch-based interaction with haptic enhancement. We then present three applications with HACHIStack: 1) chromatic percussions (xylophone and glockenspiel) with haptic feedback; 2) no-delay haptic feedback with the sensation of tapping on various simulated materials (e.g., rubber, wood and aluminum); and 3) a virtual piano instrument that allows players to perform weak and strong strokes by changing the tapping velocity.

Supplementary Material

suppl.mov (chi0701-file3.mp4)

Supplemental video

Download
77.53 MB

References

[1]

Adelstein, B. D., Begault, D. R., Anderson M. R. and Wenzel, E. M. Sensitivity to haptic-auditory asynchrony. In Proc. ICMI 2003, ACM Press (2003), 73--76.

Digital Library

[2]

Annett, M., Grossman, T., Wigdor, D. and Fitzmaurice, G. Medusa: a proximity-aware multi-touch tabletop. In Proc. UIST 2011, ACM Press (2011), 337--346.

Digital Library

[3]

Bau, O., Poupyrev, I., Israr, A. and Harrison, C. Teslatouch: electrovibration for touch surfaces. In Proc. UIST 2010, ACM Press (2010), 283--292.

Digital Library

[4]

Benko, H. Beyond flat surface computing: challenges of depth-aware and curved interfaces. In Proc. MM 2009, ACM Press (2009), 935--944.

Digital Library

[5]

Benko, H., Ishak, E. W. and Feiner, S. Cross-dimensional gestural interaction techniques for hybrid immersive environments. In Proc. IEEE Conference on Virtual Reality, IEEE (2005), 209--216.

Digital Library

[6]

Benko, H., Jota, R. and Wilson, A. D. MirageTable: freehand interaction on a projected augmented reality table top. In Proc. CHI 2012, ACM Press (2012), 199--208.

Digital Library

[7]

Benko, H. and Wilson, A. DepthTouch: using depth-sensing camera to enable freehand interactions on and above the interactive surface. Microsoft Research Technical Report MSR-TR-2009-23, Microsoft (2009).

[8]

Butler, A., Izadi, S. and Hodges, S. SideSight: multi"touch" interaction around small devices. In Proc. UIST 2008, ACM Press (2008), 201--204.

Digital Library

[9]

Cutler, L. D., Fröhlich, B. and Hanrahan, P. Two-handed direct Manipulation on the responsive workbench. In Proc. I3D 1997, ACM Press (1997), 107--114.

Digital Library

[10]

Echtler, F., Pototsching, T. and Klinker, G. An LED-based multitouch sensor for LCD screens. In Proc. TEI 2010, ACM Press (2010), 227--230.

Digital Library

[11]

Fiene, J. P. and Kuchenbecker, K. J. Shaping event-based haptic transients via an improved understanding of real contact dynamics. In Proc. IEEE World Haptics Conference 2007, IEEE (2007), 170--175.

Digital Library

[12]

Fukumoto, M. and Sugimura, T. Active Click: tactile feedback for touch panels. Ext. Abstracts CHI 2001, ACM Press (2001), 121--122.

Digital Library

[13]

Grossman, T., Hinckley, K., Baudisch, P., Agrawala, M. and Balakrishnan. Hover Widgets: using the tracking state to extend the capabilities of pen-operated devices. In Proc. CHI 2006, ACM Press (2006), 861--870.

Digital Library

[14]

Hachisu, T., Cirio, G., Marchal, M., Lécuyer, A. and Kajimoto, H. Virtual chromatic percussions simulated by pseudo-haptic and vibrotactile feedback. In Proc. ACE 2011, ACM Press (2011), 20.

Digital Library

[15]

Hachisu, T., Sato, M., Fukushima, S. and Kajimoto, H. HaCHIStick: simulating haptic sensation on tablet PC for musical instruments application. In Proc. UIST 2011, ACM Press (2011), 73--74.

Digital Library

[16]

Hachisu, T., Sato, M., Fukushima, S. and Kajimoto, H. Augmentation of material property by modulating vibration resulting from tapping. EuroHaptics 2012 Part 1, Springer (2012), 173--180.

Digital Library

[17]

Han, J. Y. Low-cost multi-touch sensing through frustrated total internal reflection. In Proc. UIST 2005, ACM Press (2005), 115--118.

Digital Library

[18]

Harrison, C. Benko, H. and Wilson, A. D. OmniTouch: wearable multitouch interaction everywhere. In Proc. UIST 2011, ACM Press (2011), 441--450.

Digital Library

[19]

Harrison, C. and Hudson, S. E. Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices. In Proc. UIST 2009, ACM Press (2009), 121--124.

Digital Library

[20]

Hilliges, O., Izadi, S., Wilson, A. D., Hodges, S., Garcia-Mendoza, A. and Butz, A. Interactions in the air: adding further depth to interactive tabletops. In Proc. UIST 2009, ACM Press (2009), 139--148.

Digital Library

[21]

Hirsch, M., Lanman, D., Holtzman, H. and Raskar, R. BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields. In Proc. SIGGRAPH Asia 2009, ACM Press (2009), 1--9.

Digital Library

[22]

Hodges, S., Izadi, S., Butler, A., Rrustemi, A. and Buxton, B. ThinSight: versatile multi-touch sensing for thin form-factor display. In Proc. UIST 2007, ACM Press (2007), 259--268.

Digital Library

[23]

Hofer, R., Nadff, D. and Kunz, A. FLATIR: FTIR multi-touch detection on a discrete distributed sensor array. In Proc. TEI 2009, ACM Press (2009), 317--322.

Digital Library

[24]

Izadi, S., Hodges, S., Taylor, S., Rosenfeld, D., Villar, N., Butler, A. and Westhues, J. Going beyond the display: a surface technology with an electronically switchable diffuser. In Proc. UIST 2008, ACM Press (2008), 269--278.

Digital Library

[25]

Janasen, Y., Karrer, T. and Borchers, J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. In Proc. ITS 2010, ACM Press (2010), 11--14.

Digital Library

[26]

Kratz, S. and Rohs, M. HoverFlow: expanding the design space of around-device interaction. In Proc. MobileHCI 2009, ACM Press (2009), Article No. 4.

Digital Library

[27]

Kyung, K. U. and Lee, J. Y. Ubi-Pen: a haptic interface with texture and vibrotactile display. IEEE Computer Graphics and Applications 29, 1 (2009), 56--64.

Digital Library

[28]

LaMotte, R. H. Softness discrimination with a tool. J. Neurophysiology 83 (2000), 1777--1786.

[29]

Lee, J. C., Dietz, P. H., Leigh, D., Yerazunis, W. S. and Hudson, S. E. Haptic Pen: a tactile feedback stylus for touch screens. In Proc. of UIST 2004, ACM Press (2004), 291--294.

Digital Library

[30]

Lylykangas, J., Surakka, V., Salminen K., Raisamo, J., Laitinen, P., Roning, K. and Raisamo, R. Designing tactile feedback for piezo buttons. In Proc. CHI 2011, ACM Press (2011), 3281--3284.

Digital Library

[31]

Mäki-Patola, T. and Hamalainen, P. Latency tolerance for gesture controlled continuous sound instrument without tactile feedback. In Proc. ICMC 2004 (2004).

[32]

Malik, S. and Laszlo, J. Visual touchpad: a two-handed gestural input device. In Proc. ICMI 2004, ACM Press (2004), 289--296.

Digital Library

[33]

Matsushita, N. and Rekimoto, J. HoloWall: designing a finger, hand, body, and object sensitive wall. In Proc. UIST 1997, ACM Press (1997), 209--210.

Digital Library

[34]

Moeller, J. and Kerne, A. ZeroTouch: an optical multitouch and free-air interaction architecture. In Proc. CHI 2012, ACM Press (2012), 2165--2174.

Digital Library

[35]

Ng, A., Lepinski, J., Wigdor, D., Sanders, S. and Dietz, P. Designing for Low-Latency Direct-Touch Input. In Proc. UIST 2012, ACM Press (2012), 453--464.

Digital Library

[36]

Ohnishi, H. and Mochizuki, K. Effect of delay of feedback force on perception of elastic force: a psychophysical approach. IEICE Transactions on Communications E90-B(1) (2007), 12--20.

[37]

Okamura, A. M., Cutkosky, M. and Dennerlein, J. Reality based models for vibration feedback in virtual environments. IEEE/ASME Transactions on Mechatronics 6 (2001), 245--252.

[38]

Rekimoto, J. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proc. CHI 2002, ACM Press (2002), 113--120.

Digital Library

[39]

Saga, S. and Deguchi, K. Lateral-force-based 2.5-dimensional tactile display for touch screen. In Proc. Haptics Symposium 2012, IEEE (2012), 15--22.

[40]

Samsung. SUR40 for Microsoft Surface 2.0. http://www.samsunglfd.com/product/feature.do?modelCd=SUR40.

[41]

Starner, T., Leibe, B., Minnen, D., Westeyn, T., Hurst, A. and Weeks, J. The perceptive workbench: computer-vision-based gesture tracking, object tracking, and 3D reconstruction for augmented desks. Machine Vision and Applications 14, Springer (2003), 59--71.

[42]

Subramanian, S., Aliakseyeu, D. and Lucero, A. Multilayer interaction for digital tables. In Proc. UIST 2006, ACM Press (2006), 296--272.

Digital Library

[43]

Tactus Technology, Inc. Taking touch screen interfaces into a new dimension. A Tactus Technology White Paper (2012).

[44]

Takeoka, Y., Miyaki, T. and Rekimoto, J. Z-Touch: an infrastructure for 3D gesture interaction in the proximity of tabletop surface. In Proc. ITS 2010, ACM Press (2010), 91--94.

Digital Library

[45]

Wellman, P. and Howe, R. D. Towards realistic display in virtual environments. In Proc. ASME Dynamic Systems and Control Division 57, ASME (1995), 713--718.

[46]

Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J. and Shen, C. LucidTouch: a see-through mobile device. In Proc. UIST 2007, ACM Press (2007), 269--278.

Digital Library

[47]

Wintergerst, G., Jagodzinski, R., Fabian, H., Müller, A. and Joost, G. Reflective haptics: enhancing stylus-based interactions on touch screens. EuroHaptics 2010 Part 1, Springer (2010), 360--366.

Digital Library

[48]

Wilson, A. TouchLight: an imaging touch screen and display for gesture-based interaction. In Proc. ICMI 2004, ACM Press (2004), 69--76.

Digital Library

[49]

Wilson, A. D. Using a depth camera as a touch sensor. In Proc. ITS 2010, ACM Press (2010), 69--72.

Digital Library

[50]

Wilson, A. and Benko, H. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In Proc. UIST 2010, ACM Press (2010), 273--282.

Digital Library

[51]

Zampini, M., Brown, T., Shore, D. I., Maravita, A., Röder, B. and Spence, C. Audiotactile temporal order judgments. Acta Psychologica 118 (2005), 277--291.

Cited By

Fan NXiao R(2022)Reducing the Latency of Touch Tracking on Ad-hoc SurfacesProceedings of the ACM on Human-Computer Interaction10.1145/35677306:ISS(489-499)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567730
Ujitoko YBan YYokosaka T(2022)Getting Insights From Twitter: What People Want to Touch in Daily LifeIEEE Transactions on Haptics10.1109/TOH.2021.310597915:1(142-153)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TOH.2021.3105979
Bermejo CHui P(2021)A Survey on Haptic Technologies for Mobile Augmented RealityACM Computing Surveys10.1145/346539654:9(1-35)Online publication date: 8-Oct-2021
https://dl.acm.org/doi/10.1145/3465396
Show More Cited By

Index Terms

HACHIStack: dual-layer photo touch sensing for haptic and auditory tapping interaction
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Helping older adults locate 'lost' cursors using FieldMouse
ASSETS '10: Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility

This paper describes how a standard optical mouse was augmented by the addition of a touch sensor inside the body of the mouse. When the mouse is released and subsequently touched it generates a Windows message which can then be used to execute an ...
Cursor relocation techniques to help older adults find 'lost' cursors
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Older adult computer users often lose track of the mouse cursor and so resort to methods such as mouse shaking or searching the screen to find the cursor again. Hence, this paper describes how a standard optical mouse was modified to include a touch ...
Investigating the effectiveness of tactile feedback for mobile touchscreens
CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

This paper presents a study of finger-based text entry for mobile devices with touchscreens. Many devices are now coming to market that have no physical keyboards (the Apple iPhone being a very popular example). Touchscreen keyboards lack any tactile ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2013

3550 pages

ISBN:9781450318990

DOI:10.1145/2470654

General Chair:
Wendy E. Mackay
INRIA
,
Program Chairs:
Stephen Brewster
Glasgow University
,
Susanne Bødker
University of Aarhus

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '13

Sponsor:

SIGCHI

CHI '13: CHI Conference on Human Factors in Computing Systems

April 27 - May 2, 2013

Paris, France

Acceptance Rates

CHI '13 Paper Acceptance Rate 392 of 1,963 submissions, 20%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
766
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)2

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fan NXiao R(2022)Reducing the Latency of Touch Tracking on Ad-hoc SurfacesProceedings of the ACM on Human-Computer Interaction10.1145/35677306:ISS(489-499)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1145/3567730
Ujitoko YBan YYokosaka T(2022)Getting Insights From Twitter: What People Want to Touch in Daily LifeIEEE Transactions on Haptics10.1109/TOH.2021.310597915:1(142-153)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TOH.2021.3105979
Bermejo CHui P(2021)A Survey on Haptic Technologies for Mobile Augmented RealityACM Computing Surveys10.1145/346539654:9(1-35)Online publication date: 8-Oct-2021
https://dl.acm.org/doi/10.1145/3465396
Park CYoon JOh SChoi SIqbal SMacLean KChevalier FMueller S(2020)Augmenting Physical Buttons with Vibrotactile Feedback for Programmable FeelsProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379337.3415837(924-937)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3379337.3415837
Kawazoe ALuca MVisell Y(2019)Tactile Echoes: A Wearable System for Tactile Augmentation of Objects2019 IEEE World Haptics Conference (WHC)10.1109/WHC.2019.8816099(359-364)Online publication date: Jul-2019
https://doi.org/10.1109/WHC.2019.8816099
Hachisu TFukumoto M(2018)SpiroSurface: A Repulsive and Attractive Force Display for Interactive Tabletops Using a Pneumatic SystemIEEE Computer Graphics and Applications10.1109/MCG.2018.04273165938:4(54-70)Online publication date: Jul-2018
https://doi.org/10.1109/MCG.2018.042731659
Shionoiri HSakuragi RKodama RKajimoto H(2018)Vibrotactile Feedback to Combine with Swing Presentation for Virtual Reality ApplicationsHaptics: Science, Technology, and Applications10.1007/978-3-319-93399-3_11(114-124)Online publication date: 6-Jun-2018
https://doi.org/10.1007/978-3-319-93399-3_11
Spelmezan DSahoo DSubramanian SMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)SparkleProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025782(3705-3717)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025782
Hachisu TKajimoto H(2017)Vibration Feedback Latency Affects Material Perception During Rod Tapping InteractionsIEEE Transactions on Haptics10.1109/TOH.2016.262890010:2(288-295)Online publication date: 1-Apr-2017
https://doi.org/10.1109/TOH.2016.2628900
Park JChae SYang YHan T(2017)Investigating pre-touch for sound generation on multi-touch surfaces using blob area detection2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC.2017.8122752(1064-1068)Online publication date: Oct-2017
https://doi.org/10.1109/SMC.2017.8122752
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten