research-article

28 frames later: predicting screen touches from back-of-device grip changes

Authors:

Mohammad Faizuddin Mohd Noor,

Roderick Murray-SmithAuthors Info & Claims

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 2005 - 2008

https://doi.org/10.1145/2556288.2557148

Published: 26 April 2014 Publication History

Get Access

Abstract

We demonstrate that front-of-screen targeting on mobile phones can be predicted from back-of-device grip manipulations. Using simple, low-resolution capacitive touch sensors placed around a standard phone, we outline a machine learning approach to modelling the grip modulation and inferring front-of-screen touch targets. We experimentally demonstrate that grip is a remarkably good predictor of touch, and we can predict touch position 200ms before contact with an accuracy of 18mm.

Supplementary Material

MP4 File (p2005-sidebyside.mp4)

Download
112.69 MB

References

[1]

Baudisch, P., and Chu, G. Back-of-device interaction allows creating very small touch devices. CHI '09 (2009), 1923.

Digital Library

Google Scholar

[2]

Cheng, L.-P., Hsiao, F.-I., Liu, Y.-T., and Chen, M. Y. iRotate grasp. In UIST Adjunct Proceedings '12 (2012), 15.

Digital Library

Google Scholar

[3]

Cheng, L.-P., Liang, H.-S., Wu, C.-Y., and Chen, M. Y. iGrasp. In CHI '13 (2013), 3037.

Digital Library

Google Scholar

[4]

Goel, M., Wobbrock, J., and Patel, S. Gripsense: Using built-in sensors to detect hand posture and pressure on commodity mobile phones. In UIST '12 (2012), 545--554.

Digital Library

Google Scholar

[5]

Hotelling, H. Relations between two sets of variates. Biometrika 28 (1936), 321--377.

Crossref

Google Scholar

[6]

Kim, K., Chang, W., Cho, Sung-jung Shim, J., and Lee, H. Hand grip pattern recognition for mobile user interfaces. IAAI '06 (2006), 1789--1794.

Digital Library

Google Scholar

[7]

Luca, A. D., von Zezschwitz, E., Nguyen, N. D. H., Maurer, M.-E., Rubegni, E., Scipioni, M. P., and Langheinrich, M. Back-of-device authentication on smartphones. In CHI '13 (2013), 2389.

Digital Library

Google Scholar

[8]

Ono, M., Shizuki, B., and Tanaka, J. Touch and activate: Adding interactivity to existing objects using active acoustic sensing. In UIST '13 (2013), 31--40.

Digital Library

Google Scholar

[9]

Rasmussen, C. E., and Williams, C. Gaussian Processes for Machine Learning. MIT Press, 2006.

Digital Library

Google Scholar

[10]

Taylor, B. T., and Bove, V. M. The bar of soap: a grasp recognition system implemented in a multi-functional handheld device. Ext. Abstracts CHI'08 (2008), 3459--3464.

Digital Library

Google Scholar

[11]

Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., and Shen, C. Lucid touch. In UIST '07 (2007), 269.

Digital Library

Google Scholar

[12]

Wimmer, R., and Boring, S. HandSense: discriminating different ways of grasping and holding a tangible user interface. TEI '09 (2009), 16--19.

Digital Library

Google Scholar

Cited By

View all

Fan YPietroni NFerguson S(2023)A Neural Network-based Low-cost Soft Sensor for Touch Recognition and Deformation CaptureProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595963(889-903)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595963
Li RSeyed TMarquardt NOfek EHodges SSinclair MRomat HPahud MSharma JBuxton WHinckley KRiche N(2023)AdHocProx: Sensing Mobile, Ad-Hoc Collaborative Device Formations using Dual Ultra-Wideband RadiosProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581300(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581300
Renna MPeruch ASunwoo JStarkweather ZMartin AFranceschini M(2022)A Contact-Sensitive Probe for Biomedical OpticsSensors10.3390/s2206236122:6(2361)Online publication date: 18-Mar-2022
https://doi.org/10.3390/s22062361
Show More Cited By

Index Terms

28 frames later: predicting screen touches from back-of-device grip changes
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Detecting Swipe Errors on Touchscreens using Grip Modulation
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We show that when users make errors on mobile devices they make immediate and distinct physical responses that can be observed with standard sensors. We used three standard cognitive tasks (Flanker, Stroop and SART) to induce errors from 20 ...
Exploring back-of-device interaction
UIST '13 Adjunct: Adjunct Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology

Back of device interaction is gaining popularity as an alternative input modality in mobile devices, however it is still unclear how the back of device is related to other interactions. My research explores the relationship between hand grip from the ...
Improving accuracy in back-of-device multitouch typing: a clustering-based approach to keyboard updating
IUI '14: Proceedings of the 19th international conference on Intelligent User Interfaces

Recent work has shown that a multitouch sensor attached to the back of a handheld device can allow rapid typing engaging all ten fingers. However, high error rates remain a problem, because the user can not see or feel key-targets on the back. We ...

Comments

Information & Contributors

Information

Published In

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2014

4206 pages

ISBN:9781450324731

DOI:10.1145/2556288

General Chairs:
Matt Jones
Swansea University, Wales, UK
,
Philippe Palanque
Université Paul Sabatier, France
,
Program Chairs:
Albrecht Schmidt
University of Stuttgart, Germany
,
Tovi Grossman
Autodesk Research, Canada

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '14

Sponsor:

SIGCHI

CHI '14: CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2014

Ontario, Toronto, Canada

Acceptance Rates

CHI '14 Paper Acceptance Rate 465 of 2,043 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
602
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Fan YPietroni NFerguson S(2023)A Neural Network-based Low-cost Soft Sensor for Touch Recognition and Deformation CaptureProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3595963(889-903)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3595963
Li RSeyed TMarquardt NOfek EHodges SSinclair MRomat HPahud MSharma JBuxton WHinckley KRiche N(2023)AdHocProx: Sensing Mobile, Ad-Hoc Collaborative Device Formations using Dual Ultra-Wideband RadiosProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581300(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581300
Renna MPeruch ASunwoo JStarkweather ZMartin AFranceschini M(2022)A Contact-Sensitive Probe for Biomedical OpticsSensors10.3390/s2206236122:6(2361)Online publication date: 18-Mar-2022
https://doi.org/10.3390/s22062361
Schmitz MMüller FMühlhäuser MRiemann JLe HKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Itsy-Bits: Fabrication and Recognition of 3D-Printed Tangibles with Small Footprints on Capacitive TouchscreensProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445502(1-12)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445502
Fan ZCoutrix C(2020)Impact of Hand Used on One-Handed Back-of-Device PerformanceProceedings of the ACM on Human-Computer Interaction10.1145/34273164:ISS(1-19)Online publication date: 4-Nov-2020
https://dl.acm.org/doi/10.1145/3427316
Le HMayer SWeiß MVogelsang JWeingärtner HHenze N(2020)Shortcut Gestures for Mobile Text Editing on Fully Touch Sensitive SmartphonesACM Transactions on Computer-Human Interaction10.1145/339623327:5(1-38)Online publication date: 17-Aug-2020
https://dl.acm.org/doi/10.1145/3396233
Kulshreshtha SArif A(2020)Woodpecker: Secret Back-of-Device Tap Rhythms to Authenticate Mobile Users2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC42975.2020.9283239(2727-2733)Online publication date: 11-Oct-2020
https://doi.org/10.1109/SMC42975.2020.9283239
Le HMayer SSteuerlein BHenze N(2019)Investigating Unintended Inputs for One-Handed Touch Interaction Beyond the TouchscreenProceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3338286.3340145(1-14)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1145/3338286.3340145
Sarcar SSilpasuwanchai CDelamare WOno AOulasvirta ARen X(2019)Exploring performance of thumb input for pointing and dragging tasks on mobile deviceProceedings of Asian CHI Symposium 2019: Emerging HCI Research Collection10.1145/3309700.3338434(38-45)Online publication date: 4-May-2019
https://dl.acm.org/doi/10.1145/3309700.3338434
Le HMayer SHenze NFu WPan SBrdiczka OChau PCalvary G(2019)Investigating the feasibility of finger identification on capacitive touchscreens using deep learningProceedings of the 24th International Conference on Intelligent User Interfaces10.1145/3301275.3302295(637-649)Online publication date: 17-Mar-2019
https://dl.acm.org/doi/10.1145/3301275.3302295
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Detecting Swipe Errors on Touchscreens using Grip Modulation

Exploring back-of-device interaction

Improving accuracy in back-of-device multitouch typing: a clustering-based approach to keyboard updating