research-article

Next-Point Prediction Metrics for Perceived Spatial Errors

Authors:

Mathieu Nancel,

Bruno De Araujo,

Géry CasiezAuthors Info & Claims

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

Pages 271 - 285

https://doi.org/10.1145/2984511.2984590

Published: 16 October 2016 Publication History

Abstract

Touch screens have a delay between user input and corresponding visual interface feedback, called input 'latency' (or 'lag'). Visual latency is more noticeable during continuous input actions like dragging, so methods to display feedback based on the most likely path for the next few input points have been described in research papers and patents. Designing these 'next-point prediction' methods is challenging, and there have been no standard metrics to compare different approaches. We introduce metrics to quantify the probability of 7 spatial error 'side-effects' caused by next-point prediction methods. Types of side-effects are derived using a thematic analysis of comments gathered in a 12 participants study covering drawing, dragging, and panning tasks using 5 state-of-the-art next-point predictors. Using experiment logs of actual and predicted input points, we develop quantitative metrics that correlate positively with the frequency of perceived side-effects. These metrics enable practitioners to compare next-point predictors using only input logs.

Supplementary Material

ZIP File (uist4939-file5.zip)

Download
2.09 MB

suppl.mov (uist4939-file3.mp4)

Supplemental video

Download
54.93 MB

MP4 File (p271-nancel.mp4)

Download
185.06 MB

References

[1]

Touch Prediction Parameters structure documentation. Retrieved July 25, 2016 from https://msdn.microsoft.com/en-us/library/windows/desktop/hh969214(v=vs.85).aspx.

[2]

What's new in Cocoa Touch. Retrieved April 10, 2016 from https://developer.apple.com/videos/wwdc/2015/'id=107, watch from 13:10 to 15:24.

[3]

Information Retrieval for Music and Motion. Springer Berlin Heidelberg, 2007, ch. Dynamic Time Warping, 69--84.

[4]

Multi-touch trajectory tracking method, June 15 2011. CN Patent App. CN 201,110,030,430.

[5]

Curve fitting based touch trajectory smoothing method and system, July 2 2014. CN Patent App. CN 201,210,585,264.

[6]

Anderson, G., Doherty, R., and Ganapathy, S. User perception of touch screen latency. In Proc. Design, User Experience, and Usability, DUXU '11 (2011), 195--202.

[7]

Annett, M., Ng, A., Dietz, P., Bischof, W. F., and Gupta, A. How low should we go': Understanding the perception of latency while inking. In Proc. Graphics Interface 2014, GI '14, Canadian Information Processing Society (2014), 167--174.

Digital Library

[8]

Asano, T., Sharlin, E., Kitamura, Y., Takashima, K., and Kishino, F. Predictive interaction using the delphian desktop. In Proc. 18th Annual ACM Symposium on User Interface Software and Technology, UIST '05, ACM (2005), 133--141.

Digital Library

[9]

Bérard, F., and Blanch, R. Two touch system latency estimators: High accuracy and low overhead. In Proc. 2013 ACM International Conference on Interactive Tabletops and Surfaces, ITS '13, ACM (2013), 241--250.

Digital Library

[10]

Biswas, P., Aydemir, G., Langdon, P., and Godsill, S. Intent recognition using neural networks and Kalman filters. In Human-Computer Interaction and Knowledge Discovery in Complex, Unstructured, Big Data, A. Holzinger and G. Pasi, Eds., vol. 7947 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013, 112--123.

[11]

Braun, V., and Clarke, V. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101.

[12]

Casiez, G., Conversy, S., Falce, M., Huot, S., and Roussel, N. Looking through the eye of the mouse: A simple method for measuring end-to-end latency using an optical mouse. In Proc. 28th Annual ACM Symposium on User Interface Software and Technology, UIST '15, ACM (2015), 629--636.

Digital Library

[13]

Cattan, E., Rochet-Capellan, A., and Bérard, F. A predictive approach for an end-to-end touch-latency measurement. In Proc. 2015 International Conference on Interactive Tabletops & Surfaces, ITS '15, ACM (2015), 215--218.

Digital Library

[14]

Cattan, E., Rochet-Capellan, A., Perrier, P., and Bérard, F. Reducing latency with a continuous prediction: Effects on users' performance in direct-touch target acquisitions. In Proc. 2015 International Conference on Interactive Tabletops and Surfaces, ITS '15, ACM (2015), 205--214.

Digital Library

[15]

Deber, J., Jota, R., Forlines, C., and Wigdor, D. How much faster is fast enough': User perception of latency & latency improvements in direct and indirect touch. In Proc. 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, ACM (2015), 1827--1836.

Digital Library

[16]

Dilger, D. E. Agawi TouchMark contrasts iPad's fast screen response to laggy Android tablets, Oct. 2013.

[17]

Jota, R., Ng, A., Dietz, P., and Wigdor, D. How fast is fast enough': A study of the effects of latency in direct-touch pointing tasks. In Proc. SIGCHI Conference on Human Factors in Computing Systems, CHI '13, ACM (2013), 2291--2300.

Digital Library

[18]

Kaaresoja, T., and Brewster, S. Feedback is... late: Measuring multimodal delays in mobile device touchscreen interaction. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI '10, ACM (2010), 2:1--2:8.

Digital Library

[19]

Kim, B., and Lim, Y. Mobile terminal and touch coordinate predicting method thereof, Aug. 28 2014. WO Patent App. PCT/KR2014/000,661.

[20]

Lank, E., Cheng, Y.-C. N., and Ruiz, J. Endpoint prediction using motion kinematics. In Proc. SIGCHI Conference on Human Factors in Computing Systems, CHI '07, ACM (2007), 637--646.

Digital Library

[21]

LaValle, S., Yershova, A., Katsev, M., and Antonov, M. Head tracking for the oculus rift. In Robotics and Automation (ICRA), 2014 IEEE International Conference on (May 2014), 187--194.

[22]

LaViola, J. J. Double exponential smoothing: An alternative to Kalman filter-based predictive tracking. In Proc. Workshop on Virtual Environments 2003, EGVE '03, ACM (2003), 199--206.

Digital Library

[23]

Leigh, D., Forlines, C., Jota, R., Sanders, S., and Wigdor, D. High rate, low-latency multi-touch sensing with simultaneous orthogonal multiplexing. In Proc. 27th Annual ACM Symposium on User Interface Software and Technology, UIST '14, ACM (2014), 355--364.

Digital Library

[24]

Liang, J., Shaw, C., and Green, M. On temporal-spatial realism in the virtual reality environment. In Proc. 4th Annual ACM Symposium on User Interface Software and Technology, UIST '91, ACM (1991), 19--25.

Digital Library

[25]

LINCOLN, J. Position lag reduction for computer drawing, Oct. 17 2013. US Patent App. 13/444,029.

[26]

MacKenzie, I. S., and Ware, C. Lag as a determinant of human performance in interactive systems. In Proc. INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, CHI '93, ACM (1993), 488--493.

Digital Library

[27]

Meyer, D. E., Keith-Smith, J., Kornblum, S., Abrams, R. A., and Wright, C. E. Speed-accuracy tradeoffs in aimed movements: Toward a theory of rapid voluntary action. Attention and performance 13: Motor representation and control (1990), 173--226.

[28]

Miller, R. B. Response time in man-computer conversational transactions. In Proc. December 9--11, 1968, Fall Joint Computer Conference, Part I, AFIPS '68 (Fall, part I), ACM (1968), 267--277.

Digital Library

[29]

Mine, M. Characterization of end-to-end delays in head-mounted display systems. Tech. rep., University of North Carolina at Chapel Hill, 1993.

Digital Library

[30]

Moussavi, F. Methods and apparatus for incremental prediction of input device motion, July 1 2014. US Patent 8,766,915.

[31]

Nelson, W. T., Roe, M. M., Bolia, R. S., and Morley, R. M. Assessing simulator sickness in a see-through hmd: Effects of time delay, time on task, and task complexity. Tech. rep., DTIC Document, 2000.

[32]

Ng, A., Annett, M., Dietz, P., Gupta, A., and Bischof, W. F. In the blink of an eye: Investigating latency perception during stylus interaction. In Proc. 32Nd Annual ACM Conference on Human Factors in Computing Systems, CHI '14, ACM (2014), 1103--1112.

Digital Library

[33]

Ng, A., Lepinski, J., Wigdor, D., Sanders, S., and Dietz, P. Designing for low-latency direct-touch input. In Proc. 25th Annual ACM Symposium on User Interface Software and Technology, UIST '12, ACM (2012), 453--464.

Digital Library

[34]

Pasqual, P. T., and Wobbrock, J. O. Mouse pointing endpoint prediction using kinematic template matching. In Proc. SIGCHI Conference on Human Factors in Computing Systems, CHI '14, ACM (2014), 743--752.

Digital Library

[35]

Pavlovych, A., and Gutwin, C. Assessing target acquisition and tracking performance for complex moving targets in the presence of latency and jitter. In Proc. Graphics Interface 2012, GI '12, Canadian Information Processing Society (2012), 109--116.

Digital Library

[36]

Pavlovych, A., and Stuerzlinger, W. The tradeoff between spatial jitter and latency in pointing tasks. In Proc. 1st ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS '09, ACM (2009), 187--196.

Digital Library

[37]

Pavlovych, A., and Stuerzlinger, W. Target following performance in the presence of latency, jitter, and signal dropouts. In Proc. Graphics Interface 2011, GI '11, Canadian Human-Computer Communications Society (2011), 33--40.

Digital Library

[38]

Shneiderman, B. Response time and display rate in human performance with computers. ACM Computing Surveys 16, 3 (Sept. 1984), 265--285.

Digital Library

[39]

Tsoi, P., and Xiao, J. Advanced touch input on ios. Technical report, Apple Inc., 2005.

[40]

Wang, W. Touch tracking device and method for a touch screen, Jan. 24 2013. US Patent App. 13/367,371.

[41]

Ware, C., and Balakrishnan, R. Reaching for objects in VR displays: Lag and frame rate. ACM ToCHI 1, 4 (Dec. 1994), 331--356.

Digital Library

[42]

Welch, G., and Bishop, G. An introduction to the Kalman filter. Tech. rep., 1995.

Digital Library

[43]

Westerman, W., and Elias, J. Multi-touch contact tracking using predicted paths, Dec. 18 2012. US Patent 8,334,846.

[44]

Wu, J.-R., and Ouhyoung, M. On latency compensation and its effects on head-motion trajectories in virtual environments. The Visual Computer 16, 2 (2000), 79--90.

[45]

Xia, H., Jota, R., McCanny, B., Yu, Z., Forlines, C., Singh, K., and Wigdor, D. Zero-latency tapping: Using hover information to predict touch locations and eliminate touchdown latency. In Proc. 27th Annual ACM Symposium on User Interface Software and Technology, UIST '14, ACM (2014), 205--214.

Digital Library

[46]

Zhao, W., Stevens, D., Uzelac, A., Benko, H., and Miller, J. Prediction-based touch contact tracking, Aug. 16 2012. US Patent App. 13/152,991.

[47]

Zhou, S. Electronic device and method for providing tactile stimulation, June 19 2014. US Patent App. 13/729,048.

Cited By

Yamanaka SStuerzlinger W(2024)The Effect of Latency on Movement Time in Path-steeringProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642316(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642316
Shao QWang JZhou BTran VKrishnan GNayar S(2023)N-euro PredictorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108847:3(1-25)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610884
Nishida NIkematsu KSato JYamanaka STsubouchi K(2023)Single-tap Latency Reduction with Single- or Double- tap PredictionProceedings of the ACM on Human-Computer Interaction10.1145/36042717:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604271
Show More Cited By

Index Terms

Next-Point Prediction Metrics for Perceived Spatial Errors
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Next-Point Prediction for Direct Touch Using Finite-Time Derivative Estimation
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

End-to-end latency in interactive systems is detrimental to performance and usability, and comes from a combination of hardware and software delays. While these delays are steadily addressed by hardware and software improvements, it is at a decelerating ...
Software-reduced touchscreen latency
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

Devices with touchscreens have an inherent latency. When a user's finger drags an object across the screen the object follows with a latency of around 100ms for current devices. Previous work showed that latencies down to 25ms reduce users' performance ...
Improving software-reduced touchscreen latency
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

The latency of current mobile devices' touchscreens is around 100ms and has widely been explored. Latency down to 2ms is noticeable, and latency as low as 25ms reduces users' performance. Previous work reduced touch latency by extrapolating a finger's ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

October 2016

908 pages

ISBN:9781450341899

DOI:10.1145/2984511

General Chairs:
Jun Rekimoto
The University of Tokyo
,
Takeo Igarashi
The University of Tokyo
,
Program Chairs:
Jacob O. Wobbrock
University of Washington
,
Daniel Avrahami
FXPAL

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

UIST '16

Sponsor:

UIST '16: The 29th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2016

Tokyo, Japan

Acceptance Rates

UIST '16 Paper Acceptance Rate 79 of 384 submissions, 21%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
733
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)4

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yamanaka SStuerzlinger W(2024)The Effect of Latency on Movement Time in Path-steeringProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642316(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642316
Shao QWang JZhou BTran VKrishnan GNayar S(2023)N-euro PredictorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108847:3(1-25)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610884
Nishida NIkematsu KSato JYamanaka STsubouchi K(2023)Single-tap Latency Reduction with Single- or Double- tap PredictionProceedings of the ACM on Human-Computer Interaction10.1145/36042717:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604271
Wiese JHenze N(2023)Predicting Mouse Positions Beyond a System’s Latency Can Increase Throughput and User Experience in Linear Steering TasksProceedings of Mensch und Computer 202310.1145/3603555.3603556(101-115)Online publication date: 3-Sep-2023
https://dl.acm.org/doi/10.1145/3603555.3603556
Wilmott JErkelens IMurdison TRio K(2022)Perceptibility of Jitter in Augmented Reality Head-Mounted Displays2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00063(470-478)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00063
Ham ALim JKim S(2021)Do We Need a Faster Mouse? Empirical Evaluation of Asynchronicity-Induced JitterThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474783(743-753)Online publication date: 10-Oct-2021
https://dl.acm.org/doi/10.1145/3472749.3474783
Gamage NIshtaweera DWeigel MWithana A(2021)So Predictable! Continuous 3D Hand Trajectory Prediction in Virtual RealityThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474753(332-343)Online publication date: 10-Oct-2021
https://dl.acm.org/doi/10.1145/3472749.3474753
Kushnirenko RAlkhimova SSydorenko DTolmachov I(2021)Using Real-Pen Specific Features of Active Stylus to Cope with Input LatencyHuman-Computer Interaction. Interaction Techniques and Novel Applications10.1007/978-3-030-78465-2_2(17-31)Online publication date: 3-Jul-2021
https://doi.org/10.1007/978-3-030-78465-2_2
Stauffert JNiebling FLatoschik M(2020)Latency and Cybersickness: Impact, Causes, and Measures. A ReviewFrontiers in Virtual Reality10.3389/frvir.2020.5822041Online publication date: 26-Nov-2020
https://doi.org/10.3389/frvir.2020.582204
Schmid PMalacria SCockburn ANancel MIqbal SMacLean KChevalier FMueller S(2020)Interaction InterferencesProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379337.3415883(516-528)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3379337.3415883
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents