Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2642918.2647379acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Sensing techniques for tablet+stylus interaction

Published: 05 October 2014 Publication History

Abstract

We explore grip and motion sensing to afford new techniques that leverage how users naturally manipulate tablet and stylus devices during pen + touch interaction. We can detect whether the user holds the pen in a writing grip or tucked between his fingers. We can distinguish bare-handed inputs, such as drag and pinch gestures produced by the nonpreferred hand, from touch gestures produced by the hand holding the pen, which necessarily impart a detectable motion signal to the stylus. We can sense which hand grips the tablet, and determine the screen's relative orientation to the pen. By selectively combining these signals and using them to complement one another, we can tailor interaction to the context, such as by ignoring unintentional touch inputs while writing, or supporting contextually-appropriate tools such as a magnifier for detailed stroke work that appears when the user pinches with the pen tucked between his fingers. These and other techniques can be used to impart new, previously unanticipated subtleties to pen + touch interaction on tablets.

Supplementary Material

suppl.mov (uistf2557-file3.wmv)
Supplemental video

References

[1]
Annett, M., et al. The Pen is Mightier: Understanding Stylus Behaviour While Inking on Tablets. Graphics Interface (GI 2014).
[2]
Annett, M., et al. Is it Intended or Unintended? Palm Rejection during Direct Pen Interaction. (Currently submitted for review), 2014.
[3]
Becchio, C., et al. Grasping intentions: from thought experiments to emprical evidence. Frontiers in Human Neuroscience, 2012. 6(117).
[4]
Bi, X., et al. An Exploration of Pen Rolling. UIST'08.
[5]
Brandl, P., et al. Occlusion-aware menu design for digital tabletops. CHI'09 Extended Abstracts.
[6]
Buxton, W. Integrating the Periphery and Context: A New Taxonomy of Telematics. Proc. Graphics Interface '95.
[7]
Cheng, L.-P., et al. iRotate grasp: automatic screen rotation based on grasp of mobile devices. UIST Adjunct Proc. '12.
[8]
Cheng, L.-P., et al. iGrasp: grasp-based adaptive keyboard for mobile devices. CHI '13 Extended Abstracts.
[9]
Chun Yat Li, F., et al. The 1Line Keyboard: A QWERTY Layout in a Single Line. UIST'11.
[10]
FiftyThree, I. A closer look at zoom. http://tinyurl.com/mxqdshz.
[11]
FiftyThree, I. Pencil (for Paper). http://www.fiftythree.com/pencil.
[12]
Goel, M., et al. GripSense: Using Built-In Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones. UIST'12.
[13]
Guiard, Y., Millerat, F., Writing Postures in Left-Handers: Inverters are Hand-Crossers. Neuropsychologia, 1984. 22(5): p. 535--538.
[14]
Harrison, B., et al. Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces. CHI'98.
[15]
Hasan, K., et al. A-coord input: coordinating auxiliary input streams for augmenting contextual pen-based interactions. CHI '12.
[16]
Hinckley, K. Synchronous Gestures for Multiple Users and Computers. UIST'03.
[17]
Hinckley, K., et al. Motion and Context Sensing Techniques for Pen Computing. Proc. Graphics Interface 2013 (GI'13).
[18]
Hinckley, K., et al. Codex: A Dual-Screen Tablet Computer. CHI'09.
[19]
Hinckley, K., Pierce, J., Sinclair, M., Horvitz, E. Sensing techniques for mobile interaction. UIST 2000.
[20]
Hinckley, K., Song, H. Sensor Synaesthesia: Touch in Motion, and Motion in Touch. CHI 2011.
[21]
Hinckley, K., et al. Pen + Touch = New Tools. UIST 2010.
[22]
Holmquist, L., et al. Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artifacts. Ubicomp'01.
[23]
Hudson, S., et al. Whack Gestures: Inexact and Inattentive Interaction with Mobile Devices. TEI 2010.
[24]
Kim, K.-E., et al. Hand Grip Pattern Recognition for Mobile User Interfaces. Proceedings of AAAI/IAAI-2006: Innovative Appl.
[25]
Kurtenbach, G., et al. The design of a GUI paradigm based on tablets, two-hands, and transparency. CHI'97.
[26]
Lee, D., et al. PhantomPen: virtualization of pen head for digital drawing free from pen occlusion & visual parallax. UIST'12.
[27]
Lester, J., et al. Are You With Me? Using Accelerometers to Determine if Two Devices are Carried. Pervasive'04.
[28]
Liu, S., Guimbretiere, F. FlexAura: A Flexible Near-Surface Range Sensor. UIST '12.
[29]
Luff, P., Heath, C. Mobility in collaboration. CSCW '98.
[30]
Mackenzie, C., Iberall, T., The Grasping Hand. Advances in Psychology 104. 1994, Amsterdam: North Holland.
[31]
Mahoney, R., Nonlinear Complementary Filters on the Special Orthogonal Group. IEEE Trans. on Automatic Control, 2008. 53(5).
[32]
Marquardt, N., et al. Cross-Device Interaction via Micro-mobility and F-formations. UIST '12. 2012. Cambridge, MA.
[33]
Mason, A., MacKenzie, C., Grip forces when passing an object to a partner. Exp. Brain Res, 2005(163): p. 173--187.
[34]
Matulic, F., Norrie, M. Supporting active reading on pen and touchoperated tabletops. Proc. Int'l Working Conf. on Advanced Visual Interfaces (AVI'12). 2012. Capri Island, Italy.
[35]
Moleskine Journal (iPad app). 2014. http://tinyurl.com/kqhtomz.
[36]
Penultimate by Evernote (iPad app). 2014. http://tinyurl.com/kx82c8a.
[37]
Premerlani, W., Bizard, P. Direction Cosine Matrix IMU: Theory. 2009. http://gentlenav.googlecode.com/files/DCMDraft2.pdf.
[38]
Rofouei, M., et al. Your Phone or Mine? Fusing Body, Touch and Device Sensing for MultiUser Device-Display Interaction. CHI 2012.
[39]
Roudaut, A., et al. TimeTilt: Using Sensor-Based Gestures to Travel through Multiple Applications on a Mobile Device. Interact '09.
[40]
Schmidt, A., et al. Advanced interaction in context. Handheld and Ubiquitous Computing (HUC'99).
[41]
Schmidt, D., et al. PhoneTouch: A Technique for Direct Phone Interaction on Surfaces. UIST'10.
[42]
Schwarz, J., et al. Probabilistic Palm Rejection Using Spatiotemporal Touch Features and Iterative Classification. CHI'14.
[43]
Selin, A., Pencil Grip: A descriptive Model and Four Empirical Studies. 2003: Akademi University Press.
[44]
Sellen, A., Shaw., K. How Knowledge Workers Use the Web. CHI'02.
[45]
Siio, I., Tsujita, H. Mobile interaction using paperweight metaphor. UIST '06.
[46]
Song, H., et al. Grips and Gestures on a Multi-Touch Pen. CHI'11.
[47]
Sun, M., et al. Enhancing Naturalness of Pen-and-Tablet Drawing through Context Sensing. Interactive Tabletops and Surfaces (ITS '11).
[48]
Suzuki, Y., et al. Stylus Enhancement to Enrich Interaction with Computers. HCII 2007, Part II. 2007: Springer-Verlag.
[49]
Taylor, B., Bove Jr., V. Graspables: Grasp-Recognition as a User Interface. CHI'09.
[50]
Tian, F., et al. Tilt Menu: Using the 3D Orientation Information of Pen Devices to Extend the Selection Capability. CHI'08.
[51]
Vogel, D., Casiez, G. Conte: Multimodal Input Inspired by an Artist's Crayon. UIST 2011.
[52]
Wagner, J., et al. BiTouch and BiPad: Designing Bimanual Interaction for Hand-held Tablets. CHI'12.
[53]
Wimmer, R., Boring, S. HandSense Discriminating Different Ways of Grasping and Holding a Tangible User Interface. TEI '09.
[54]
Xin, Y., et al. Natural Use Profiles for the Pen: An Empirical Exploration of Pressure, Tilt, and Azimuth. CHI'12.
[55]
Yoon, D., Chen, N., Guimbretiere, F. TextTearing: Expanding Whitespace for Digital Ink Annotation. UIST'13.

Cited By

View all
  • (2024)Exploring the Design Space of Input Modalities for Working in Mixed Reality on Long-haul FlightsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661560(2267-2285)Online publication date: 1-Jul-2024
  • (2024)Opportunistic Nudges for Task Migration Between Personal DevicesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651037(1-8)Online publication date: 11-May-2024
  • (2024)Elicitation and Evaluation of Hand-based Interaction Language for 3D Conceptual Design in Mixed RealityInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103198183:COnline publication date: 14-Mar-2024
  • Show More Cited By

Index Terms

  1. Sensing techniques for tablet+stylus interaction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology
    October 2014
    722 pages
    ISBN:9781450330695
    DOI:10.1145/2642918
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 October 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. bimanual input
    2. grip
    3. motion
    4. pen+touch
    5. sensing
    6. tablet

    Qualifiers

    • Research-article

    Conference

    UIST '14

    Acceptance Rates

    UIST '14 Paper Acceptance Rate 74 of 333 submissions, 22%;
    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)38
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 03 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Exploring the Design Space of Input Modalities for Working in Mixed Reality on Long-haul FlightsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661560(2267-2285)Online publication date: 1-Jul-2024
    • (2024)Opportunistic Nudges for Task Migration Between Personal DevicesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651037(1-8)Online publication date: 11-May-2024
    • (2024)Elicitation and Evaluation of Hand-based Interaction Language for 3D Conceptual Design in Mixed RealityInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103198183:COnline publication date: 14-Mar-2024
    • (2023)AdHocProx: Sensing Mobile, Ad-Hoc Collaborative Device Formations using Dual Ultra-Wideband RadiosProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581300(1-18)Online publication date: 19-Apr-2023
    • (2023)The tactile dimension: a method for physicalizing touch behaviorsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581137(1-15)Online publication date: 19-Apr-2023
    • (2022)Video Interaction on Tablet Computers: Browsing with Pinch Gesture and Pen TiltProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia10.1145/3568444.3570598(285-287)Online publication date: 27-Nov-2022
    • (2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
    • (2021)PenShaft: Enabling Pen Shaft Detection and Interaction for Touchscreens12th Augmented Human International Conference10.1145/3460881.3460934(1-9)Online publication date: 27-May-2021
    • (2021)Punch-Sketching E-textilesProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440640(1-12)Online publication date: 14-Feb-2021
    • (2021)GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip DeformationProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445727(1-10)Online publication date: 6-May-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media