research-article

PianoText: redesigning the piano keyboard for text entry

Authors:

Anna Maria Feit,

Antti OulasvirtaAuthors Info & Claims

DIS '14: Proceedings of the 2014 conference on Designing interactive systems

Pages 1045 - 1054

https://doi.org/10.1145/2598510.2598547

Published: 21 June 2014 Publication History

Abstract

Inspired by the high keying rates of skilled pianists, we study the design of piano keyboards for rapid text entry. We review the qualities of the piano as an input device, observing four design opportunities: 1) chords, 2) redundancy (more keys than letters in English), 3) the transfer of musical skill and 4) optional sound feedback. Although some have been utilized in previous text entry methods, our goal is to exploit all four in a single design. We present PianoText, a computationally designed mapping that assigns letter sequences of English to frequent note transitions of music. It allows fast text entry on any MIDI-enabled keyboard and was evaluated in two transcription typing studies. Both show an achievable rate of over 80 words per minute. This parallels the rates of expert Qwerty typists and doubles that of a previous piano-based design from the 19th century. We also design PianoText-Mini, which allows for comparable performance in a portable form factor. Informed by the studies, we estimate the upper bound of typing performance, draw implications to other text entry methods, and critically discuss outstanding design challenges.

Supplementary Material

ZIP File (dis0264-file4.zip)

This is the supplementary material for the paper: PianoText: Redesigning the Piano Keyboard for Text Entry It contains the following files: AppendixA.pdf A pdf file describing the algorithm for creating a letter-to-key mapping AppendixB.pdf A pdf file describing the training program of the long-term study of PianoText Example_Study1.pdf An example stimulus used in study 1. The music sheet shows sentences translated into notes. Example_Study1.mp4 A video of an example performance in study 1 - a professional pianist enters sentences on the piano by playing from a sheet of music. Performance was 82 wpm. Example_Study2.mp4 A video of the last test conducted in study 2. It shows the trained piano-typist transcribing sentences. Overall performance was 81 wpm. Required software: Pdf viewer Video player

Download
52.95 MB

MP4 File (dis0264-file3.mp4)

Download
6.23 MB

References

[1]

Adler, M. Antique Typewriters, from Creed to QWERTY. Schiffer Pub., 1997.

[2]

Andersen, T. H., and Zhai, S. writing with music: Exploring the use of auditory feedback in gesture interfaces. TAP 7, 3 (2010).

Digital Library

[3]

Beddoes, M. P., and Hu, Z. A chord stenograph keyboard: a possible solution to the learning problem in stenography. IEEE Transactions on Systems, Man and Cybernetics 24, 7 (1994), 953--960.

[4]

Bowers, B. Electrical engineering 100 years ago. British Journal of Audiology 13, S2 (1979), 1--4.

[5]

Brewster, S. Overcoming the lack of screen space on mobile computers. Personal and Ubiquitous Computing, 3 (2002).

Digital Library

[6]

Buxton, B. Chord keyboards. http://www.billbuxton. com/input06.ChordKeyboards.pdf, 2012.

[7]

Deutsch, D. Pitch circularity. deutsch.ucsd.edu.

[8]

Dunlop, M., and Levine, J. Multidimensional pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking.

[9]

Engelbart, D. C. Design considerations for knowledge workshop terminals. In Proc.1973, national computer conference and exposition, ACM (1973), 221--227.

Digital Library

[10]

Ericsson, A. K. The Cambridge handbook of expertise and expert performance, Chapter 26: Music. Cambridge University Press, 2006.

[11]

Ericsson, K. A., and Williams, A. M. Capturing naturally occurring superior performance in the laboratory: translational research on expert performance. Journal of Experimental Psychology: Applied 13, 3 (2007), 115.

[12]

Findlater, L., Wobbrock, J. O., and Wigdor, D. Typing on flat glass: examining ten-finger expert typing patterns on touch surfaces. In Proc. CHI'11, ACM (2011), 2453--2462.

Digital Library

[13]

Furuya, S., and Soechting, J. F. Role of auditory feedback in the control of successive keystrokes during piano playing. Experimental Brain Research, 2 (2010), 223--237.

[14]

Gentner, D. Expertise in typewriting. Tech. rep., DTIC Document, 1984.

[15]

Gopher, D., and Raij, D. Typing with a two-hand chord keyboard: will the qwerty become obsolete? IEEE Transactions on Systems, Man and Cybernetics, 4 (1988), 601--609.

[16]

Lepinski, G. J., Grossman, T., and Fitzmaurice, G. The design and evaluation of multitouch marking menus. In Proc. CHI'10, ACM (2011), 2233--2242.

Digital Library

[17]

Lifton, J., and Knight, M. Plover, the open source steno program. http://plover.stenoknight.com/, 2012.

[18]

Lyons, K. e. a. Experimental evaluations of the twiddler one-handed chording mobile keyboard. Human-Computer Interaction 21, 4 (2006).

Digital Library

[19]

MacKenzie, I., and Zhang, S. The design and evaluation of a high-performance soft keyboard. In Proc. CHI'99, ACM (2011), 25--31.

Digital Library

[20]

MacKenzie, S. I., and Soukoreff, W. R. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction 17, 2--3 (2002), 147--198.

[21]

MacKenzie, S. I., T.-I. K. Text entry systems: Mobility, accessibility, universality. Morgan Kaufmann, 2010.

Digital Library

[22]

Norman, D. A., and Fisher, D. Why alphabetic keyboards are not easy to use: Keyboard layout doesn't much matter. Human Factors, 5 (1982).

[23]

Oulasvirta, e. a. Improving two-thumb text entry on touchscreen devices. In Proc. CHI'13, 2765--2774.

Digital Library

[24]

Rumelhart, D. E., and Norman, D. A. Simulating a skilled typist: A study of skilled cognitive-motor performance. Cognitive Science, 1 (1982), 1--36.

[25]

Schmidt, R. A., and Lee, T. Motor Control and Learning, 5E. Human kinetics, 1988.

[26]

Schmuckler, M. A., and Bosman, E. L. Interkey timing in piano performance and typing. Canadian Journal of Experimental Psychology, 2 (1997), 99.

[27]

Schneider, W. Training high-performance skills: Fallacies and guidelines. Human Factors, 3 (1985), 285--300.

[28]

Seibel, R. Data entry through chord, parallel entry devices. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2 (1964), 189--192.

[29]

Vertanen, K., and Kristensson, P. A versatile dataset for text entry evaluations based on genuine mobile emails. In Proc. MobileHCI'11, ACM (2011), 295--298.

Digital Library

[30]

Zhai, S., Hunter, M., and Smith, B. Performance optimization of virtual keyboards. Human-Computer Interaction, 2--3 (2002), 229--269.

Cited By

Rossmy BRümelin SWiethoff AKaltenbrunner MMurer MWolf KOakley I(2021)StringTouch - From String Instruments towards new Interface MorphologiesProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440628(1-10)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3440628
Rossmy Bvan den Hoven ELoke LShaer Ovan Dijk JKun A(2020)Metaphors and Technologies for New Tangible User InterfacesProceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3374920.3374963(925-929)Online publication date: 9-Feb-2020
https://dl.acm.org/doi/10.1145/3374920.3374963
Feit ABrewster SFitzpatrick GCox AKostakos V(2019)SIGCHI Outstanding Dissertation AwardExtended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290607.3313773(1-4)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290607.3313773
Show More Cited By

Index Terms

PianoText: redesigning the piano keyboard for text entry
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

PianoText: redesigning the piano keyboard for text entry
DIS Companion '14: Proceedings of the 2014 companion publication on Designing interactive systems

Inspired by the high keying rates of skilled pianists we study the design of piano keyboards for rapid text entry. We present PianoText, a computationally designed mapping that assigns letter sequences of English to frequent note transitions in music. ...
PianoText: transferring musical expertise to text entry
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

We present PianoText, a text entry method based on a piano keyboard with an optimized mapping between notes and chords of music to letters of the English language. PianoText exemplifies the idea of transferring musical expertise to a text entry task by ...
Invisiboard: maximizing display and input space with a full screen text entry method for smartwatches
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services

The small displays of smartwatches make text entry difficult and time consuming. While text entry rates can be increased, this continues to occur at the expense of available screen display space. Soft keyboards can easily use half the display space of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '14: Proceedings of the 2014 conference on Designing interactive systems

June 2014

1102 pages

ISBN:9781450329026

DOI:10.1145/2598510

General Chairs:
Ron Wakkary
Simon Fraser University
,
Steve Harrison
Virginia Tech
,
Program Chairs:
Carman Neustaedter
Simon Fraser University
,
Shaowen Bardzell
Indiana University
,
Eric Paulos
UC Berkeley

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

DIS '14

Sponsor:

SIGCHI

DIS '14: Designing Interactive Systems Conference 2014

June 21 - 25, 2014

BC, Vancouver, Canada

Acceptance Rates

DIS '14 Paper Acceptance Rate 107 of 402 submissions, 27%;

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
304
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)6

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rossmy BRümelin SWiethoff AKaltenbrunner MMurer MWolf KOakley I(2021)StringTouch - From String Instruments towards new Interface MorphologiesProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440628(1-10)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3440628
Rossmy Bvan den Hoven ELoke LShaer Ovan Dijk JKun A(2020)Metaphors and Technologies for New Tangible User InterfacesProceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3374920.3374963(925-929)Online publication date: 9-Feb-2020
https://dl.acm.org/doi/10.1145/3374920.3374963
Feit ABrewster SFitzpatrick GCox AKostakos V(2019)SIGCHI Outstanding Dissertation AwardExtended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290607.3313773(1-4)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290607.3313773
Montano-Murillo RHan TIrani PMartinez-Plasencia DSubramanian S(2019)Designer Led Computational Approach to Generate Mappings for Devices with Low Gestural ResolutionHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29381-9_38(623-643)Online publication date: 2-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-29381-9_38
Feit AOulasvirta AWakkary RHarrison SBardzell SNeustaedter CPaulos E(2014)PianoTextProceedings of the 2014 companion publication on Designing interactive systems10.1145/2598784.2602800(129-132)Online publication date: 21-Jun-2014
https://dl.acm.org/doi/10.1145/2598784.2602800

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents