research-article

Learning to feel words: a comparison of learning approaches to acquire haptic words

Authors:

Robert Turcott,

Pablo Castillo,

Wahyudinata Setiawan,

Ali IsrarAuthors Info & Claims

SAP '18: Proceedings of the 15th ACM Symposium on Applied Perception

Article No.: 11, Pages 1 - 7

https://doi.org/10.1145/3225153.3225174

Published: 10 August 2018 Publication History

Abstract

Recent studies have shown that decomposing spoken or written language into phonemes and transcribing each phoneme into a unique vibrotactile pattern enables people to receive lexical messages on the arm. A potential barrier to adopting this new communication system is the time and effort required to learn the association between phonemes and vibrotactile patterns. Therefore, in this study, we compared the learnability and generalizability of different learning approaches, including guided learning, self-guided learning, and a mnemonic device. We found that after 65 minutes of learning spread across 3 days, 67% of participants, including both native and non-native English speakers, following the guided learning could identify 100 haptic words with over 90% accuracy, while only 20% of participants using the self-guided learning paradigm could do so.

References

[1]

Sophia Alcorn. 1932. The Tadoma Method. The Volta Review 34 (1932), 195--198.

[2]

Francis S. Bellezza. 2012. Imagery and Related Mnemonic Processes. Springer New York, New York, NY. 470 pages.

[3]

Francis S. Bellezza and B. Goverdhan Reddy. 1978. Mnemonic devices and natural memory. Bulletin of the Psychonomic Society 11, 5 (1978), 277--280.

[4]

Vladimir V. Bochkarev, Anna V. Shevlyakova, and Valery D. Solovyev. 2015. The average word length dynamics as an indicator of cultural changes in society. Social Evolution and History 14, 2 (2015), 153--175. arXiv:1208.6109

[5]

P. L. Brooks and B. J. Frost. 1983. Evaluation of a Tactile Vocoder for Word Recognition. Journal of the Acoustical Society of America 74, 1 (1983), 34--39.

[6]

P. L. Brooks, B. J. Frost, J. L. Mason, and K. Chung. 1985. Acquisition of a 250-word vocabulary through a tactile vocoder. The Journal of the Acoustical Society of America 77, 4 (apr 1985), 1576--1579.

[7]

P L Brooks, B J Frost, J L Mason, and D M Gibson. 1986. Continuing evaluation of the Queen's University tactile vocoder II: Identification of open set sentences and tracking narrative. Journal of rehabilitation research and development 23, 1 (1986), 129--138.

[8]

Mei-Mei Chang. 2005. Applying Self-Regulated Learning Strategies in a Web-Based Instruction - An Investigation of Motivation Perception. Computer Assisted Language Learning 18, 3 (2005), 217--230.

[9]

Roger W Cholewiak and Amy A. Collins. 2003. Vibrotactile localization on the arm: Effects of place, space, and age. Perception & Psychophysics 65, 7 (oct 2003), 1058--1077.

[10]

Shelda Debowski, Robert E Wood, and Albert Bandura. 2001. Impact of Guided Exploration and Enactive Exploration on Self-Regulatory Mechanisms and Information Acquisition Through Electronic Search. Journal of Applied Psychology 86, 6 (2001), 1129--1141.

[11]

Mary Jo Dondlinger. 2007. Educational Video Game Design : A Review of the Literature. Journal of Applied Educational Technology 4, 1 (2007).

[12]

David Eagleman. 2014. Plenary talks: A vibrotactile sensory substitution device for the deaf and profoundly hearing impaired. In 2014 IEEE Haptics Symposium (HAPTICS). IEEE, xvii--xvii.

[13]

Donald H. Ecroyd. 1966. Making the separate sounds of speech. In Voice and Articulation: A Handbook. Scott Foresman & Co, Chapter 5, 63--87.

[14]

Mario Enriquez, Karon MacLean, and Christian Chita. 2006. Haptic phonemes: basic building blocks of haptic communication. Proceedings of the 8th international conference on Multimodal interfaces (2006), 302--309.

Digital Library

[15]

Robert H. Gault. 1924. Progress in experiments on tactual interpretation of oral speech. Journal of Abnormal Psychology and Social Psychology 19, 2 (1924), 155--159.

[16]

Frank A Geldard. 1957. Adventures in tactile literacy. American Psychologist 12, 3 (1957), 115--124.

[17]

Jan H. Hulstijn. 2005. Theoretical and Empirical Issues in the Study of Implicit and Explicit Second-Language Learning: Introduction. Studies in Second Language Acquisition 27, 02 (jun 2005), 129--140.

[18]

Reinhold Kliegl, Jacqui Smith, and Paul B. Baltes. 1990. On the Locus and Process of Magnification of Age Differences During Mnemonic Training. Developmental Psychology 26, 6 (1990), 894--904.

[19]

Nate Kornell and Robert A Bjork. 2007. The promise and perils of self-regulated study. Psychonomic Bulletin & Review 14, 2 (2007), 219--224.

[20]

Nate Kornell and Robert A Bjork. 2008. Optimising self-regulated study: The benefits and costs of dropping flashcards. Memory 16, 2 (2008), 125--136.

[21]

Lori F. Lamel, Robert H. Kassel, and Stephanie Seneff. 1989. Speech Database Development: Design and Analysis of the Acoustic-Phonetic Corpus. Speech Input/Output Assessment and Speech Databases 2 (1989), 2161--2170.

[22]

Yi-Chi Liao, Yi-Ling Chen, Jo-Yu Lo, Rong-Hao Liang, Liwei Chan, and Bing-Yu Chen. 2016. EdgeVib: effective alphanumeric character output using a wrist-worn tactile display. Proceedings of the 29th Annual Symposium on User Interface Software and Technology - UIST'16 (2016), 595--601.

Digital Library

[23]

Björn Lindblom. 1996. Role of articulation in speech perception: Clues from production. The Journal of the Acoustical Society of America 99, 3 (1996), 1683--1692.

[24]

Lara L. MacQuarrie, James A. Tucker, Matthew K. Burns, and Brian Hartman. 2002. Comparison of retention rates using traditional, drill sandwich, and incremental rehearsal flash card methods. School Psychology Review 31, 4 (2002), 584--595. http://scholar.google.com/scholar?hl=en{&}btnG=Search{&}q=intitle:Comparison+of+Retention+Rates+Using+Traditional, +Drill+Sandwich+, +and+Incremental+Rehearsal+Flash+Card+Methods{#}0

[25]

Scott D. Novich and David M. Eagleman. 2015. Using space and time to encode vibrotactile information: toward an estimate of the skin's achievable throughput. Experimental Brain Research 233, 10 (2015), 2777--2788.

[26]

A S Reber and R Allen. 2000. Individual differences in implicit learning: Implications for the evolution of consciousness. In Individual differences in conscious experience: Advances in consciousness research, Robert G. Kunzendorf and Benjamin Wallace (Eds.). John Benjamins Publishing Company, 227--247.

[27]

Maura L. Roberts, Timothy L. Turco, and Edward S. Shapiro. 1991. Differential effects of fixed instructional ratios on students' progress in reading. Journal of Psychoeducational Assessment 9, 4 (1991), 308--318.

[28]

Robert Turcott, Jennifer Chen, Pablo Castillo, Brian Knott, Wahyudinata Setiawan, Forrest Briggs, Keith Klumb, Freddy Abnousi, Prasad Chakka, Frances Lau, and Ali Israr. 2018. Efficient Evaluation of Coding Strategies for Transcutaneous Language Communication. In Proceedings of Eurohaptics.

[29]

Siyan Zhao, Ali Israr, Frances Lau, and Freddy Abnousi. 2018. Coding Tactile Symbols for Phonemic Communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '18).

Digital Library

[30]

Barry J Zimmerman. 1990. Self-Regulated Learning and Academic Achievement : An Overview Self-Regulated Learning and Academic Achievement: An Overview. Educational Psychologist 25, 1 (1990), 3--17.

Cited By

Jung JReed CMartinez JTan H(2024)Tactile Speech Communication: Reception of Words and Two-Way Messages through a Phoneme-Based DisplayVirtual Worlds10.3390/virtualworlds30200103:2(184-207)Online publication date: 7-May-2024
https://doi.org/10.3390/virtualworlds3020010
Macklin AYau JFischer-Baum SO'Malley M(2023)Representational Similarity Analysis for Tracking Neural Correlates of Haptic Learning on a Multimodal DeviceIEEE Transactions on Haptics10.1109/TOH.2023.330383816:3(424-435)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TOH.2023.3303838
Reed CTan HJones L(2023)Haptic Communication of LanguageIEEE Transactions on Haptics10.1109/TOH.2023.325753916:2(134-153)Online publication date: Apr-2023
https://doi.org/10.1109/TOH.2023.3257539
Show More Cited By

Index Terms

Learning to feel words: a comparison of learning approaches to acquire haptic words
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Feeling Speech on the Arm
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

In this paper, we introduce a transcutaneous language communication (TLC) system that transmits a tactile representation of spoken or written language to the arm. Users receive messages without looking at their smart devices, and feel them through their ...
Learning to lemmatise slovene words
Learning language in logic

Automatic lemmatisation is a core application for many language processing tasks. In inflectionally rich languages, such as Slovene, assigning the correct lemma to each word in a running text is not trivial: nouns and adjectives, for instance, inflect ...
Wearable head-mounted 3D tactile display application scenarios
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct

Current generation virtual reality (VR) and augmented reality (AR) head-mounted displays (HMDs) usually include no or only a single vibration motor for haptic feedback and do not use it for guidance. In a previous work, we presented HapticHead, a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAP '18: Proceedings of the 15th ACM Symposium on Applied Perception

August 2018

162 pages

ISBN:9781450358941

DOI:10.1145/3225153

Conference Chairs:
Cindy Grimm
Oregon State University
,
Peter Willemsen
University of Minnesota
,
Program Chairs:
Joseph Kearney
University of Iowa
,
Bernhard Riecke
Simon Fraser University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 August 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAP '18

Sponsor:

SIGGRAPH

SAP '18: ACM Symposium on Applied Perception 2018

August 10 - 11, 2018

British Columbia, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 43 of 94 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
217
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jung JReed CMartinez JTan H(2024)Tactile Speech Communication: Reception of Words and Two-Way Messages through a Phoneme-Based DisplayVirtual Worlds10.3390/virtualworlds30200103:2(184-207)Online publication date: 7-May-2024
https://doi.org/10.3390/virtualworlds3020010
Macklin AYau JFischer-Baum SO'Malley M(2023)Representational Similarity Analysis for Tracking Neural Correlates of Haptic Learning on a Multimodal DeviceIEEE Transactions on Haptics10.1109/TOH.2023.330383816:3(424-435)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TOH.2023.3303838
Reed CTan HJones L(2023)Haptic Communication of LanguageIEEE Transactions on Haptics10.1109/TOH.2023.325753916:2(134-153)Online publication date: Apr-2023
https://doi.org/10.1109/TOH.2023.3257539
Luzhnica GKrajnc AVeas E(2022)Exploring Feedback-based Testing Effects for Skin ReadingProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519393(212-217)Online publication date: 13-Mar-2022
https://dl.acm.org/doi/10.1145/3519391.3519393
Luzhnica GVeas E(2022)A Visualisation Driven Training for Vibrotactile Skin ReadingIEEE Transactions on Haptics10.1109/TOH.2021.313877315:1(103-108)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TOH.2021.3138773
Kappers APlaisier M(2021)Hands-Free Devices for Displaying Speech and Language in the Tactile Modality — Methods and ApproachesIEEE Transactions on Haptics10.1109/TOH.2021.305173714:3(465-478)Online publication date: 1-Jul-2021
https://doi.org/10.1109/TOH.2021.3051737
Dunkelberger NSullivan JBradley JManickam IDasarathy GBaraniuk RO'Malley M(2021)A Multisensory Approach to Present Phonemes as Language Through a Wearable Haptic DeviceIEEE Transactions on Haptics10.1109/TOH.2020.300958114:1(188-199)Online publication date: 1-Jan-2021
https://doi.org/10.1109/TOH.2020.3009581
Luzhnica GVeas EFarrahi KHarle RLane N(2019)Boosting word recognition for vibrotactile skin readingProceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341163.3347715(135-144)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341163.3347715
Luzhnica GVeas EBrewster SFitzpatrick GCox AKostakos V(2019)Optimising Encoding for Vibrotactile Skin ReadingProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300465(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300465
Camarillo-Abad HAlfredo Sánchez JStarostenko O(2019)Organizing Knowledge on Nonverbal Communication Mediated Through Haptic TechnologyHuman-Computer Interaction10.1007/978-3-030-37386-3_20(269-283)Online publication date: 15-Dec-2019
https://doi.org/10.1007/978-3-030-37386-3_20

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents