Article

Enabling multimodal communications for enhancing the ability of learning for the visually impaired

Authors:

Francisco OliveiraAuthors Info & Claims

ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

Pages 326 - 332

https://doi.org/10.1145/1180995.1181056

Published: 02 November 2006 Publication History

Abstract

Students who are blind are typically one to three years behind their seeing counterparts in mathematics and science. We posit that a key reason for this resides in the inability of such students to access multimodal embodied communicative behavior of mathematics instructors. This impedes the ability of blind students and their teachers to maintain situated communication. In this paper, we set forth the relevant phenomenological analyses to support this claim. We show that mathematical communication and instruction are inherent embodied; that the blind are able to conceptualize visuo-spatial information; and argue that uptake of embodied behavior is critical to receiving relevant mathematical information. Based on this analysis, we advance an approach to provide students who are blind with awareness of their teachers' deictic gestural activity via a set of haptic output devices. We lay forth a set of open research question that researcher in multimodal interfaces may address.

References

[1]

Williams, J. M., Nationwide shortage of teachers for blind students must be corrected. National Federation of the Blind: Advocates for Equality, Canadian Blind Monitor, 2002. 14.

[2]

McNeill, D. and S. Duncan, Growth points in thinking-for-speaking, in Language and Gesture, D. McNeill, Editor. 2000, Cambridge University Press: Cambridge, MA. p. 141--161.

[3]

McNeill, D., Gesture and language dialectic. Acta Linguistica Hafniensia, 2002. 34: p. 7--37.

[4]

Beattie, G. and H. Shovelton, Iconic hand gestures and the predictability of words in context in spontaneous speech. British Journal of Psychology, 2000. 91: p. 473--491.

[5]

Beattie, G., Visible Thought: The New Psychology of body Language. 2003, Hove: Routledge.

[6]

Furuyama, N., Gestural interaction between the instructor and the learner in origami instruction, in Language and gesture. 2000, Cambridge University Press: Cambridge. p. 99--117.

[7]

Penrose, R., The Emporer's New Mind. 1989, New York: Oxford University Press.

[8]

McNeill, D., Hand and Mind: What Gestures Reveal about thought. 1992, Chicago: University of Chicago Press.

[9]

Smith, N., Gesture and Beyond, in Cognitive Science. 2003, University of California at: Berkeley.

[10]

Haber, R. N., Haber, L. R., Levin, C. A. and Hollyfield, R., Properties of spatial representations: Data from sighted and blind subjects. Perception and Psychophysics, 1993. 54: p. 1--13.

[11]

Millar, S., Movement cues and body orientation in recall of locations by blind and sighted children. Quarterly Journal of Psychology, 1985(A 37): p. 257--279.

[12]

Landau, B., E. Spelke, and H. Gleitman, Spatial knowledge in a young blind child. Cognition, 1984. 16: p. 225--260.

[13]

Kennedy, J. M., Drawing and the Blind. 1993, New Haven, CT: Yale Press.

[14]

Iverson, J. M. and S. Goldin-Meadow, Why people gesture as they speak. Nature, 1998. 396: p. 228.

[15]

Heller, M. A., Tactile picture perception in sighted and blind people. Behavioural Brain Research, 2002. 135: p. 65--68.

[16]

Craig, R. H., Learning the Nemeth Braille Code: A Manual for Teachers. 1980, Salt Lake City: Brigham Young University Press.

[17]

Eramian, M. G., et al. Talking Tactile Diagrams. in Universal Access in HCI: Inclusive Design in the Information Society, Proceedings of HCI International 2003. 2003. Hersonissos, Crete, Greece.

[18]

Williams, A., E. Kabish, and P. Dourish. From Interaction to Participation: Configuring Space through Embodied Interaction. in Proc. Intl. Conf. Ubiquitous Computing (Ubicomp 2005). 2005. Tokyo, Japan.

Digital Library

[19]

Tall, D., A Theory of Mathematical Growth through Embodiment, Symbolism and Proof, in International Colloquium on Mathematical Learning from Early Childhood to Adulthood. 2005: Nevellis, Belguim.

Cited By

Bargerhuff MCowan HOliveira FQuek FFang B(2019)Haptic Glove Technology: Skill Development through Video Game PlayJournal of Visual Impairment & Blindness10.1177/0145482X1010401103104:11(688-699)Online publication date: 7-Jan-2019
https://doi.org/10.1177/0145482X1010401103
Oliveira FCowan HFang BQuek F(2010)Fun to develop embodied skillProceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments10.1145/1839294.1839313(1-8)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.1145/1839294.1839313
Oliveira FQuek F(2008)A multimodal communication with a haptic gloveProceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments10.1145/1389586.1389630(1-8)Online publication date: 16-Jul-2008
https://dl.acm.org/doi/10.1145/1389586.1389630

Index Terms

Enabling multimodal communications for enhancing the ability of learning for the visually impaired
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices

Recommendations

Embodiment and multimodality
ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

Students who are blind are typically one to three years behind their seeing counterparts in mathematics and science. We posit that a key reason for this resides in the inability of such students to access multimodal embodied communicative behavior of ...
Authoring Communicative Behaviors for Situated, Embodied Characters
ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

Embodied conversational agents hold great potential as multimodal interfaces due to their ability to communicate naturally using speech and nonverbal cues. The goal of my research is to enable animators and designers to endow ECAs with interactive ...
A multimodal communication with a haptic glove: on the fusion of speech and deixis over a raised line drawing
PETRA '08: Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments

Mathematics instruction and discourse typically involve two modes of communication: speech and graphical presentation. For the communication to remain situated, dynamic synchrony must be maintained between the speech and dynamic focus in the graphics. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

November 2006

404 pages

ISBN:159593541X

DOI:10.1145/1180995

General Chairs:
Francis Quek
Virginia Tech, USA
,
Jie Yang
Carnegie Mellon University, USA
,
Program Chairs:
Dominic Massaro
University of California, Santa Cruz, USA
,
Abeer Alwan
University of California, Los Angeles, USA
,
Timothy J. Hazen
Massachusetts Institute of Technology, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI06

Sponsor:

ICMI06: 8th International Conference on Multimodal Interfaces 2006

November 2 - 4, 2006

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
353
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bargerhuff MCowan HOliveira FQuek FFang B(2019)Haptic Glove Technology: Skill Development through Video Game PlayJournal of Visual Impairment & Blindness10.1177/0145482X1010401103104:11(688-699)Online publication date: 7-Jan-2019
https://doi.org/10.1177/0145482X1010401103
Oliveira FCowan HFang BQuek F(2010)Fun to develop embodied skillProceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments10.1145/1839294.1839313(1-8)Online publication date: 23-Jun-2010
https://dl.acm.org/doi/10.1145/1839294.1839313
Oliveira FQuek F(2008)A multimodal communication with a haptic gloveProceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments10.1145/1389586.1389630(1-8)Online publication date: 16-Jul-2008
https://dl.acm.org/doi/10.1145/1389586.1389630

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents