research-article

Talk and Draw: Bundling Speech and Graphics

Authors:

Mark W. Salisbury,

Joseph H. Hendrickson,

Terence L. Lammers,

Caroline Fu,

Scott A. MoodyAuthors Info & Claims

Computer, Volume 23, Issue 8

Pages 59 - 65

https://doi.org/10.1109/2.56872

Published: 01 August 1990 Publication History

Publisher Site

Abstract

A description is given of a workstation with integrated graphical and voice I/O (input/output), to improve the man-machine interface (MMI) of the Airborne Warning and Control System (AWACS). Incremental prototype development is used to improve the MMI for AWACS. Operator tasks were analyzed to identify functions that would be better performed with voice I/O, and a 70-word vocabulary and corresponding grammar were developed for an ITT VRS-1280 speech recognizer. Responses for a DEC Talker speech synthesizer were also developed. The system architecture for integrating voice and graphical input in the AWACS interface prototype and its implementation are described and development issues are examined. Future developments are considered.

References

[1]

1. O. Sacks, "The President's Speech," in The Man Who Mistook His Wife for a Hat, Summit Books, New York, 1985, pp. 76-80.

Google Scholar

[2]

2. G. Rohr, "Using Visual Concepts," in Visual Languages, S. Chang, T. Ichikawa, and P. Ligomenides, eds., Plenum Press, New York, 1986, pp. 325-348.

Google Scholar

[3]

3. S. Guastello, M. Traut, and G. Korienek, "Verbal Versus Pictorial Representations of Objects in a Human-Computer Interface," Int'l J. Man-Machine Studies, July 1989, Vol. 31, No. 1, pp. 99-120.

Digital Library

Google Scholar

[4]

4. C. Fu, "An Independent Workstation for a Quadriplegic," in International Exchange of Experts and Infomation in Rehabilitation , Monograph #37, Richard Foulds, ed., World Rehabilitation Fund, New York, 1986, pp. 42-44.

Google Scholar

[5]

5. R. Bolt, "'Put That There': Voice and Gesture at the Graphics Interface," tech. report, MIT, Cambridge, Mass., 1980.

Google Scholar

[6]

6. C.D. Anderson, "Application of Speech Recognition and Touch-Screen Input Systems to Airborne C<sup>3</sup> Operations--Results of Mission Simulator Evaluation," Document No. 10180-28809-1, The Boeing Co., Seattle, Wash., 1985.

Google Scholar

[7]

7. S.R. Young et al., "High-Level Knowledge Sources in Usable Speech Recognition Systems," Comm. ACM, Vol. 32, Feb. 1989, pp. 183-194.

Digital Library

Google Scholar

[8]

8. R. Jackendoff, Semantics and Cognition, MIT Press, Cambridge, Mass., 1983.

Google Scholar

[9]

9. A. Requicha and J. Vandenbrande, "Automated Systems for Process Planning and Part Programming," in Artificial Intelligence: Implications for CIM, A. Kusiak, ed., Springer-Verlag, Berlin, 1988, pp. 301- 326.

Google Scholar

[10]

10. H.-J. Novak, "Strategies for Generating Coherent Descriptions of Object Movements in Street Scenes," in Natural Language Generation , G. Kempen, ed., Martinus Nijhoff, Dordrecht, Netherlands, 1987, pp. 117-132.

Google Scholar

[11]

11. C. Fillmore, "The Case for Case," in Universals in Linguistic Theory, E. Bach and R. Harms, eds., Holt, Rinehart & Winston, New York, 1968, pp. 1-90.

Google Scholar

[12]

12. S.A. Moody, T.H. Hudson, and M.W. Salisbury, "Rapid: A Prototyping Environment for Battle Management Information Systems," Proc. Third Annual User-System Interface Conf., Austin, Tex., 1988, pp. 109-116.

Google Scholar

Cited By

View all

Cohen POviatt S(2017)Multimodal speech and pen interfacesThe Handbook of Multimodal-Multisensor Interfaces10.1145/3015783.3015795(403-447)Online publication date: 24-Apr-2017
https://dl.acm.org/doi/10.1145/3015783.3015795
Cohen PBuchanan MKaiser ECorrigan MLind SWesson MEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Demonstration of sketch-thru-planProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2531754(69-70)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2531754
Sharma AMadhvanath SShekhawat ABillinghurst MBourlard HHuang TVidal EGatica-Perez DMorency LSebe N(2011)MozArtProceedings of the 13th international conference on multimodal interfaces10.1145/2070481.2070538(307-310)Online publication date: 14-Nov-2011
https://dl.acm.org/doi/10.1145/2070481.2070538
Show More Cited By

Index Terms

Talk and Draw: Bundling Speech and Graphics

Recommendations

Cross-talk reduction
IJCAI '24: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence

While far-field multi-talker mixtures are recorded, each speaker can wear a close-talk microphone so that close-talk mixtures can be recorded at the same time. Although each close-talk mixture has a high signal-to-noise ratio (SNR) of the wearer, it has ...
Recognising speakers from the topics they talk about

We investigate how a speaker's preference for specific topics can be used for speaker identification. In domains like broadcast news or parliamentary speeches, speakers have a field of expertise they are associated with. We explore how topic information ...
Speech Speed Awareness System Slows Down Native Speaker’s Talk
Collaboration Technologies and Social Computing
Abstract
Conversation between native speakers and non-native speakers is not always easy. Non-native speakers sometimes feel hard to understand what native speakers talk because of their speaking speed. Even kind native speakers who speak slower in the ...

Comments

Information & Contributors

Information

Published In

Computer Volume 23, Issue 8

August 1990

88 pages

ISSN:0018-9162

Editor:
Bruce D. Shriver
Univ. of Southerwestern Louisiana,

Issue’s Table of Contents

Publisher

IEEE Computer Society Press

Washington, DC, United States

Publication History

Published: 01 August 1990

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cohen POviatt S(2017)Multimodal speech and pen interfacesThe Handbook of Multimodal-Multisensor Interfaces10.1145/3015783.3015795(403-447)Online publication date: 24-Apr-2017
https://dl.acm.org/doi/10.1145/3015783.3015795
Cohen PBuchanan MKaiser ECorrigan MLind SWesson MEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Demonstration of sketch-thru-planProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2531754(69-70)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2531754
Sharma AMadhvanath SShekhawat ABillinghurst MBourlard HHuang TVidal EGatica-Perez DMorency LSebe N(2011)MozArtProceedings of the 13th international conference on multimodal interfaces10.1145/2070481.2070538(307-310)Online publication date: 14-Nov-2011
https://dl.acm.org/doi/10.1145/2070481.2070538
Irawati SGreen SBillinghurst MDuenser AKo H(2006)"Move the couch where?"Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality10.1109/ISMAR.2006.297812(183-186)Online publication date: 22-Oct-2006
https://dl.acm.org/doi/10.1109/ISMAR.2006.297812
Irawati SGreen SBillinghurst MDuenser AKo H(2006)An evaluation of an augmented reality multimodal interface using speech and paddle gesturesProceedings of the 16th international conference on Advances in Artificial Reality and Tele-Existence10.1007/11941354_28(272-283)Online publication date: 29-Nov-2006
https://dl.acm.org/doi/10.1007/11941354_28
Kim HSeo J(2003)Resolution of referring expressions in a Korean multimodal dialogue systemACM Transactions on Asian Language Information Processing10.1145/1007551.10075532:4(324-337)Online publication date: 1-Dec-2003
https://dl.acm.org/doi/10.1145/1007551.1007553
Billinghurst M(1998)Put that where? voice and gesture at the graphics interfaceACM SIGGRAPH Computer Graphics10.1145/307710.30773032:4(60-63)Online publication date: 1-Nov-1998
https://dl.acm.org/doi/10.1145/307710.307730
Grasso MFinin T(1997)Task integration in multimodal speech recognition environmentsXRDS: Crossroads, The ACM Magazine for Students10.1145/270974.2709823:3(19-22)Online publication date: 1-Apr-1997
https://dl.acm.org/doi/10.1145/270974.270982
Billinghurst MSavage J(1996)Adding intelligence to the interfaceProceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96)10.5555/832290.836047Online publication date: 30-Mar-1996
https://dl.acm.org/doi/10.5555/832290.836047
Faconti GBordegoni MKansy KTrahanias PRist TWilson M(1996)Formal framework and necessary properties of the fusion of input modes in user interfacesInteracting with Computers10.1016/0953-5438(96)01029-68:2(134-161)Online publication date: 1-Jun-1996
https://dl.acm.org/doi/10.1016/0953-5438%2896%2901029-6
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

Cross-talk reduction

Recognising speakers from the topics they talk about

Speech Speed Awareness System Slows Down Native Speaker’s Talk

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations