Abstract
A methodology for developing a multimodal interactive system is required to be implemented as the number of input and output devices is increasing. Previous MMI (MultiModal Interaction) description languages used a state transition model for defining interaction, which has low maintainability and extensibility. In this paper, we design and implement a new MMI description language named MrailsScript from the software engineering point of view. MrailsScript consists of the data model definition that can be inherited from existing semantic web class. By expanding the existing web application development framework, our Mrails framework can automatically generate prototype code of MMI applications that are based on the MVC (Model-View-Controller) model referring to the annotations of task type and dialogue initiative. We also develop a helper application for the MMI system, MrailsBuilder, which assists in the coding of MrailsScript and contents management using semantic web standards, such as OWL, RDFS, and RDF.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Hori, C., Ohtake, K., Misu, T., Kashioka, H., & Nakamura, S. (2009). Statistical dialog management applied to WFST-based dialog systems. In Proceedings of ICASSP 2009, Taipei, Taiwan, pp. 4793–4796.
Williams, J. D., & Young, S. (2007). Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21(2), 393–422.
Heinroth, T., Denich, D., & Bertrand, G. (2009). Ontology-based spoken dialogue modeling. In Proceedings of the IWSDS 2009, Irsee, Germany
Kogure, S., & Nakagawa, S. (2001). A development tool for spoken dialogue systems and its evaluation. In Proceedings of TSD2001 (LNAI 2166), Zelezna Ruda, Czech Republic, pp. 373–380.
Dean, M., & Schreiber, G. (Eds.). (2004). OWL Web Ontology Language Reference, W3C Recommendation. 10 February 2004. https://www.w3.org/TR/2004/REC-owl-ref-20040210/.
Araki, M., & Mizukami, Y. (2011). Development of a data-driven framework for multimodal interactive systems. In Proceedings of IWSDS 2011, Granada, Spain, pp. 91–101.
Araki, M. (2012a). Rapid development process of spoken dialogue systems using collaboratively constructed semantic resources. In Proceedings of SIGDial 2012, Seoul, South Korea, pp. 70–73.
Takegoshi, D., & Araki, M. (2014). Development environment for multimodal interactive system based on ontological knowledge. In Proceedings of IIAI AAI 2014, Kitakyushu, Japan, pp. 785–788. doi:10.1109/IIAI-AAI.2014.158.
Prud’hommeaux, E., & Seaborne, A. (Eds.). (2008). SPARQL Query Language for RDF. W3C Recommendation 15 January 2008. https://www.w3.org/TR/rdf-sparql-query/.
Shires, G., & Wennborg, H. (Eds.). (2012). Web Speech API Specification. https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html.
Araki, M. (2012b). Multimodal motion learning system for traditional arts. In Proceedings of AHFE2012, San Francisco, USA, pp. 5274–5281.
Araki, M., & Takegoshi, D. (2012). A rapid development framework for multilingual spoken dialogue systems. In Proceedings of COMPSAC 2012, Izmir, Turkey, pp. 195–202.
Acknowledgment
The collaborator of this research is Mr. Daisuke Takegoshi.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Araki, M. (2017). Multimodal Interaction Description Language Based on Data Modeling. In: Dahl, D. (eds) Multimodal Interaction with W3C Standards. Springer, Cham. https://doi.org/10.1007/978-3-319-42816-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-42816-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42814-7
Online ISBN: 978-3-319-42816-1
eBook Packages: EngineeringEngineering (R0)