Automatic Extraction of Semantic Information for a Context Sensitive Multimodal Framework for VR

The capability of processing spoken commands is one of the most important features of modern multimodal AR/VR environments. This feature requires programmers to compile some human supplied knowledge in the form of grammars which are used at runtime to process spoken utterances into complete commands. Further speech recognition (SR) must be hard-coded into the application. This time-consuming, error-prone process is repeated every time modifications to the code are introduced. This paper presents a completely automatic process to build a body of knowledge from the information embedded within the application source code. The programmer in fact often embeds, throughout the coding process, a vast amount of semantic information by defining classes, reference names, or through method definitions. This research work exploits this semantic richness and it provides a self-configurable system, which automatically adapts its understanding of human commands according to the semantic information within the application's source code.