3D User Interfaces

Possibilities of virtual reality (VR) technology have gained considerable attention recently due to technical advances in affordable head-mounted displays. Yet the use of VR technology has not become mainstream, and there still does not exist a “killer application” for VR. One reason for this situation could be the inherent difficulty of VR software development.
This thesis investigates challenges specific to VR software development, and explores methodology for such research. The thesis includes some of the earliest quantitative analysis on VR software development challenges, identifies the most severe development issues, and proposes solutions to them. This has implications on how VR software development could be eased. The analysis is based on data collected from 132 developers of VR application programs, which forms the backbone of the research.
The thesis introduces RUIS, a software toolkit for facilitating hobbyist innovation by simplifying the development of VR application programs that rely on immersive displays and spatial interaction devices. Case studies employing VR application programs created with RUIS are included, describing different ways how 3D user interfaces can affect the experience and performance of VR software users.
Methodology for benchmarking VR toolkits is presented, RUIS is contrasted with other toolkits, and multiple VR application programs created by students with RUIS are juxtaposed. The results demonstrate the importance of the chosen VR toolkit for the development process in two ways: 1) by presenting several comparisons that show how different VR toolkits can significantly affect the experienced development challenges, and 2) by highlighting the quantifiable distinctions in VR application programs created with different toolkits.
Additionally, this thesis features an extensive survey on the developers of 3DUI application programs, revealing their demographics, the software and the hardware that they use, and an overview of the 3DUI application programs that they create. The survey also points out those development challenges that particularly affect inexperienced developers, and illustrates that the reuse of high-level 3D user interface features is low. Potential solutions to these issues are proposed in the thesis.

One of the goals of much virtual reality (VR) research is to increase realism. In particular, many techniques for locomotion in VR attempt to approximate real-world walking. However, it is not yet fully understood how the design of more realistic locomotion techniques affects user task performance. We performed an experiment to compare a semi-natural locomotion technique (based on the Virtusphere device) with a traditional, non-natural technique (based on a game controller) and a fully natural technique (real walking). We found that the Virtusphere technique was significantly slower and less accurate than both of the other techniques. Based on this result and others in the literature, we speculate that locomotion techniques with moderate interaction fidelity will often have performance inferior to both high-fidelity techniques and well-designed low-fidelity techniques. We argue that our experimental results are an effect of interaction fidelity, and perform a detailed analysis of the fidelity of the three locomotion techniques to support this argument.

Virtual reality (VR) locomotion techniques that approximate real-world walking often have lower performance than fully natural real walking due to moderate interaction fidelity. Other techniques with moderate fidelity, however, are intentionally designed to enhance users' abilities beyond what is possible in the real world. We compared such hyper-natural techniques to their natural counterparts on a wide range of locomotion tasks for a variety of measures. The evaluation also considered two independent components of interaction fidelity: bio-mechanics and transfer function. The results show that hyper-natural transfer functions can improve locomotion speed and some aspects of user satisfaction , although this can come at the expense of accuracy for complicated path-following tasks. On the other hand, hyper-natural techniques designed to provide biomechanical assistance had lower performance and user ac ceptance than those based on natural walking movements. These results contribute to a deeper understanding of the effects of interaction fidelity and designer intent for VR interaction techniques.

The purpose of this study is to determine the parameters of the best possible spatial user interface design for the online shopping experience in mixed reality. In the first phase of the study, ten existing augmented and virtual reality shopping applications are examined, and the spatial relationships of the interfaces used in these experiences and the possibilities of natural interactions provided to the user are compared. In the next phase of the study, the interfaces are evaluated to propose a three-dimensional spatial interface powered by the mixed reality that improves the spatial relationship and neutrality of these interfaces.

Ambient Interface research has the goal of embedding technology that disappears into the user’s surroundings. In many ways Augmented Reality (AR) technology is complimentary to this in that AR interfaces seamlessly enhances the real environment with virtual information overlay. The two merge together in context aware Ambient AR applications, which allow users to easily perceive and interact with Ambient Interfaces by using AR overlay of the real world. In this paper we describe how Tangible Interaction techniques can be used for Ambient AR applications. We will present a conceptual framework for Ambient Tangible AR Interface, a new generation of software and hardware tools for development and methods for evaluating Ambient Tangible AR Interfaces.

The purpose of this thesis is to propose and evaluate a markerless system for capturing hand movements in real time to allow 3D interactions in virtual environment (VE). Tools such as keyboard and mouse are not enough for interacting in 3D VE; current motion capture systems are expensive and require wearing equipments. Systems based on cameras and image processing partially fill these gaps, but do not yet allow an accurate, efficient and real-time 3D motion capture. Our system provides a solution to this problem with a 3D camera. We have implemented modalities that allow a more natural interaction with objects and VE. The goal of our system is to obtain performances at least equal to those of common tools in virtual reality while providing a better overall acceptability (i.e., usefulness, usability, immersion).
To achieve this goal, we conducted three experimental studies involving over 100 participants. With the first study, we compared the first version of our system (based on a 3D Camera MESA SwissRanger) to a traditional mouse, for a selection task. The second experiment was focused on the study of the realization of objects-manipulation tasks (position, orientation, scaling) and navigation tasks in VE. For this study, we compared the improved version of our system (based on Microsoft Kinect) with data gloves associated with magnetic sensors. An additional study concerns the evaluation of new modalities of interaction, implemented based on participants’ feedback of the second study.
Keywords: Hand motion capture, 3D interaction, bimanual interaction, markerless device, real-time, 3D camera.

Advanced 3D virtual environment technology, similar to that used by the film and computer games industry can allow educational developers to rapidly create realistic 3D, virtual environments. This technology has been used to generate a range of interactive learning environments across a broad spectrum of industries and educational application areas.
Virtual Reality (VR) simulators represent a powerful tool for learning and teaching. The idea is not new. Flight simulators have been used for decades to train pilots for both commercial and military aviation. These systems have advanced to a point that they are integral to both the design and the operation of modern aircraft [1, 2]. There are a number of lessons that can be learned from other industries that have successfully utilised virtual training and learning systems for a number of years. Generic rules of thumb regarding the specification, development, application and operation of these learning environments can be garnered from other industrial training systems and examined in an educational context [3, 4, 5, 6, 7].
This paper will introduce a virtual learning environment which has been developed by the authors. During the implementation of this, and other, visual learning environments a number of complex operational problems have been encountered, these have required a number of innovative solutions and management procedures to be developed.
The paper will also discuss the implementation of these systems and extrapolate the lessons learnt into general guidelines to be considered for the development of VR based educational learning resources. These guidelines will then be discussed in the context of the development of ViRILE (Virtual Reality Interactive Learning Environment). This software is designed for use by undergraduate chemical engineers and simulates the configuration and operation of a polymerisation plant [4].

With this paper we document an experiment that is defined to test to what extent stereoscopic depth may control a user’s gaze pattern while performing a search task. The experiment consisted in displaying a search engine result page created in a 2D and 3D side by side format, and a questionnaire that gathers information about participants’ opinion towards a stereoscopic 3D experience. While this experiment is a first pilot and more research is needed, it is indicative of user behavior. In this paper we describe preparations for the study, hardware and software tools used to carry out the experiment, and results analysis.

This paper investigates some important visual factors of immersive 3D user interfaces on the latest 3D displays and studies how they can be applied for 3D TVs at home. The vergence-accommodation conflict in im mersive 3D user interfaces is addressed in order to theoretically estimate some perceived parameters of the user interfaces under this constraint. To demonstrate the proposed immersive 3D user interfaces, a 3D TV browse and control demo is built and some stereo screenshot results are presented.

This paper presents a novel interaction technique to combine multiple inputs and output modalities with a 3D printed open disc which is tracked and augmented with a virtual information. Props have been used in virtual and augmented environments to provide tangible associations to virtual elements. When used in a larger immersive environment or where a user is walking around, users may not have the capability to set these objects/props down or are limited to the number of hands, so users are reduced to the capability of using only two props or controllers. In our technique, users can instead use their body to store and organize the virtual objects associated with tangible props by physically hanging them on or attaching them to the body. This type of interaction is well-suited for both immersive visualizations and immersive virtual environments. In this paper, we present the results of an experimental evaluation of our Body Prop Interaction technique for each type of immersive environment and corresponding task type, demonstrating more effective usage in each scenario than a joystick or gesture-based interaction.

We propose an inherently three-dimensional interaction paradigm which allows individuals to manage their personal digital artifact collections (PAC) regardless of the specific devices and means they are using. The core of our solution is to provide unified access to all user artifacts normally spread across several repositories and devices. Not till then individuals may foster and evolve persistent multi-hierarchical artifact structures (PAS) fitting their cognitive needs. PAS subsets can be arranged and meaningfully related to virtual habitats or even mapped to physical contexts and environments they are frequenting to solve their tasks.

A method of Selecting and displaying an item on a display
Screen uses a pointing and Selecting device including a
control button. Displayed, on the Screen, are (i) a multifac
eted three dimensional object having a fixed center and
having at least one side including at least a representative
window of a first item to select and display, and (ii) a fixed
pointer. The representative window includes a link to the
item to be Selected and displayed. The three dimensional
object is rotated on the screen about the fixed center with
respect to three free orthogonal axes, by the manipulation of
the pointing device. The representative window of the item
to be Selected and displayed is Superimposed with the
displayed pointer at the center of the three dimensional
object, for the Selection and display on the display Screen of
the item to be Selected and displayed by pushing the control
button of the pointing device.

Procédé de navigation en images de synthèse 3D par la manipulation d'image 3D. L'invention concerne un dispositif technique de sélection d'un fichier par la manipulation d'un objet 3D virtuel (2). La manipulation en 3D permet d'accéder rapidement à de nombreuses données aux arborescences complexes et exponentielles, par une interface intuitive, comme si nous tournions un cube dans notre main. Ce procédé de navigation 3D est caractérisé en ce qu'il comporte un volume 3D (2), volume (2) dont le centre est fixe et confondu avec le centre de la surface d'affichage du dispositif et symbolisé par un repère: 'le pointeur' (3), volume (2) sur les faces duquel sont affichées les représentations des éléments sélectionnables 'les objets' (4), volume (2) manipulé en temps réel dans les trois degrés de liberté en rotation grâce à un dispositif de pointage (6) (trackball, ...) afin de positionner l'élément choisi sur le pointeur (3). La sélection est déclenchée par un click sur un bouton (5) du dispositif de pointage (6). Les objets (4) sont représentés par un élément graphique issu d'un fichier numérique. A chaque objet (4) est associé un lien. Le point de vue de l'observateur est fixe. Le dispositif selon l'invention est particulièrement destiné à faciliter l'accès rapide à un fichier numérique dans une base de données numérique importante.