3D User Interfaces

The purpose of this study is to determine the parameters of the best possible spatial user interface design for the online shopping experience in mixed reality. In the first phase of the study, ten existing augmented and virtual reality shopping applications are examined, and the spatial relationships of the interfaces used in these experiences and the possibilities of natural interactions provided to the user are compared. In the next phase of the study, the interfaces are evaluated to propose a three-dimensional spatial interface powered by the mixed reality that improves the spatial relationship and neutrality of these interfaces.

Possibilities of virtual reality (VR) technology have gained considerable attention recently due to technical advances in affordable head-mounted displays. Yet the use of VR technology has not become mainstream, and there still does not exist a “killer application” for VR. One reason for this situation could be the inherent difficulty of VR software development.
This thesis investigates challenges specific to VR software development, and explores methodology for such research. The thesis includes some of the earliest quantitative analysis on VR software development challenges, identifies the most severe development issues, and proposes solutions to them. This has implications on how VR software development could be eased. The analysis is based on data collected from 132 developers of VR application programs, which forms the backbone of the research.
The thesis introduces RUIS, a software toolkit for facilitating hobbyist innovation by simplifying the development of VR application programs that rely on immersive displays and spatial interaction devices. Case studies employing VR application programs created with RUIS are included, describing different ways how 3D user interfaces can affect the experience and performance of VR software users.
Methodology for benchmarking VR toolkits is presented, RUIS is contrasted with other toolkits, and multiple VR application programs created by students with RUIS are juxtaposed. The results demonstrate the importance of the chosen VR toolkit for the development process in two ways: 1) by presenting several comparisons that show how different VR toolkits can significantly affect the experienced development challenges, and 2) by highlighting the quantifiable distinctions in VR application programs created with different toolkits.
Additionally, this thesis features an extensive survey on the developers of 3DUI application programs, revealing their demographics, the software and the hardware that they use, and an overview of the 3DUI application programs that they create. The survey also points out those development challenges that particularly affect inexperienced developers, and illustrates that the reuse of high-level 3D user interface features is low. Potential solutions to these issues are proposed in the thesis.

The purpose of this thesis is to propose and evaluate a markerless system for capturing hand movements in real time to allow 3D interactions in virtual environment (VE). Tools such as keyboard and mouse are not enough for interacting in 3D VE; current motion capture systems are expensive and require wearing equipments. Systems based on cameras and image processing partially fill these gaps, but do not yet allow an accurate, efficient and real-time 3D motion capture. Our system provides a solution to this problem with a 3D camera. We have implemented modalities that allow a more natural interaction with objects and VE. The goal of our system is to obtain performances at least equal to those of common tools in virtual reality while providing a better overall acceptability (i.e., usefulness, usability, immersion).
To achieve this goal, we conducted three experimental studies involving over 100 participants. With the first study, we compared the first version of our system (based on a 3D Camera MESA SwissRanger) to a traditional mouse, for a selection task. The second experiment was focused on the study of the realization of objects-manipulation tasks (position, orientation, scaling) and navigation tasks in VE. For this study, we compared the improved version of our system (based on Microsoft Kinect) with data gloves associated with magnetic sensors. An additional study concerns the evaluation of new modalities of interaction, implemented based on participants’ feedback of the second study.
Keywords: Hand motion capture, 3D interaction, bimanual interaction, markerless device, real-time, 3D camera.

We present an immersive 3D modeling application with
stereoscopic graphics, head tracking, and 3D input devices. The
application was built in three weeks on top of Blender, an open
source 3D modeling software, and relies solely on affordable, offthe-shelf hardware like PlayStation Move controllers. Our goal was to create an easy to use 3D modeling environment that employs both 2D and 3D interaction techniques and contains several modeling tools. We conducted a basic user study where novice and professional 3D artists created 3D models with our application. The study participants thought that the application was fun and intuitive to use, but accurate posing of objects was difficult. We also examined the participants’ beliefs about future use of immersive technology in 3D modeling. The short implementation time of the application, its many features, and the 3D models created by the study participants set an example of what can be achieved with open source software and off-the-shelf hardware.

One of the goals of much virtual reality (VR) research is to increase realism. In particular, many techniques for locomotion in VR attempt to approximate real-world walking. However, it is not yet fully understood how the design of more realistic locomotion techniques affects user task performance. We performed an experiment to compare a semi-natural locomotion technique (based on the Virtusphere device) with a traditional, non-natural technique (based on a game controller) and a fully natural technique (real walking). We found that the Virtusphere technique was significantly slower and less accurate than both of the other techniques. Based on this result and others in the literature, we speculate that locomotion techniques with moderate interaction fidelity will often have performance inferior to both high-fidelity techniques and well-designed low-fidelity techniques. We argue that our experimental results are an effect of interaction fidelity, and perform a detailed analysis of the fidelity of the three locomotion techniques to support this argument.

Three-dimensional point clouds are generated by devices such as laser scanners and depth cameras, but their output is a set of unstructured, unlabeled points. Many scenarios require users to identify parts of the point cloud through manual annotation. Inspired by the current generation of " natural user interface " technologies, we present Slice-n-Swipe, a technique for 3D point cloud annotation based on free-hand gesture input. The technique is based on a chef's knife metaphor, and uses progressive refinement to allow the user to specify the points of interest. We demonstrate the Slice-n-Swipe concept with a prototype using the Leap Motion Controller for free-hand gesture input and a 3D mouse for virtual camera control.

This paper investigates some important visual factors of immersive 3D user interfaces on the latest 3D displays and studies how they can be applied for 3D TVs at home. The vergence-accommodation conflict in im mersive 3D user interfaces is addressed in order to theoretically estimate some perceived parameters of the user interfaces under this constraint. To demonstrate the proposed immersive 3D user interfaces, a 3D TV browse and control demo is built and some stereo screenshot results are presented.

A method of Selecting and displaying an item on a display
Screen uses a pointing and Selecting device including a
control button. Displayed, on the Screen, are (i) a multifac
eted three dimensional object having a fixed center and
having at least one side including at least a representative
window of a first item to select and display, and (ii) a fixed
pointer. The representative window includes a link to the
item to be Selected and displayed. The three dimensional
object is rotated on the screen about the fixed center with
respect to three free orthogonal axes, by the manipulation of
the pointing device. The representative window of the item
to be Selected and displayed is Superimposed with the
displayed pointer at the center of the three dimensional
object, for the Selection and display on the display Screen of
the item to be Selected and displayed by pushing the control
button of the pointing device.

Virtual reality (VR) locomotion techniques that approximate real-world walking often have lower performance than fully natural real walking due to moderate interaction fidelity. Other techniques with moderate fidelity, however, are intentionally designed to enhance users' abilities beyond what is possible in the real world. We compared such hyper-natural techniques to their natural counterparts on a wide range of locomotion tasks for a variety of measures. The evaluation also considered two independent components of interaction fidelity: bio-mechanics and transfer function. The results show that hyper-natural transfer functions can improve locomotion speed and some aspects of user satisfaction , although this can come at the expense of accuracy for complicated path-following tasks. On the other hand, hyper-natural techniques designed to provide biomechanical assistance had lower performance and user ac ceptance than those based on natural walking movements. These results contribute to a deeper understanding of the effects of interaction fidelity and designer intent for VR interaction techniques.

In this paper we focus on the creation of a tool that alows us to put stereoscopic depth into a web page. We also explore with an experiment to what extent this stereoscopic depth may control a user's gaze pattern while performing a search task. The tool uses HTML5 to create 3D stimuli compatible with stereo displays. The stimuli created consists of a search engine result page created in a 2D and 3D side by side format. We then test the stimuli with eye tracking equipment and perform a questionnaire that gathers information about participants' opinion towards a stereoscopic 3D experience. While this experiment is a first pilot and more research is needed, it is indicative of user behavior. In this paper we describe preparations for the study, hardware and software tools used to carry out the experiment, and results analysis.

Procédé de navigation en images de synthèse 3D par la manipulation d'image 3D. L'invention concerne un dispositif technique de sélection d'un fichier par la manipulation d'un objet 3D virtuel (2). La manipulation en 3D permet d'accéder rapidement à de nombreuses données aux arborescences complexes et exponentielles, par une interface intuitive, comme si nous tournions un cube dans notre main. Ce procédé de navigation 3D est caractérisé en ce qu'il comporte un volume 3D (2), volume (2) dont le centre est fixe et confondu avec le centre de la surface d'affichage du dispositif et symbolisé par un repère: 'le pointeur' (3), volume (2) sur les faces duquel sont affichées les représentations des éléments sélectionnables 'les objets' (4), volume (2) manipulé en temps réel dans les trois degrés de liberté en rotation grâce à un dispositif de pointage (6) (trackball, ...) afin de positionner l'élément choisi sur le pointeur (3). La sélection est déclenchée par un click sur un bouton (5) du dispositif de pointage (6). Les objets (4) sont représentés par un élément graphique issu d'un fichier numérique. A chaque objet (4) est associé un lien. Le point de vue de l'observateur est fixe. Le dispositif selon l'invention est particulièrement destiné à faciliter l'accès rapide à un fichier numérique dans une base de données numérique importante.

Advanced 3D virtual environment technology, similar to that used by the film and computer games industry can allow educational developers to rapidly create realistic 3D, virtual environments. This technology has been used to generate a range of interactive learning environments across a broad spectrum of industries and educational application areas.
Virtual Reality (VR) simulators represent a powerful tool for learning and teaching. The idea is not new. Flight simulators have been used for decades to train pilots for both commercial and military aviation. These systems have advanced to a point that they are integral to both the design and the operation of modern aircraft [1, 2]. There are a number of lessons that can be learned from other industries that have successfully utilised virtual training and learning systems for a number of years. Generic rules of thumb regarding the specification, development, application and operation of these learning environments can be garnered from other industrial training systems and examined in an educational context [3, 4, 5, 6, 7].
This paper will introduce a virtual learning environment which has been developed by the authors. During the implementation of this, and other, visual learning environments a number of complex operational problems have been encountered, these have required a number of innovative solutions and management procedures to be developed.
The paper will also discuss the implementation of these systems and extrapolate the lessons learnt into general guidelines to be considered for the development of VR based educational learning resources. These guidelines will then be discussed in the context of the development of ViRILE (Virtual Reality Interactive Learning Environment). This software is designed for use by undergraduate chemical engineers and simulates the configuration and operation of a polymerisation plant [4].

Ambient Interface research has the goal of embedding technology that disappears into the user’s surroundings. In many ways Augmented Reality (AR) technology is complimentary to this in that AR interfaces seamlessly enhances the real environment with virtual information overlay. The two merge together in context aware Ambient AR applications, which allow users to easily perceive and interact with Ambient Interfaces by using AR overlay of the real world. In this paper we describe how Tangible Interaction techniques can be used for Ambient AR applications. We will present a conceptual framework for Ambient Tangible AR Interface, a new generation of software and hardware tools for development and methods for evaluating Ambient Tangible AR Interfaces.

This paper presents a novel interaction technique to combine multiple inputs and output modalities with a 3D printed open disc which is tracked and augmented with a virtual information. Props have been used in virtual and augmented environments to provide tangible associations to virtual elements. When used in a larger immersive environment or where a user is walking around, users may not have the capability to set these objects/props down or are limited to the number of hands, so users are reduced to the capability of using only two props or controllers. In our technique, users can instead use their body to store and organize the virtual objects associated with tangible props by physically hanging them on or attaching them to the body. This type of interaction is well-suited for both immersive visualizations and immersive virtual environments. In this paper, we present the results of an experimental evaluation of our Body Prop Interaction technique for each type of immersive environment and corresponding task type, demonstrating more effective usage in each scenario than a joystick or gesture-based interaction.

This paper describes a novel hardware/software platform dedicated to design review process simplification, using immersive reality technologies.
The proposed platform allows designers to interface with CAD engines and visualize different data types simultaneously into an immersive and stereoscopic multi-view visualization system. This research focuses on the development of novel immersive user interfaces, including a virtual keyboard and a smart 3D disk with a set of widgets.
During the immersive sessions, the user can activate functionalities using
cost effective pointing devices and the conceived 3dUIs projected into the virtual environment. In this way, it is possible to easily manipulate virtual objects, perform basic operations such as rotations and translations but also more complex CAD functionalities such as surfaces shape modification. Each feature can be selected inside the virtual world using the smart 3D disk. Users evaluations show that the use of a virtual environment may enhance the perception of designers ideas during the design process and the use of smart 3D interfaces simplifies the interaction among user and virtual objects.

With this paper we document an experiment that is defined to test to what extent stereoscopic depth may control a user’s gaze pattern while performing a search task. The experiment consisted in displaying a search engine result page created in a 2D and 3D side by side format, and a questionnaire that gathers information about participants’ opinion towards a stereoscopic 3D experience. While this experiment is a first pilot and more research is needed, it is indicative of user behavior. In this paper we describe preparations for the study, hardware and software tools used to carry out the experiment, and results analysis.

In this paper we present 3DSjQ, a tool used to implement stereoscopic 3D in web pages. It provides HTML developers the possibility to create static and dynamic content that interacts with depth. We uncover the algorithm used for the tool, describe the method of operation and discuss future work including further development and implementations.