Presence Production in a Distributed Shared Virtual Environment for Exploring Mathematics Claus J. S. Knudsen Division of Media Technology and Graphic Arts, Dept. of Numerical Analysis,and Computing Science (NADA), Royal Institute of Technology (KTH), Drottning Kristinas v.47 D, SE-100 44 Stockholm, Sweden (Tel: +46-8-790 6376; Fax: +46-8-791 8793; E-mail: clausk@gt.kth.se) Ambjørn Naeve Centre for user-oriented IT Design (CID), Dept. of Numerical Analysis,and Computing Science (NADA), Royal Institute of Technology (KTH), 10044 Stockholm, Sweden (Tel: +46-8-790 6896; Fax: +46-8-10 24 77; E-mail:amb@nada.kth.se) Abstract: It is well known that the current state of mathematics education is problematic in many countries. The Interactive Learning Environments group at CID (Centre for user-oriented IT Design) at the Royal Institute of Technology (KTH) has developed an avatar-based shared virtual environment called CyberMath, aimed at improving this situation by allowing interaction with mathematical content in new and exciting ways. CyberMath is suitable for exploring and teaching mathematics in situations where both the teacher and the students are co-present and physically separated. In this virtual reality environment the participants are represented by avatars. The space concept in virtual environments is different from that of any known physical space. Yet people seem to perceive, for example, chat rooms and bulletin board systems as places. Still, avatars have limited possibilities for non-verbal expressions, such as body language, which are important in order to improve the communication quality. To investigate the importance of human-to-human expression and eye-contact between actors in the CyberMath environment, a test lecture in mathematics was carried out between students at Uppsala University and a lecturer at the Royal Institute of Technology (KTH) in Stockholm. The Media Environment group at the KTH Learning Lab was responsible for the production of a sense of presence involving the lecturer and the students, using distance technology such as networked twoway television systems and interactive storytelling. Empirical material was collected from recordings of 5 video sources and through a questionnaire given to the participants in the test. The main goal of the study was to investigate whether students at a distance could adapt to a combination of different shared virtual environments. It was found that presence production mediated as two-way television is a good way to build trust and to enhance non-verbal communication between the actors. The students treated the avatars on the computer screen and the lecturer on the display in front of them as real persons. In the same way, they treated the virtual reality space and the space distributed through two-way television as real spaces. Key words: Virtual environment, interactive learning, virtual reality, mathematics, non-verbal expression 1 2 1. INTRODUCTION 1.1 The virtual lecture At the Centre for user-oriented IT Design (CID) [17] an avatar-based 3D learning environment called CyberMath [7], [8], [9] has been developed for the interactive exploration of mathematics. This paper focuses on an assessment study of a learning experience in CyberMath carried out between Uppsala University and the Royal Institute of Technology, both universities partners in the Swedish Learning Lab [20] financed by the Wallenberg foundation [21]. The study connected two tracks of the organized research activity within SweLL called, CVEL (3D communication and visualization environments for learning) [6] and DILS (Distributed Interactive Learning Spaces). On 5th of October 2000, a preliminary assessment study was performed within the course “Interactive Graphical Systems”, organized by Stefan Seipel at DIS [30] in Uppsala. As a preparation for this study, the CyberMath system was installed at the DIS computer lab in Uppsala, and at CID/KTH in Stockholm. It was used by 12 students from the above mentioned course. These students were divided into 2 groups, and each of these groups were split into 3 subgroups of 2 persons each. The average age of the participants was 23.5 years. They all claimed that they work with computers on a daily basis. Ambjörn Naeve from CID/KTH held a virtual lecture on “generalized cylinders” [2]. He was physically located in Stockholm, while in Uppsala three subgroups at a time were using the 3D computer graphics display to attend the lecture in the Cybermath environment. Voice communication was provided through Cybermath as well as visual communication amongst the participants using their respective avatars. In addition to the pure avatar-based communication, an “augmented reality interface” was provided by Claus Knudsen, in order to produce a sense of presence and reality [12] between the teachers and the students [13], [14]. The lecture, which took approximately 30 minutes, was performed twice (once for each group). The entire experimental lecture was video-recorded for documentation. Prior to this virtual lecture, the same topic was taught in the form of a conventional lecture by Ambjørn Naeve given in Uppsala to the rest of the students of the course, (20 students). 1.2 The main goal of the study The main goal of the study described in this paper was to investigate whether students at a distance could adapt to a combination of two shared virtual environments. The virtual test lecture was focusing on the following research questions. • • How can a feeling of presence be achieved by technical and narrative means in a distributed learning experience? Will an increased feeling of presence and reality increase motivation and improve the learning results? The second goal was to deliver a concept for the effective usage CyberMath system in the context of on the topic of generalized cylinders evaluation is described in [6]. 2. proof-ofof the lecturing [2]. This METHODS After the test lecture, the 12 participants were given a questionnaire to fill in and return. This questionnaire served as a tool for the educational evaluation and assessment. 3 2.1 Methods for telepresence analysis To measure the experienced degree of presence is not easy. Many factors like the installation characteristics, individual preconditions, sensory environment and content characteristics influence the degree of mental attention [12]. The human factor is essential. According to Sheridan [10], presence is a subjective sensation or mental manifestation that is not easily amenable to objective physiological definitions and measurements. He indicates that “subjective report is the essential basic measurement”. Witmer and Singer have carried out research on the measuring of presence in virtual environments. They have presented a Presence Questionnaire in the MIT Presence Journal [11]. In addition to this, they have developed an immersive tendencies questionnaire in order to measure differences in the tendencies of individuals to experience presence. The term “immersive” often refers to certain types of sensory reproduction systems used in VR and telepresence, where the users actually become part of the experience - which causes exclusion of their immediate reality - as opposed to being a mere observe. For example, a person using a head mounted display or CAVE [27] system would be immersed in the experience, whereas a person viewing a remote location on a simple computer monitor would not. In this paper a subjective report method has been used for telepresence [29] analysis of the empirical video-recorded material. the physical distance between the teacher and the students, see Figure 1. The three spaces involved had both physical and virtual cameras operating from subjective and objective points of view depending on positions and functions. The students operating in CyberMath could choose between an objective point of view, looking over the shoulder of their avatars or a subjective point of view, looking through the eyes of their own avatar. The cameras for the telepresence production were positioned to achieve a subjective point of view for the participants. Two cameras, with an objective point of view, were used for documentation, one in Uppsala and one in Stockholm. In addition to this, the other video sources were also recorded. The recorded material was used for analyses and edited for a documentary. Figure 1. Connectivity diagram for the virtual lecture and documentation 3.1 3. THE TEST INSTALLATION COMPONENTS The virtual components of the test lecture consisted of the CyberMath system and an eyeto-eye telepresence production system bridging The CyberMath environment The CyberMath system is built on top of DIVE [28], which has the ability to display interactive three-dimensional graphics as well as to distribute live audio between standard desktop PCs. It also supports a number of other hardware configurations, ranging from headmounted displays to CAVE environments. It is possible to allow different users to access the same virtual environment from workstations 4 with different hardware configurations. In the CyberMath environment avatars can gather and share their experience of mathematical objects. When a person points to an object, a red beam running from his avatar to the object appears on the screens of all the other participators and when the person manipulates the object in some way, the result is directly distributed and therefore visible to all participators. Since the sound is distributed as well, a person can point, act and talk - much as she would do in real reality – as if the mathematical objects were hanging there in front of her. Hence a mathematics teacher is provided with a tool that integrates the best of both the virtual and the real world. Virtual (mathematical) objects can be shared, manipulated and discussed in a realistic way. The spatial architecture of CyberMath [19] consists of a number of different exhibition halls, each of which contains a collection of mathematical installations expressing a common theme. Each wall is equipped with a projection system, where traditional OH material can be presented. At the same time the avatars are free to move around and study the mathematical objects in practice. So far, four exhibition halls have been completed. Three of these are filled with content concerning the differential geometry of curves and surfaces [1], [2]. The fourth exhibition hall is devoted to the dynamic exploration of mathematical transformations. Here an arbitrary transformation (from R3 to R3) can be specified and the effects of this transformation can be studied interactively by manipulating different objects in the domain and observing what happens in the image. The animations are controlled through an interface which enables starting and stopping as well as changing between displaying the animations in rendered or wire frame mode. Figure 2. CyberMath, the generalized cylinders exhibit, where the distributed lecture was held. 3.2 Augmented reality interface The two physical spaces in Uppsala and Stockholm were connected by a telepresence system using two-way video on a simple H. 261 protocol with a 128 kbps dialled up ISDN line. The CyberMath system provided communication through the Internet. Figure 3. Students and the virtual teacher in Uppsala In Uppsala, 3 workstations were placed side by side with 2 students at each station. The incoming video was back-projected on a videowall mounted 2 metres in front of the students. The video projection was adjusted in size and position to optimise the sense of presence of the teacher sitting in front of the students meeting them face-to-face. The students should 5 experience the teacher as sitting in front of his own workstation, at the same level as the students, meeting their eyes when looking up. Because of this, the teacher in Stockholm was produced with a black background in order to achieve a video signal level just beneath the video black level, so that only the body of the teacher would appear on the screen in Uppsala. An advantage caused by isolating the body of the teacher was the reduction of distractions on the incoming video in Uppsala. Another advantage was that the H.261 MPEG compression could work with just the changes in the picture from the movement of the teacher. A remote-controlled camera was placed as low as possible in front of the back-projected screen in Uppsala, representing the eyes of the teacher. The camera was pre-programmed to 3 positions, each one framing a two-shot student group sitting in front of their workstation. It was important to position the camera as close as possible to the position of the projected eyes of the teacher in order to achieve an eye-to-eye contact. Moderate lighting was used in order to achieve better quality of the outgoing video signal from Uppsala. monitor screen in order to achieve eye-to-eye contact. 4. RESULTS 4.1 Vital requirements Our results indicate that in order to establish verbal communication, face-to-face contact is a vital requirement in virtual environments when individual personal appearance is restricted to avatar representations. To be talked to by a third person who is not included in ones visual frame seems to be particularly annoying when many participants share the same virtual space. We have found that manual self-locomotion in virtual environments for the purpose of initiating a verbal contact appears to be difficult. One reason for this might be that it takes quite some time to navigate towards an intended dialogue partner, and while doing so, the target avatar is quite liable to shift its location. 4.2 . Figure 4. The studio of the teacher in Stockholm In Stockholm the teacher was sitting 3 metres in front of a 28` TV monitor with the computer just in front of him. The camera was positioned in the lower part of the video Subjective telepresenceobservations in Uppsala The position of the projector for the telepresence production in Uppsala was derived from the calibrated position, which resulted in a reduced sense of eye-to-eye contact between the participants. The Uppsala projection was approximately 20-25 cm off, up to the right, compared with the calibrated position. Optimal eye contact is essential for the feeling of presence between the participants, and the calibrated position must be fixed in future testinstallations. The light sources are also important for the quality of the video and should have fixed colour temperature and be adjustable both in direction and level. 6 4.3 Subjective observations in KTH, Stockholm The image format of the students appearing on the screen in Stockholm was too small and should be enlarged in future test-installations in order to achieve a better sense of presence and reality. The remote-control of the far-end camera should be made much easier for the teacher to handle during the lecture, so that adjustments can be made in a more intuitive way. The lighting of the teacher in Stockholm should be improved in order to achieve better picture quality. Moreover, the telepresence system should have provided a connection on the Internet using the H. 323 protocol in order to achieve a more flexible use of the data networks. 4.4 The planned lecture The teacher introduced himself in a dialogue using the telepresence system in order to build trust between the participants of the learning experience. While acting in the CyberMath system, telepresence was used for confirmation of understanding by eye-to-eye contact between the teacher and individual students. Often the students replied by just nodding their heads for a yes or shaking them for a no. The telepresence experience of the first lecture influenced the teacher in strengthening the initial trust-building process of the second lecture. Whereas the students of the first lecture were only identified by their group (1, 2, 3), the students of the second lecture were invited to introduce themselves by their individual names. These names were then recorded by the teacher on a map that related them to their virtual space positions in Uppsala. This enabled the teacher to personalize the lecture by establishing communication with the students on a first name basis, which resulted in an easier dialogue and more active participation (= questions) from the students. 4.5 The Informal Lecture After the planned lecture something interesting happened which we had not prepared for in our experiment. The students discovered how to take the heads off their avatars, which produced roaring laughter and established a relaxed and informal atmosphere. The participants then started to mingle and this turned the formal lecture mode into a social interaction mode. When the students were relieved of their learning roles, they became much more talkative and dared to express themselves verbally in a way that they did not do before. During the formal lecture, the students were repeatedly asked if they had any questions, but they did not come up with any questions at all. But during the spontaneous conversation, several questions came up, including one that was not related to the original lecture, but concerned an installation in another part of the exhibition hall. This triggered an improvised lecture on the topic of solar energy. This event clearly showed the importance of informal social interaction as a means of establishing trust and supporting quality of communication in virtual learning environments. 5. CONCLUSIONS AND FUTURE WORK In our study we found that presence production on the two-way television provides an effective way to build trust and to enhance non-verbal communication between the participators in a distributed learning experience. The students treated the avatars on the computer screen and the lecturer on the display in front of them as real persons [16]. In the same way, they treated the virtual reality space and the space distributed through twoway television as real spaces. 7 From the perspective of the teacher, the prepositioning of the camera and the nameposition map provided the technological support for communicating with the students. Although they were useful, they can both be substantially improved. The pre-positioning system was locked into three fixed views, which limited the intuitive dialogue process. The discontinuity inherent in jumping between these three positions resulted in the teacher losing part of his spatial orientation with regard to the physical learning space in Uppsala. Moreover, the organization of this space made it impossible to capture the overall presence of all of the students within a single frame, which added to this spatial confusion. The name-position map made it necessary for the teacher to look down every time he needed to remind himself of a student's name. This resulted in a certain loss of contact, which reduced the transparency of the underlying presence technology and made it harder to maintain the intensity of the non-verbal communication. In order to improve the presence production in virtual environments, the following improvements should be made: • • • • • • Fixed colour temperature on all light sources. Better quality routines for the calibration of camera and projectors. Development of eye-to-eye applications. Development of an intuitive transparent human computer interface for remote control of the far end cameras Development of a HUD (head up display) system that supports personalization by superimposing the names on the corresponding images of the students Designing a spatial configuration that supports these types of learning experiences that integrate both the physical and the non- physical environment. 6. ACKNOWLEDGEMENTS Several people have contributed to the experiment reported in this paper. Gustav Taxén has created the CyberMath environment. The mathematical installations (objects and animations) have been created by Ambjörn Naeve in Mathematica® and translated into DIVE by Gustav Taxén. Olle Sundblad has handled the complexity of DIVE, especially the networking aspects. Bosse Westerlund drew the connectivity diagram (Figure 1), and Sinna Lindqvist handled the video camera in Uppsala. The Advanced Media Technology Laboratory [25] hosted the experiment at KTH in Stockholm and DIS [30] did the same in Uppsala. Many thanks to you all! 7. REFERENCES 7.1 Papers [1] Naeve, A., Focal Shape Geometry of Surfaces in Euclidean Space, CVAP-130, TRITA-NA-P9319, Dissertation, Department of Numerical Analysis and Computing Science, KTH, Stockholm, 1993. [2] Naeve., A & Eklundh, J.O., Representing Generalized Cylinders, Proceedings of the EuropeChina Workshop on Geometrical Modeling and Invariants for Computer Vision, pp. 63-70, Xi'an April 27-29, 1995. Published by Xidian University Press, Xi'an, China, 1995. [3] Naeve, A., The Garden of Knowledge as a Knowledge Manifold - A Conceptual Framework for Computer Supported Subjective Education, CID-17, TRITA-NA-D9708, Department of Numerical Analysis and Computing Science, KTH, Stockholm, 1997, http://cid.nada.kth.se/sv/pdf/cid_17.pdf. [4] Naeve, A., Conceptual Navigation and Multiple Scale Narration in a Knowledge Manifold, CID-52, TRITA-NA-D9910, Department of Numerical Analysis and Computing Science, KTH, 1999. http://cid.nada.kth.se/sv/pdf/cid_52.pdf. 8 [5] Naeve, A., The Work of Ambjörn Naeve within the Field of Mathematics Educational Reform, CID-110, TRITA-NA-D0104, KTH, Stockholm, 2001, www.amt.kth.se/projekt/matemagi/mathemathics_ed ucational_reform.doc. [6] Naeve, A. & Seipel, S., APE track C, Progress Report, July-Dec. 2000, available from SweLL[20]. [7] Taxén G. & Naeve, A., CyberMath - A Shared 3D Virtual Environment for Exploring Mathematics, presented within Course-31, Geometric Algebra new foundations, new insights, Siggraph2000, New Orleans July 2000. [8] Taxén G. & Naeve, A., CyberMath - A Shared 3D Virtual Environment for Exploring Mathematics, CID/KTH, 2000, the 20:th ICDE world conference on distance education and e-learning, Düsseldorf, April 1-5, 2001, on Compact Disc. [9] Taxén G. & Naeve, A., CyberMath - Exploring Open Issues in VR-based Learning, SIGGRAPH 2001 Educators Program, In SIGGRAPH 2001 Conference Abstracts and Applications, pp. 49-51. [10] Sheridan, T. B. (1992). Musings on Telepresence and Virtual presence. Presence: Teleoperators and Virtual Environments, 1(1), p120-125. [11] MIT Presence Journal Volume 7, Number 3 · June 1998, Measuring presence in virtual environments, A Presence Questionnaire by Bob G. Witmer and Michael J. Singer. [12] Enlund, N., The Production of Presence - Distance techniques in Education, Publishing and Art, ACS'2000 Proceedings, Szczecin, 2000, pp. 44-49. [13] Handberg, L., Knudsen C., Sponberg H., New Learning modes in the production of presence – distance techniques for education, Proceedings of the 20th World Conference on Open Learning and Distance Education, ICDE01, Düsseldorf, 2001, on Compact Disc. [14] Knudsen, C., Interaction between musicians and audience in a learning process on the Internet, ISTEP 2000 Proceedings, Kosice, 2000, pp. 159-164. [15] Wann, J., & Mon-Williams, M., What does virtual reality NEED? Human factors issues in the design of three-dimensional computer environments, International Journal of Human-Computer Studies, 44, 1996, pp. 829–847. [16] Reeves, B., Nass, C., The media equation, Cambridge University Press, New York, 1996. 7.2 Relevant web sites [17] CID, Center for user oriented IT design URL:http://www.nada.kth.se/cid/ [18] CID/Interactive Learning Environments: http://cid.nada.kth.se/il. [19] CyberMath: www.nada.kth.se/~gustavt/cybermath. [20] SweLL (Swedish Learning Lab): www.swedishlearninglab.org. [21] WGLN (Wallenberg Global Learning Network): www.wgln.org. [22] PADLR (Personalized Access to Distributed Learning Repositories) proposal to WGLN, Granted March 2001: www.learninglab.de/pdf/L3S_padlr_17.pdf. [23] ICDE-2001: www.fernuni-hagen.de/ICDE/D-2001. [24] The synchronous virtual space installation, URL:http://www.r1.kth.se/epresence/ [25] The Advanced Media Technology Lab., URL:http://www.amt.kth.se [26] Royal Institute of Technology, URL: http://www.kth.se [27] KTH, the PDC CUBE, URL:http://www.pdc.kth.se/projects/vr-cube/ [28] DIVE, Swedish Institute of Computer Science, www.sics.se/dive [29] Transparent Telepresence Research Group (TTRG), URL:http://telepresence.dmem.strath.ac.uk/index.htm [30] DIS (Department of Information Science), Uppsala University, www.dis.uu.se