Collaborative Virtual Environment:

An Application for Civil Engineering

By Mauricio Capra

Thesis submitted to the University of Nottingham for the degree of

Doctor of Philosophy, April 2007

Index
INDEX......................................................................................................................................................II

LIST OF FIGURES................................................................................................................................V

LIST OF TABLES..............................................................................................................................VII

LIST OF ACRONYMS........................................................................................................................IX

ABSTRACT........................................................................................................................................XIII

ACKNOWLEDGEMENTS................................................................................................................XV

CHAPTER 1..........................................................................................................................................18

1.1 COMMUNICATION, COLLABORATION, AND MUTUAL UNDERSTANDING..............................19

1.2 POTENTIAL GAPS DURING THE CEN PHASES – COMMUNICATION, COLLABORATION AND
MUTUAL UNDERSTANDING.................................................................................................................21
1.3 CIVIL ENGINEERING LIFE CYCLE.........................................................................................23
1.4 THESIS OBJECTIVES..............................................................................................................26
 1.5 PROPOSED SOLUTION...........................................................................................................27
1.6 STEPS FOLLOWED TO ACHIEVE THE OBJECTIVES................................................................28
1.7 THESIS STRUCTURE..............................................................................................................29

CHAPTER 2..........................................................................................................................................36

2.1 SOLVING CEN CHALLENGES – DESIGN AND CONSTRUCTION PHASES................................38

2.1.1 Building Information Model...........................................................................................38
2.1.2 Researches addressing Communication, Collaboration and Mutual Understanding in
CEn Projects.................................................................................................................................42
2.2 IMPLEMENTING NEW TECHNOLOGIES IN A CEN ENVIRONMENT.........................................49
2.3 IMPROVING CEN PROJECTS – COMMUNICATION, COLLABORATION AND MUTUAL
UNDERSTANDING................................................................................................................................49

CHAPTER 3..........................................................................................................................................52

3.1 VIRTUAL REALITY................................................................................................................53

3.2 AUGMENTED REALITY..........................................................................................................55
3.3 AUGMENTED VIRTUALITY....................................................................................................56
3.4 MIXED REALITY...................................................................................................................57
3.5 COLLABORATIVE VIRTUAL ENVIRONMENTS........................................................................58
3.6 RELATED PROJECTS..............................................................................................................63
3.7 GLOBAL POSITIONING SYSTEM............................................................................................94
3.8 GPS FREQUENCIES AND DATA – BRIEF REVIEW.................................................................95
3.9 STAND-ALONE GPS POSITIONING........................................................................................97
3.10 DIFFERENTIAL GPS – DGPS..............................................................................................100
3.11 REAL TIME KINEMATICS GPS - RTK-GPS........................................................................101
3.12 AUGMENTATION SYSTEMS AND NETWORK SERVICES.......................................................105
3.13 TESTING SINGLE FREQUENCY GPS, DGPS AND RTK-GPS..............................................109
3.14 OTHER POSITIONING SYSTEMS...........................................................................................112
3.15 COMBINING AND INTEGRATING TECHNOLOGIES................................................................115

CHAPTER 4........................................................................................................................................117

4.1 CONCEPTUALISATIONS OF USER GROUPS..........................................................................118

4.2 DESIGN OVERVIEW.............................................................................................................120
4.2.1 Office User Design........................................................................................................122
4.2.2 Field User Design.........................................................................................................123
4.3 HARDWARE AND SOFTWARE..............................................................................................125
4.3.1 Hardware......................................................................................................................125
4.3.2 Software........................................................................................................................136
4.4 FINAL ANALYSIS OVER THE DESIGN..................................................................................140

CHAPTER 5........................................................................................................................................142

5.1 FIRST PHASE – OFFICE USER AND CVE.............................................................................144

5.1.1 Survey and Data Conversion........................................................................................144
5.2 FIRST PHASE OF PRACTICAL WORK...................................................................................149
5.3 SECOND PHASE OF THE PRACTICAL WORK........................................................................154
5.3.1 Integrating Software, Hardware, Field and Office Users............................................158
5.3.2 Notes of How to Use the System...................................................................................167

iii
CHAPTER 6........................................................................................................................................168

6.1 THREE EXERCISES FOR THE EVALUATION..........................................................................170

6.2 EVALUATION......................................................................................................................180
6.2.1 Pre-Test before the Final Evaluation – Final Adjustments..........................................182
6.2.2 The Final Evaluation....................................................................................................184
6.2.3 Outcome from the Exercises.........................................................................................187
6.3 FINAL POINTS OVER THE EVALUATION..............................................................................202

CHAPTER 7........................................................................................................................................212

7.1 SUMMARY AND REFLECTIONS............................................................................................213

7.2 THESIS CONTRIBUTION.......................................................................................................218
7.3 DISSEMINATION..................................................................................................................220
7.4 FUTURE WORK...................................................................................................................221
7.4.1 Future on Hardware.....................................................................................................221
7.4.2 Future on Software.......................................................................................................222
7.4.3 Future Applications......................................................................................................222
7.4.4 Further Evaluation........................................................................................................224
7.4.5 Concluding Remarks.....................................................................................................224

APPENDIX A......................................................................................................................................226

A.1 FEATURES FILE...................................................................................................................227

A.2 SOURCE CODE REFERENCE.................................................................................................229

APPENDIX B.......................................................................................................................................235

B.1 RTK-GPS CLASS................................................................................................................236

B.1.1 GPSReveiver.java.........................................................................................................236
B.1.2 GPSPublisher.java........................................................................................................237
B.2 TILT-SENSOR AND MAGNETIC COMPASS CLASS................................................................238
B.2.1 TCM2.java....................................................................................................................238
B.2.2 TCM2Publisher.java.....................................................................................................239
B.3 READING FROM THE DATABASE.........................................................................................240
B.3.1 Integrator.java..............................................................................................................240
B.4 KALMAN FILTER.................................................................................................................242
B.4.1 Matriz.java....................................................................................................................242
B.5 OFFICE USER KEY STROKE COMMAND..............................................................................244
B.5.1 ReadKeyStroke.java......................................................................................................244
B.5.2 KeyStroke.java..............................................................................................................245

APPENDIX C......................................................................................................................................246

C.1 MERGING REAL AND DIGITAL ENVIRONMENT...................................................................247

C.2 USABILITY..........................................................................................................................249
C.3 HARDWARE.........................................................................................................................249
C.4 DIGITAL ENVIRONMENT.....................................................................................................250
C.5 OFFICE USER AND FIELD USER..........................................................................................252
C.6 DIFFERENT/COMPLEMENTARY APPROACHES.....................................................................254

Bibliography..........................................................................................................................................258

iv
v
List of Figures
FIGURE 1.1 CEN LIFE-CYCLE AND INTERRELATIONSHIPS (NEIL S. GRIGG ET AL., 2001).....................24
FIGURE 1.2 GENERAL VIEW OF THE PROJECT.........................................................................................27
FIGURE 1.3 THESIS STRUCTURE..............................................................................................................30
FIGURE 2.1 REDUCTION OF COSTS AND TIME IN THE PROJECT EVALUATED ON (KHANZODE ET AL.,
2008)..............................................................................................................................................42
FIGURE 3.1 THE “VIRTUALITY CONTINUUM” (MILGRAM AND KISHINO, 1994).....................................54
FIGURE 3.2 POTENTIAL OIL FIELD – PETROBRAS....................................................................................66
FIGURE 3.3 VRML MODEL OF THE SRI CAMPUS (SRI INTERNATIONAL, 2000A)..................................68
FIGURE 3.4 VRML WIND VECTOR OVERLAY (SRI INTERNATIONAL, 2000B).......................................68
FIGURE 3.5 DOWNTOWN OF LA MODELLED BY UST TEAM (UST, 2003).............................................70
FIGURE 3.6 LAX ORIGINAL CONDITION (UST, 1998)............................................................................70
FIGURE 3.7 LAX PROPOSED CONDITION (UST, 1998)...........................................................................70
FIGURE 3.8 SIMCITY – VIEW FROM OLYMPIA (CITYSCAPE, 2004).........................................................72
FIGURE 3.9 VISUALISING DATA FROM A GIS (CAPRA, 2000).................................................................73
FIGURE 3.10 TEXTURED 3-D VIEW OF MUNICH (ESRI, 2004B).............................................................75
FIGURE 3.11 ARCGIS SCREENSHOT (ESRI, 2004A)................................................................................76
FIGURE 3.12 SHIODOME CYBERDOME STRUCTURE (MATSUSHITA ELECTRONIC WORKS LTD, 2003)...78
FIGURE 3.13 SHIODOME CYBERDOME USER’S INTERFACE (MATSUSHITA ELECTRONIC WORKS LTD.,
2003)..............................................................................................................................................78
FIGURE 3.14 ADDITIONAL INFORMATION OVERLAYING A HISTORIC BUILDING.....................................82
FIGURE 3.15 ADDITIONAL INFORMATION EXPLORED BY THE USERS (REITMAYR AND SCHMALSTIEG,
2004C)............................................................................................................................................83
FIGURE 3.16 TINMITH WEARABLE SYSTEM (PIEKARSKI AND THOMAS, 2002).......................................87
FIGURE 3.17 AUGURSCOPE (BENFORD ET AL., 2003B)............................................................................89
FIGURE 3.18 AR SUB-SURFACE SYSTEM (ROBERTS ET AL., 2004).........................................................91
FIGURE 3.19 VE PROJECTED OVER THE REAL ENVIRONMENT (ROBERTS ET AL., 2004)........................92
FIGURE 3.20 ARMOBILE (WOODWARD ET AL., 2007B)..........................................................................93

vi
FIGURE 3.21 ARWEBCAM (WOODWARD ET AL., 2007C).......................................................................94
FIGURE 3.22 RANGE MEASUREMENT TIMING RELATIONSHIPS (KAPLAN, 1996)....................................98
FIGURE 3.23 CARRIER PHASE AND INTEGER AMBIGUITY (AQUINO, 1992)..........................................103
FIGURE 3.24 SINGLE FREQUENCY RECEIVER........................................................................................110
FIGURE 3.25 SINGLE FREQUENCY RECEIVER – A SINGLE COORDINATE..............................................110
FIGURE 3.26 DUAL FREQUENCY RECEIVER PERFORMING DGPS.........................................................111
FIGURE 3.27 DUAL FREQUENCY RECEIVER PERFORMING RTK-GPS...................................................112
FIGURE 4.1 PROPOSED CARS................................................................................................................119
FIGURE 4.2 DESIGN OVERVIEW OF THE PROJECT – FIELD AND OFFICE USERS....................................121
FIGURE 4.3 GPS RECEIVER, ANTENNAS, FLASH CARD, AND RADIO....................................................128
FIGURE 4.4 TCM2-50 CIRCUIT BOARD.................................................................................................130
FIGURE 4.5 PLM-S700..........................................................................................................................131
FIGURE 4.6 VFOV CALCULATION.........................................................................................................132
FIGURE 4.7 TWIDDLER2.........................................................................................................................134
FIGURE 4.8 HELMET, PCGLASSTRON, LEICA BATTERY, SEE-THROUGH HMD AND CABLE................135
FIGURE 4.9 EQUIP PLATFORM (GREENHALGH, 2001F)........................................................................138
FIGURE 5.1 STEPS FOR THE SYSTEM IMPLEMENTATION........................................................................143
FIGURE 5.2 LANDSCAPE SURVEYED AROUND IESSG BUILDING..........................................................146
FIGURE 5.3 VIEW OF A COMPLETED VE ON VRML BROWSER............................................................147
FIGURE 5.4 SYSTEM REQUIREMENT BETWEEN OFFICE USERS ONLY...................................................150
FIGURE 5.5 VIEW OF THE USING MASSIVE-3.....................................................................................153
FIGURE 5.6 FRONT VIEW OF THE IESSG BUILDING..............................................................................153
FIGURE 5.7 RED EMBODIMENT REPRESENTING A SECOND OFFICE USER.............................................154
FIGURE 5.8 SYSTEM REQUIREMENT BETWEEN OFFICE AND FIELD USERS............................................154
FIGURE 5.9 MASSIVE-3 CARTESIAN SYSTEM.....................................................................................155
FIGURE 5.10 MAGNETIC COMPASS DATA..............................................................................................166
FIGURE 5.11 PITCH DATA......................................................................................................................166
FIGURE 5.12 ROLL DATA.......................................................................................................................166
FIGURE 6.1 BLUEPRINT OF PHOENIX TRAM STATION (CARILLION, 2000)............................................173
FIGURE 6.2 SIGN AND BIN.....................................................................................................................174
FIGURE 6.3 SIGN AND LAMP POST........................................................................................................174
FIGURE 6.4 LAMP POST.........................................................................................................................174
FIGURE 6.5 TRAM SHELTER...................................................................................................................174
FIGURE 6.6 PLATFORM..........................................................................................................................174
FIGURE 6.7 TRAM STATION...................................................................................................................175
FIGURE 6.8 TRAM STATION – TOP VIEW...............................................................................................175
FIGURE 6.9 TRAM STATION – CVE.......................................................................................................175
FIGURE 6.10 VIRTUAL TRAIN APPROACHING THE STATION.................................................................176
FIGURE 6.11 PLATFORM IN THE PHYSICAL WORLD (NOTTINGHAM TRAM CONSORTIUM, 2004).........176
FIGURE 6.12 INTERNAL DIVISION (LUCENA, 1986)...............................................................................177
FIGURE 6.13 RIGHT SIDE SCHEME (LUCENA, 1986).............................................................................177
FIGURE 6.14 COLOURS REPRESENTING DIFFERENT STAGES OF THE ENGINEERING PROJECT..............178
FIGURE 6.15 THE COMPLETE CONSTRUCTION (DESIGN AND CONSTRUCTION PHASES).......................178
FIGURE 6.16 THE HOUSE AS IT LOOKS IN THE PHYSICAL WORLD........................................................178
FIGURE 6.17 THE STRUCTURE OF THE HOUSE.......................................................................................179
FIGURE 6.18 WINDOWS AND STAIRS.....................................................................................................179
FIGURE 6.19 DIFFERENT COLOURS OF THE PIPES INDICATING DIFFERENT USES.................................179
FIGURE 6.20 TOPICS COVERED BY THE EVALUATION...........................................................................186
FIGURE 6.21 GROUPING POINTS OF INTEREST FOR THE COMPLEMENTARY OUTCOME.........................202
Figure B.1Kalman Filter Dataflow........................................................................................................243

vii
List of Tables
TABLE 3.1 DIFFERENT POSITIONING SYSTEMS..................................................................................115
TABLE 4.1 LLQ – LEICA LOCAL POSITION AND QUALITY....................................................................127
TABLE 4.2 PIN FUNCTION OF TCM2-50.............................................................................................128
TABLE 4.3 TCM2-50 MESSAGE............................................................................................................130
TABLE 5.1 AN EXAMPLE OF .CVE FILE..................................................................................................148
TABLE A.1 FEATURES’ FILE STRUCTURE..............................................................................................227
TABLE A.2 VRML HEADER’S MODIFIED..............................................................................................228
TABLE A.3 GLOBAL VARIABLE.............................................................................................................229
TABLE A.4 FUNCTION MAIN..................................................................................................................230
TABLE A.5 FUNCTION OPEN_ARQUIVO.................................................................................................230
TABLE A.6 FUNCTION TAKE_INFORMATION..........................................................................................230
TABLE A.7 FUNCTION VRML_GENERATOR............................................................................................230
TABLE A.8 FUNCTION INPUT_FEATURE.................................................................................................231
TABLE A.9 FUNCTION BUILD_CVE........................................................................................................231
TABLE A.10 FUNCTION TABELA_POLIGONO..........................................................................................231
TABLE A.11 FUNCTION FBGUPNPOLY...................................................................................................231
TABLE A.12 FUNCTION INPUT_ROAD....................................................................................................232
TABLE A.13 HIGH LEVEL ALGORITHM.................................................................................................232
TABLE B.1 GPSRECEIVER.JAVA ALGORITHM.......................................................................................236
TABLE B.2 GPSPUBLISHER.JAVA ALGORITHM.....................................................................................237
TABLE B.3 TCM2.JAVA ALGORITHM....................................................................................................238
TABLE B.4 TCM2PUBLISHER.JAVA ALGORITHM..................................................................................239
TABLE B.5 INTEGRATOR.JAVA ALGORITHM..........................................................................................241
TABLE B.6 READKEYSTROKE.JAVA ALGORITHM.................................................................................244
Table B.7 KeyStroke.java Algorithm....................................................................................................245

viii
ix
List of Equations
EQUATION 3.1..........................................................................................................................................98
EQUATION 3.2..........................................................................................................................................99
EQUATION 3.3..........................................................................................................................................99
EQUATION 3.4..........................................................................................................................................99
Equation 4.1..........................................................................................................................................132

x
List of Acronyms

American Standard Code for Information Interchange (ASCII)

Application Program Interface (API)
Architecture, Engineering and Construction (AEC)
Architect1 (A1)
Augmented Reality (AR)
Augmented Virtuality (AV)
Building Information Model (BIM)
centimetre (cm)
Central Processor Unit (CPUs)
Chemical Environmental and Mining Engineering (SChEME)
Civil Engineering (CEn)
Closed Circuit Television (CCTV)
Coarse/Acquisition (C/A code)
Collaborative Augmented Reality Systems (CARS)
Collaborative Environments (CE)
Collaborative Virtual Environments (CVE)
Communication Port (COM1)
Computational Fluid Dynamics (CFD)
Computer Aided Design (CAD)

xi
Computer Automatic Virtual Environment (CAVE)
Computer Scientist1 (C1)
Computer Scientist2 (C2)
Constructive Solid Geometry (CSG)
Dave’s Graphic Library (DGL)
Differential Global Positioning System (DGPS)
Digital Terrain Modelling (DTM)
Disk Operational System (DOS)
encrypted P P(Y)
Engineer1 (E1)
Engineer2 (E2)
Engineer3 (E3)
Engineer4 (E4)
Engineering and Physical Sciences Research Council (EPSRC)
European Geostationary Navigation Overlay Service (EGNOS)
European Space Agency (ESA)
Federal Aviation Administration (FAA)
Field of View (FoV)

Frequency Modulation (FM)

Full Operational Capability (FOC)
General Packet Radio Service (GPRS)
Geographical Information Systems (GIS)
Global Orbiting Navigation Satellite System (GLONASS)
Global Positioning System (GPS)
Global System for Mobile Communication (GSM)
Graphical Unit Interface (GUI)
Head Mounted Display (HMD)
Heating, Ventilating and Air Conditioning (HVAC)
hertz (Hz)
Identifications (IDs)
Inertial Navigation System (INS)
kilo hertz (kHz)
kilobytes (k)
Knowledge Base System (KBS)
Leica Local Positioning and Quality (LLQ)
Level of Detail (LoD)
Local Area Network (LAN)
Los Angeles (LA)
Los Angeles International Airport (LAX)
Mega Hertz (MHz)

xii
Metre (M)
Mixed Reality (MR)
Mixed Reality Laboratory (MRL)
Mobile Augmented Reality System (MARS)
Multi-Functional Satellite-based Augmentation System (MSAS)
Multi-Functional Transport Satellite (MTSAT)
Multiple User Dimensions/Dungeons (MUD)
National Marines Electronics Association (NMEA)
Nottingham Express Transit (NET)
Object Oriented (OO)
On The Fly (OTF)
Ordinary User1 (O1)
Ordnance Survey (OS)
Packed Internet Gopher (PING)
Personal Computer (PC)
Personal Digital Assistant (PDA)
Precise (P)
Precise Positioning System (PPS)
Pseudo Random Noise (PRN)
Quasi-Zenith Satellite System (QZSS)
Radio Data System (RDS)
Random Access Memory (RAM)
Real Environment (RE)
Real Time Kinematic Global Positioning System (RTK-GPS)
Standard Positioning System (SPS)
Stanford Research Institute (SRI)
Super Video Graphics Array (SVGA)
Swedish Network of Permanent GPS Reference Station (SWEPOS)
Technical Research Centre of Finland (VTT)
Tile Set Manager Application Program Interface (tsmApi)
Uniform Resource Locator (URL)
United States Department of Defence (USA-DoD)
United States Geological Survey (USGS)
Universal Time Coordinated (UTC)
University of California (UCLA)
Urban Simulation Team (UST)
Video Graphics Array (VGA)
Virtual Environment (VE)
Virtual Design Construction (VDC)
Virtual Reality (VR)
Virtual Reality Modelling Language (VRML)

xiii
Visual Simulations (VizSims)
WAAS Master Station (WMS)
Watts (W)
Wide Area Augmentation System (WAAS)
Wide Area DGPS (WADGPS)
Wide Area Reference Stations (WRS)
World Geodetic System 1984 (WGS84)
World Wide Web/File Transfer Protocol (WWW/FTP)

xiv
Abstract
The present thesis designs, implements and evaluates a collaborative augmented
reality system for civil engineering purposes oriented to design and construction
phases. The work starts with a survey of the potential gaps and solutions presented so
far regarding to communication, collaboration and mutual understanding. Such gaps
can occur on multidisciplinary teams of architects, engineers and constructors when
working in the same civil engineering project. In the context of this thesis, outdoor
users are equipped with a real-time kinematic global positioning system receiver, a
notebook, a head-mounted display, a tilt sensor and a compass. A virtual environment
representing parts of a civil engineering work is projected before their eyes. Outdoor
users share this collaborative virtual environment with indoor users. They can talk
and see each other through an embodiment. Indoor users can be at any place where
internet is available. The goal of the thesis is to show that a networked solution of at
least two users (between indoor and outdoor users) is a requirement when outdoor
users are performing complex tasks at the same time that they are experiencing an
immersive augmented reality application. Indoor users interact with outdoor users
when manipulating the virtual environment, working as a guide through the scene,
making clear obscure points of understanding, etc. The thesis evaluates how users
interact with the system using a formative approach. Users are introduced to the
system and motivated to “talk loudly” what they are experiencing during the tests. All

xv
users are video-recorded while performing the exercises and interviewed straight
afterwards. The evaluation reveals that users end experiencing a too immersive
system, narrowing their “attentional spotlight” to the virtual environment and not
experiencing an augmented reality system. The evaluation also makes clear that the
design of the virtual environment is eventually more important for users than the
system itself and it is completely dependent of the kind of application that it is being
used to and who the users are.

xvi
Acknowledgements

The author would like to thank the Engineering and Physical Sciences Research
Council (EPSRC) for the scholarship awarded to this project.

To Professors Steve Benford, Alan Dodson, Tom Rodden and Chris Greenhalgh, who
always believed in my potential, thank you very much for your guidance, help,
support and encouragement.

To Dr. Marcio Aquino (and his family) who from the very beginning (1999, in
Salvador, Brazil) was the person to always show me the right path, thank you very
much for your friendship.

To Dr. Boriana Koleva for the help and patience in the early stages.

To Dr. Norman Teferle and his family, Marcos Andreotti and his family, and Stefan
Egglestone, thank you for your friendship.

xvii
To my wife Laiz Chen Capra, thank you for all your love, dedication and specially
your patience.

To my parents Mauro Paseto Capra and Sonia Erdmann Capra and my brother
Marcos Capra, who showed me from the beginning the need to stand up, proud of
ourselves and confident, even in the worst scenarios.

To my parents-in-law Chen Wen Yii and Maria Sylvia Rubinger Chen for the support
and love.

To Leif Oppermann, Matt Watkins, Rachel Jacobs and Robin Shackford for the really
good moments together.

To Sensei Andreas Santschi, Sensei Masaki Negishi, Dominique Schmid, Nadia

Probst, Andre Borges, Simone Bueno, Fred Goede, Anneke Boom. You are amazing
people that help me a lot at the right time. Thank you very much!

To all my friends and family, who I did not name here but who motivated me to do
this work.

xviii
This thesis is dedicated to the memory of the best fighter
I ever saw. She only fell in her very last battle,
the one that everyone succumbs to.
My mom Sonia Erdmann Capra

xix
Chapter 1
Introduction
Civil Engineering (CEn) as many other professions relies over a complex network of
knowledge and information among the parts involved. As mention in (Ugwu O. O. et
al., 2001) in this collaborative design environment the real challenge is to find a
common representation of the problem-solving domain from the various experts. The
difference in terminologies can guide the communication between parts to a dead end.
Professionals can use the different terms to same concepts or same terms to different
concepts. The online dictionary (Merrian-Webster, 2008) translate “ontology” as “a
branch of metaphysics concerned with the nature and relations of being”. Augenbroe
(Augenbroe et al., 2002) mentions the difficulties of communication and
collaboration between beings. He exemplifies the intensiveness and paper work-flow
in design and construction business. Any change of request could take weeks to be
approved between contractors, engineers and architects. Heitz (Heintz, 2002)
mentions in his paper the “fail to capture the collaborative nature of the design
process and do not support the efficient and effective management of the
dependencies between designs tasks carried out by different actors”. The problem of
communication and collaboration is also explored by (Wilkinson, 2001). Her paper
describes the resistance of acceptance of project managers and their work by different
actors in the CEn environment.

20
 Although there are several other points that could also create some influence for
the relation between beings do not work as smooth as it is desired, see (Heintz, 2002)
for some other examples, the citations illustrates where collaboration, communication
and mutual understanding between those involved in the numerous steps of CEn cycle
do not work.

This relation (ontology) between parts or actors creates the motivation for the
present work. The present work aims to explore and evaluate a tool created for users
to enhance communication, collaboration, and mutual understanding between parts
involved in a CEn project. Such tool is based on a Augmented Reality (AR) system
concept.

1.1 Communication, Collaboration, and Mutual Understanding

The three key worlds for this thesis are communication, collaboration, and mutual
understanding. Along the chapters this thesis explores its objectives as a motivation,
based on the three key worlds, aiming to reduce some of the gaps identified on CEn
environments.

A general definition of communication can be found in Wikipedia (Public

Domain / Wikipedia). Communication is a process to transfer information from “A”
(also know as sender) to “B” (also know as receiver) through the use of a medium. It
is possible to exemplify ways to perform communication as:

 Auditory means: speaking, tone the voice, etc.

 Physical means: body language, eye contact, writing, etc.

Communication is defined as a process by which we assign and convey meaning in an

attempt to create shared understanding.”…“It is through communication that
collaboration and cooperation occur. (Office of Superintendent of Public Instruction)

The online dictionary Merrian-Webster (Merrian-Webster, 2008) translates

communication as:

1: an act or instance of transmitting

21
 2 a: information communicated b: a verbal or written message

3 a: a process by which information is exchanged between individuals through a common

system of symbols, signs, or behaviour <the function of pheromones in insect
communication>; also: exchange of information b: personal rapport <a lack of
communication between old and young persons>

4 plural a: a system (as of telephones) for communicating b: a system of routes for
moving troops, supplies, and vehicles c: personnel engaged in communicating

5 plural but sing or plural in constr a: a technique for expressing ideas effectively (as in
speech) b: the technology of the transmission of information (as by print or
telecommunication)

The world collaboration comes almost as a consequence of communication. In

Wikipedia (Public Domain / Wikipedia) collaboration means:

Collaboration is a recursive process where two or more people work together toward an
intersection of common goals — for example, an intellectual endeavour (Merrian-
Webster, 2007, Encyclopedia Brittanica Online, 2007) that is creative in nature ((Eds.)
Simpson J. A. and Weiner E. S. C., 1989) — by sharing knowledge, learning and
building consensus.

From Merrian-Webster collaboration can be understood as:

1: to work jointly with others or together especially in an intellectual endeavour

2: to cooperate with or willingly assist an enemy of one's country and especially an
occupying force

3: to cooperate with an agency or instrumentality with which one is not immediately
connected

Godfried (Godfried Augenbroe et al., 2002) cites Gray (Gray, 1989) when defining
collaboration in his paper:

…finding common ground for multiparty problems which relates to collaboration as a

process through which parties who see different aspects of a problem can constructively
explore their differences and search for solutions that go beyond their own limited vision
of what is possible.

Mutual understanding embraces communication and collaboration. Thesaurus

(Thesaurus, 2008) resumes mutual understanding as:

Noun 1. mutual understanding – sympathy of each person for the other

22
 mutual affection

sympathy – a relation of affinity or harmony between people; whatever affects one

correspondingly affects the other; "the two of them were in close sympathy"

Merrian-Webster (Merrian-Webster, 2008) has the following translation:

1: a mental grasp : comprehension

2 a: the power of comprehending; especially : the capacity to apprehend general relations

of particulars b: the power to make experience intelligible by applying concepts and
categories

3 a: friendly or harmonious relationship b: an agreement of opinion or feeling

: adjustment of differences c: a mutual agreement not formally entered into but in some
degree binding on each side

4: explanation, interpretation

5: sympathy 3a

The citations above have shown a strong link between these three key worlds. With
good communication it is possible to achieve collaboration between two or more
people. It is the element that must be present if it is aimed collaboration (Godfried
Augenbroe et al., 2002). Collaboration just happens with mutual understanding.
Parts/actors in CEn environment (as well as in any other relations between beings)
must to find a balance to enhance these three key worlds in order to achieve the goals
of the project.

1.2 Potential Gaps during the CEn Phases – Communication, Collaboration

and Mutual Understanding

The challenges exemplified by Kamara (Kamara J. M. et al., 2000a) are a clear

example of the complexity to attend the requirements of a client when ordering a new
CEn project. Kamara’s paper describes the development of methodologies to enhance
the understanding of client requirements (Kamara J. M. et al., 2000c). To achieve full
comprehension of what the client was requesting the information should facilitate the
concurrent work of the number of disciplines involved in the project; the early
consideration of all life-cycle issues; the integration of all professional; and the
traceability of the design decisions. The paper highlights:

23
 In the context of the multidisciplinary teams, this implies that the presentation of the
requirements has to be neutral enough to allow different professionals to understand them
in the same way (i.e. in the perspectives and priorities of the client).

The communication across disciplines regarding design constraints and trade-offs,

where it is also possible to translate as different group of people with different
expertises, is mentioned on Fiksel and Haynes (Kamara J. M. et al., 2000b, Fiksel and
Hayes-Roth, 1993) cited in Kamara.

As mentioned in the beginning of this chapter Ugwu (Ugwu O. O. et al., 2001)

does reference to “ontology” for the difficulties faced by members of different groups
to work together. For him, a tool that can enable to enhance collaboration would help
to reduce the problem:

One potential solution to this communication problem is the development of an ontology

of collaborative design concepts and terms along with their unambiguous definitions. The
existence of such an ontology in a problem domain would enhance the ability of the
participating agents (both human and artificial) to inter-operate without
misunderstanding, and consequently share or re-use data, information and knowledge.

The study of ontology is very wide and it is not limited to the issues in
communication and interoperability between beings. An extended review about the
subject can be found in (Uschold and Gruninger, 1996).

The difficult faced by those involved in a CEn project is also highlighted by

Mokhtar (Mokhtar, 2002). Although construction teams work over what is called the
plans of constructions errors still being made. The lack of coordination of design
information leads to an incompatibility between the plans. Mokhtar completes his
view about the issue with:

The errors lead to change orders, contractual disputes, cost overruns, time delays,
compromise to quality, frustration and client dissatisfaction. Common types of errors are:

o Inconsistency in design information. For example, the location of a specific column

is not identical when comparing the architectural and the structural drawings.
o Mismatch between connected components. For example, Heating, Ventilating and
Air Conditioning (HVAC) ducts dimensions, which are given in the mechanical
drawings, do not match the dimensions of the related pass-holes in the structural
beams, which are given in the structural drawings.

24
 o Components malfunction. For example, electric supply in a room is designed to serve
a classroom activity while architectural drawings indicate that the same room has
been re-designed as a computer lab.

Tiley (Tilley and Barton, 1997) cited by Mokhtar complements the view of the
problem with the following citation:

Unfortunately, such errors have become very common in the construction industry,
especially with escalating complexity in buildings, the increasing specialization in design
teams, and the tightening of financial and time resources available to designers.

The increased demand for collaboration especially when construction becomes more
and more complex is also covered by Heintz (Heintz, 2002):

When it comes to collaboration, effective planning and conduct of the project occurs
informally and not through either formal planning procedures or contractually specified
relations and products. It would therefore seem essential, in a time when the design of
buildings is an increasingly collaborative effort, to find planning methods that can
support collaboration.

Although the subject was not covered to exhaustion the citations above give a solid
foundation for the motivation of this thesis: there are lacks of communication,
collaboration and mutual understanding between those involved in a CEn project.
Clearly it will depend of the size and complexity of the project. However, such gaps
can arise at critical moment during the CEn life-cycle and often when in its very
beginning – the planning phase.

1.3 Civil Engineering Life Cycle

The tool created and evaluated by the present thesis covers few of phases of what is
call CEn life cycle. The life cycle of a CEn project is not resumed to the construction
site. Neil (Neil S. Grigg et al., 2001) demonstrates the number of steps related to CEn
cycle (Figure 1.1) and their interrelationships.

25
 Figure 1.1 CEn Life-Cycle and Interrelationships (Neil S. Grigg et al., 2001)

Considering that the process starts in “planning” and runs clockwise until
“replaced/decommission” there are eight major steps for the CEn cycle. Those steps
are not restricted and can vary to different authors. For each of the phases there are
sub-phases that can be explored. Phases can move backwards (from preliminary
design to repair/rehabilitate). At the centre Figure 1.1 shows the society. Those are
the ones involved directly or indirectly by the CEn project. The following bullet
points were extracted from Neil (Neil S. Grigg et al., 2001) work and they are a
briefly review of the phases showed in Figure 1.0

 Planning – This is the very early stage of a CEn project. This phase is
assigned by the discussions and exchanges of concepts take place in the realm
of planning before the formal process begins. It is where the conceptual
design relies, it is also where the needs of the project are identified, and the
first set of options is developed. The planning phase breaks up to:

o Reconnaissance – identifies projects that meet the goals established in

the overall planning and development process.
o Feasibility – financial, technological, environmental and political.
o Defined project plans – plans, specifications and operating agreements.

26
 Design – The preliminary design can lead the project to a different direction
considering the outcome of the investigations for the feasibility of the project.
This phase uses a systematic methodology for analyzing options for the
project selecting the one that best fits the requirements (Kamara J. M. et al.,
2000c). The option selected should even have public involvement.
The final design phase extends the analysis done in the preliminary design,
like some of the calculations, to a more detailed level. This sub phase includes
the “specification of the material, dimensions of the projects components, and
details of member connections”(Neil S. Grigg et al., 2001). Finally the design
review sub phase confirms that the design in optimal for the cost.
 Construction – This phase starts with the “contractor planning all aspects of
work for finances, resources, permissions, environmental awareness, safety,
etc”(Askew, 2007). As the name suggests, it is on this phase that the project
becomes a reality in a construction site. The project management is performed
by a contracted company, an individual, or the same company hired to
develop the project. that manages the project in name of the client (Wilkinson,
2001). Construction should reach the final goal as much close of the design
phase as possible including the budget. Any change in the construction will
force in a change or at least in a design review. “All construction changes
must be evaluated to ensure that the design integrity of the project is not
compromised”(Neil S. Grigg et al., 2001). This phase also host the processes
of quality control and quality assurance leading to the final inspection and
acceptance activity (Neil S. Grigg et al., 2001). Another important aspect of
this phase is the project management. Although this position can offer a
valuable help to make a CEn project to achieve its final goal managers can
face very strong opposition as presented in (Wilkinson, 2001). This phase is
finished with the approved completion of the construction contract –
commissioning (Askew, 2007).
 Operation – The facility is monitored by one or more engineers. It is their duty
to make operational decisions like quality control for example. Greene
(Greene, 1984) cited by Neil spread operation management in the following
disciplines: production management, facilities management, maintenance
management, and information management.

27
  Maintenance – As mentioned in (Neil S. Grigg et al., 2001) “preventive
maintenance heads off problems and is the most common maintenance
operation in organisations. Corrective maintenance has minor and major
aspects, depending of the extent of the correction needed. Corrective
maintenance involves repair, replacement, and rehabilitation of facilities”.
 Rehabilitation/Replacement – CEn projects should be already designed to
support rehabilitation/replacement during its live-cycle. This has as key issue
the impact created when a major infrastructure must to pass through a
rehabilitation/replacement process (Neil S. Grigg et al., 2001). Hartshorn
(Hartshorn et al., 2005) gives successful examples of sustainability in CEn
projects.
 Demolition/Decommissioning – Neil (Neil S. Grigg et al., 2001) signs with an
example of the importance of demolition/decommissioning: “consider the
engineering challenges in decommissioning a nuclear power plant or a
hazardous waste processing facility”. Again it is also part of the design
process to prepare the CEn project for its very last step in the life-cycle.

1.4 Thesis Objectives

The objective of this thesis is to investigate, create and evaluate a Collaborative

Augmented Reality System (CARS) able to explore communication, collaboration
and mutual understanding on CEn environments.

The CARS created can be easily adapted for any of the phases of a CEn life-cycle.
However, this prototype had its focus directed to design, construction (coordination
and supervision sub-phases) phases. It can also be used for both people with technical
skills (engineers, architects, contractors, etc.) as well as any other person that is being
affected by a CEn project (society in general terms).

The selection of only two between eight phases on CEn life-cycle (Figure 1.1) is
not a singular opinion. Shin and Dunston (Shin and Dunston, 2008) describes in their
paper an wide research about where AR projects best fits on CEn environments.
Although they sub categorise CEn life-cycle they could identify that: layout,

28
excavation, positioning, inspection, coordination, supervision, commenting and
strategising are the best matches for applications based on AR in CEn projects.

1.5 Proposed Solution

Once it was possible to identify that there are potential gaps of communication,
collaboration and mutual understanding along of different phases of a CEn life-cycle
this thesis proposes a solution to reduce them through the development of a CARS.
The diagram on Figure 1.2 is a wide view of the CARS that was developed by this
thesis.

In the left side of the diagram it is introduced the indoors engineers. They are
equipped with a computer where they can explore a Virtual Environment (VE) with
the help of a standard mouse. Their work is to assist those that are exploring the VE
outdoors. The VE explored shows parts of a CEn project that are relevant for the
moment. They can navigate freely in this VE, walking though it, fly over it and even
move the objects.

Indoor engineers can communicate using the audio link or through the movements
of their representation inside of the VE – an avatar. They can see where outdoors
engineers are in the VE also through their avatars. Engineers when indoors do not
need to stay closed to the CEn site. They can be virtually anywhere around the globe
as much as there are Internet connection available.

Figure 1.2 General View of the Project

29
Outdoor engineers are introduced in the right side of the diagram (see Figure 1.2). As
mentioned before it could be any person with any professional qualifications 1. The
outdoor engineers must carry all the equipment with them once they are mobile in the
Real Environment (RE). In a backpack they have a notebook with wave-Local Area
Network (LAN) enable. In this computer they have the same VE running as indoor
engineers. An external set of equipment is connected to the notebook: high accuracy
Global Positioning System (GPS) receiver, an inertial sensor, and a see-through Head
Mounted Display (HMD), etc. They can communicate with other users through the
audio link or through a limit number of body movements.

Outdoor engineers can explore in the RE the VE displayed before their eyes
thanks to a see-through HMD. The images of the VE are rendered in real time
following the body movements (head and body) of the users. They can, for example,
walk inside of a house at same time that they talk and share points of view with other
users of the system.

The system processes and transmits any action inside of the VE (for both indoors
and outdoors engineers) to other participants when sharing the same VE. A wave-
LAN router is installed nearby the area where the outdoors engineers are. The data
takes whatever way of transmission available (wireless or not) from the router to
indoors engineers and vice versa.

1.6 Steps Followed to Achieve the Objectives

Several steps were performed to achieve the final objective of this project, the
investigation, creation and evaluation of CARS.

The investigation process starts with the identification of the state of art tools
created to enhance communication, collaboration and mutual understanding in CEn
projects.

1
The “engineer” word was chosen for the matter of clarification and to exemplify the use of the tool.

30
 The next step is the investigation of the resources available to develop the project
itself. This phase gives preliminary directions of how far the development of the
project can achieve.

The understanding of technologies like high accuracy GPS is also a requirement

for this project. Questions like:

 Why this technology is the best fit for the project?

 How does this technology work?
 Why there are differences between methodologies?
 How to set up the equipment (reference point and rover)?
 What kind of data can be extracted from the GPS receiver?

There are similar questions for the inertial sensor too.

The mapping of the areas where several preliminary experiments of the project
were performed is also part of the steps towards the final objectives.

For the second half of the research the steps are:

 Initial implementation of the system (integration of hardware and software).

 Testing of the system in single user mode.
 Further implementation of the system exploring more a collaborative
environment.
 Testing of the system in collaborative mode.
 Final integration of the components (hardware and software).
 Final test of system behaviour.
 Evaluation of the project with users with different expertises.

1.7 Thesis Structure

The structure of the thesis follows the diagram presented on Figure 1.3.

31
 Figure 1.3 Thesis Structure

Chapter 2 – Review of CEn Issues

Some of the challenges over CEn environment are introduced in Chapter 2. The
research looks for potential gaps on communication, collaboration and mutual
understanding when actors from Architecture, Engineering and Construction (AEC)
are together working in a project. The idea of collaboration over AEC is not new but
just now information technology (hardware and software) is becoming good enough
to attend the requirements for such complex integration/merging of disciplines. There
are already in the market solutions provided by companies trying to create a culture
and a standard of what format and platform to use. However this is a not a finished
issue and there are ongoing researches looking for what AEC professionals really
want to know/have in hands as information at the right time. The way that
information is being retrieving from previous construction scenarios, how data can be
filtered for a specific user, how to exchange formats are just few examples. It is a fact
that technology helped to create a dynamic environment over AEC but it is also a fact
that if such information does not became easily and broadly available people will still
keep it as “owners of knowledge”.

32
 In any case, the solutions so far suggested by companies and researchers are
already proving that the investments made buy AEC societies worth. There are
reductions of costs and time, reductions of injury workers and even a reduction of
time to document a project between many other advantages.

Chapter 2 shows the advances in technology for AEC but also illustrate that
information can be better explored (using visual sense). Such gap is a point for the
proposed thesis. The data can be available but the right output for some of the cases
can be better explored. Depending of the application an AR system can make the
difference for builders, managers, engineers, architects, customers and the society
affected by a new construction.

Chapter 3 – Review of MR and GPS

The concepts of Mixed Reality (MR) are introduced in the beginning of Chapter 3.
This chapter reviews not only all the concepts involved on Milgram and Kishino
(Milgram and Kishino, 1994) diagram but also gives special attention to collaborative
environments. The reason for it is the creation of a base of what this project is about –
a collaborative environment created over the communication and mutual
understanding between users the users.

Chapter 3 reviews projects with some relation of what is being proposed here. The
survey covers projects with the following aspects:

 The way that information is being explored and shared by users – the
survey reviews projects where users are working in a stand alone mode or
in collaboration with other users.
 The way that information is being display to users – users can visualise
information in computer’s screen, large screens with stereoscopic images,
handheld’s screen and see-through HMD.
 Where information is available for users – users can explore information
in stable environments like an office, in a theatre, or outdoors.
 The position system used to explore information – users can explore
information in a click of mouse devices or using complex indoor tracking

33
 systems. The research also survey outdoors AR systems with or without
GPS to track users in the field.

The chapter covers a review over GPS as well. This positioning system is the one
choose for the kind of application that is being proposed in this thesis. The text goes
over basic knowledge about how the system works and why sometimes it does not
provide the expected accuracy. Three main methodologies of GPS are explained –
single frequency GPS, Differential GPS (DGPS) and Real Time Kinematic GPS
RTK-GPS. It is also parte of this chapter a demonstration of how different results can
be when using different methodologies to survey a coordinate in the field. Such
results are useful to demonstrate why RTK-GPS is the technology used to track
outdoor users.

Chapter 3 finishes with a review of augmented systems created to improve GPS

signal or to overtake the necessity of a second receiver as base station. They are also
called augmentation systems and they are becoming more popular with wide areas
being covered by the availability of the service. The advantage is the reduction of the
costs for high accuracy GPS and a system producing corrections twenty four hours
day seven days a week.

Chapter 4 – Design Overview

Chapter 4 introduces how the system is implemented. The chapter starts with an
overview about its design, who the expected users for the system are and how the
system is design for interaction. The overview shows the complexity of how the
system works and what is necessary to make it run. The introduction of the design
requires a close look on Chapter 5. Several process mentioned in Chapter 4 are fully
explained in the next chapter. Diagrams help to understand the connections between
processes and the order that they run when a key is stroke in the keyboard or when a
new GPS coordinate arrives in the Communication Port (COM1) for example.

The hardware is then reviewed together with the messages that are expected from
their input – like RTK-GPS receiver, magnetic compass and tilt sensor. The hardware
integration together with the platform used to manage the VE completes this chapter
– Equip and MASSIVE-3.

34
Chapter 5 – First Phase of Implementation
In the practical sense during this part of thesis some knowledge about how to survey
parts of RE and convert it to a VE becomes a requirement. The survey of an area
involves techniques about setting up receivers and tripods, collecting points, pos-
processing, interpolation and conversion of coordinates to a common format. The
next step is the design of the scene based on the coordinates of terrain and the features
mapped during the survey. Such procedures are detailed in the beginning of Chapter 5
and the results are demonstrated together with the first tests using the VE.

Another key point of Chapter 5 is the description of the process involved when
the system is operational. Such documentation is important to make future users
aware about how the technology works and what is necessary to do to make it run.
Parameters for the processes are described and complement the information started on
Chapter 4.

Collaboration starts to work when two users share the same VE over a network.
This is the first step towards the final objective – to have an outdoor user exploring a
VE assisted by an indoor user enhancing the three keywords.

At the end of the chapter it is described the results achieved with the
implementation of the Kalman filter for the magnetic compass and tilt. The graphs
indicate a reduction of jerking caused by the disturbance of magnetic field around the
sensors. This reduction improves the comfort for the user when exploring the VE with
the see-through HMD.

Chapter 6 – Second Phase of Implementation and Final Evaluation

The second phase of the implementation is marked by the combination of indoor and
outdoor users sharing the same Collaborative Virtual Environment (CVE). The
system is tested for several times until become reliable for a final evaluation.

Three exercises are also created for the final evaluation. Two of the VE created
are based on real constructions and the third one is created to demonstrate some extra
potentialities of the system.

35
 Before the final evaluation a pre-test is performed in order to learn if there are any
final adjustments to be done. The results show that outdoors users cannot perform
complex tasks using the handheld keyboard. All interaction over the VE must be
performed by the indoor user. This final requirement makes the system easier to be
explored by outdoors users. Users now have their hands free and can put more
attention in exploring the AR system than to control it.

In total eight people are invited for the final evaluation. It is used a formative
approach to extract the results from the tests. As mentioned in (Vicente, 1999)
“formative approaches focus on the way things could be by identifying novel
possibilities for productive work”. Users are chosen by their knowledge about
subjects like: Information Technology, CEn, MR, new technologies, etc. Each user
has his test recorded by a cameraman, together with notes taken by the author
(performing the indoor user), and an interview is also recorded in video. Both tests
and interviews do not follow a strictly format. However they are run mostly in the
same way with users doing the similar tasks during the exercises and answering
similar questions along of the interviews.

The evaluation process shows that the composition of the VE is a very important
issue. Users become more responsive regarding to “what they are seeing” than “how
they are seeing”. There is a lack of quality in the see-through HMD that cannot be
avoided. The device is not designed for outdoors AR tasks. The evaluation highlights
that the system proposed is well accepted when used in the right application.

Chapter 7 – Conclusion and Future Works

A final conclusion is provided on Chapter 7. This overview highlights the key points
of the thesis as well as the goals achieved. The points that could not be achieved are
reviewed as well. Together with them there are the expectations if they were
completed and the implications for the thesis for not have them complete.

Along the development of the thesis several new ideas started to be elaborated as
a complement of the work. Due to the time frame these ideas are mentioned in future
works. These ideas are perhaps just a starter point for even more elaborate ideas that

36
can become new researches in a near future. The future works also demonstrate the
potentiality of the tool here suggested and the way that it can be adapted.

Appendixes
The appendixes complement the research produced by this thesis. They are created to
clear the reading of the text as well as keep more technical information concentrated
in a single point. Appendix A reviews the program created to convert the surveyed
coordinates and features of the RE in a Virtual Reality Modelling Language (VRML)
(W3C, 1995) format. It shows the parameters and a high level algorithm based on C
language (Huss, 1997).

Appendix B is an implementation reference for all other Java classes (Schildt,

2001) created to link external devices to the controller of incoming data to the data
base. There is also an explanation and a diagram of how Kalman filter (Welch and
Bishop, 2001) is implemented.

Appendix C is the transcription of the interviews of all participants during the

evaluation process. Chapter 6 – Evaluation – contains exceptions of Appendix C.

Compact Disk
This thesis is followed by a compact disk that resumes the research. It is
recommended the download of the video to a hard drive for better visualisation.

37
Chapter 2
Civil Engineering Challenges
This thesis argues that collaboration, communication and mutual understanding are
vital to better handle complex information in CEn projects and there are issues where
these key worlds are not fulfilled. Regarding to the project proposed in this thesis the
users do not suppose to explore the VE alone. They are expected to share their views,
to ask questions and to talk about what they have understood about the CEn plans.
From this interaction between users, new conclusions can be reach, adding value to
the information that is being explored.

The concept of an “augmented architect” is not new. However, it was just during
the past few years that technology has improved sufficiently for the creation of the
first prototypes. Engelbart cited by Brate (Engelbart, 1962) introduced views about
collaboration in CEn or architecture using VE:

In 60’s Douglas Engelbart imagined the “augmented architect” situated at a workstation

with a three-foot display screen. The user would communicate with the machine by
means of a keyboard or similar device. The computer would be a “clerk”, an assistant to
the human problem solver. The clerk would compute the math and display information,
both textually and graphically. Several people could work together to model, for
example, the interior of a building. Each architect would be presented with the same
information and collaborate in the same environment. The computer would manipulate
and display information in real time, and people would have an opportunity to simulate

38
 various solutions to a problem. In this way, the technology fosters intuition and creativity
as well as logic and reason. This, Engelbart said, is human augmentation.

The possibility to use computers and electronic devices as tools that support the
manipulation of huge volumes of data in a matter of seconds is nowadays feasible.
Engelbart mentioned the augmented architect as a clear example that several people
should work together in CEn projects. This means collaboration. The human
augmentation is the combination of human beings and computers. Computers are able
to process several megabytes of data allowing human beings to go even further to
collaborate with each other and to resolve any unclear points faster.

In the Augmenting Human Intellect manifesto mentioned in (Engelbart, 1962),

the collaborative environment is exemplified as:

Three people working together in this augmented mode seem to be more than three times
as effective in solving a complex problem as is one augmented person working alone –
and perhaps ten times as effective as three similar men working together without this
computer-based augmentation.

The challenges however remain and they are the motivation for this work as
mentioned in the previous chapter. It is not always that information flows smoothly to
all parts of a CEn project. Haymaker (Haymaker et al., 2004) presents some
complementary challenges in a modern CEn project:

 Multidisciplinary – engineers and architects from different companies must to

work together. Therefore companies have different priorities and projects and
there are always clash of interest representing different engineering criteria.
 Constructive – a creation is usually over the views of other creators. The
information should flows towards one direction only.
 Iterative – during the project modifications in the project could happen. Such
modification must to pass through others engineers and architects.
 Unique – projects are unique in several aspects.
 Error-prone, time-consuming and difficult – “manually constructing and
integrating dependent views from source views causes many problems on
AEC projects today”.

39
This chapter introduces what researchers and companies are creating to mitigate such
issues.

2.1 Solving CEn Challenges – Design and Construction Phases

The following subsections were created to explore the current state of art on CEn.
The reviews are focused on design and construction phases (where this thesis is
centred) and where the possibilities are for further research. The projects that are
introduced bellow come to answer some of the difficulties introduced by Kathleen
(Liston et al., 2001). Actors in a CEn project faced major difficulties to deal with
information when using conventional 2D plans. They need to flip over plans,
schedules, contract documents, etc., to find answers as quick and precise as possible.
If the answer is not there, actors need to leave the meeting rooms to bring an answer.
The environment becomes unproductive with actors spending more time trying to
understand their own information than giving proper answers, than sharing their
views about a subject.

2.1.1 Building Information Model

Salazar (Salazar et al., 2006) mentions in his paper that the disparities between AEC
are being reduced since tools pursing collaboration and coordination started to
become a reality in CEn. Also know as Building Information Model (BIM), these
tools help AEC teams to deliver CEn projects with “higher quality in shorter times
and at lower costs”.

BIM models are very extensive and cover several areas of knowledge/information
like: geometry, spatial relationships, geographic information, quantities and
properties of building components (Public Domain / Wikipedia, 2008). Part of the
BIM framework is a database to store all these information. Once such information is
digitalised and shared by AEC any handover of a project has a greatly reduction of
information loss. This is specially significant when exchanging data between different
software (White, 2006).

40
 BIM can greatly decrease errors made by design team members and the construction
team (Contractors and Subcontractors) by allowing the use of conflict detection where the
computer actually informs team members about parts of the building in conflict or
clashing, and through detailed computer visualization of each part in relation to the total
building. As computers and software become more capable of handling more building
information, this will become even more pronounced than it is in current design and
construction projects. This error reduction is a great part of cost savings realized by all
members of a project. Reduction in time required to complete construction directly
contributes to the cost savings numbers as well (Public Domain / Wikipedia, 2008).

ArchiCAD from Graphisoft (Graphisoft R&D Zrt., 2007) is an example of tool with
BIM proprieties. The company claims that the software can make users do better:
focus on design; manage change; evaluate design alternative; collaborate; coordinate.
It was created to give more “control over design, while maintaining accuracy and
efficiency in documentation”. A central database stores and shares all information
available for the project reducing the risks of replications and working over the same
issue again. Any change is automatically spread in the database keeping the integrity
of the project. The software also allows 3D interactive design sessions with
customers: edit, navigate and interact with the session under design.

Another example of BIM came from AEC Design Group (AEC Design Group,
2008) with ArTrA BIM. Their website informs that the models created can be
handled by the design team, to contractors and subcontractors, and them to the owner
of the project. AEC Design Group claims that the main advantage is the freedom for
the users to choose their own Computer Aided Design (CAD), saving money and time
of training.

Companies like Autodesk (Autodesk Inc., 2008a) are also aware of the
potentiality of BIM concept and are guiding its products towards this direction. On
their website there are several examples of how BIM can improve the productivity of
companies, the management of projects (small and big ones) reducing time to
produce paperwork. Product like Revit allows designers to interact with customers in
real time, changing plans, showing the modifications in a 3D environment.

Bentley (Bentley Systems Incorporated, 2008) puts the challenge for BIM as:

Unite the project lifecycle, bringing together design and documentation information and
workflows for the design, construction, and operation of all types of buildings and

41
 facilities around the world, from the conventional to the most inspiring projects of our
time.

Their products are aiming to bring together the life-cycle of CEn projects considering
(design/building/operation) with information about the building and its life-cycle.
“The definition and simulation of the construction, its delivery, and operation using
integrated tools”.

The study of the advantages achieved in a CEn project (in this case Mechanical,
Electrical and Plumbing – MEP) using BIM can be found in (Khanzode et al., 2008).
In this particular study, they evaluate the construction of a medical office building
with an initial contract of $96.9 millions USA dollars, 250,000 square foot, three
level building, and a car park. All these constructed in 27 months – from January
2005 to April 2007. Khanzode highlights the first impressions of CEn projects
difficulties:

o Lack of ability to identify conflicts due to the 2D representation of the designs

o Delays in construction process due to conflict being identified in the field
o Lack of trust in the fabrication offsite due to the fear of system not fitting leading
to a lot of on the site fabrication
o Rework to fix the conflict issues not identified during design and coordination
o Increased site supervision required to avoid conflicts between trade contractors
o Increased administrative burden of more request for information and change
orders due to identification of conflicts in the field after budgets are approved
o “Install first” mentality amongst trade contractors so as to avoid having to move
their systems in case conflicts arise leading
o Overall reduced productivity for everyone involved in the process

The team leaders decided to go for a BIM and Virtual Design Construction (VDC) –
specifically 3D/4D and automated clash detection tool – even though this being their
first experience with these technologies.

A the end of the project the team members created a guideline which forms – in
their point of view – the best practices for doing MEP coordination using BIM/VDC
tools:

o Clarifying the role of the general contractor and specialty contractors in the
coordination process.
o Increased site supervision required to avoid conflicts between trade contractors.
o Developing the levels of detail in the architectural, structural, and MEP models.

42
 o Setting up the coordination process – with different sub-levels.
o Managing the coordination of the installation process.

The paper describes how teams worked in small groups, organising themselves,
orchestrating their tasks over the construction.

The project team should collaboratively determine how the design will be broken down
into small enough batch sizes that allow detailers to coordinate and complete an area so
that fabrication can begin. This is an iterative process between the design and
construction teams.

Between the many benefits of the use of BIM/VDC tools that are reported in the
papers it is highlighted:

 The drastic reduction of time spent on request of information or change of

orders due to field conflicts – reliving architects and engineers.
 There was no change of orders related to field conflicts.
 Information about the building is easier to recovery.
 Savings of $6 millions USA dollars and 6 months.
 There was a drastic reduction on time spend by general contractor – from
typical 2 to 3 hours day to 10 to 15 hours in 8 months.
 The number of accidents on site was also well reduced.
 The building parts were pre-fabricated off site. It reduces the risk of injuries
(lower number of machinery) and makes the site cleaner. The delivery method
was “just in time” reducing the amount of stocked material too.
 All trades have finished their work ahead of schedule.
 “The subs could use lower-skilled labour for the field work compared to other
projects where higher-skilled field labour is necessary for installation as the
labour force typically needs to interpret 2D drawings, etc”.
 The 3D design should address not only the structure and the displacement of
the rooms of the building but also the furniture (especially in a medical
centre).
 Sub contractors spend around 2 days per week with architects and engineers to
issues being addressed. This is more than what they predicted initially.

43
Figure 2.1 (Khanzode et al., 2008) shows the savings of time and money along the
CEn project. The difference between the estimation (red line) using standard methods
of design and building and the final achievement (green line) is significant thanks to
the tools (BIM/VDC) used to help to manage the project and the collaboration
between teams. The image (Figure 2.1) is a courtesy of DPR construction, Inc., CA,
USA and extracted from (Khanzode et al., 2008).

Figure 2.4 Reduction of Costs and Time in the Project Evaluated on (Khanzode et al., 2008)

There are several other companies producing BIM products in the market. Some of
the examples are but not limited to: (Building Integration Software Company Ltd.,
2008, BulidingExplorer.com, 2007, Tekla, 2008, AceCad, 2004, Nemetschek North
America, 2008). More information about BIM can also be found in (White, 2006,
Maldovan, 2007).

2.1.2 Researches addressing Communication, Collaboration and Mutual

Understanding in CEn Projects
The projects presented in this sub-section are part of researches aiming to reduce the
problems on communication, collaboration and mutual understanding in CEn
projects. They are more focus in one point than others making them not a “total
solution” as claimed in some commercial products.

44
 Kamara (Kamara J. M. et al., 2000a) describes the difficulties in establishing and
processing client requirements on concurrent CEn. Kamara quoting Winner (Winner
et al., 1998) explains that the concurrent approaches considers all the elements of the
product life-cycle including quality, costs schedule and customer requirements. The
paper is focused in the development of methodologies to enhance the understanding
of client requirements (Kamara J. M. et al., 2000c). With this methodology client’s
requirements will be better comprehended by multidisciplinary teams to work
collaboratively. The requirements are processed and prioritised at the time that they
are also cross correlated against several other requirements that normally arise in a
CEn project like: site requirements, environment requirements, regulatory
requirements, design requirements, and construction requirements. The output of the
system “serves as the basis for adopting a particular procurement/contract strategy,
such as design and build. This approach focuses on clarifying the ‘problem’ using
methods other than design, as is usually the case in current briefing practice”.

Ugwu (Ugwu O. O. et al., 2001) work describes that “multiple-agent systems

paradigm allows distributed organizations and domain experts to be modelled as sets
of collaborating (but not necessarily co-operative) agents that interact in a problem-
solving space to achieve set goals”. The goal mentioned by Ugwu is:

…often the agreement on a product configuration that is mutually satisfactory or

satisfying to the design team. The design team includes: Architect, Structural Engineer,
Steel Fabricator, Building Services Engineer, Quantity Surveyor, Health and Safety
Personnel, and other specialist trades that are required in a given project.

The paper describes the early stages of a website creation – or a portal to facilitate a
collaborative design – helping to spread knowledge between the multiple-agents.
Data and information processing improves the decision of the agents in a
collaborative design area. An important point in the paper is about the understanding
of the concepts of collaboration and ontology as well as the functionality of the
system by the participants. Even in an environment like this agents/participants can
act like owners of the knowledge not using the potentiality of the collaboration
between parts. The challenges in these systems are in translating what experts know
as owner of knowledge, capture, translate, and represent it using digital forms. The
complexity to create systems like this is part of the research presented by Ammar-

45
Khodja (Ammar-Khodja et al., 2008). In this paper the research has focus on make
actors from different parts work collaboratively aiming knowledge be transferable
between actors using an automated system. The paper reviews the concepts of
Knowledge Base System (KBS) and its complexity to be used it. More refined
products systems like the Knowledge Base Engineering complements what can be
produced by CAD systems, automating design activities and creating a structure of
knowledge for the product.

Tuzmen (Tuzmen, 2002) presents a conceptual model of a distribute process

management environment in the establishment of a collaborative design environment.
The tool supports remote access where users can filter information to see only what is
relevant for their work – like the activities that they need perform for example. Users
can create a new process definition or reuse an existing one modifying it when
necessary. The process definitions are composed by activities and the relationship
between each other and are linked to each phase of CEn life-cycle. A process
enacting service interprets the execution of the tasks completed identifying the next
one. Users are advised by the system each new update.

Similar project can also be found in (Mokhtar, 2002, Augenbroe et al., 2002,
Heintz, 2002).

Sarshar (Sarshar et al., 2002) describes seven themes that projects like the ones
mentioned in this sub section that should address:

 Model driven as opposed to document driven information management on

projects – information is not centralised/unique to everyone but shaped for the
requirements of each case.
 Life cycle thinking and seamless transition of information and processes
between life cycle phases – communication and knowledge should be shared
between phases of CEn life-cycle.
 Use of past knowledge (information) in new developments – reuse of past
knowledge in present and future projects.

46
  Dramatic changes in procurement philosophies, as a result of the Internet –
new business opportunities should arise with the advance of technology.
 Improved communications in all life cycle phases, through visualization –
taking advantage of visualisation to improve communication between parts
making it more effective and user friendly.
 Increased opportunities for simulation and what if analysis – simulation can
improve the understanding that helps to predict feasibility, planning and
scheduling.
 Increased capabilities for change management and process improvement – it
is not only the technology that should improve. As mentioned before the
structure of the organisation must also understand the concepts involved, what
technology has to offer and restructured itself for this new era.

Another approach to integrate AEC project models is presented by Haymaker

(Haymaker et al., 2004). This research “investigates the conceptual simplicity of
formalizing a project model as a directed acyclic graph of views and their
dependencies, allowing engineers to manually manage the cycles”. Their work
introduces an approach to “formalise methods to construct views for other views and
control the integration of a directed acyclic graph of views and dependencies as the
project progresses”. The Perspective tool logs information like: “sources, status, and
nature of its dependencies” of a CEn projects. Engineers can compose and reuse
information from past projects creating then new “perspectors”. The tool automates
or assists engineers to add more information analysing if all part of the project are
integrated and making sure that all requirements of the project were satisfied. The
prototype created allows users to see and interact with perspectors and analyse the
geometry of the construction in 3D using VRML.

Shelbourn (Shelbourn et al., 2006) claims that the challenges do not relies “only”
on “capturing, storing, sharing, and re-using” information. The paper describes the
“creation of tools that enable the development of organisational practices in the
construction sector to promote knowledge creation, prior to sharing and re-use,
along with the tools to support such a process”. They work over the knowledge

47
management and rise reasons for the limitation on creation and dissemination of the
knowledge between actors in CEn projects. Some of the reasons are:

 Knowledge residing in the head of single individuals. People can move to

different projects or change some of the ideas.
 Decisions are not often recorded in documents.
 Information is only recorded at the end of the project instead of and not
during the events.

To improve these practices the paper addresses the C-SanD project (University of
Loughborough et al., 2001). C-Sand is driven by the possibility of sustainability of
CEn in favour of environment. Creating and sharing knowledge is one of the
possibilities for the project to achieve its goals. To purse this objective they report the
creation of a website portal called SMAZ. The results after evaluation demonstrate
that although the research would be useful for actors in CEn project they need to
improve considerably the design of the portal user interface. As also a result they
report that groupwork functions are apparently better accepted than information
retrieval functions.

Kubicki (Kubicki et al., 2006) investigates the cooperative work between

heterogeneous actors in a CEn project. The paper includes to the discussion the
dynamic of a CEn project especially in the construction phase and the difficulty to
create rules when new actors (contractors, security, etc.), objects (material,
equipments, tools), and documents are in constant change. Eventually requirements
and coordination becomes informal and eventually not registered:

Coordination activities have to determine elementary construction tasks and their time
sequence. Planning has to take into account resources (human and material) and technical
constraints (Kubicki et al., 2006).

Kubicki quotes Brousseau (Brousseau and Rallet, 1995) on the challenges faced on
the codification of actors activities in CEn projects:

The random nature of the activity, the singular requirements of CEn projects and the
participation of actors from different sources make difficult a precise codification of rules
of interactions. In addition, the effort to represent procedures in a codified way is not

48
 justified because of the transient nature of relations. As a consequence, coordination
procedures implemented in construction are relatively few formalized. This does not
mean that written contracts do not exist, or that the regulations do not define certain
coordination procedures. This means that any texts are subject to greater margin of
interpretation in the industry (Brousseau and Rallet, 1995).

The paper (Brousseau and Rallet, 1995) finishes reporting the achievements of the
designing of a assistance tool for cooperative engineering. A first prototype called
“Image.Chantier” is an assistance tool created to coordinate the diffusion of meeting
documents between actors. The tool can spread “personalised access to information”,
meaning that actors receive only the relevant information for their work. The tool is
based on a web browser view with four split windows: Gantt planning view, 3D
mock-up, text, and graph. The windows are dedicated to different users like architects
(with focus on 3D visualisation) and coordinators (with focus of
documentation/organisation).

Kathleen (Liston et al., 2001) also argues that actors should concern only to what
is really important to share as information. Such sharing should happen only when
time requires and tools should be more specific instead of broad designed to all cases.
Researchers evaluated the difficulties for actors to share information when using
conventional paper-based meeting. Actors spend more time trying to find the
information than explaining it to other actors. The situation improves considerably
when the meetings are hosted in a Computer Automatic Virtual Environment
(CAVE). Information becomes more interactive and easy to find. A large and shared
visualisation between AEC helps to keep them with focus in the meeting also
improving the interaction between them. The disadvantage is the limitation of
essential project information available that can occur. The final evaluation was on
meetings with product model support. This technology purses the compatibility of
data between different packs of software used by actors making data more consistent
and reliable (updated). The major disadvantage is:

Pre-defined semantic models are unlikely to support all the information needs by multi-
disciplinary project teams because each project needs to address a unique set of
requirements, which require the generation of project-specific information views to
address project-specific decisions. They (the models) do not yet scale to the multi-
disciplinary, multi-phase, multi-objective, and multiple level of detail nature of projects.

49
The paper describes a project called iRoom. This interactive workspace is shaped by
interactive touchscreens, notebooks, projectors, laser pointers, etc. The meeting room
accepts many ways at the hardware level to input and output information. The
software controls the information making it easier to share between meeting
participants. Information is automatically updated and replicated when users select an
item in one of the many screens. The results suggest that users are more focus in
relevant information than looking for answers in their on documentation.

Complex building designs and the challenges of use 3D tools between actors from
different design teams is studied by Ku (Ku et al., 2008). The paper describes 3 cases
of studies where companies had to share information between them to implement a
CEn project. Usually companies do not have the same set of software making
sometimes files incompatibles and improving the chances of misunderstanding.
Problems start to increase when the complexity of the structure in construction can
not be clearly represented by 2D graphs (some companies still using only 2D graphs
for representation) giving margin to ambiguities on the interpretation. The lack of a
common channel of communication between the platforms used by constructors and
designers can also introduce a lack of coordination on changes to the project. Ku also
highlight the liability issues when companies share the same project. Such barrier can
force one company to redesign a model increasing the chances of mistakes and
misunderstanding due to transcription errors.

Scheer (Scheer et al., 2007) describes the results of a found when several
professionals involved in a design team worked collaboratively in a system called
“System for Project Management” developed at the Federal University of Paraná –
Brazil. The system logs in a data base exchangeable information with other actors and
users can extract information for particular interest like (personal projects, activities
and tasks). The actors of the design team did never have previous experience in tools
like this. They were trained to use the system and encouraged to use it. Results of the
evaluation report that some of the actors did not fully understand the tool or they were
not available to use it showing a moderate optimism in the potentiality of system.

50
2.2 Implementing New Technologies in a CEn Environment

Although companies usually claim that their products are the “best of the market”
(sub-chapter 2.1.1) for a range of solutions the papers presented on sub-chapter 2.1.2
showed a different reality. It was shown that researchers still need to face the number
of challenges to implement a system that can help to enhance collaboration,
communication and mutual understanding between those involved on a CEn project.
They also showed the difficulties to introduce/implement information and
communication technology in a company when actors are already used to a culture.
The situation can get worst when the solution provided does not attend the real users’
desires (Peansupap and Walker, 2005, Mathieson and Keil, 1998, Nitithamyong et al.,
2007).

In a survey proposed by Nitihamyong (Nitithamyong et al., 2007) with 82

companies evaluating 14 different project management system – application manager
providers concluded that benefits are limited to “strategy, time, and communication”.
Benefits like “cost savings, quality improvement, and risk management” still need to
be improvement – although (Khanzode et al., 2008) proved the opposite regarding to
cost and time savings at least. On a collaborative environment system there are also
risk to be calculated regarding the coordination of distributed products as explained
on Qiu (Qiu et al., 2007a, Qiu et al., 2007b). To know more about the limitation on
implementing new technologies in CEn environments the author recommends
Peansupap (Peansupap and Walker, 2005) with an extended list of papers reviewing
the subject.

2.3 Improving CEn Projects – Communication, Collaboration and Mutual

Understanding

The works and products reviewed above confirm that even though the technology in
general become more accessible to companies there are still issues to be solved.
Problems like how to model the data that will be shared between actors, how to make
the software intelligent enough to recognise a problem and provide a right solution
from previous cases, how data can be visualised and understood, or even how to input

51
data without loosing long time to do it, are just some of the tasks to be implemented
or improved.

Some of the projects also mentioned the fact of using 3D visualisation (some of
them were using even 4D – time) actors can easily indentify problems and find a
solution (Khanzode et al., 2008, Brousseau and Rallet, 1995). This is particular true
when investigating complex buildings where 2D representation becomes insufficient
(Ku et al., 2008).

This thesis concerns about the methodology of how data was collected from AEC
and put in the data base but does not get involved on it. The presented research is
already a next step that goes beyond the challenges faced by researchers above and
explores the possibility of using a CARS to visualise information that most of the
time has being explored only in a computer screen. The investigation is directed to
user behaviour when exploring such VE in a CEn context. This thesis aims to create
and study the initial impact of CARS in a CEn environment, exploring if this
technology would be welcome by actors and to instigate communication,
collaboration and mutual understanding.

Based on the works presented above and the challenges faced by researches this
thesis understands that the topics above should drive a research that aims the use of
CARS in a CEn context:

 Communication – the topic communication in this thesis aims to instigate

verbal and no verbal communication. Users should talk asymmetric with no
rules imposing who should speak and who should listen when in a
conversation. They are encouraged to ask and to answer and specially to make
their on conclusions about what they are exploring in the VE. There is no
logging or annotation system to what is being saying. The research is focused
first if actors can understand the information. This is only possible with
communication available all the time and in special during the final
evaluation.

52
 Collaboration – the second key world for this thesis only happen if firstly
communication is in place. Collaboration is placed when users can help each
other inside of the VE. It could happen when the user outdoors cannot find a
specific target for instance. A discussion about what is being presented or not
is also part of this key world. As mentioned in Chapter 1 collaboration is
characterised when two or more people work towards the same objective.
 Mutual Understanding – this can only be achieved if communication and
collaboration were previous reached. The third key world can be exemplified
when outdoor actors cannot understand 3D sections of a building for
instances. Through communication and the exchange of information
(collaboration) users can talk and explain what each phase of the visualisation
represents making the understanding finally happen.
 Exploring the VE – users should be able to explore the VE in their
computers in stand alone mode. There is no path to be followed or limited
area to be explored. There is no need to render view points to create a movie.
Everything must be rendered in real time even with the cost the quality of
image being produced. The same is true with users exploring the VE outdoors.
The system must support one or more users (indoor or outdoor) and allow
them to share the same context in real time.
 Moving objects in VE – virtual objects can be moved at any time. All
modifications are automatically reported to all users that are sharing the VE in
that moment.
 Animation – this feature help to bring the VE as close as possible of
reality. Animation helps also to demonstrate simulations to users like wind
corridors, water elevation, increase of traffic, etc.
 Accuracy – users and virtual objects should have their positioning as much
accurate as possible. The accuracy should happen in real time with no
limitations like: pre-determinate surveyed points, huge discrepancies of the
real positioning and the calculated one, etc.
 User friendly – although the system is a prototype it should be as user
friendly as possible. Very few or not instructions should be necessary to use it.
Users should be able to rely in their natural instincts to explore the VE. This is
particularly true when using the system outdoors.

53
Chapter 3

Technology Review – VR and GPS

As mentioned by James Vandezande (Autodesk Inc., 2008b): “Design and
visualisation is an important aspect of a project as people are driven by the visual
experience of the architecture”. This state is particularly true in a project that works
over the visualisation of VE in complex area like CEn. Lange (Lange and Bishop,
2005) quotes Bruce (Bruce et al., 1996) when talking about the importance of
visualisation. They estimate that “80 percent of our impression of our surrounding
comes from sight”. Back to the 3 key world of this thesis the work presented by
Lange indicates three bullet points very closed related to what this project is focused.
Lange and Bishop summarise the idea of visualisation to enhance communication,
collaboration and mutual understanding:

o We want visualization to give us the opportunity to see, experience and

understand environment changes before they occur.
o Through the ability to share their experience and potential for exploration,
visualization will help communities (of whatever size) to build consensus and to
make decisions about their future.
o The relationship of people to their environment is a key contributor to
environmental decisions and visualization can help us learn more about that
relationship.

54
In VR perspectives, CEn and Architecture are recognised by the wide field of
applications that can be created (Woodward et al., 2007a). Tools based on VR
concepts are natural medium for ACE professionals to express ideas, manipulate 3D
objects in real-time, explore VE from different angles and stages (Bouchlaghem et al.,
2004).

This chapter makes clear the potential use of VR and in special CARS in CEn
projects. The following sub-chapters introduce a review of VR and related work with
direct or indirect application in CEn. Although this thesis is not involved in a research
aiming to collect information from AEC (like those presented in Chapter 2) it is a
complement of it in a near future. The next natural step is the convergence between
those researches presented in the Chapter 2 and AR systems (Woodward et al.,
2007a).

This chapter also introduces an overview of the concepts related to positioning

system – Global Positioning System (GPS) – which are relevant to this thesis. It is not
intended to include in depth description of the techniques required to generate
positioning information from GPS signals. Instead, the following paragraphs provide
an overview of the three usual procedures which are commonly applied in AR
systems: stand-alone GPS positioning, DGPS and RTK-GPS. Since this chapter is an
overview of the applications of GPS, some knowledge of the working of the system is
assumed.

3.1 Virtual Reality

The term VR was first coined in the 80’s by Jaron Lainer (Brate, 2002). As
mentioned in Koleva (Koleva, 2001), the term MR is proposed by Milgram and
Kishino (Milgram and Kishino, 1994) in 1994. They noticed that the term VR is
being used not only for full synthetic environments but also for all kinds of computer
generated scenarios that combine real and virtual.

Milgram and Kishino proposed the Virtuality Continuum to demonstrate where

each of the concepts lay (Figure 3.1).

55
 Figure 3.5 The “Virtuality Continuum” (Milgram and Kishino, 1994)

The RE, on the left side of the diagram, is defined by Milgram and Kishino based on
the difference between real and virtual objects according to three different aspects:

 Real objects are any objects that have an actual objective existence. Virtual
objects are objects that exist in essence or effect, but not formally or actually.
Real objects can be viewed directly. Virtual objects must be simulated.
 In order to be seen, virtual objects must be synthesised and then projected or
displayed by an electronic device. Just because an image “looks real” it does
not mean that the object being represented is real.

Milgram and Kishino developed these aspects on their paper, defining real objects by
means of a sort of counterproof. Even though they clarified that virtual and REs are
much more comprehensive than visual perception, there is a tendency to work with
examples focused on vision.

The VE, on the right side of the Virtuality Continuum diagram (Figure 3.1), or
VR, are both concepts that can be found in the literature as: “an experience in which
a person is surrounded by a three-dimensional computer-generated representation,
and is able to move around in the virtual world and see it from different angles, to
reach into it, grab it, and reshape it” (Novak, 1999). Brooks (Brooks, 1999) suggests
that: “a VR experience is any in which the user is effectively immersed in a
responsive virtual world. This implies user dynamic control of viewpoint”. It must be
clear that a VR experience is not only related to vision and viewpoint. There are
environments where stimulation of the haptic, audio, olfactory and palate senses also
exist. All the five senses can work together or separately. The final structure produced
for the VE is deeply dependant on the research’s focus.

56
 Apart from these concepts, there are three key words to better understand the VR
theme: immersion, interactivity and involvement.

 Immersion – The idea of immersion comes from the sensation of staying

inside the environment, which means being part of it. This key word depends
basically on the quality and accuracy of stimulation of the five human senses.
It could be achieved by a high graphical resolution on the scene with textures,
shapes and simulations to make it very similar to what is known in the
physical world, for example. Very accurate devices could be added to it,
enabling the user to forget that they are connected on a computer. Such
devices must be fast enough to update the changes, light enough in order not
to be perceived, and ergonomic enough to allow freedom of movements.
 Interactivity: In his report, Pinho (Pinho, 1999) exemplifies a VE where the
user can visualise, handle and explore in real-time all the features of the VE
without a rigid control of these actions. The user applies his natural and
intuitive movements to control the VE. The interactivity is not only related to
the object but it is also completed with other users of the system and virtual
agents. They share the same senses of space, presence and time.
 Involvement: In the same report, Pinho (Pinho, 1999) describes how
motivated the user is inside the virtual world. The user can have an active
situation when there are interactions or engagements between other users or
existing elements. A passive situation is always possible when there is not
mutual or reciprocal action (between users or existing elements) in the VE.

3.2 Augmented Reality

If the intention is to overlay a perception of the actual physical and real world with
images, sounds, smells, haptic sensations and tastes created by a computer system, the
terminology used is named AR. Azuma (Azuma, 1995) gave the following concept
about AR:

AR is a variation of VE, or VR as it is more commonly called. VE technologies

completely immerse a user inside a synthetic environment. While immersed, the user
cannot see the real world around him. In contrast, AR allows the user to see the real

57
 world, with virtual objects superimposed upon or composite with the real world.
Therefore, AR supplements reality, rather than completely replacing it.

In AR systems, the perceptions can be increased or decreased. Besides adding objects

to the RE, AR also has the potential to remove them (Azuma, 1995).
It is possible to differ between VE and AR as:

 VE is a situation where the user is in a place without external contact. That is,
if the VE has the focus in audio, for example, the user will not listen to
anything directly from the external world. The sound generated is digital. A
cockpit of an aircraft simulator is an example of VE. The user has the
sensation of being contained in the VE.
 The integration of current information and computer generated (digital) data
in a designated area is the concept for AR. This combination supplies the
user’s necessity of information that can not be perceived at the moment. The
added digital data can represent information from the past, present or future.
The inverse could be achieved by taking out information from the RE. It will
depend on the applicability of each project. An AR system can, for example,
capture sounds from the RE and reproduce it in a directional 3-D VE with a
partial inclusion or exclusion of the audio.

3.3 Augmented Virtuality

Milgram and Kishino (Milgram and Kishino, 1994) exemplified the concept of AV
on the right side of the Virtuality Continuum diagram (Figure 3.1). The concept
proposes integration of information retrieved from the RE on the VE. Schnädelbach
et al. (Schnädelbach, 2001) presents an example of AV application in Augurscope
project.

58
3.4 Mixed Reality

MR is commonly understood as “seamless integration between the real world and a

virtual world”(Novak, 1999). As shown on Figure 3.1, MR is a generic concept
involving both AR and AV running from RE to VE.

Possible Applications – MR and VE

There are different types of applications for MR. Visual applications are the most
common work related to this subject. Audio and haptic MR projects are being
increasingly explored. The development of new devices that can produce all sorts of
smells and tastes opens a wide area to be investigated by researchers. Applications
that could be enhanced with works on MR projects are presented on the list bellow.
MR combined with other technologies brings information to more accessible way
helping researchers with the possibility of simulate substances behaviour,
displacement of features, animations, sound simulations, etc. The list is bellow is
neither extensive nor complete but illustrates where MR can be used as a tool to
enhance understanding over information.

 Scientific Visualisation: Planets surface visualisation; Virtual tunnel test;

Molecular synthesis .
 Health and Medical Applications: Surgery simulation; Radiotherapy planning;
Virtual health; Anatomic visualisation; Training for disabled people.
 Architecture and CEn: CAD; Decoration of environments; Acoustic
estimation; Design of tools.
 Education: Virtual Laboratories; Planet Exploration; Distance learning;
Education for people with mental disabilities.
 Entertainment: Virtual games; Virtual Cinemas.
 Training: Flight simulators; Military operations planning; Astronauts training.
 Arts: Painting; Virtual Sculpture; Music; Virtual museum.
 Information Control: Financial view; Information view; Virtual information.
 Telepresence and Telerobotics: Remote control systems; Teleconference;
Virtual lecture.

59
3.5 Collaborative Virtual Environments

As mentioned before, one of the main goals of this thesis is related to the aspects of
collaboration between the users of the system. A successful co-operation between
actors in a CEn environment is achieved easily when well regulated and managed
than when developed for a self-contained entity. As cited on (Churchill and Snowdon,
1998): “in ‘real world’ domains, collaborative work involves considerable complex
information exchange. Individuals need to negotiate shared understandings of task
goals, of task decomposition and sub-task allocation, and task/sub-task progress”.

Therefore, collaboration does not mean that there is a complete agreement of the
parts involved on the progress of a task. People can have different points of view over
a resolution of a task for instance, disagreeing in some topics, but because they are
also looking for the same welfare, the dialog and the co-operation must have place for
the mutual understanding. No one holds the true on a CE and neither there is
imposition of a perspective over a person. There is mutual respect and a well-
balanced division of tasks. Thus, it is important that collaborators know what is
currently being done and what has been done in the context of the task goals
(Churchill et al., 2001).

Collaboration can be observed in several ways in our society: personal contacts,

gestures, body expressions, work groups, letters, telephone calls are just few
examples which could help to achieve a common goal. More recently, computers
have been used on the creation of CVE.

CVEs are a digital “space” created over a net of computers where collaboration
and interaction create links between users. Each space has a connotation that
constitutes a “place”. It gives the users the opportunity to understand the activities
performed on the environment as well as on the real world. The places are created to
support multiple groups developing multiple activities. As on the real world, people
(or the users on a CVE context) understand that even without straight relations the

60
spaces and places are dependent on their actions, behaviour, social skills,
manifestations, etc.
According to Churchill (Churchill and Snowdon, 1998, Churchill et al., 2001)
CVEs are explained as:

CVE’s represent the computer as a malleable space, a space in which to build and utilize
shared places for work and leisure. CVE provide a terrain or digital landscape that can be
‘inhabited’ or ‘populated’ by individuals and data, encouraging a sense of shared space
or place. Users, in the form of embodiments or avatars, are free to navigate through the
space, encountering each other, artefacts and data objects and are free to communicate
with each using verbal and non-verbal communication through visual and auditory
channels.

The users that explore a CVE are not restricted to a desktop PC. CAVE, wearable
computers and PDA, are some examples where CVE can be shared and explored by
the users.

Greenhalgh (Greenhalgh, 1999a) mentions some common characteristics on

CVEs:

 Computers spread over the planet are connected over a network sharing the
same CVE. The system, as itself, can or cannot be running in a specific
machine or set of machines. It can be distributed over the network supporting
multiple users.
 Collaboration can be done on different ways like: gestures, audio, video, etc.
 The idea of space or world is conveyed to the users. The users soon understand
the boundaries of the VE and the context of the place.
 Users are represented, in a way, as a form inside the VE in order to make them
visible. It could be a textual representation as it is done on Multiple User
Dimensions/Dungeons (MUD) or as an embodiment (avatar), a humanoid
representation on 3D environments. Often the users can choose a
representation more appropriate with the VE.
 VE is a free space for users to freely go and explore the environment.
Certainly, the VE has a virtual boundary delimiting the space. Depending on
how detailed the VE is, the users will need to respect basic conventions of the

61
 physical world, such as walking over a lake, going straight through a wall;
thus, marking the internal boundaries of the system.

Natural and Spatial Communication

The idea of natural and spatial communication is connected to the knowledge of
where the users are positioned in relation to the others and to the objects. On the real
world, for example, a simple look around can give to person enough information to
identify the approximate number of people on the place, who is physically close to
him and who is not, where he is located in relation to the place, the obstacles around,
etc. This “glance” over the area that surrounds him is converted to the knowledge of
space and time.

The CVE should support natural spatial communication. Based on the knowledge
of space and time, “…gestures, posture and speed of movement consciously and
unconsciously convey information such as availability for conversation, interest and
intended actions” (Greenhalgh, 1999a) .

The paragraphs above reveal two important points on CVEs. The users on the
VEs must have at least an embodiment. This generates the notion of space and time.
Comparing two extremes, both 3D and textual CVE follow the same principle. On 3D
environments, the perception of scale or obstacles is the most notorious way to know
if another user can be reached or not. On textual CVEs inhabited by objects, the user
knows if what is being said is reaching the others by a “proximity sensor”. On a
richer scenario, space and time are revealed through the information of the object and
places that surround a specific user.

The second important point is the support of body movements. A CVE can only
offer this possibility if, and only if, the embodiment of one user can be observed by
the others. On the real world, a good percentage of our communication is done via
gestures, movements (it includes body hair movements), expressions, level of voice,
colour of skin, transpiration, etc. Currently, much of this “communication” can not be
provided by CVEs. However, simple movements are already integrated on the
system. On MASSIVE-3 (Greenhalgh, 2000b) the embodiment of the user has its
head down if the user is not actively interacting with his avatar for a few minutes. For

62
other users sharing the CVE, it indicates that the person represented by the avatar
maybe is not in front of the computer.

Sharing Context
As cited on (Churchill et al., 2001) a “shared context can mean many things; it can
mean shared knowledge of each other’s current activities, shared knowledge of
other’s past activities, shared artefacts and shared environment”.

Collaboration is based on sharing. When a gesture is done or an object is shared

“not only do they become the subject of communication between users, but also the
medium of communication” (Churchill et al., 2001).

The way that an object is being carried can have different meanings. In a dramatic
example, if the sharp blade of a knife is pointing to the person that is holding it in the
direction of another, there is no particular reason for a precaution movement from the
receptor. The first basic intuitive reaction is that one person is passing a knife to
another. In contrast, if the sharp blade is pointing to the receptor, the object transmits
the information that something bad is about to happen. The information contained on
that specific object being carried on different ways is disseminated to other users. The
context of the situation is then shared over the participants. The visual perception of
the scene collaborates with the logic of the facts.

Peripheral Awareness
When a person is driving a car, the environment that surrounds the car provides
enough information for the driver to know whether it is time to brake or push the
accelerator pedal. What the driver has is a natural and spatial communication that
must be shared with all potential hazards that surround him. If the traffic light turns
red and the cars in front stop suddenly, the peripheral awareness lets the driver know
if it is feasible to avoid the car taking the other lane. In a fraction of time, several
calculations must take place, such as: velocity of the car, distance from the car in
front, condition of the other lane, break distance of the vehicles behind, etc.

63
 Hughes et al., cited on (Churchill et al., 2001), “observed that co-located workers
maintain and make use of an ongoing awareness of the activities of others with their
environment, even when not explicitly co-operating”.

Negotiation and Communication

Churchill (Churchill et al., 2001) explains that:

Conversations are crucial for negotiation and communication about collaborative

activities. Collaborative work requires the negotiation not only of task-related content,
but also of task structure in terms of roles and activities and task/sub-task allocations.

The exchange of information through limited number of body gestures (vision) and
voice (hearing) has been developed with great results on CVE. Such approaches
always enhance the comprehension of what is being shared. Wearable tracking
devices like (Xsens, 2008, Ubisense, 2008) are examples of how gestures can be
added to VE.

Freedom
Another premise of VE is freedom. The users can explore the VE, maintaining the
autonomy of their acts. There are no pre-programmed movements where users should
go and scenes are rendered on real time.

Eventually, multiple view points can be explored. Instead of being restrained by

the embodiment’s line of sight, other shoots can be taken from “virtual eyes”. They
could be placed on specific points, either following or not the embodiment
movements.

Extra powers (flying, diving, etc.) can also be given to the embodiment with the
objective of turning the VE flexible, enhancing the perspective of collaboration.

Large Number of Participants

The size and complexity of the VE added to the number of participants suggests how
powerful the computational system is on the background. Usually CVEs are complex,
with thousands of polygons to render and textures to add. Video and voice channels

64
are also included on the system’s performance. As cited on Greenhalgh (Greenhalgh,
1999a) “the final advantage of CVEs from the perspective of collaboration is that
there is a clear potential to support extremely large number of simultaneous
participants”.

As a reproduction of the real world, a CVE system must only pay attention to the
strict necessities of each user. If an embodiment is placed on a room with some
furniture spread around and is talking with a couple of other users, there is no
requirement for the system to render the whole building for the three users. The
system spends resources on what is relevant for each user – this process is also called
Level of Detail (LoD) (Wikipedia, 2008b). As cited on (Greenhalgh, 1999a), “…it
allows them to be aware of and to interact with a relatively large, continuous and
consistent subset of the total environment”.

3.6 Related Projects

As presented in the previous sub-chapters, MR is an open and wide area of study. The
concept can be applied to most of the areas that need to interpret information, which
is sometimes not easily understandable by charts, tables, numbers, text, etc.

A selection of projects were choose to exemplify how MR, and in special VR, can
help CEn actors to perform their works. Although some of the examples introduced at
this first moment do not have a direct relation with CEn application it is imaginable
that they can be adapted to this area. This selection is based and presented using the
following criteria:

 From VE to AR – the overview cover the aspects of researches on VE and

going towards AR. This approach was used to demonstrate the limits of VE
and when AR becomes a necessity.
 Visualisation Modes – from computer screens to large concave screens for 3D
visualisations. The examples cover the idea of how the VE can be visually
explored. The final researches presented the use of HMD devices for AR
proposes.

65
  Collaboration – some of examples are for users exploring the VE in a
computer screen without direct interaction with other users. There is no real
sense of presence inside of the VE. The information can be shared but not the
VE. The examples progress up to where users can share information in a
collaborative way. Users can share the same physical or/and virtual space at
same time that they collaborate over the information that is being presented.
 Positioning System – researches have different approaches to calculate the
position of a user, especially when outdoors. This survey exemplifies how
researchers suggested the resolution of positioning. The examples are focused
on GPS solutions once this thesis is working with this methodology for
outdoor positioning.

From numerous projects surveyed, the following projects were chosen for review.

 Interactive Visual Simulations (VizSims) (Delaney, 1999) – Applied to Oil

Exploration – Petrobras.
 TerraVision II (Reddy, 1999).
 VizSims – Applied on Urban Planning (Delaney, 2000, UST, 2004).
 IMAGINE VirtualGIS – ArcGIS 3-D Analyst (ESRI, 2003, ESRI, 2001).
 Shiodome CyberDome (Masanori and Sawada, 2004).
 Cyber Walker 2001 – Wisteria World 2001(Tamura, 2001a, Tamura, 2001b).
 Towards Outdoor Wearable Navigator (Tamura, 2001a, Satoh et al., 2001) –
Air Hockey (Tamura, 2001a) – AquaGauntlet (Tamura, 2001a).
 Studierstube Augmented Reality Project (Interactive Media Systems Group
Institute for Software Technology and Interactive Systems Vienna University
of Technology, 2004).
 Columbia University AR Projects (Columbia University Computer Graphics
and User Interfaces Laboratory, 2004b).
 Tinmith (Piekarski, 1999, Piekarski, 2001b, Piekarski, 2001d, Piekarski,
2001c, Piekarski, 2001a, Piekarski, 2002).
 The Augurscope (Koleva et al., 2001).
 Augmented Reality for Subsurface Data Visualisation (Roberts, 2002, Roberts
et al., 2001, Evans, 2001).

66
  ARMobile, AROnSite, ARWebCam, ARPhone (Woodward et al., 2007a)

VizSims – Applied to Oil Exploration – Petrobras

Delaney (Delaney, 1999) describes on his paper a technology that helps find oil faster
by using a 3D interactive visualisation. The device collects data from the ground
based on acoustic imaging and has its patent dated back to 1923 – US patent
no.1451080. Initially, these devices used strip recorders to record the acoustic signal
that bounced off from underground rocks. Soon, the number of acoustic receivers
increased in order to give a better idea for Petroleum Geologists of what they can
expect to find on a possible oil-bearing ground. The big issues were the volume of
data captured by the acoustic receivers and the noise caused by the reflection of the
signal by the subterranean structures. The data, usually presented as layers of
information in 2D charts, was improved in 3D charts with the quality limited by the
skill of the geologist, the quality of the data and the time available.

In the late 1990s, computers were powerful enough to handle huge amounts of
data in real time. 3D charts were substituted by 3D immersive environments where
data could be analysed and explored in different angles, and teams collaborated with
each other with different expertise. Delaney (Delaney, 1999) cited Loftin – Director
of the Virtual Environment Technology Laboratory at the University of Houston –
described the rationale:

No one person owns all the knowledge. You have five or six people who work together
to manage a reservoir. They have different vocabularies and knowledge. If you put them
together in the same data, they communicate better.

The researchers of the Brazilian Oil Company, Petrobras, have worked with the
previously described system in order to help geologists and engineers to explore oil
on deep oceans.2 With the data from the layers of the ocean ground, 3D images are
projected on concave screens. Stereoscope glasses create the sensation of 3D objects
“flying” in front of the research team, thus allowing the discussion of ways to
perform a new perforation. The system runs on an Onyx with 5 Central Processor
2
This was observed during my visit to this company in 2000.

67
Units (CPU) IP27 250 Mega Hertz (MHz), 8 gigabytes of Random Access Memory
(RAM) and a graphic board InfinityReality2E. Even with the high cost of the
equipment and the area where the visualisation is performed (estimated at that time in
U$2,000,000.00 USA dollars), the cost of positioning an oil platform in the ocean
along several days is even higher. With this system they know exactly where the oil
platform should be positioned according to the acoustic data received. Figure 3.2
shows a selected number of layers of information collected from the deep ocean to be
analysed by Petrobras team. Different colours represent distinct soil layers found by
the survey.

Figure 3.6 Potential Oil Field – Petrobras

The system explores an idea created around 1917 when Reginald Fessended received
a patent for his “Method and Apparatus for Locating Ore Bodies” which also used
acoustic energy to makeup underground formations (Delaney, 1999). Nowadays,
users are placed in a collaborative environment that creates a synergy of knowledge,
dramatically cutting time and budged needed to analyse a block of seismic data, in
some cases from months to hours.

TerraVision II
Reddy et al. (Reddy, 1999) describe the browser TerraVision that uses the
potentialities found in VRML. Their research created the Tile Set Manager
Application Program Interface (tsmApi) where they pointed out four fundamental
design criteria to develop the browser to explore a virtual terrain:

 Scalability – the browser must support large and small data sets.
 Composability – the browser must support multiple types of geo-referenced
data such as images, cultural features, etc.

68
  Efficiency – the easy navigation to explore the VE on a standard VRML
browser must be maintained or a customisation is required to further increase
the navigation efficiency.
 Data Interchange – the development of generic data representations for geo-
referenced data in VRML.

TerraVision II (Reddy, 1999) was created as a solution to retrieve and merge massive
volumes of remotely located data, including aerial and satellite imagery, topography,
weather data, buildings and other cultural features. Such data can be terabytes in size
and could be recovered from multiple servers across the Internet. The interface to the
data used by TerraVision is GeoWeb, a distributed searchable database of metadata
also created by the Stanford Research Institute (SRI) group. TerraVision II constantly
queries the indices for any relevant data in the current view. The system starts the
most suitable interface depending on the kind of metadata returned from GeoWeb.

In order to handle massive data sets TerraVision II does not download the whole
high resolution data once, but instead employs a tiled multiresolution data hierarchy.
The data is divided in rectangular tiles over a range of resolutions creating the
sensation of LoD. This implementation avoids the transmission of dozens of
megabytes over the network, leaving the whole system faster even when working
with highly accurate data. The browser interprets and predicts the user’s movement
over the data and manages to download only the subsequent four tiles in a higher
resolution. The process is repeated if the user wants to explore more information from
a specific area. Unlike VRML, this approach avoids overspending huge network
communication or system processing.

The system allows the user to explore a data set through multiple windows from
different points of view or different combination of sets. Three-dimensional models
can overlay the terrain data adding extra information on the visualisation. Features
including buildings and roads, and even simulations of wind vectors and air
turbulence can be added.

69
 Other features of TerraVision II include (Leclerc et al., 2002): open source,
massive scalable dataset, multiple datasets, gazetteer support, OpenGIS Consortium
Web Mapping Server support, flight path, viewpoint bookmarks, heads-up display,
documentation and cross-platform.

Figure 3.3 shows an example of a TerraVision II application. The image

represents, with a resolution of one metre, United States Geological Survey (USGS)
imagery and elevation for the Menlo Park area. A three dimensional VRML model of
the SRI Campus was superimposed by a texture-map extracted from aerial imagery
(SRI International, 2002).

Figure 3.7 VRML Model of the SRI Campus (SRI International, 2000a)

Figure 3.4 shows an example of an application created through different sources of

information. The view shows an area of Southern California with a high resolution
image inserted around the Camp Pendleton area and a VRML model of wind
conditions for that region.

70
 Figure 3.8 VRML Wind Vector Overlay (SRI International, 2000b)

As described on (Leclerc et al., 2002) “the high performance system enables the user
to browse massive, rich, distributed terrain; view embedded multimedia data; and
collaborate with other users in the virtual 3D space”.

VizSims – Applied to Urban Planning

Interactive VizSims were explored in a paper presented by Delaney (Delaney, 2000)
with the focus on urban planning. His examples started with Bill Jepson who founded
the Urban Simulation Team (UST) at the University of California (UCLA), one of the
pioneers of urban simulation.

The UST (UST, 2004) produces solutions for real time visual simulation in the
field of design, urban planning, emergency response and education. However, the
focus of the group is on the development of a model of Los Angeles (LA), an area of
several hundred square miles (Delaney, 2000). UST claims that the system is
extremely efficient when compared to the time and labour spent to construct a model.
The system can be used to visualise any urban environment. The data that feeds the
digital environment is acquired through aerial photographs, GPS data, street maps,
eye-level photos and three dimensional geometry and creates a realistic 3D visual
simulation of the extremely dense Los Angeles urban environment (Delaney, 2000).

The VizSim eliminates the necessity of blueprints, charts and other “hard-to-
understand” traditional representational methods. The VE explored allows the users

71
to fly, drive or walk-through in real time giving a better impression of what the
architects are doing. On Delaney’s paper (Delaney, 2000), Jepson highlighted the
UST work, saying:

We empower the clients to understand what the architects are doing. The architect wants
to control the information flowing to the client. We come between the architect and the
client. Once the client walks through (the planned building), he feels empowered to take
a larger role in the design process…When (their preference) differs from the client’s
agenda, problems happen. The model shows the problems.

Even with the focus in modelling LA, the UST team has completed a number of
transportation related studies and recreation of historic environments. Transport
simulations run from new metro stations to LA freeways development. Some studies
are also related with the beautification of Los Angeles International Airport (LAX)
(UST, 2003). Figures 3.5 shows an image produced with the results achieved with the
modelling of LA. The enhancement proposed to LAX is exemplified by Figures 3.6
and 3.7.

Figure 3.9 Downtown of LA Modelled by UST Team (UST, 2003)

72
 Figure 3.10 LAX Original Condition (UST, 1998)

Figure 3.11 LAX Proposed Condition (UST, 1998)

The LAX project is a combination of efforts from different teams of architects,

environment graphic designers, landscape, lighting, engineers and art consultants.
The first phase of the project was to register the actual condition of the entire
environment where LAX is placed. Later, the enhancements on the area were
designed and presented to the Los Angeles World Airport’s Board of Commissioners
and at public meetings held throughout the process (UST, 2004).

Delaney (Delaney, 2000) also reports as experiment with the architect Mike
Rosen, in which they created 3D models of houses and office buildings for sales and
planning purposes. When working in a virtual baseball stadium, the mayor of
Philadelphia, Ed Rendell, was faced with a scoreboard obscuring the City Hall. The
solution was to move the scoreboard 40 feet away. It was a simple solution but the
system proved to be useful on situations like this. As Delaney (Delaney, 2000)
affirmed:

73
 …simulation is quickly becoming an essential tool for planners at the town, city and
regional levels. As computers become more powerful, we can expect planet-wide
simulations to extend our understanding of the impact of development of large areas of
the globe.

Interactive VizSim is a term that can be applied in much wider prospects than only
urban planning. The application of VizSim for urban design is a facilitator tool that
makes the ideas introduced by engineers and architects more accessible. Generally,
the public who is affected by the changes produced by CEn schemes have difficulties
in assimilating the information presented on 2D layouts, blueprints, charts, and even
scale models. Significant changes on the VE can be done almost instantaneously,
generating objective and efficient answers.

Another example of how far an urban simulator can be easy to understand is the
game created by Will Wright titled SimCity. Wright started the project in 1985
(SC3000.COM, 1998) and released the first computer game simulation to accurately
model a city in 1989. On a CNN online interview (SC3000.COM, 1998), Will
explained:

…in the first two years I worked on "SimCity," I did quite a bit of research and read 20
books at least on urban planning theory. In addition to that, we did a fair amount of
research into simulation techniques. And for the next version of "SimCity" (2000), I
spent quite a bit of time talking and meeting with various people who had expertise,
including city planners, police administrators, public works people, teachers, etc.

Nowadays, SimCity is a VizSim urban simulator played by millions of people around

the world. The players can delimit the virtual city areas where industry, shops, rescue
teams, etc. will be placed. Roads, power plants, water supply are also allocated and
placed by the players. The high graphic resolution and the dynamic way that the
virtual city evolves contributes to a sense of reality. Figure 3.8 shows a virtual city
named Olympia created by CityScape.

74
 Figure 3.12 SimCity – View from Olympia (CityScape, 2004)

IMAGINE VirtualGIS – ArcGIS 3D Analyst

The urban visualisation tools that have been developed are getting closer to
Geographical Information Systems (GIS) which are, according to (ESRI, 2001): “a
system of computer software, hardware and data, and personnel to help manipulate,
analyse and present information that is tied to a spatial location”. The same concept
is understood by the U.S. Census Bureau (Bureau, 2001) as: “A system of hardware,
software and procedures designed to support the capture, management,
manipulation, analysis, modelling and display of spatially-referenced data for
solving complex planning and management problems”.

The following list gives a better idea of what is involved on a GIS system:

 Spatial location – all the data contained on a GIS database must be

georeferenced. A reliable result depends on reliable surveys of data collection.
 Information – a chart of an area containing different colours representing
distinct layers of information is better comprehended than a long table with
lots of columns and numbers. A visual graph result is one of the goals
explored by GIS. Fast analyses of databases, different combinations of layers
and a precise result in colour charts, maps, or even 3D VEs contributes to the
popularity of this system.
 System – the whole system can be exemplified as a bridge between software,
hardware and data. The hardware encompasses not only the computers that are

75
 part of the management system but also those that capture information that
feeds the database.
 Personnel – they are the people involved in exploring the potentialities of GIS.

It is not the intention of this thesis to conceptualise GIS. The objective is to describe
GIS as an information source to be explored on VEs. The results can be presented to
the users in large or reduced scales depending on the application and general
specification. Urban simulators are huge databases with the shape, locations and
textures of buildings. Telephone and electrical cables, water pipes, street planning,
population density, the number of traffic lights; all are examples of data that can be
embedded in the visualisation. Figure 3.9 introduces an idea of a GIS visualisation,
exploring 3D potentials. The image was created by a combination of layers of
information as the Digital Terrain Modelling (DTM), boundaries of the forest, water
level, position of the fences, and the position of energy cables. This VE is explored
with the help of a VRML browser.

Figure 3.13 Visualising Data from a GIS (Capra, 2000)

The following two examples are commercial products designed for GIS applications.
The purpose is to demonstrate how this technology uses VR for visualisation.

IMAGINE VirtualGIS by ERDAS is an example (Leica Geosystems LLC., 2001)

of a package that allows the user to make an accurate registration between the DTMs
and images acquired by aerial photography, satellite imagery and airborne scanner
data. Users can add annotations, vector GIS layers and symbols, and can create a VE
with significant realism by adding texture on 3D objects. By using mouse control, it is
possible to navigate through the VE, exploring different points of view – as if the user

76
had the freedom and eye-sight of a bird – and to record this movement to create a
movie. ERDAS improved the LoD algorithms in order to minimise the effect when
presenting complex polygons in the DTMs. The software can render datasets with
different resolutions that can be merged together on the same visualisation. The
screenshot shown in Figure 3.10 is an example of results produced by the software.
Some advantages exposed by ERDAS include:

 Terrain Database Export – the software can export the objects and its
proprieties to VRML, OpenFlight (OSG Community, 2004) and RAW
(Wikipedia, 2008d).
 Performance Setting – the users can control the quality and performance of the
system to the level that they would like to achieve.
 Animation Tool – the user can create movies simulating aircraft flight paths,
ground vehicle routes, shipping lanes, etc.

VirtualGIS is a complementary package of a full system named IMAGINE. The full

system is composed of: Advantage, Essential, VirtualGIS and Analyst.

Figure 3.14 Textured 3-D View of Munich (ESRI, 2004b)

ArcGIS 3D Analyst (ESRI, 2004b) is part of a complete system GIS named ArcGIS.
This is a browser that integrates the ArcScene application, providing an interface to
view multiple layers in a 3D view. With ArcGIS 3D the users can analyse 3D data

77
from several viewpoints to create a realistic perspective, draping raster and vector
data over the DTM. The release 9.0 (ESRI, 2003) can manage extremely large set of
3D geographic data. As an extension to the ArcGIS Desktop products, the system
allows users to leverage the GIS analysis tool available in ArcView, ArcEditor and
ArcInfo to perform geoprocessing tasks in a 3D environment (ESRI, 2003). Some of
the characteristics of ArcGIS are:

 It can work with both vector and raster data.

 It can create 3D fly-through animations.
 It can apply various effects to layers of data, including: transparency, lighting,
shading and priority on layers overlap.
 It allows the visualisation of multiple perspectives simultaneously.

Figure 3.11 shows a screenshot of the ArcGIS 3D Analyst.

Figure 3.15 ArcGIS Screenshot (ESRI, 2004a)

These two tools introduced are a subset of a complete GIS package composed of
several other tools. Both IMAGINE Virtual GIS and ArcGIS 3D Analyst are in fact
browsers and interfaces between the queries made to the dataset and the 3D
visualisation. The major advantage is the full integration between the packages,
making it possible to combine 2D and 3D data. The software (Sapeta, 2003) creates
an integrated, real-time visual interface for the analysis, mining, linking and

78
visualisation of geographic and related data sources. Also, image registration is a
strong component on these tools. It enables the production 3D DTMs with
photogrammetric works placed upon the model, providing extra realism to the scene.
As cited by Sapeta (Sapeta, 2003), the next generation of GISs with applications for
urban analyses are limitless and can include: simulation on emergency and disaster,
land development, police operations, urban planning, environmental impact,
transportation planning, telecommunications, etc.

Shiodome CyberDome
Apart from 3D digital environments being explored on computer screens,
stereoscopic visualisation presents another possibility for users to be immersed and
surrounded by 3D objects.

Matsushita Electronics Hemispheric Dome – Shiodome CyberDome – is one of

the largest immersive projection based VR systems in the world. The users receive a
pair of polarised glasses at the beginning of the presentation. The dome is a hemi-
spherical screen of 8.5m in diameter, displaying to the users a Field of View (FoV) of
180 degrees on the horizontal and 150 degree vertical. On the background, there are
18 liquid crystal projectors, one master Personal Computer (PC) and 18 rendering
PCs computing the 3D environment.

The high quality stereoscopy projection makes a perfect match between the
screens projected by each projector. The software-based distortion-correction
technology does not require an expensive special lens, and the projector placement,
expressed in parameters, enables projection on a screen dome of any size (Masanori
and Sawada, 2004).

In a visit to Shiodome, the author could experience the illusion of being immersed
in the VE is convincing 3. In on of the possible applications, the users can explore part
of a city going between buildings, streets and submerging into a river. The sensation
is that of a real 3D scale environment, rendered in real-time and commanded by a
logical and simple user interface. The interface consists of a ball on a stand where just
half of the sphere can be seen. This control allows the forward, backward, left and

3
Based on my visit to this complex on December 2003.

79
right movements, without any body movements from the user, who faces ahead
exploring the environment by moving just the sphere. Around the hemisphere there is
a disc that controls the right and left turns. Finally, a stick shifts the view up and
down. Everything is easy to understand and control, so that users are confident
enough to have a close experience and to manipulate it.

Figure 3.12 shows a layout of how the CyberDome looks and where the
participants are placed. The users are free to explore the 3D VE and to contribute new
ideas based on what they see. If the visualisation is related to a new urban plan, for
example, the engineers, architects and the other groups involved can maximise effort
to find a better solution for ongoing problems.

Figure 3.16 Shiodome CyberDome Structure (Matsushita Electronic Works Ltd, 2003)

Figure 3.13 shows three different users’ interface to control the movements inside the
VE. Each interface is strictly related to the approach where the 3D VE is being
applied. The frame on the left illustrates the ball interface explained previously. On
the middle and on the right, respectively, a virtual walk and a scooter ride are
explored as interfaces.

80
 Figure 3.17 Shiodome CyberDome User’s Interface (Matsushita Electronic Works Ltd., 2003)

Shiodome CyberDome is a huge state-of-the-art interface created to explore 3D VEs.

As mentioned on Virtools (Virtools, 2004), the core application of this CyberDome is
the "VR Environmental Project Support Too", which is used for reviewing and
consensus-building in CEn and urban developmental projects by construction
companies, engineers, planners, designers, residents and users. The range of
applicability is unlimited, as are the ways to explore the user’s controls over the VE.

Cybercity Walker 2001 and Wisteria World 2001

The Cybercity Walker 2001 by Tamura and Kotake (Tamura, 2001a) uses AV
technology to generate a large-scale virtual city. The basic idea was to equip a car
with several sensors, including seven video cameras on its roof. The cameras were
distributed around a circle on the roof of the car to progressively record images at the
same time that the car is driven around the city. The other sensors indicate the car’s
position and the angles that the images are being captured.

The visualisation is done on a widescreen projection that can change the point of
view when the user simulates walking on the street or driving a fast car. The velocity
of the projection depends on the amount of force that is being applied to the controls.
Virtual buildings can be added to the scene for city planning purposes and virtual
tours.

Another example introduced by Tamura is the Wisteria World 2001 (Tamura,

2001a), which incorporates MR functions into a telepresence system. Instead of using
pre-captured images, the users control a remote video camera pointed at a 1/87 scale
model. The scale model is precise enough to resemble a city. The simulation of
walking and bird’s eye viewing can be seen on a widescreen projection. As the
previous project, virtual buildings can be added to the scene.

Both projects work with widescreen projection and can be easily associated with
CEn applications. Some enhancements, such as the development of a dedicated
hardware to render the images and a robust registration scheme can also described in

81
the paper (Tamura, 2001a). As illustrated on section 3.6.5, the advantage of
projections on widescreen is noticed by the number of people in the audience.

Towards Outdoor Wearable Navigator - Air Hockey - AquaGauntlet

The projects discoursed so far have introduced VEs through computer screens or
widescreens with or without stereoscopy. The following sections will explore projects
related to AR. Even though some of them are not linked to CEn applications, they are
helpful to exemplify the basis of the research proposed of in this thesis, at present, to
seamlessly join VE and RE.

“Towards Outdoor Wearable Navigator” (Tamura, 2001a, Satoh et al., 2001) is a

mobile system where the user is equipped with a HMD and optical fibre gyro, and a
portable PC. Instead of the users being tracked by a GPS receiver for example, they
announced a system that uses vision-assisted registration to adjust the drift of the
optical fibre gyro and presently performs with one of the best spatial registration
accuracy in the world. However, the users can only visualise the environment on pre-
surveyed places. There is no flexibility to walk around exploring the VE.

Applications including the insertion of new features over specific points in the
city were demonstrated in Tamura and Satoh (Tamura, 2001a, Satoh et al., 2001).

Tamura and Yamamoto (Tamura, 2000, Yamamoto, 1999) introduced the

experience of a case study of collaborative AR. They developed the AR Air Hockey,
a game where the players hit a virtual puck with mallets on a table. Thanks to a see-
through HMD, the players can recognise the opponent’s movements (on the real
world) at the same time that they see the virtual puck. A sensor and a camera are
mounted on the HMD to register the user’s movements. After recognising the
player’s movements, the system produces the behaviour of the virtual puck and shares
the information on a VE. The authors claim that AR Air Hockey became the see-
through AR system experienced by the largest number of users worldwide.

Tamura (Tamura, 2001b, Tamura, 2000) also comments on a multi-player MR

entertainment entitled AquaGauntlet. Their objective was to create a system able to
support more than three players, multiple virtual objects, occlusion between virtual

82
and real, special sound effects, gestures’ recognition, etc. The players’ movements are
logged through magnetic trackers positioned on the HMD and on gloves. The HMD is
not a see-through type but a block view. The real environment is recorded by a
camera placed on the HMD and then blended on the VE before it is displayed on the
glasses. The gloves track the arms’ movements, which are recognised by the system,
resulting in different actions to attack the virtual monsters.

These three projects exemplify the multipurpose use of AR. The project entitled
“Towards Outdoor Wearable Navigator” only works on delimited areas of the city.
The users always need to stop over a pre-surveyed base point. Then the users need to
tell the system which base point they are stepping over. By choosing this
methodology, there is no need of a GPS receiver. Positive points on the research
presented by Tamura and Satoh (Tamura, 2001a, Satoh et al., 2001), are as follows:

 The wearable equipment is considerably light.

 There is never the problem of GPS signal being blocked or multipath
distortion.
 There is no need for radio links to use DGPS or RTK-GPS.

However, there are also the following disadvantages:

 The users must stop at one of the base points in order to identify their location.
 The system does not allow the exploration of collaborative environments.

Air Hockey and AquaGauntlet are two examples of AR systems which allow the
exploration of CVE. Users are immersed on the system due to the large amount of
interactivity and involvement between players and the objects to be hit. Another
positive aspect is the development of interaction in the VE through recognition of a
user’s arm movements. In AquaGauntlet (Tamura, 2000), for example, the users have
three gestures commands: ready, fire and defence in order to attack virtual jellyfishes,
sharks and rockets.

Studierstube Augmented Reality Project

83
The Studierstube (Columbia University Computer Graphics and User Interfaces
Laboratory, 2004b) is a system that interfaces humans and VE in a CARS context. It
was named after the study room where Goethe's famous character, Faust, tried to
acquire knowledge and enlightenment. This system has been widely explored over the
years, having been involved in around forty projects, according to its website. The
most relevant ones are described below.

The Mobile Collaborative Augmented Reality is a system created to combine 3D

computer generated stereographic images, interaction with virtual objects,
collaboration and data sharing between the users (stationary or not). The application
integrates a see-through HMD, a web camera, an orientation sensor to be placed on
the user’s helmet, a notebook, a pen and a tablet with markers (Reitmayr and
Schmalstieg, 2001). Based on ARToolkit (Human Interface Technology Laboratory
at the University of Washington, 2004), this system processes the images captured by
a camera looking for specific targets. The system overlays the recognised image by a
virtual object every time that it recognises a target.

The user’s movements are also recognised by the system through the cameras and
acted over the virtual objects. In one scenario, two players with different points of
view sit at a virtual chess table and manipulate virtual pieces. The Studierstube is the
platform used by the players to share the VE.

Reitmayr and Schmalstieg describe a CARS for outdoor navigation and

information browsing (Reitmayr and Schmalstieg, 2004a). The equipment used to
perform the AR system outdoors does not differ much from the one presented on the
previous project. Additional integrated hardware consists of a GPS receiver and a
wave-LAN network adapter to enable communication with a second rover user. As an
example of the application, they developed a tourist guide: the system helps the
tourist by indicating the best route to visit interesting locations spread in an area of
the City of Vienna, and provides access to useful information related to the area.

Information about the city is presented to the users in different ways on the HMD,
working as an enhancement or complementary information over a building, images or
even text. The users interact with the interface through a 2D touchpad placed on the

84
user’s belt or carried as a handheld. All the devices connected to the notebook are
managed by the Studierstube, as the collaborative environment.

Figure 3.18 Additional Information Overlaying a Historic Building

(Reitmayr and Schmalstieg, 2004b)

Figure 3.14 depicts an AR view of a tourist attraction overlaid with extra information.
The yellow shapes indicate that more information can be acquired about the site
through the user’s interface.

Figure 3.15 illustrates a box containing text and images supplied through the AR
visualisation when the yellow shapes shown on Figure 3.14 are explored.

On their website (Interactive Media Systems Group Institute for Software

Technology and Interactive Systems Vienna University of Technology, 2004) it is
possible to explore other projects. There are researches involving: AR aided surgery,
AR locator for publications in a library, VR slide show, occlusion in collaborative
augmented environments, etc.

85
 Figure 3.19 Additional Information Explored by the Users (Reitmayr and Schmalstieg, 2004c)

In the collaborative AR for outdoor, introduced by Reitmayr and Schmalstieg

(Reitmayr and Schmalstieg, 2004a), the authors mentions the low accuracy on GPS
positioning. The problem could be reduced, but not completely solved, once the
DGPS feature is enabled on the receiver (see more about this subject in §3.7).

Columbia University AR Projects

Columbia University (Columbia University Computer Graphics and User Interfaces
Laboratory, 2004b) has introduced an AR project named Spaceframe (Webster et al.,
1996). The project is a test bed of a spaceframe construction aided by an AR system
that indicates which piece of a collection should be manipulated in order to assemble
a diamond shape. The pieces look very similar, generating confusion about which
should be handled by the users. The hardware used consists of a see-through
stereoscopic colour HMD with integral headphones and orientation tracker. The
position tracking is provided by an Origin Instruments DynaSight (Origin
Instruments, 2002) optical radar that tracks small LED targets on the HMD
(Columbia University Computer Graphics and User Interfaces Laboratory, 2004a).
The system uses verbal or textual instructions to guide a user to a pile containing
required parts. It sends confirmation through a barcode reader on whether the user has
the correct piece, as well as indicating how to assemble the component. At the same
time that the audio and text instructions are conveyed to the user, a virtual object is
positioned where the frame should be placed.

86
 Feiner (Feiner, 1997) also described the Touring Machine, created to combine 3D
graphics of AR with the freedom of mobile computing. The hardware design is based
on a DGPS receiver, a see-through HMD, an orientation tracker, a notebook, a
handheld PC and a radio modem. This project was developed under the Mobile
Augmented Reality System (MARS) concepts (Höllerer, 1999).

The application scenario in Feiner’s paper (Feiner, 1997) introduces a user who is
wearing the equipment and is standing in the middle of Columbia University. Around
the user, buildings are labelled with virtual texts displayed on the see-through HMD.
The interface between the user and the system is composed of a menu displaying a set
of options. With the help of a touchpad, mounted on the back of the handheld, the
user can select the options that give information related to the buildings around,
departments and the user’s location. Finally, the chosen option is sent as a Uniform
Resource Locator (URL) to a web-browser running on the handheld. The system can
communicate via wireless with the web-servers to collect any data that is not cached.

If the user stops for a short time in front of a building for example, the label
displayed on the middle of the screen changes from grey to yellow and then to green,
opening a second menu bar, which offers more links about the place. When the
building is selected an indicator remains pointing at the position, then it changes
colour, from green to red, if the user’s head turns more than 90 degrees from the
target.

The system runs on two different platforms: the COTERIE distributed VE

infrastructure, based on Modula-3 (Wikipedia, 2008c), for the outdoor backpack
system and its indoor simulation version, and Java/Java 3D (Sun Microsystems,
2008) for the indoor desktop and immersive user interface, as well as for the main
database interface (Höllerer, 1999).

Improvements made in related projects, including the positioning of labels and

information on the VE, are described on Bell (Bell, 2001). The distribution of
information and annotations inside the VE needs to be displaced in a way to avoid
self-occlusion. Excess information can also cause confusion in the VE, making it

87
almost impossible to navigate through the real and virtual data. The paper describes
the study of positioning of the information, size, transparency, priority, etc.

Höller (Höllerer, 2001b, Höllerer, 2001a) introduced in his paper the early stages
of an experimental mobile AR system that employs technologies to position the users
in indoor and outdoor areas. These technologies include a ceiling-mounted ultrasonic
tracker (used on covered areas) and a RTK-GPS system for open areas. For areas out
of range of both systems, he described the results of the development of a dead-
reckoning approach that combines a pedometer and orientation tracker with
environment knowledge expressed in spatial maps and accessibility graphs.
According to the example cited in his paper (Höllerer, 2001b), the results achieved
with the combination of pedometer and inertial orientation tracker were strongly
disturbed by the magnetic field inside of the building. In the second example, an
inertial orientation tracker was used and the results were more convincing, apart from
the linear degradation of the orientation information due to drift. Finally, good results
were achieved when using the combination of pedometer, inertial orientation tracker
and environment knowledge. The system is based on a map of the area where the
users is walking. It checks whether there is the possibility of a collision or if the
directions given by the sensors are just noise caused by orientation drift.

Spaceframe (Webster et al., 1996), is an example of an AR system designed for

potential industrial applications. Any kind of situation where the users need to
integrate their skills with instructions provided by a machine can be shaped on the
base of this project. Examples of its application are: automobile industry, aerospace
industry, CEn, etc. The system is being modified to support more than one user
working on the same frame in real-time, as stated on the website (Columbia
University Computer Graphics and User Interfaces Laboratory, 2004b).

The Touring Machine (Feiner, 1997) introduced an AR application with a suitable

user interface. The evolutions described on Bell (Bell, 2001) and Höller (Höllerer,
2001b) complements the issues of an AR system where the GPS signals are not
available.

Tinmith

88
The Tinmith system (Piekarski, 1999, Piekarski, 2001b, Piekarski, 2001d, Piekarski,
2001c, Piekarski, 2001a, Piekarski, 2002) from the University of South Australia is a
complete architecture project developed for AR purposes. As a wearable system, the
equipment attached to the notebook does not differ much from what has been
previously introduced in the other AR systems.

The main differential is in an interface approach named Tinmith-Hand. Custom-

made pinch gloves are used to control a finger-mapped menu through sensors placed
on the tip of the fingers. The menu system handles eight complex pre-programmed
tasks displayed in the bottom of the see-through HMD. Also on the extremity of the
thumb in the gloves there are two targets (two small squares with a symbol painted in
black and white) that are recognised by the system through to ARToolkit’s (Human
Interface Technology Laboratory at the University of Washington, 2004) image
recognition. The images captured by a web camera mounted on the user’s head are
analysed by the system. Each time that ARToolkit recognises the target it calculates
the position according to the tilt sensor, also placed on the user’s head. Figure 3.16
shows the wearable system.

Figure 3.20 Tinmith Wearable System (Piekarski and Thomas, 2002)

With the glove’s targets locked, the user can explore the enhancements achieved by
Tinmith-Metro (for outdoors environment) and Tinmith-VR (for indoors
environment). Tinmith-Metro (Piekarski, 2002) allows users to capture the outdoor
design of a building and its structures using beyond arms reach manipulation
techniques. Through Constructive Solid Geometry (CSG) the users can shape

89
complex objects using their hands. The results can be achieved from unions,
intersections, or differences between primitive 3D objects (as pyramids or spheres,
for example). The user has a choice to position and select the object using 2D or 3D
techniques (Piekarski, 2002). If the object to be designed is more complex than can be
made with the primitive forms, the users have the option to use the “infinite planes
technique” (Piekarski, 2002, Piekarski, 2001d). Two examples are given in order to
clarify how the user can design the objects:

 Given the location of the thumbs (Piekarski, 2002), the system overlays
registered 3D axes. At the same time, a 2D flat cursor is overlaid on the top.
After the user has positioned the cursor with his hands it is possible to activate
the select mode using one of the pre-programmed tasks offered in the menu.
The users create a virtual mask that superimposes the real object selecting one
point for each corner that shapes its boundary. The plane created can be
extruded in order to be modelled, with the same dimension of the original, or
assembled with other planes created during the process.
 A 2D cross-named eye cursor is displayed in the centre of the see-through
HMD. The user needs to match the cross with the corners of the object and
select the point. The final result is an object with irregular shape.

The process to acquire the planes is repeated until the whole building is modelled.
Once set, the planes can be grouped in just one object and processed to be combined
with another one. When the model is completed, the VE is fed with the new virtual
object.

The Tinmith-VR system is used for indoor environments and has the same
modelling techniques offered by the Tinmith-Metro system.

As cited on Piekarski and Thomas (Piekarski, 2002), the graphical quality of

objects created is largely limited by the accuracy of the tracking hardware employed.
DGPS has been used for positioning, with a claimed accuracy that ranges from 50
centimetres to 5 metres.

Augurscope

90
The Augurscope (Koleva et al., 2001) has been developed at the Mixed Reality
Laboratory (MRL) as part of the Equator (Equator, 2005) project. It is a portable MR
interface for outdoor use, supporting both AR and AV.

The Augurscope is a wheeled stand designed to carry equipment involved in the

positioning and display of the VE. On the base of the wheeled stand there is a CPU,
which is powered by batteries, which are covered by a shell. Arising from the centre
of the base, an arm supports a screen which functions as a window to the VE. This
small platform is movable. A compass and a tilt sensor are used to give the
orientation of the Augurscope. A DGPS receiver calculates its position. The sensors
are integrated on the screen of a CPU that has a wave-LAN card transmitting the
information to the computer placed on the base. The data are computed, rendered and
then re-transmitted to the display.

The system has an attractive and user-friendly interface which facilitates the
manipulation and exploration of the VE. The users can manoeuvre the wheeled stand
with the advantage of not having to carry any equipment in a backpack. Figure 3.17
shows the Augurscope.

Figure 3.21 Augurscope (Benford et al., 2003b)

Currently the project has its test-bed at Nottingham Castle, where users can see and
explore how the outdoor environment of the castle was in the past. For each direction

91
that Augurscope points the user can see the walls, houses, and towers that were once
placed there.

Two potential applications for the Augurscope are: cultural heritage and
environmental planning. In terms of planning, it could be used to explore scenes from
the future as a consultation over designs for new buildings, transport systems and
other public facilities.

Augurscope is marked by the design. It is an easy-to-handle device, which almost

invites the users to play with it4. The intuitive handle allows even children to
manipulate it. If the user does not like to use the wheeled stand the display can be
detached from the stand arm’s support. Because its communication is done via wave-
LAN with the base, the display always stays within range of the wave-LAN
transmission.

The display tilt has been exaggerated so that for every sensed degree of tilt two
degrees are rendered (Benford et al., 2003a). Additionally between 20˚ to 45˚ the
camera pulls upwards. At 45˚ (the limit of sensible movement), the camera tilts to
90˚, raising several metres into the air giving a bird’s eye view to the users. This
procedure not only masks the limits of the tilt sensor as well as helps the users to
locate areas that were not yet explored.

The lack of accuracy of the DGPS was substituted by an interpolation between the
previous point and the actual positioning. Thus, with each three or four metre
movement the system computes the new positioning. For instance, if the user moves
only one or two steps and stops, the system does not compute this change of position.
Whereas if the user pushes the trolley two more steps and again stop, the system
identifies this movement, not as a drift of the DGPS system, but as a real change of
user’s position.

Augmented Reality for Subsurface Data Visualisation

The Institute of Engineering Survey and Space Geodesy (IESSG) (IESSG, 2008) and
the School of Chemical Environmental and Mining Engineering (SChEME) have
4
Based on a demonstration on October 2003

92
developed in cooperation a prototype of an AR system aiming to show the location of
underground pipes for engineers and workers involved in a excavation. As cited in
Roberts (Roberts, 2002), the risk to a worker of striking a live power cable is very
high when there are no plans describing what lays under the ground. This situation
has already claimed eighty six lives and caused more than three thousand injures over
the last ten years – based on a study conducted by the Automobile Association of
Great Britain in 1997. Even if 2D-charts exist, they can be misinterpreted, and the
cost per day at digging a hole in the street is high.

The developed system integrates RTK-GPS and an Inertial Navigation System

(INS), enabling centimetre-level positioning accuracy. Apart from the receiver, which
weighs 1.2Kg, the other equipment involved is very light and compact, which makes
the whole set suitable to be carried in two small packs. Figure 3.18 shows the RTK-
GPS receiver placed in the user’s backpack. Another pack then contains a
minicomputer connected to a pair of digital goggles, a radio antenna, a digital
compass and a tilt sensor.

Figure 3.22 AR Sub-Surface System (Roberts et al., 2004)

Slightly different from the see-through HMD, the digital goggles have a camera on
the front, where the RE is recorded. The images received from the camera are placed
as textures over a digital rectangle, wide and far enough to be set as background on
the VE. Between the background and the user’s view the VE is positioned also as a

93
digital image, respecting the angles of the compass, tilt sensor and RTK-GPS
positioning. Figure 3.19 shows the VE projected over the RE.

Roberts mentions two other applications for the project, which have been tested
in a talc mine in Luzenac, France. The first idea is to supply information through the
goggles that is not easily visible for workers in the mine, such as faults, aquifers, old
workings, obscured rock surfaces, etc.

The second idea is the visualisation of a grid with the geology of the terrain
superimposed on the mine face and the visual location of different minerals found but
not extracted from the ground.

Figure 3.23 VE Projected over the Real Environment (Roberts et al., 2004)

It is clear that the system, with its focus on sub-surface work, could be used in the
same way to explore any sort of AR environment. The author had the opportunity to
test the equipment. The system passed the impression of being sufficiently precise for
the proposed objective.

Dr. Hollands is one of the collaborators in the development of AR for Subsurface

Data Visualisation. In an informal interview, the differences from this project to the
proposed research introduced in this thesis are:

94
  The visualisation approach is slightly different. As previously explained, the
images projected onto the pair of goggles are totally digital, making this
quality dependent on the quality of the camera and displayers.
 The platform used for the development of the system was DirectX and the
source code is not in the public domain.
 There is no collaboration between users. The equipment runs in a stand alone
mode. There are no audio or video channels for the users to exchange ideas
and points of view.

ARMobile, AROnSite, ARWebCam, ARPhone

The Technical Research Centre of Finland (VTT) worked over several researches
exploring VR and AR in CEn applications. In the paper (Woodward et al., 2007a)
they document the use of applications like ARMobile, AROnSite and ARPhone in the
planning and construction phases of VTT’s new head offices named Digitalo.

Using basically off the shelf hardware devices the system explored allowed users
to explore a VE outdoors. For the positioning perspective they report the use of image
pattern recognition through ARToolkit (Human Interface Technology Laboratory at
the University of Washington, 2004) placing big printed markers in the streets to be
recognised by the system. This solution is combined to another system that also
indentifies other features in the RE to help to stabilise the system – when markers are
not detected.

ARMobile (Figure 3.20) is a system created over a miniature PC enabling

rendering of 3D models, adding new features or light adjusting in real time. The paper
describes the enhanced version of ARMobile called AROnSite (VTT, 2008,
Woodward, 2007) that does not require markers to perform outdoors AR but a GPS.
With this system, users are able to place features (i.e. a building) in a specific place
using Google Earth (Google, 2006) and explore it using the AR system.

95
 Figure 3.24 ARMobile (Woodward et al., 2007b)

ARWebCam (Figure 3.21) is an experiment based in a web camera placed over the
roof of one of the buildings that surrounds the construction of Digitalo. The system
had some knowing land marks and superimposed virtual images of the new building
during the construction phase. Users could compare the actual development of the
construction with what is really done or how the place will look after the conclusion
of the works. The video streamed from VTT’s website already contained the VE
blended, making no requirement of an extra software in the users side.

Figure 3.25 ARWebCam (Woodward et al., 2007c)

ARPhone is a prototype of a system aiming to delivery AR models in pocket devices

like mobile phones. They are expecting to delivery 3D view of objects in real time
once such devices become more powerful and common between people. Users will
need only to point the phone camera to the place where the building is being planned
to see how the environment will look in the future.

96
3.7 Global Positioning System

The presented thesis was co-funded by IESSG at The University of Nottingham (The
University of Nottingham, 2008). The institute is well known as a centre of
excellence in satellite navigation and positioning system, photogrammetry, remote
sensing, sensor integration, and GIS. The idea for this project was created inside of
IESSG and GPS as positioning system become a natural choice to track outdoor users
when testing the CARS created. It is part of the technology review a research to
compare most used GPS approaches on AR projects. The results guided this research
to the RTK-GPS methodology due to:

 Accuracy – RTK-GPS can provide coordinates with approximately 5cm in

real time. No other system outdoor can provide such accuracy with such
reduced infrastructure.
 Infrastructure – to make RTK-GPS coordinates available the user need a
reference and a rover equipment – in some cases the reference equipment is
substituted by augmented systems. Both equipment should not be more than
20Km far from each other for the radio transmission of the corrections. There
is no need of triangulation, cameras for partner recognition, several cell
signals, etc.
 Availability – GPS signal works 24h days, 7 days week in any weather
condition.
 Size of Equipment – compared with other high accuracy GPS systems like
DGPS the size of the equipment is the same, as well as the weight.
 Restrictions – RTK-GPS works over the GPS signal. If there is a limitation to
receive the signal like multipath for example, the same limitation will affect
other GPS methodologies.

The following paragraphs are a review of GPS methodologies. This review is

important once it aims to inform the reader (especially those without strong GPS
background) the basic principles of GPS that eventually are misunderstood or not
comprehended at all. GPS has a great value but also has its limitations. This sub-
chapter was created to “briefly” explain why there are limitations and how different

97
GPS methodologies can provide results with so big differences in accuracy. Some
tests were performed to demonstrate how far the coordinates could be when using
different methodologies.

3.8 GPS Frequencies and Data – Brief Review

The GPS was primarily designed to provide worldwide instantaneous positioning to

US military and allies. It was developed by the United States Department of Defence
(USA-DoD), and it reached Full Operational Capability (FOC) on 27 April 1995.
GPS is operational 24 hours a day and as a passive method it allows a virtually
infinite number of users in any weather condition. At the moment there are twenty-
nine operational satellites in orbit (United States Naval Observatory, 2005). There are
different techniques to handle the GPS signals for civilian use. Each technique has
been devised to fulfil distinct necessities of the user with different equipment
configurations and at varied costs.

The DoD of USA created the GPS system for military purposes. However, a
second service was also created for civilian users. Following its original purpose, the
military system is known as the Precise Positioning System (PPS) and the civilian
system is called the Standard Positioning System (SPS).

Non-military receivers are enabled to sense GPS signal through L1 and L2

frequencies. L1 frequency contains three different types of data: Coarse/Acquisition
(C/A code) and Precise (P) code, the ephemeris and the almanac. L2 frequency
contains the same data but not the C/A code.

The P code is encrypted (Y code – anti-spoof) which makes it partially useless for
civilian users. The encrypted P code is known as P(Y). The C/A code is composed by
a sequence of 1023 “0s” and “1s” also called chips. The P code (without the Y code)
is 23547x10 14 chips long. Both C/A and P(Y) codes are PRN and they are
deliberately added to the signal. Then the signal becomes more reliable against noise
and spoofing. The C/A code is repeated every millisecond and it is different for each
satellite. Due to its length, the P(Y) code can be transmitted for 266.4 days without

98
repetition (Weimer, 1998). Instead of making the transmission continuously, the code
is broken in 37 segments (one for each expected satellite) and reset every seven days
to prevent non-military users to understand and to break the code. PRN is not truly
random. The codes are known due to their retransmission over a period of time. Both
C and P timing-codes are not a secret but the problem lays over the Y code, which is
unknown. Nowadays, the algorithms in the receivers that are able to sense both L1
and L2 try to extract the Y code from the P code, making it as useful as possible for
timing and positioning.

Each satellite sends its status on the ephemeris data. It does not contain only the
satellite’s “health” but the current date, orbit parameters and timing.

The almanac of each satellite contains in a simplified form, an approximation of

its orbit over a period of time and it is logged in the receivers. This information is
fundamental when a GPS receiver is switched on due the need to know for example
which satellites are crossing the sky at that moment. The observation centres on Earth
often update the satellites’ almanac in order to improve the computation of its
position by the receivers.

The military use of the GPS signal enables the receivers to understand the binary
code (Y code – anti-spoof) contained in the P(Y) code. The length of the P code helps
the receivers to be more precise due to the algorithms embedded in hardware that can
identify the location of the received segments along the entire code.

The frequencies L1 and L2 are always the same. The differences are in the
approaches used to work with them.

3.9 Stand-Alone GPS Positioning

GPS receivers can be single-frequency (tracking only L1) or double-frequency

receivers (tracking both L1 and L2 frequencies). It depends on the hardware, on the
embedded algorithm and on the type of antenna that are in the set.

99
 The receiver in stand-alone mode uses the signals from GPS satellites to compute
position without any sort of correction coming from other receivers or geostationary
satellites. As any other receiver, it needs at least the GPS signal from four satellites to
compute its three-dimensional position.

Stand-alone positioning is based on the C/A-code and on the navigation message.

The C/A-code is extracted from the signal received and aligned – cross correlation
process – with a copy of the same code produced by the receiver. The difference in
the alignment is the time taken by the signal to travel from the satellite to the receiver.
This calculated amount of time can be converted to range if multiplied by the velocity
of the signal propagation, however it is contaminated by several error sources as
mentioned in (Bingley, 2003), such as atmospheric delays biases and clock offsets.
This is why such ranges are known as pseudo-ranges. The atomic clock on the
satellite is much more accurate than the quartz clock embedded on the GPS receiver,
so with the C/A-code aligned, the algorithm must consider the satellite’s time frame
and the receiver’s time frame. Figure 3.22 (Kaplan, 1996) shows an overview of what
must be considered on the range measurement timing relationships:

Figure 3.26 Range Measurement Timing Relationships (Kaplan, 1996)

Where:

t = geometric range time equivalent

Ts = System time at which the signal left the satellite
Tu = system time at which the signal would have reached the user receiver
without t D (theoretical)
T u
'
= System time at which the signal reached the user receiver with t D
t = offset of the satellite clock for system time (advanced is positive;

100
 retardation (delay) is negative) – this information is transmitted in
the ephemeris
tu = offset of the receiver clock from system time
Ts  t = satellite clock reading at time which the signal left the satellite
Tu'  t u = user receiver clock reading at time when the signal reached the user
receiver

The pseudo-range is then calculated as shown in the Equations 3.1 to 3.3.

t  Tu  Ts

Equation 3.1

pseudo range  Tu'  t u   Ts  t 

Equation 3.2
pseudo range  t  t D  tu  t

Equation 3.3

The receiver clock offset determination does not solve the problem of error inherent
on the signal. The description of t D is given by equation 3.4 (Kaplan, 1996). The
Equation 3.1 shows the other error sources involved on the total time offset:

t D  t atm  t noise&res  t mp  t hw

Equation 3.4

Where:

t atm = delays due to the atmosphere

t noise&res = receiver’s noise and resolution offset
t mp = multipath offset
t hw = receiver’s hardware offset

The biases caused by the atmosphere effects on the GPS signal can be partially
mitigated by mathematical models, like the Klobuchar model, embedded into the
hardware of the receiver. The Klobuchar models’ parameters are filled by the
information received in the ephemeris, achieving up to 50% of success in the
reduction of error.

101
 Even with this number of errors sources over the signal, companies that produce
GPS receivers claim accuracy between five to twenty-five metres on single frequency
receivers when in clear view of sky. The error is even smaller when the receiver can
use the signal from augmentation systems as discussed in §3.12.

The pseudo-range errors can be reduced but not completely solved, even when
using double-frequencies receivers. In this case the receiver uses the carrier phase to
calculate errors caused by ionosphere delays. Sub-chapter 3.10 gives a review of the
carrier phase approach.

For further details on how to reduce the error to a minimum in single frequency
receivers the reader is recommended the following literature: (Kaplan, 1996, Bingley,
2001, Aquino, 1992, Aquino, 1988, Aquino, 1999).

3.10 Differential GPS – DGPS

DGPS is by far the most frequently used positioning system in AR system

development. There are handheld GPS receivers on the market able to compute and
transmit DGPS corrections. Some of them are cheaper and lighter when compared to
the receivers that will be introduced in §3.11. According to Roberts (Roberts, 1997)
“DGPS was developed to increase the accuracies through the elimination of the
clock, atmospheric and orbital errors”.

Most of the DGPS receivers work with both timing codes and carrier phase
positioning. The carrier phase frequencies correspond to wavelengths of the order of
19cm for L1 (1575.42MHz) and 24cm for L2 (1227.60MHz) (Pendleton, 2002).
However, only timing code corrections are broadcasted to rover receivers.

With DGPS positioning we will normally refer to receivers called base stations
and rovers. The base station is placed on a pre-surveyed coordinate known as a base
point. As the name indicates this receiver is left static on the base point, calculating
and transmitting the GPS corrections for the rovers. The rover is the receiver that

102
changes its position from time to time. A rover equipment could be a receiver that is
being used to guide a person, a car, an airplane or an AR system for example.

The pseudo-ranges are contaminated due to the presence of several variables,

including: clock offset, atmospheric delays, multipath offset and receiver’s hardware
offset. The solution proposed by DGPS is to use the pseudo-range corrections
calculated in a reference station for each visible satellite. The results are compared
with the true range and the biases are broadcasted via radio-link or other sorts of
dissemination to other receivers. The rover equipment also calculate their own
pseudo-ranges for each visible satellite, which are then corrected using the
adjustments received from the reference station. This type of positioning can achieve
the accuracy in the range of two to five metres depending on the distance of the
receivers.

Pseudo-range corrections can only be applied if the reference station is receiving

signal also from all satellites that reach the rover equipment. There is the possibility
of the transmitted corrections not being appropriate for the rover equipment if it is too
far from the reference station, as the method assumes the errors are the same at both
reference and rover. In this case, ephemeris and atmospheric errors cannot be
completely solved by this technique. As mentioned in (Aquino, 1999), “an
increasing separation makes the ephemeris errors to propagate differently into the
user and the reference station positions”. In addition, tropospheric and ionospheric
delays can be different for receivers placed a significant distance apart. This problem
is known as “spatial decorrelation” and poses the most significant limitation to the
technique. A final concern is related to the time that the corrections are applied. A
truly real-time correction is not possible due to computational processing delay and
the transmission of the corrections. This is generally known as “latency” (Aquino,
1999).

(Aquino, 1999, Kaplan, 1996, Bingley, 2003) are recommended for further
reading about DGPS.

103
3.11 Real Time Kinematics GPS - RTK-GPS

The pseudo-range methods of GPS positioning have advantages and disadvantages.

Single-frequency (stand-alone) receivers work only with timing codes.
Hypothetically, if the C/A code was available on the L2, dual frequencies (in stand-
alone mode) receivers could improve the computation by comparing the time taken
for the code to travel from the satellite to the receiver in different frequencies (L1 and
L2). The atmospheric delays could then be calculated with more accuracy using only
one receiver.

DGPS accent for this error by receiving corrections from reference stations.
However, none of these methods can achieve a centimetre level of precision in real
time. High accuracy in real-time can only can be achieved through the RTK-GPS
method.

RTK-GPS receivers process both L1 and L2 frequencies working exclusively

over the carrier phase. The general concept of carrier phase positioning is reviewed
bellow.

Carrier Phase Positioning

As cited in Bingley (Bingley, 2003), “high precision relative positioning requires the
use of the carrier phase”. The first step to obtain the carrier phase is to remove the
timing codes from the signal. As with the pseudo-range, the phase is also
contaminated by errors related to clock synchronisation, ephemeris, ionosphere, etc.
For more details in the techniques developed to model or minimise these errors the
reader is referred to (Kaplan, 1996, Aquino, 1992).

GPS receivers can only calculate the fractional part of the carrier phase leaving
the integer number of cycles between the receiver’s antenna and the GPS satellite
unknown. This integer number of cycles is called “integer ambiguity”. Figure 2.2
(Aquino, 1992) illustrates the carrier phase and the integer ambiguity on the signal
received by the GPS equipment.

104
 The measured carrier phase (Figure 3.23) is the fraction limited by the green and
red lines on time zero (t0). On the next two epochs, t1 and t2, the satellites’ have
moved, altering the fraction value. As demonstrated on Figure 3.23, the integer
ambiguity does not change as long as the lock is maintained on the satellite
(represented by the number of cycles between the two red lines). As the integer
ambiguity is not known the receiver then tries to “guess” the number of integer cycles
between the receiver and the satellite (Roberts, 1997, Bingley, 2003) when the
satellite was first locked. As cited on Roberts (Roberts, 1997), “the range between
the satellite and GPS antenna is a result of the fractional carrier phase plus the
integer ambiguity”.

This technique is better suited for static relative positioning otherwise the
movements produced by the rover could affect the number of “guessed” cycles. This
method normally calculates the coordinates of one station in post-processing, using
the carrier phase data from adjacent stations.

Integer
Ambiguity
Integer
Ambiguity

Fractional Carrier Phase

calculated by the receiver

Figure 3.27 Carrier Phase and Integer Ambiguity (Aquino, 1992)

The approaches raised for integer ambiguity resolution can be found on (Kaplan,
1996, Aquino, 1988, Bingley, 2003).

As long as the GPS signal is recorded continuously, the accuracy of the

coordinates remains high. But it is difficult to maintain a clear line of sight between

105
the receiver and the satellites when in movement. Trees, bridges, narrow streets with
high buildings and changes of height are some examples of what can contribute to
block or reflect the GPS signal causing “cycle slips”.

The cycle slips can be identified, for example, through the technique called phase
smoothing. This technique uses the carrier phase to smooth the pseudo-ranges
identifying when the GPS signal was blocked. Phase smoothing prevents the receiver
to compute segments of the signal that suffered some disruption helping to keep the
number of integer ambiguity. Eventually the technique invalidates itself due to the
time that the signal was blocked. The receiver then needs to restart the computation of
the integer ambiguity again. The reader is referred to (Roberts, 1997) for further
reading about cycle slips recovery techniques.

A solution for applications where movement is essential involves the use of a

carrier phase base method known as On The Fly (OTF) Integer Ambiguity Resolution
as cited on (Roberts, 1997). This method runs in real time and as explained by
(Roberts, 1997):

the positioning can be established whilst in the field, and any poor reading and data can
be rectified and taken again. The raw data may be recorded, in addition to the positions,
and the positions computed by post processing using different variables and options as
well as forward and backward processing.

This method requires more expensive equipment if compared with the methods
suggested in §3.9 and §3.10. The main advantage is the centimetre level of accuracy
in real time. To set up the RTK GPS, the coordinates of a reference point are
necessary to transmit the corrections for the rover receiver. A vicinity of up to 20Km
is the limit of the range in which the corrections from the static GPS receiver can be
effectively used. The limit is based in the differences that could occur in the
troposphere over the reference and the rover stations. Such differences could make
the corrections transmitted not appropriate, resulting in a poor resolution for the
integer ambiguity.

Differently from the DGPS positioning system, RTK requires at least five
satellites and transmits, from base stations to rovers, not only timing-code corrections

106
but also the corrections for possible solutions for the integer ambiguity. This solution
can be based on stochastic estimation process or through the Hatch filter (Yunchun
Yang et al., 2002) and there is a wide number of methods and variation aiming to
improve accuracy and reduce the time to compute the integer ambiguity. In any case,
the solutions go through a mesh of ambiguous combinations rejecting all those with
higher residuals and selecting the smallest. If more than one solution suits the final
answer the algorithm computes the next epoch’s data.

Details of RTK integer ambiguity resolution are beyond the scope of this thesis.
Further reading is suggested in (Yunchun Yang et al., 2002, Roberts, 1997).

3.12 Augmentation Systems and Network Services

Perhaps one of the main issues for those aiming to have a better accuracy when using
a GPS receiver is the cost of the equipment. As shown on the previous subchapters
there are different methods with an extensive list of different approaches to get better
resolution in the positioning. The better the accuracy, the more expensive is the
receiver. The problem of cost becomes more serious if a base station is required for
the solution. If the designers aim to cover a large area with the corrections, more than
one base station will be necessary to create a network.

If the set up of a network of base stations is not viable, another solution can be
provided through services offered by governments or private companies. The
following paragraphs exemplify some of the services that help users, equipped with
single or double frequency receivers, without a private base station, to obtain better
precision on their positioning. The services offered have significant impact on the
adjustment of pseudo-ranges especially for navigation (airplanes, boats, cars,
personal, etc.), and safety issues. A private base station is only then necessary for
remote areas where these systems cannot cover. In the near future, it is expected that
the dual frequencies receivers have their prices dropped and their size reduced,
providing general users with sub-metre positioning accuracy.

Augmentation Systems

107
The Federal Aviation Administration (FAA) is a department of the government of the
USA with a mission to provide safety and efficiency on American skies. The FAA
has created the WAAS, declared operational in late 2003. As mentioned in (Trimble
Navigation, 2002) “the system augments GPS with additional signals that increase
the reliability, integrity, precision and availability of GPS signals required for the
aviation industry”. The system works with twenty-five Wide Area Reference Stations
(WRS) spread over the USA with the mission to receive all possible GPS signals
from the GPS constellation. According to (Federal Aviation Administration, 2005a),
the information collected is then sent to a WAAS Master Station (WMS). The WMS
has the responsibility to create the messages that help GPS receivers to remove errors,
increasing the system’s precision. The messages are transmitted using geostationary
communication satellites. Perhaps one of the most reliable features in the system is a
notification for the users whenever there is any error on the GPS system that could
cause serious miscalculation on the user’s position. It is claimed by (Federal Aviation
Administration, 2005b) that “messages broadcasted by WAAS improve GPS signal
accuracy from one hundred meters to approximately seven meters”. Even though this
system was designed for airplane landing (Category 1 – precision approach) non-
aviation users can also be benefited. To make use of WAAS corrections, it is just
necessary to have a receiver with a particular chipset.

The European Space Agency (ESA) also offers, free of charge, the SISNET
(European Space Agency, 2005), “a platform to provide worldwide access to the
EGNOS System signal through the Internet”. The system works through the WAAS’
principle. The main difference is that the corrections transmitted to the users also
contain information about the Global Orbiting Navigation Satellite System
(GLONASS). The system comprises thirty base references, four master control
centres and six up-link stations. According to (European Space Agency, 2005):

EGNOS signal is broadcast by two Inmarsat-3 satellites, one over the eastern part of the
Atlantic, the other over the Indian Ocean, and the ESA Artemis satellite which is in
Geostationary Earth orbit above Africa.

European GPS users are supposed to achieve a precision of five metres in their
positioning without any extra costs of using this service. All they would need is an
EGNOS GPS chipset enabled receiver on the GPS receiver. However, EGNOS’ users

108
can also receive corrections that are being transmitted via Internet. Torán-Marti
(Torán-Marti et al., 2003) describes a handheld Personal Digital Assistant (PDA)
integrated with a low-cost GPS receiver (without EGNOS chipset enabled) and a
General Packet Radio Service (GPRS) wireless link to the Internet. The result is a
Graphical Unit Interface (GUI) with pseudo-ranges, ephemeris and ionospheric errors
being corrected based on the SISNet information. They claim that the correction
applied on the signals received has its accuracy down to one to two metres horizontal
and two to three metres vertical.

Japan also has its augmented system called Multi-Functional Satellite-based

Augmentation System (MSAS), which is based on the same principles described in
WAAS system. MSAS is augmented with geostationary satellites called Multi-
Functional Transport Satellite (MTSAT).

Network Services
In Japan, approximately two million GPS navigation-equipped cars are sold annually
(Petrovski, 2003), and currently, there are 3.8 million GPS-equipped cellular phones
being used. The chipsets implemented for cars and mobile phones are basically the
same which are in handheld GPS receivers or even smaller. Petrovski (Petrovski et
al., 2000) cites two approaches used to provide DGPS and RTK-GPS corrections in
Japan: radio broadcast or the Internet.

Asahi TV broadcasts DGPS and RTK-GPS corrections services encoded into a

TV audio sub-channel signal. The main disadvantage lies with the covering distances
between 40km to 100km. The RTK-GPS system usually works with base points
positioned not more than 20km far from the rover equipment. With Internet based
services, the disadvantage is the latency to transmit the corrections. Petrovski
(Petrovski et al., 2000) proposes a solution using virtual reference stations taking
advantage of the low latencies and high capacity of sub-channel signals where the
users can select the nearest base point available to receive the corrections choosing
which piece of information is necessary for particular purposes. This service is also
supported by the augmented system MSAS.

109
 By 2008, plans have been made to launch the Quasi-Zenith Satellite System
(QZSS) with a constellation of three satellites, where at least one of them will be
covering near zenith over Japan at the times. The advantage is a better coverage for
GPS augmentation in urban canyons providing high accuracy for rapid moving
vehicles. This system will also provide a wide range of services based on
communication. Petrovski (Petrovski, 2003) gives a wide overview of the Japanese
project.

SWEPOS is the Swedish network that provides DGPS and RTK corrections to
real time users and carrier phase raw data for post-processing. The network consists
of 25 permanent base stations covering the whole country. SWEPOS network offers
the following services for the users (Jonsson et al., 2003):
 Post processing data via World Wide Web/File Transfer Protocol
(WWW/FTP) in Receiver Independent Exchange Format (RINEX)-format.
 An Automatic Computation Service on the SWEPOS web. The data collected
on dual frequencies in RINEX format is sent to the service. After five to ten
minutes the user receives via e-mail the corrected data.
 The private company Cartesia broadcasts the DGPS corrections via Radio
Data System (RDS) channel on the Frequency Modulation (FM) network. The
accuracy achieved is between one to two metres.
 The Wide Area DGPS (WADGPS) is also explored by the private company
Fugro. It uses geostationary satellites for the distributions of the corrections.
The accuracy between one to two and a half metres is achievable.

There are several studies of RTK-GPS network corrections described in (Jonsson et

al., 2003). The corrections are broadcasted via an FM-sub carrier. Apart from the
advantages of this kind of system, such as ‘easy to use’ and no need of base station,
the most significant achievement is the millimetre accuracy in real time even with
strong indications of a disturbed ionosphere. More about SWEPOS can be found in
(SWEPOS Control Centre, 2005).

Ordnance Survey (OS) and Leica Geosystems offered the SmartNet to the British
market. Currently using 96 reference stations, this system is designed to provide
accuracy from centimetre level (RTK-GPS) to sub-metre (DGPS); or raw data for

110
post-processing (Leica Geosystems LLC., 2006). The new models of Leica GPS
receivers are able to communicate to the servers via Internet GPRS and Global
System for Mobile Communication (GSM). The communication can be done 24
hours per day, 7 days a week, with the promise of a highly dense network and highly
redundant. The controller of the network, SpiderNet (Leica Geosystems LLC., 2005),
decides which base station should send the correction based on the coordinates of the
rover receiver. Usually there is a master station and few more auxiliaries working as
backup in case of disconnection. IESSG is involved in this project “undertaking
independent monitoring and integrity checking, to validate the user experience”. The
institute is using its own technology to compare the results offered by SmartNet. This
process also includes several receivers placed strategically, working as users to check
the quality.

Some of the advantages for users of SmartNet are (Leica Geosystems LLC.,
2006):

 High speed initialisation for RTK solution.

 Redundancy due to the number of reference stations.
 Quality assurance once an independent body is also monitoring the service
provided.
 The cost of RTK-GPS solution is cut almost in half, since a base station is no
longer necessary.

3.13 Testing Single Frequency GPS, DGPS and RTK-GPS

Even though the RTK-GPS methodology was adopted as standard for this project due
to its high accuracy in real time, some tests were performed to show how much
accuracy other GPS approaches can provide. The intention here is just to show
visually the differences between the approaches. The test was performed with
receivers that were pushed in a trolley following a circular path with 40 metres of
diameter. The centre of the circle was surveyed by all methods and the path shaped
with the help of a tape measure. The tests performed are:

111
  A single frequency GPS receiver (see §3.9).
 A dual frequency GPS receiver working in DGPS mode (see §3.10).
 A dual frequency GPS receiver working in RTK-GPS mode (see §3.11).

The single frequency receiver had its data accumulated for two minutes for each
point. Because of this, the graph shown on Figure 3.24 has more coordinates
displayed then other graphs. This configuration was chosen because of the popularity
of such receivers especially because the low price. The graph shows the instability of
the system to accurately retrieve the circular shape of the trekking. A spread of about
10 metres is observed at each of the surveyed points. All data was processed using the
software Gringo (IESSG, 2002) which was used to create the images. This solution is
clearly not the best if in a given project the expectation is for the positioning data to
be consistent and accurate.

Figure 3.28 Single Frequency Receiver

Figure 3.25 shows the example of a single coordinate logged for approximately two
minutes using a single frequency receiver. It is important to say that the receiver in
this case is not moving. The several samples of this unique coordinate are the result of
errors involving this methodology (see §3.9). The figure below clarifies the reason an
AR system, as proposed in this thesis, would not work as desired if using this
methodology.

112
 C:\CAPRA\COORDENADAS\045446.G12.04O
(Mobile Stand-Alone Solution)
Lat (rms = 2.7 m), Lon (rms = 2.3 m)
Distance (rms = 3.5 m, 95% = 5.4 m)

Latitude (metres from mean)

2

-2

-4

-6

-8
-10

-8

-4

-2

8
-6
Longitude (metres from mean)
P4 - Pseudorange and Phase Post-Processor © Copyright The University of Nottingham 1999

Figure 3.29 Single Frequency Receiver – A Single Coordinate

The results of the performance of the DGPS in a dual frequency receiver are shown in
Figure 3.26. The circular shape has now been retrieved even though it is not exactly
centred. The error is spread to all coordinates as a constant drift only in the Latitude.

Figure 3.30 Dual Frequency Receiver Performing DGPS

A final decision about which receiver would be the best can only be achieved if GPS
receivers from different brands could be tested together at the same time and day. The
data could then be post-processed and analysed. However, there is always a chance
that the solution found would not be definitive, once there are different conditions
that the receiver could be exposed to. Narrow urban canyons, different weather
conditions, magnetic solar storms or even the chosen antenna of the receiver are just

113
some of the examples that would affect the results. There is no perfect equipment, but
one that suits better the needs and budget of the project.

The RTK-GPS solution is by far the best amongst all tested methods. The graph
shows (Figure 3.27) a circumference drawn with the logged coordinates from the
RTK-GPS receiver. All coordinates were collected and processed in real time. No
post-processing was required, only Ski-Pro (Leica Geosystems LLC., 2003) to plot
the graph.

Figure 3.31 Dual Frequency Receiver Performing RTK-GPS

These tests were particularly important, since they proved that different
methodologies can provide totally distinct results. Researchers often try to implement
projects requiring good accuracy using the wrong approach in relation to positioning.
Eventually they misjudge the limitations of the methodologies and when the results
are not as expected, an alternative solution, as reported in (Benford et al., 2004,
Tamura, 2001a), is required.

3.14 Other Positioning Systems

For outdoor AR applications GPS is the best available solution. However other
positioning techniques like cell phone or Wi-Fi triangulation and self-report
positioning are also available to be explored.

114
 The main advantage of using a mobile phone for positioning is the availability of
the service. Mobile phones are connected to one or more antennas wherever the
service is available, inside or outside buildings. However, the accuracy is based on
the number of cells that the phone has in its range (Gomez M. et al., 2002) and on
how much the signal was affected by the distortion along its travel transmission.

Wi-Fi triangulation can use either the tags distributed in a delimited area, or the
reference of pre-surveyed wireless points. In both cases, the places where tags or the
wireless points are, must be pre-surveyed. The use of the tags gives more accuracy to
the system, but it is limited to the area where they were placed. They are normally
used as positioning systems inside buildings. Nowadays, wireless points are common
in places with high density of offices due to the price and practicality of the system.
The same happens in wealthy areas where houses have more than one computer.
However, that does not occur when middle and working class areas are surveyed. The
result is a positioning system with very poor accuracy because of the lack of wireless
points.

Self-reported positioning (Benford et al., 2004) is based on the users saying where
they are. They explicitly add a new input each time that they are over a pre-surveyed
point. This kind of positioning has as its major disadvantage the lack of freedom for
users to explore AR environments from different angles.

The limitations of GPS when used with AR systems are well known and perhaps
one of the main frustrations for AR researchers is when the positioning system cannot
achieve the desirable accuracy. It is often observed that researchers who need
centimetre precision in real time would wrongly choose to use a modest set up of
equipment. As introduced on sub-chapter 3.9, the number of variables that would
cause errors on GPS is large, which makes single frequency receivers unable to
produce results with good accuracy on positioning in real time.

DGPS is the most used methodology, perhaps because most of the medium priced
sensors available in the market already come with this feature. There are also an
extensive number of libraries on the Internet that can help researches to create a

115
network of base stations and transmit corrections to the rovers. With the complement
of additional software, it is possible to explore the hardware of the receiver and
extract information that is not often available to ordinary users. However, this takes
time for configuration and eventually the final result (the coordinates) is not as good
as when using RTK-GPS.

This thesis adopted the RTK methodology for positioning. It is a fact that this is
the best positioning method for real time. Even though this could be the main reason
for using it, this thesis also evaluates the advantages and disadvantages of using this
method. Some of the advantages that can be mentioned without further research are:

 The high accuracy can be achieved using only a couple of receivers.

 Usually the receiver falls back to DGPS when the signal’s lock is lost. In this
case, the positioning error increases but the system keeps running.
 Top of the range equipment are not cosmetically different from receivers
enabled to compute DGPS only. The size, weight and shapes are the same.

The disadvantages when compared with other systems are:

 The price of this technology is far higher than any other.

 RTK-GPS system requires the GPS signal of five satellites, at least. If the
receivers are being used on narrow streets or in regions with poor satellite
constellation (low or high latitude) the results may not satisfy the user’s
necessities.
 Receivers should not be more than 20Km far from each other.
 Items of top-of-the-range equipment are heavy – about 1.2Kg and big – the
size of a shoe box. The antenna is not embedded to the receiver, forcing the
user to carry a pole for the antenna or a backpack with an adapted pole. There
are a few cables that need to be connected and the battery consumption is
higher than in small devices, which make them require stronger and heavier
batteries. The final pack becomes more expensive because of the high-
technology embedded in the peripheral equipment. Antennas, for example, are
highly sensitive for dual frequencies. The user must be aware that in order to

116
 carry this equipment, he needs to have a bag that can carry the antenna, the
receiver and batteries without damaging the equipment.

Table 3.1 shows the systems that were reviewed on this thesis and the average
accuracy in each of those.

Table 3.1 Different Positioning Systems

Positioning System Accuracy Comment
Mobile phone 30m to 500m Good precision is only achieved when 16 cells are being
tracked at once (Gomez M. et al., 2002). Only few
mobile phones can do it and Global System for Mobile
Communication (GSM) modems.
Wi-Fi Around 20m This system is based on a pre-survey and in areas well
populated with Wi-Fi spots.
Self-positioning precise The system is based on pre-surveyed points taking the
freedom of the user to explore the digital environment.
Single frequency 5 to 20 metres Minimum of 4 satellites and clear sky for better GPS
receivers (C/A signal reception.
code)
Single frequency < 5m vertical The receiver requires a special chipset that enables it to
receivers using and < 3m acquire augmentation signal as EGNOS or WAAS.
augmented system horizontal
(C/A code) 95% of time
DGPS (C/A code) 2 to 5 metres Same as above but requiring two receivers.
RTK-GPS Centimetre Expensive technique. The system requires at least 5
(carrier phase – level satellites available to calculate it’s coordinate.
double frequencies)

3.15 Combining and Integrating Technologies

All the researches presented above have one or more features that overlaps with the
CARS introduced in this thesis. They are aiming in common the delivery of
information in the most understandable way for users. Some of them aspire for an
increasing of communication, collaboration and mutual understanding between users,
but not all explore CVE. The way that users explore the VE also changes dramatically
from system to system specially between VR and AR. Systems also divert when
covering aspects like moving objects, animation, accuracy and user friendly.

117
 For the perspective of this thesis such aspects are important and are integrated at
the end of the research.

 Aspects like the communication, collaboration and mutual understanding

can only be achieved as a result of the implementation/integration of hardware
and software as well as the way that the system is exposed to users.
 The VE explored by users when indoor requires a system flexible enough
to allow other users working from different computers both indoors and
outdoors. The position of users when indoors is controlled by a handheld
device – mouse. When outdoors users must have a reliable and high accurate
positioning system to identify where they are, which direction they are going
to and which direction they are looking at.
 Very few researches mentioned the possibility of animation and
simulation. This is also an important aspect once it brings the CVE closest to
the RE. Animation also helps users to understand better information like wind
corridors or how a particular object moves after become operational.
 The accuracy is not only necessary to know exactly where outdoor users
are. It must be as much stable as possible avoiding jerking through the
selection of the right methodology of positioning as well as the
implementation of filters to reduce false movements.
 As easy to work with the system as quick users start to interact and
produce results. Users should not face any complex interfaces, especially
outdoors users. For complex tasks outdoors users should have the remote
assistance from indoor users.

118
Chapter 4

Design, Hardware and Software Overview

Previous chapters exposed the difficulties faced by researchers to make AEC working
closer. The review also showed that companies try to push products and standards
although the lack of a solid consolidation for what is best solution for CEn market.
Researchers with focus on BIM solutions are more concentrated on what kind of
information is relevant to store, how to drill correct answers from previous CEn
developments or how to exchange information between different solutions – between
many other topics. Although there are already tools enhancing 3D visualisation of
data, like collision between layers for instance, it does still encapsulated on a VE
explored on a PC where only few actors can have access.

The market is experiencing a technological revolution and this process can make
companies easily behind of high tech solutions. Even today communication in CEn
projects is sometimes only possible through traditional methods making 2D sketches
very much in use (Woksepp and Olofsson, 2006). Once technology is being
introduced gradually in CEn companies (not only because the investment required but
also which is the right technology a company should invest on) VR is becoming the
first natural step to be explored. As mentioned in (Bouchlaghem et al., 2004)

119
 VR models are one way to improve information handling, communication and understanding in
construction projects. VR offers a natural medium for the users providing a three-dimensional
view that can be manipulated in real-time and used collaboratively to explore design options and
simulations of the construction process.

However, in a near future it is not difficult to imagine AR systems displaying BIM

information for outdoors users (Woodward et al., 2007a, Woodward, 2007). This
thesis shares the view of Woodward, standing perhaps one or two steps ahead of what
CEn market is receiving as solution today.

The following paragraphs extend the idea of how this thesis aims to achieve the
enhancement of communication, collaboration and mutual understanding using
CARS. The design overview introduce here creates bridges that integrates the several
technologies reviewed in Chapter 3 and some knowledge from Chapter 2 in order to
address what is proposed as requirements of the project on Chapter 1.

This chapter also reveals the hardware and software used during the
implementation of this thesis. The description of what was used complements the idea
of why this design was choose and already introduces the way that it was
implemented.

4.1 Conceptualisations of User Groups

Figure 4.1 is an illustration of how the project should be presented when completed.
In the left side of the illustration there are a couple of actors exploring the CVE and
talking about what they can see. They share this CVE with a user outdoors through
the computer screen. The actor outdoor (Figure 4.1 right side) explores the CARS
though the images projected before his eyes.

This exchange of information is explored by interaction between actors through

the use of a voice channel and the movements of the avatars (human representation)
in the CVE. Such interactivity stimulates: the comprehension of obscure points, the

120
perception of new ones and the creation of new conclusions, helping actors to make
decisions of what to do next in the area that is being remodelled.

There are two main groups that benefits from the results of this thesis: the office
users and the field users. Initially, these groups were not created to distinguish people
with different levels of knowledge in CEn. However it is imagined that eventually the
combination of who is working in the system as actors in the office and who is in the
field reflects different levels of expertise. Example might be the situation where an
engineer (office user) explains to a potential customer (field user) where the stairs of
the house would be placed.

Figure 4.32 Proposed CARS

Office User
The office user (Figure 4.1 – left side) is the actor that explores the CVE displayed on
a computer screen with the help of a standard mouse. This actor could be either an
AEC or a customer wanting to know more about the new construction. The objects
designed to populate this CVE are the representation in the natural scale of everything
that could be placed in the RE. The actor has the freedom to explore the CVE without
restrictions. The avatar can be directed to go through the objects, take panoramic
views and even fly.

121
 The first example of configuration can be exemplified when only office users are
working on CVE. They can explore the CVE at the same time with the freedom to go
to all directions or move all objects. These actors are connected via Internet with the
facility of a voice channel.
A second configuration is the CVE being explored by both office and field users.
Again the voice channel is available and users can interact at the same time that they
are exploring the CVE.

Field User
The actor that explores the RE where a new construction is being built is called field
user (Figure 4.1 – right side). As with the office user, the field user is also represented
by an avatar inside of the CVE. Field user explores the CARS using the movements
of his body instead of a mouse from a computer. The person in the field has with him
a notebook and a RTK-GPS receiver in the backpack. The GPS coordinates are
associated to the embodiment’s movements. A crash helmet, adapted to fit the
inclinometer and the electronic compass, logs the movement of the user’s head
updating the angles of where the avatar should look at. A see-through HMD is also
adapted in the crash helmet to alleviate the weight over the user’s nose. The images
generated are then transmitted to a see-through HMD giving to the field user the view
of both the RE and VE projected before his eyes.

The see-through HMD also provides an output for an earphone. A microphone is

attached to the notebook and they are used to voice communication with other users.

A wave-LAN link is used to create the transmission between computers. The

office user does not necessarily need to be in the wave-LAN’s range. For instance,
users could collaborate over the Internet connected to a host that is in range of the
wave-LAN router.

4.2 Design Overview

The diagram on Figure 4.2 is a more detailed view of the design of the project
showing what actors (field and office users) have, as hardware and software, when

122
the project is fully operational. The diagram also shows how the connections are
between different parts of the system created. Two platforms are presented in this
graphic:

 MASSIVE-3 – working as the manager and render of the CVE (see §4.3).
 Equip – working as integrator between external devices and the CVE,
controls the data base Coordinates, and control the queries and the replies of
clients (see §4.3).

As general information, each computer involved has a copy of MASSIVE-3 and

Equip. They also have a copy of a file containing the entire CVE and the design of the
objects that populates it.

Figure 4.33 Design Overview of the Project – Field and Office Users

The diagram above does not represent the design used between office users only. This
diagram is presented in §5.2.

The following paragraphs go through the design of the project although some of
the elements of the diagram (trader, API/comm port and Integrator) receive
complementary information on chapter 5 due to their complexity and the level of
details.

123
 Firstly it is presented the left side of Figure 4.2 (office user) and then the
components of the right side (field user).

4.2.1 Office User Design

As mentioned before each computer used by office and field users to interact with the
CVE contains the files that populate the VE as well as the files that shape the avatar.
Both the files are interpreted and rendered by MASSIVE-3.

MASSIVE-3 – Office User Design

Actors can use the mouse to navigate inside of the CVE. All the rendering process of
the CVE is perform by MASSIVE-3. It contains a manager that keep continuously
listening for new event inside of the CVE World data base (Figure 4.2 – right side).
Office users can click and hold the right button of the mouse and move it forwards,
backwards, and both sides. All these movements are logged and replicated to clients
sharing the same CVE. If the avatar is facing to an object the actor can click and hold
the middle button. A ray comes from the front head of the avatar towards the object
selected. The object can be repositioned in another place and angle. Such movements
and clicks are handled by MASSIVE-3.

The repository of the CVE is managed and hosted by only one of the computers
involved (Figure 4.2 – CVE World).

Equip – Office User Design

Equip contributes in different ways:

 The handling of connection between computers.

 The connection with the data base (Figure 4.2 – Coordinates).
 The handling of incoming data from external devices.

The use of this platform is not compulsory in the office user side. It could be
discarded with actors interacting with the CVE in a simple way, like exploring the

124
CVE or moving objects. However it is requested if office users would like that an
external device have its input logged by the Coordinates data base.

In the diagram, it is presented the input from a keyboard. To make the keys
pressed available in the Coordinate data base (Figure 4.2 – right side) a Java code was
created (see §5.3 – keystroke) to keep listening from the keyboard. The following
steps document what happens when an office user presses a key in the keyboard:

1. The key pressed is read and sent through Equip Application Program
Interface (API) (see §5.3.1 equip_net_trader, trader).
2. The key stroked is then logged in the data base (Figure 4.2 – Coordinates
– see §5.3.1 equip_data_server).
3. This new log triggers an event in Integrator (Figure 4.2 – left side – see
§5.3.1 Integrator).
4. Integrator processes this new information (translating the key stroked for
the real meaning of it).
5. Integrator updates the management of MASSIVE-3 (see §5.3.1
equip_mas3_mgmtserver).
6. MASSIVE-3 management logs the data in CVEWorld (Figure 4.2 – CVE
World).
7. MASSIVE-3 clients keep listening for any change in the data base (Figure
4.2 – CVE World). When there is a change the CVE is then modified – in this
example CARS window and CVE window.

All these steps cover a view of the process involved to make the system running. The
complete set of programs running as well as their functionality can be found in
§5.3.1.

Audio Channel – Office User Design

The audio channel (see §5.3.1 – audio_server and audio_test) works completely
independent of what is being performed by Equip and MASSIVE-3. The
communication is directed between computers through the data communication. It is
only necessary to create a server in both computers and to set the acceptance between
parts to exchange the audio packages.

125
4.2.2 Field User Design
The field user side (Figure 4.2 – right side) is slight more complexes than the office
user side. On this side there are two data bases running as well as two external
devices being systematically checked for new inputs. The configuration of the data
bases was selected on this way because historically the project always performed with
the faster computer in the field user side. A new device is placed straight to field user
whenever a part is exchanged (new hard disk for instance) or a new notebook is
purchased.

MASSIVE-3 – Field User Design

The CVE manager running in the field user side is exactly the same version of the one
running in the office user’s computer. Field users could also manipulate the CARS
using a handheld keyboard (see §5.3) (it runs in the same way described for office
users) and mouse. However this option proved be very difficult for users to get used
to (see §6.2.1). All the manipulation of the CVE is performed then by users in the
office (see §6.2.2). The CVE World data base is automatically updated when a new
image is rendered in CARS. Field users can see the changes before their eyes through
a see-thought HMD.

The difference relies over an external application that feeds MASSIVE-3 with
new information to be rendered. This application is triggered by new data arriving in
Coordinates data base logging the data in CVE world (see §5.3.1 –
equip_mas3_mgmtserver).

Equip – Field User Design

As in the office users’ keyboard interactions, field users’ coordinates (GPS,
inclinometer and compass) are stored in a data base created and managed by Equip
(Figure 4.2 – Coordinates).

Although there are two devices connected with Equip API the process is basically
the same described when an office user presses a key in the keyboard. The RTK-GPS
updates its position every one second as set in the receiver. A Java class keeps

126
reading from the communication port every new data. When a new position arrives it
is processed and then published in the data base – Coordinates. The data is then
transferred to the CVE world. The same process is performed with the inclinometer
and the magnetic compass used to indicate the direction that field user is looking – all
these steps are introduced on §5.3.1.

Animated objects are also controlled in the right side of the diagram. A file
containing the coordinates of where the animated object should follow is interpreted
by equip_mas3_entitygenerator (see §5.3.1). The new coordinates are published in
the data base Coordinates triggering equip_mas3_mgmserver. The process then is the
same as described when a key is stroked or a new GPS coordinate is received.

Audio Channel – Field User Design

The process is exactly the same as described with the office user. A server is
established and the connection between computers set. Actors can then talk to each
other with good sound quality.

4.3 Hardware and Software

The following paragraphs describe the hardware, the platforms and interpolator
software used on the implementation of this thesis. This text goes through with more
details over:

 The GPS receiver and the messages that were extracted from it.
 The compass and inclinometer and also the messages that came from it.
 The see-through HMD and the angles to achieve parity between what is
displayed with what is seen.
 The notebook used to process all the integration.
 The handheld keyboard.
 The wave-LAN set up.
 The interpolator software used to create the models after the survey.
 Equip.

127
  MASSIVE-3.

4.3.1 Hardware
In most of AR projects the price of the receivers may be a relevant factor when the
GPS technique is to be used. It is possible to buy for a few hundred dollars good
handheld GPS receivers that can compute the position through the C/A code, as
mentioned in §3.9. Such equipment is good enough for people that are walking in the
moors, for example. The accuracy is around a few tens of meters making this device a
useful tool when combined with a map for example. However, such receivers are not
convincingly good enough for what this work is proposed to. As described before, the
drifts can cause not only discomfort to the users but also a misalignment between the
real and VE.

GPS Receiver
The Leica SR530 is a twenty four channel receiver that tracks both the L1 and the L2
carrier phases with high accuracy and with an on-board RTK capability (Leica
Geosystems LLC., 1999). The receiver can offer in RTK mode, two centimetres of
accuracy in normal conditions of use, which is appropriate for the application that is
being developed. The hardware is modular but reasonably heavy when in conjunction
with the other equipment in the backpack. A mini-pack with the receiver, plug-in
battery, radio modem, radio antenna and GPS antenna weights around 3.7Kg. The
radio for the RTK data transmission can be easily connected with the receiver and the
reception is good even for long distances. Leica reports that the maximum RTK range
is usually 10Km. The receiver has 16Mb of space in a flash card. A QWERTY
keyboard, a numerical keypad, function keys, and arrow keys are available to the
system’s interface. There is a screen coupled with the keyboard to activate the sub-
menus, each one with different procedures and very clear, avoiding possible
confusions or even mistakes from the user. After the receiver is setup, it is possible to
take out the screen-keyboard and leave the receiver working without the risk of
accidentally pressing a key. The connection between the receiver and the notebook is
done using standard communication interface RS-232C and a 9600 baud rate. It is
possible to configure the messages in National Marines Electronics Association
(NMEA) or American Standard Code for Information Interchange (ASCII) format.

128
For the present project the second option was chosen. More details describing how
the messages are collected can be found on §5.3.1. For the reference receiver, a set of
two small Camcoder batteries is enough for approximately one hour and a half. The
short life time of the batteries is due to the fact that the reference receiver is sending
RTK messages to the rover receiver via radio, which makes it necessary to connect
another external battery for long periods of use. The rover receiver can work well for
a few hours using only two small batteries.

The AT502 is a dual frequency antenna used both in the reference receiver and
the rover receiver. With high-accuracy, it is small, light, precision centered and has a
build-in groundplane. The antenna orientation is not required.

Using the system flexibilities, it was possible to convert the output coordinates
from the World Geodetic System 1984 (WGS84) to OS datum with the help of a
small program stored in the receiver’s memory. The output format chosen is the Leica
Local Positioning and Quality (LLQ). The description of this format is shown in
Table 4.1 as was presented in (Leica Geosystems LLC., 1999):

Table 4.2 LLQ – Leica Local Position and Quality

$GPLLQ, Header, include Talker ID, message sent form Receiver
hhmmss.ss, Universal Time Coordinated (UTC) time of position
ddmmyy, UTC date
xxxx.xxxx, Grid Easting, metres
M, Metre (fixed text “M”)
xxxx.xxxx, Grid Northing, metres
M, Metre (fixed text “M”)
x, GPS Quality, “0” = not valid, “1” = GPS Navigation
Fix, “2” = DGPS Fix, “3” = RTK Fix
x, Number of satellites used in computation
xx.xx, Position Quality, metres
xxxx.xxxx, Height, metres
M Metre (fixed text “M”)
*hh Checksum
<CR> Carriage Return
<LF> Line Feed

Figure 4.3 shows the Leica SR530 connected with the screen-keyboard. In the same
figure: GPS antenna (white plate), radio antenna (black stick), flash memory (in the

129
bottom of the receiver) and radio transmitter/receiver (with a red label on centre of
the figure) are shown.

Figure 4.34 GPS Receiver, Antennas, Flash Card, and Radio

The receivers were lent and supported by IESSG that has the know-how necessary to
supervise the use of this equipment for this project.

Magnetic Compass and Inclinometer

The TCM2-50, by Precision Navigation, is a very light circuit board equipped with an
embedded RS-232C interface on its 10 pin connector (PNI Corporation, 2004, Heike,
2002). It was used to capture the movements of the user’s head through the values
calculated by the compass and inclinometer. Table 4.2 shows the pins and describes
its functions:

Table 4.3 Pin Function of TCM2-50

Pin Function
1 V Supply (5V +/-)
2 V Supply (6 – 18V unregulated)
3 Power Ground
4 RxD (RS232) Input
5 TxD (RS232) Output
6 Mouse Output
7 Data Ground
8 Analog Output 2
9 Analog Output
10 Data Ground

The RX–input is tolerant to signals from -15V to +15V. The minimum signal
difference is ±5V. The TX–output only produces a -5V to +5V signal (Heike, 2002).
The electronic compass and tilt sensor utilizes a fixed 3-axis magnetometer in

130
conjunction with a 2-axis inclinometer (tilt sensor). As the system moves and tilts, the
TCM2’s strapped-down magnetometer continues to calculate accurate field
measurements. The inclinometer provides accurate pitch and roll information to the
TCM2’s microprocessor, which then mathematically corrects the tilt (PNI
Corporation, 2004). As output the device gives:

 Compass heading (360 degrees).

 Roll output (±50 degrees).
 Pitch output (±50 degrees).
 Temperature output (-25° C to 100° C).

The following limitations have to be considered when operating the TCM2-50

module in any application (PNI Corporation, 2004):

 Acceleration: earth gravity and artificial acceleration can become a problem

on situations like banking turns and take offs of an airplane.
 Settling Time: after a sudden change in heading the settling time for the
inclinometer (limiting factor) is approximately 300ms.
 Vibration: the natural frequency of the standard fluid in the inclinometer is
approximately 20Hz. Inaccurate readings occur if vibration in and around this
frequency is present.

Continuous output of the circuit board draw as low as 7mA of power consumption
from the battery. A 12V battery was used and it took approximately a month to
discharge.

The user is able to configure the TCM2-50 with the software offered by the
manufacturer based on Disk Operational System (DOS) mode. After the circuit board
is configured, it can be switched off as the parameters are stored in an internal
memory.

The description of how the messages from TCM2-50 are being processed can be
found on §5.3.1. The message provided by the circuit board is shown in Table 4.3.

131
 Table 4.4 TCM2-50 Message
$C compass
P pitch
R roll
X x – component of magnetic field
Y y – component of magnetic field
Z z – component of magnetic field
T temperature
E error code
* checksum

Figure 4.4 shows the TCM2-50 circuit board.

Figure 4.35 TCM2-50 Circuit Board

The magnetic compass is the vector result of three solenoids displaced as X, Y, and Z
axes. The circuit is sensitive enough to compute any magnetic distortion when the
circuit board is rotated generating a new position for this vector. The three solenoids
are isolated from the other parts of the circuit board and located on the upper left side
of Figure 4.4. The tilt sensor is the small dome placed on the upper middle section of
Figure 4.4. There is a liquid that makes three resistors become half immersed in this
dome. Each time that the circuit board is turned, the liquid follows the movement
allowing the difference of power in each resistor to be calculated.

See-Through Head Mounted Display

A see-through HMD (more specifically a SONY PCGlasstron HMD) – PLM-S700
(SONY, 1998), was employed to augment the user’s vision.

The following bullet points indicate some characteristics of this HMD:

132
  The size of the projection is comparable to watching a 30-inch screen from
approximately 4 feet away.
 Minimum screen resolution for PC video card: Video Graphics Array (VGA)
with 640 x 400 (dot x line); 31.5 horizontal frequency kilo hertz (kHz) and 70
vertical frequency hertz (Hz).
 Maximum screen resolution for PC video card: Super Video Graphics Array
(SVGA) with 800 x 600 (dot x line); 53.7 horizontal frequency (kHz) and 85
vertical frequency (Hz).
 A pair of stereo earphones is provided with the unit.
 It is possible to power the see-through display with an electrical socket. For
outdoors the use of a battery pack model NP-F750 is needed. The charging
time is approximately 8 hours and the battery runs for up to 2 hours.
 The unit weights around 120g and is made of magnesium alloy.

Figure 4.5 shows the PLM-S700.

Figure 4.36 PLM-S700

With the features described above it is possible to calculate the Vertical Field of View
(VFoV) of the see-through HMD. This value provides the angle to be followed by the
virtual camera inside the VE giving the illusion that the VE has a compatible scale.

Figure 4.6 shows the parameters of the VFoV.

133
 Figure 4.37 VFoV Calculation

The VE is projected into a screen represented by the green rectangle, as shown in

Figure 4.6. The user’s eye is placed behind the lens on the centre of the green
rectangle. The VE is explored respecting this window and the whole system must
follow this guideline in order to produce a compatible scale sensation. The VE is
represented as the black rectangle shown on Figure 4.6. It has been assumed that the
user’s eyes would be 122cm far from the projector (4 feet – the blue arrow) and the
screen has a diagonal of 76.2cm (30 inches – doted line), with an aspect ratio of 4:3.
These values allow the computation of the VFoV as shown below (Equation 4.1):

3
arctan    0.75    36.87 o
4

Height
sin 36.87o   Height  45.72cm
76.2

22.86
arctan  '    '  10.61o
122

  VFoV  2 x '  21.22 o

Equation 4.5

This angle is placed on the description of the configuration file that will be used by
the VE management (MASSIVE-3) to create and maintain the virtual camera.
Notebook
A notebook, Toshiba Satellite Pro – 6100 (TOSHIBA, 2002), was used in the project
to host the two platforms worked (see below) and on the creation of Java classes and
methods to make the communication between this and the external set of equipment.
This computer was also used to be the server of the CVE between office and field
user as mentioned before.

134
 Some of the features are listed bellow:

 1.6 GHz Mobile Intel Pentium 4 processor with 256 MB of RAM.

 16 or 32 MB of RAM is provided for video display on a 3-D accelerator
graphic board.
 The battery pack made of lithium-ion can supply approximately 1:30h usage
time.

Keyboard
One of the requirements of a new project while in the test phases is the constant
changes in the parameters and source code. For AR projects it is crucial to have
access to the computer without taking out all the equipment from the backpack or
having to go back to the laboratory. The evaluation of the system is only possible
when it is being tested outdoors, where there are GPS signals. Any corrections need
to be made in situ, hence the importance of an easily accessible keyboard.

During the early phase of this project a desktop keyboard had been used for a
while in order to access the computer. This was not practical because of its large size,
heavy weight and the difficulties to “carry and type” at the same time. Another
problem appeared when the Personal System (PS)/2 was used with the keyboard,
leaving no spare port for the mouse.

The solution was found on the handheld keyboard-mouse twiddler2 by Handykey

Corporation (Handykey Corporation, 2006). The device is connected in the PS/2 port
and can manage both keyboard and mouse at the same time. With only 16 buttons it is
possible to get a standard 101 desktop keyboard.
Figure 4.7 shows the twiddler2.

135
 Figure 4.38 twiddler2

Wave-LAN
The communication between the field and office users is done via wave-LAN. In the
early stages of this thesis usually both computers are not far away from each other,
making it unnecessary to have a gateway to connect them. The configuration chosen
was peer-to-peer wireless workgroup.

The model of the wave-LAN card used was an Orinoco Silver PC Card. Orinoco
guide (Lucent Technologies - Bell Labs Innovations, 2000) describes all features of
the device.

For the final evaluation of this thesis (see Chapter 6) a wave-LAN router was
adopted. The advantage is the power of the router that can let field user go further
without the concern that the link could be break in few metres. A D-
Link router was chose for any particular reason. The features of this equipment can be
found in (D-Link Systems Inc., 2006).

Integrating the Crash Helmet to the System

Some adaptations were necessary to prevent devices like the TCM2-50 from being
damaged by static shock or accidental drop. This circuit board does not have a
protection to avoid damages or short circuit caused by static electricity. A plastic box,
originally designed for business cards, was used to protect the circuit board. There are
4 plastic pins to fix it inside the box and several small holes in the cover to avoid
over-heating. Only plastic material was used to prevent distortions caused by metal
pieces common in boxes (like screws, for example).
Another precaution was taken with the cable used to connect TCM2-50 with the
notebook and battery. This cable should be flexible and at the same time robust. Leica

136
supplies a kind of cable with 5 wires inside and 1.8m long. This cable is the only one
in the market that has a connector that fits with the Leica 12V battery. Two of the five
pins were used for the power supply. The remaining pins were used to connect the
circuit board with the RS232-C connector, splitting the cable in two. Even though the
battery is not too light to carry, the result was satisfactory. The advantage to use this
configuration was that the power consumption of the circuit board was very low,
allowing the battery to last for days without interruption.

A few more adaptations were necessary to fit some of the equipment giving more
comfort and freedom for the field user.

It was noticed that the see-through HMD becomes very uncomfortable on the top
of the nose after some time due to its weight. At the same time the TCM2-50 should
be placed over the user’s head to track its movements. The solution was to cut the top
of a crash helmet to fit the TCM2’s and in helmet’s peak two holders have been made
to loop a couple of stripes of Velcro, where the see-through glass could be fixed. The
Velcro keeps the PCGlasstron out of contact with the user’s nose, making the
experience of visualising the VE more comfortable.

Figure 4.8 shows the helmet with some of the devices connected to it.

Figure 4.39 Helmet, PCGlasstron, Leica Battery, See-Through HMD and Cable

137
4.3.2 Software
During the process of implementation of this project some extra knowledge were
required to create the VE and the displacement of the features that configure the area
that is being represented. This fidelity of what is being displayed in the VE is
particular important when dealing with CEn projects. A misplaced of a pipe
representation in the VE for example could expose the CEn project to risks like:
financial, time consuming, rebuilding, and the most serious: threat of life (Roberts et
al., 2004).

The creation of the VE in the scope of this thesis started with a survey of the area
where the examples run for the first time. The survey was particular important
because:

 To get use to the GPS equipment:

o What parts of the equipment should be selected. Between many
parts including: batteries, cables, antennas, radios, devices models,
tripods, nuts for landmarks, etc.
o The setting up of the equipment in the field: like how to calibrate
tripods.
o The setting up of the equipment configuration. The device should
be set to output the right information.
o The logistic: charging batteries, transport, packing, booking
system, safety issues, etc.
 The area where the equipment is being placed. There are issues like: thief
hazard, people that would like to touch the equipment, traffic hazard, field dispute
(green area can be booked by other users to play soccer or gardeners can arrive to
mow the grass).
 How the information can be extracted from the GPS receiver and then
processed.

The first example created between office users (see §5.1.1) was based in a well know
area inside of The University of Nottingham Main Campus. It was important for users
to identify the area that they are exploring.

138
 The use of a DTM technique to collect the points helps to extract extra features
from the terrain, giving more realism to the VE created. The quality of the results is
dependant on how the control points that represent the terrain’s features were chosen
and which technique was used to collect them (Capra, 2000). They can be established
on a regular or irregular form. If the control points were surveyed using regular
method, several details of the terrain could be missed such as canyons and steps. This
technique is usually applied on flat areas. In irregular terrains it is recommended that
control points are collected in an irregular form. In this configuration the control
points have a stronger meaning for what can be found in that particular area. The
coordinate collected are then interpolated in order to create a mesh (regular or
irregular) that will represent the area surveyed.

Interpolator Software
The software Surfer (Golden Software, 2004) was used to interpolate these GPS
coordinates. This accessible software leaves the user free to manipulate the data
without deep knowledge about methods of interpolation, for example. From a set of
control points placed in a table, the data is interpolated following the method chosen
by the user. The output is another table with a square mash containing intermediate
points. A wireframe view can be produced as an option to see if the result matches
with what was expected. Because there was not interested in a specific method but in
fact in the file format for a further conversion, the method chosen is not be
commented in this thesis.

A small program was created in order to convert the DTM generated by Surfer to
a file that can be explored on the VE.

EQUIP
To manage all the devices connected to the computer (i.e. GPS receiver, electronic
compass, etc.) and to share the information in a data space, the MRL has developed a
platform known as EQUIP. This system can provide the distribution of data in real
time for different types of applications running in different machines. It is compatible
with C++ and Java, in which it is possible to share and select information from other
programs using a simple interface of communication. The distribution of data is

139
organised into “data spaces” which are named using a URL-like scheme. As cited on
(Greenhalgh, 2001f):

There can be any number of data space and each data spaces can contain an arbitrary
number of data items. Any number of processes can connect to a data space and add to
and access the data item in it. Data items are passed between processes using EQUIP’s
own serialisation protocol together with dynamic code loading in both C++ and Java.

Once the data is published on the data space one or more clients are notified that there
is something new to be collected. This new data is then distributed for clients that are
sharing the common resource.

Figure 4.9 provides a general view of the system. Through EQUIP’s API the data
can be shared on a data space illustrated by the grey rectangle. The green and red
blocks represent data coming from different sources. Using basically the same API to
publish classes in C++ or JAVA the VE is updated with these new data. The reverse
direction can also be achieved when data comes from the VE and is read by external
devices.

Figure 4.40 EQUIP Platform (Greenhalgh, 2001f)

Generally, when there is the need to include another device in the system, a new Java
or C++ class or method that will enable the communication with EQUIP needs to be
implemented, supporting different formats and particular characteristics.

In the context of this thesis EQUIP has the function of reading the information
from external devices connected to the computer, publishing it in a data bank, as
mentioned. The clients read from the data space and update the VE with the

140
oncoming information. The outdoor user’s movements are translated into
embodiment actions into the VE. Every time the field user starts to walk, for example,
the new coordinates from the RTK-GPS receiver update the data space handled by
EQUIP and consequently the position of the embodiment in the VE. The same
process is done for the user’s head movements. In summary, EQUIP is the link
between external devices and the VE.

MASSIVE-3
The creation of the data repository does not mean that the VE is ready. The VE is
created by another system, namely MASSIVE-3, which was also developed at the
MRL. The VE is composed by objects and avatars that are managed by MASSIVE-3.

As cited on (Greenhalgh, 2000d):

MASSIVE-3 is a multi-user CVE system. It allows multiple users to view and become
embodied in a 3D audio-graphical virtual world. It supports real-time audio
communication between users, and dynamic modification of virtual worlds.

MASSIVE-3 allows the user to construct a CVE easily. First, it is necessary to design
all the objects that compose it. It can be done using a VRML 1.0 browser designer.
The positioning of each element inside the CVE is made by a descriptive file
informing the coordinates of the objects, scale, rotation, etc. This file is exemplified
in §5.1.1.

Because these two platforms (EQUIP and MASSIVE-3) were developed on such
a complex level, the reader is encouraged to read the following papers in order to
understand not only the contained in them, but the possibilities that these platforms
present: (Greenhalgh, 2001f, Greenhalgh, 2000d, Greenhalgh, 1999b, Greenhalgh,
1999a, Greenhalgh, 2000c, Greenhalgh, 2000b, Greenhalgh, 2001d, Greenhalgh et al.,
2001, Greenhalgh, 2001c, Greenhalgh, 2001b, Greenhalgh, 2001a, Greenhalgh,
2001e).

141
4.4 Final Analysis over the Design

The project’s design introduced above is created to improve the gaps faced by AEC
regarding to communication, collaboration, and mutual understanding during the CEn
project with special attention to design and construction phases.

The platforms used to manage and manipulate the CVE (MASSIVE-3) as well as
the one use to bridge external devices to the computer, handle one of the data bases
and deal with clients requests (Equip) are the best options available due to the know-
how and support inside of MRL. These technologies were being in used for a number
of years in different projects and proved that are good options for the implementation
of the CARS proposed.

Again, as mentioned in the beginning to the chapter, this thesis shares the views
presented by (Woodward et al., 2007a, Woodward, 2007, Shin and Dunston, 2008)
and strongly believe that technology is increasing its capacity of processing complex
data at same time that its reduce its size. AR systems and in special CARS are
becoming a common tool for CEn actors during the implementation of CEn projects.

As an example of real application of CARS, architect offices propose outstanding

design all over the world like David Fisher (Rotating Tower Technology International
Limited, 2008) or Skidmore, Owings and Merrill (Wikipedia, 2008e) office
responsible for the design of the Freedom Towers, between many others. It is feasible
to imagine that AEC cannot stay all the time in the same place for every meeting set
during the whole cycle of the CEn project.

The CARS proposed on this thesis explores the practicality of a wearable

computing to enhance the contact between CEn actors. The field users can receive
real time assistance from office users saving time and money, clearing doubts,
enhancing once again the 3 key worlds of this thesis. AEC actors can collaborate also
using only the CVE between office users. The way that a discussion about a project is
set is depending of the requirements of the meetings and what kind of assistance
actors need at the time.

142
From the hardware perspective it is possible to extend the following points:

 The hardware involved can produce the positioning of users at centimetre

level of accuracy. This is exactly what an outdoors AR system needs.
 Inclinometer and magnetic compass is eventually affected by magnetic
field distortions. The main problem is the jerking of the CVE visualisation.
This can be compensated using a Kalman filter (see §5.3.1).
 The see-through HMD was not design for AR solutions although it works
very well for a prototype. It is expected that coming technology will address
problems of resolution, automatic contrast, weight, battery consumption, etc.
 Notebooks are becoming each time more powerful, light, small, and
inexpensive.

143
Chapter 5

System Implementation
The practical implementation of the system is performed in three steps (Figure 5.1).
The first cycle covered the following points.

 The familiarisation with the equipment that is being used: RTK-GPS,

inclinometer and magnetic compass, see-through HMD, wave LAN cards, etc.
 The familiarisation with the platform used to manage the CVE – MASSIVE-3.
 The design of a VE representing the surrounding area of IESSG building
inside of The University of Nottingham Main Campus.

The first phase finishes with the implementation of office users able to share a CVE,
base on a RE.

The second phase extends the idea of the CVE to outdoors, creating then the
CARS proposed on this thesis. The following steps are taken to achieve this
objective:

144
  Familiarisation with the platform used to bridge external devices with the
CVE – Equip.
 Implementation of Java classes to link external devices with Equip API.
 Tests of performance and reliability.
 Creation of new CVEs.
 Logistical and orchestration management of how to deal with a
considerable large number of expensive equipment (particularly RTK-GPS
receivers).
 Final pre-test.

Figure 5.41 Steps for the System Implementation

The final stage is the evaluation process. Although several small evaluations and
demonstration of the project were performed to different users along the whole
project cycle, only the last one is documented. This final evaluation is created over a
small number of people. The invited users were choose due to they professional
competence in their respective areas of knowledge.

This chapter does not only report the work perform during the implementation
process of this thesis. It also documents how to do it in a simple and practical way. It
is noticed that such documentation is not easy understandable, especially for
researchers not used to MASSIVE-3 and Equip environments. The following sub-
chapters may do not describe all functionalities of these two platforms (there are

145
plenty of literature for it) but they are extremely useful for users to setup, start up and
make run the processes involved in the creation of a CVE as well as the connections
of external devices with the data base managed by Equip. Another positive point to
document this technology is the change of researchers inside of MRL. Once
researchers start to work in different projects or move to other laboratories the
knowledge of “how to make it – in a simple way” is in risk to be lost.

5.1 First Phase – Office User and CVE

The following paragraphs describe the first phase of the project implementation.

5.1.1 Survey and Data Conversion

The first practical work of this research was the creation of a DTM in the
surroundings area of IESSG building at The University of Nottingham Main Campus.
Once the IESSG is a centre of excellence in survey and positioning system and holds
all the necessary hardware, software, and knowledge, it is a natural procedure to
create this DTM using in-house expertise. The data acquisition stage starts with the
setup of a base point where the RTK-GPS receiver would work. This base station was
setup in the third pole at the top of the IESSG tower. The coordinates based on
WGS84 for that base point are:

 Latitude North: 52°.56’.26”.49612.

 Longitude West: 01°.11’.32”.32484.
 Height: 98.388m.

The same coordinates translated to OS National Grid are:

 East: 454363.647m.
 North: 338474.259m.
 Height: 40.679m.

The area that surrounds the IESSG building is rich in features. Some of the mapped
features were: the slope where the building is, the stairs of the footpath used by the

146
general public, lamp posts, fixed bins, roads around the institute and the car park, and
the building itself.

SKI-Pro software, by Leica (Leica Geosystems LLC., 2003), was used to extract
and convert the data from the RTK-GPS receiver in the manipulation stage. This mass
of coordinates was then interpolated using the software Surfer (Golden Software,
2004) to generate a mesh with regular spaces between the coordinates. Control points
from previous surveys carried out in (Evans, 2001) were included. Such control
points were representing the road kerb that surrounds the IESSG building and shape
the streets.

Due to the incompatibly between the file format that specifies the VE in
MASSIVE-3 and the Surfer’s output, a program in C language was created to do the
conversion to an intermediary solution. In this case the output was in VRML file
format. This software (Appendix A) was designed to receive the following inputs:

 Surfer’s output. This file contains the landscape representation in a regular

grid.
 A file describing the terrain’s features like: trees and building. The objects’
descriptions contain the coordinates logged in the acquisition stage.

The software then generates three outputs:

 A file that contains the landscape and the features as they are in the physical
world – VRML file.
 A file with only the landscape. It proved useful for computers with low
processing capacity. This file was used to verify if the landscape created was
reflecting the RE.
 A file containing the .cve file description. More about the .cve file format is
described in the paragraphs to follow.

The software is started with the following command line at the DOS:

147
 MIX006 <grid>.grd <feature>.txt <outfile>.wrl

Where:

 grid – file where the Surfer’s output is saved

 feature – file with the features and related coordinates
 outfile – name of the output file

The file with the surveyed features requires an especial attention if the users of the
system want to include a road when creating the VE. The feature “road” is not an
object but a set of coordinates that bounds it. These boundaries require a descriptive
file that is read by the program changing the colour of the terrain (from green to grey)
where the boundaries of the road are. This solution helps to reduce the number of
polygons to be processed by the browser used in the visualisation. The structures of
these files are described in Appendix A.1. Figure 5.2 shows the terrain (green area)
and the roads (grey area) when visualised on a VRML browser.

The flat lower area at the base of the landscape, shown in Figure 5.2, is generated
by the interpolation process. Once the survey collected the control points in an
irregular form, the interpolator software, when processing the date, creates a regular
grid where the flat areas show no data.

Figure 5.42 Landscape Surveyed Around IESSG Building

148
A partially finished VE containing a number of features that were surveyed in the
surroundings of the IESSG is shown on Figure 5.3. Features including trees and
IESSG building were logged.

Figure 5.43 View of a Completed VE on VRML Browser

Both the IESSG building and the trees were modelled for the VE using the AC3D
(AC3D, 2004) tool. In order to give a more realistic appearance, digital pictures were
taken around the building, which created a recognisable texture (Figure 5.4 later in
this section– where the VE is already in MASSIVE-3 environment). The texture does
not have the intention to be a photogrammetric work but an enhancement in the
visualisation for those who know the area and the IESSG building.

The elevation of the real terrain is accurately reproduced in the VE. Differences
like the ground level between the road that runs in front of the building and the car
park in the back of it can be identified, conveying information with real features for
the user and reassuring that the data acquisition and the manipulation stages were
successfully completed.

The .cve File Format

As mentioned previously, the .cve file used on this thesis must run under MASSIVE-
3 specification. That includes:

 The structural description of the CVE specified in a .cve file.

 The objects that fill the CVE specified in a .dgl file.

149
Both files must be stored on each computer involved on the CVE exploration. The
structural description consists of a textual file that specifies the objects that fill the
CVE. The description of each object contains: the path where the .dgl file is stored,
position, scale, rotation, etc. Greenhalgh (Greenhalgh, 2000b) offers a full description
of how to generate these files and the commands that can be passed to MASSIVE-3.
For clarification, Table 5.1 shows a very simple example of a .cve file.

Table 5.5 An Example of .cve File

/*An example of .cve file*/
{{
{{
{ GEOMETRY_FILE: "cube.dgl"}
{ GEOMETRY_SCALE: 1.00 }
{ POSITION: {0 0 0} }
}}
}}

Where:
 /**/ comment
 {{ begin
 GEOMETRY_FILE: .dgl file with the object’s description that compose the
VE
 GEOMETY_SCALE: scale associate with the object
 POSITION: object location in the VE
 }} end

DGL’s Description
With a VRML file for each one of the objects created to feed the VE, it is possible to
use the program available on MASSIVE-3 library to make the final conversion
between VRML to Dave’s Graphic Library (DGL) file format.

Dave (Snowdon and Greenhalgh, 1996) gives the following description for the
DGL (.dgl extension) library:

DGL was designed to support network applications. It has two components a renderer /
platform independent part which provides graphics primitives and basic geometric
operations and one or more renderer dependant parts which perform rendering for
specific platform / renderer combinations. The renderer supports user input using the

150
 mouse and keyboard in addition to notifying clients of other events such as resizing and
window close events.

DGL files can be created through the MASSIVE-3 library using a single command
line in DOS. The conversion can be done from VRML file specification to .dgl. More
information can be found in (Greenhalgh, 1997).

The utility process used to do the conversion is initialised in the context of this
thesis using the command line:

dgl_convert <infile>.wrl <outfile>.dgl

Figures 6.3 and 6.4 show the area around the IESSG and the building through screen
shots of MASSIVE-3.

With the files converted to DGL format and the CVE description finished, the VE
can now be explored using the MASISVE-3 environment. Three elements must run:
trader, cveworld and bodyexampleclient. All these process are well described in
(Greenhalgh, 2000d). The following paragraphs are an overview of what each one of
these processes mean and how they were ran within the context of this thesis. The
first experiments done using MASSIVE-3 were performed using a single user. Later
the CVE was explored for more than one user at the same time.

5.2 First Phase of Practical Work

The sub-chapter 4.2 already introduced how the messages go through the data bases,
how they are recovered and how the CVE and CARS windows are refreshed. The
complexity of the system required that a more detailed view should be presented. The
following paragraphs introduce the complete set of processes involved to make the
system run. It fully complements the introduction gave during the system design. The
way that the programs below are organised represents the order that they should be
started up.

Figure 5.4 describes the programs necessary to run the CVE only between office
users. This configuration is just a subset of the full design presented on §4.2. At this

151
level Equip is not necessary. Users using this design have limited power over the
CVE. They can explore, move objects, and talk to each other.

Figure 5.44 System Requirement between Office Users Only

To start a CVE between two or more office users the following processes need to be
started:

 trader
 cveworld – this process is initialised in only one machine that handles the
requests of other clients.
 bodyexampleclient
 audio_server
 audio_test

The processes are described below.

trader
For any CVE started using MASSIVE-3 resources, the trader process is required to
bootstrap it (Greenhalgh, 2000d). As cited on (Purbrick, 2001):

152
 …all environments created on MASSIVE-3 are databases which contain part or all of a
virtual world as a hierarchical scene graph. The two main types of MASSIVE-3
applications are clients and servers. Servers create an agent and an environment and then
publish the existence of the environment with a trader service. Clients create an agent,
locate a server by querying the trader service and then ask the server for a replica of its
environment. Having received the replica the client can query and update its local replica
which will generate events which are first sent to the server and from there to other
clients.

This means that the trader creates a “space” for the database leaving a port of
communication ready to transmit and receive messages to/from the server. At first,
trader processes start in all machines although they cannot communicate with each
other, since it has not been established which computer is setup as server (the one
holding the VE World data base) and which are the client. When the VE is finally
loaded and created – a “place” the VE receives a name and a unique system identifier
(Greenhalgh et al., 2000). The trader translates environment names to Identifications
(IDs) and IDs to network addresses (Greenhalgh et al., 2000).

The trader process is initialised in the context of this thesis using the command
line:

trader

cveworld
The VE is started when the cveworld process is activated. This process reads the .cve
file in order to place the objects that will compose the VE. This process also reads the
.dgl files that describe each one of the objects referenced in the .cve file. For each VE
described in the .cve file an ID (as a name for example) is required. The trader is then
notified that a new VE was created and all requests of communication are directed by
the ID. A complete description of this command can be found in (Greenhalgh, 1997,
Greenhalgh, 2000d). The cveworld command is only applied once – for the server.
Clients do not start this process.

Cveworld process is initialised in the context of this thesis using the command
line:

cveworld <IP_address>/<ID> <CVE_world>.cve

153
Where:

 ID is any significant name that will easily identify the CVE created.

bodyexampleclient
The bodyexampleclient is a process started for all clients aiming to have access, with
their own embodiment, to a specific CVE through the socket offered by the trader
service. Note that the machine that is hosting the CVE World is also a client of itself.
The embodiment follows the description given in a file named massive3.config that is
fully described on (Greenhalgh, 2000a). The process starts by calling a specific
address and ID.

The bodyexampleclient process is initialised in the context of this thesis using the
command line:

bodyexampleclient <IP_address>/<ID>

Where:

 ID is the same name included in the cveworld command.

From the client’s perspective, this request first goes to the local trader that redirects it
to the server’s machine. In the server, the trader identifies the request by the ID.
Finally the server returns a copy of the VE requested indicating the displacement,
scale, rotation, etc, for each one of the objects. Each client has its own copy of the
objects that fill the VE. This process is fully described on (Greenhalgh, 2000d,
Greenhalgh, 2000a).

After the process has started a window pops up in the computer’s screen
displaying a CVE ready to be explored.

The VE based on the processes of acquisition and manipulation can now be

explored as shown on Figure 5.5 and 5.6. These MASSIVE-3 screenshot also shows

154
the textures created with the pictures that were taken around the IESSG building in
the Main Campus of The University of Nottingham.

Figure 5.45 View of the Using MASSIVE-3

In these two images (Figure 5.5 and 5.6) the VE is being explored by a single user
who controls the movements using a mouse.

Figure 5.46 Front View of the IESSG Building

The next step was the setup of a CVE where more than one computer could share the
same place. Figure 5.7 presents both office users sharing the same environment. The
screen shot is a view from one avatar to another. The red avatar in Figure 5.7
represents the second user.

155
 Figure 5.47 Red Embodiment Representing a Second Office User

The first stage of this thesis finalised with two actors (office users) sharing the same
CVE. Although the audio link was already available it was not explored at this stage.

5.3 Second Phase of the Practical Work

Figure 5.8 shows the design of the system required when field and office users are
active. This diagram is the same presented in Figure 4.2 however it is now introduced
the complete number of processes required to make the system run.

Figure 5.48 System Requirement between Office and Field Users

From the field users perspective it was necessary the implementation of new Java
classes, necessary to connect GPS receives, inclinometer and magnetic compass to
the data base Coordinates (Figure 5.8 – right side). From the office user side a class
was implemented to continuously read from the keyboard. The following sub-chapter

156
explores the MASSIVE-3, Equip and Java classes created to integrate external
devices to the system. The high level algorithm and the class’ constructor are
described on Appendix B. Some of these classes are based on/or adapted from
previous works developed also in the MRL, like Augurscope (Koleva et al., 2001),
and were eventually modified to fulfil the requirements of this project.

RTK - GPS Connection

Two Java classes were created to handle the information from the RTK-GPS receiver.
The first one reads and writes the messages from the receiver and it is named
GPSReceiver.java. The second class, GPSPublisher.java, requests the update of
Coordinates data base through Equip processes.

GPSReceiver.java (Appendix B.1.1) reads the data using a COM1 from the RTK-
GPS receiver. Before the communication is established, the transferring velocity and
parity bits are setup to start to receive the GPS messages. The GPS receiver first
converts the coordinates from WGS84 to OS grid in an internal program. The string
received is formatted using LLQ message format (see §4.3.1) and decoded to extract
east, north and height coordinates.

The normalisation between the base point and the coordinates received is the next
step. The user’s position is converted to the Cartesian system used in MASSIVE-3.
Figure 5.9 shows the axes orientation of Cartesian system used by the platform.

Figure 5.49 MASSIVE-3 Cartesian System

Figure 5.9 shows the North (physical world) represented by a negative Z in the VE.
When the RTK-GPS receiver is moved to north, the positive signal of the coordinate
should be changed to negative, otherwise the embodiment will be moved to South.

157
 The GPSPublisher.java constructs an instance of GPSReceiver.java and acts as
the interface between the data coming from the GPS receiver and the Coordinates
data space using the services provided by Equip. The GPSPublisher.java requests
from the library the creation of a data space where the coordinates coming from the
GPSReceiver.java are to be published. This data space is then ready to be accessed by
Integrator that interfaces with the CVE World data base. MASSIVE-3 is then finally
triggered updating the visualisation (Appendix B.1.2).

In resume, the processes involved when a new GPS data is read from the receiver
are:

 GPSReceiver
 GPSPublisher
 equip_net_trader
 equip_data_server – logging and reading from Coordinates data base
 Integrator
 equip_mas3_mgmtserver – logging and reading from CVE World data
base. This process also renders the CARS window.

All clients sharing the CVE World data base are advised about the new data. The
process bodyexampleclient receives and update the view on office user side.

Magnetic Compass and Inclinometer Connection

Like the RTK-GPS receiver, the magnetic compass and inclinometer circuit board has
two main Java classes to control communication and data logging: TCM2.java and
TCM2Publisher.java. The device was configured to send information only for the
compass direction, pitch and roll.

TCM2.Java (Appendix B.2.1) starts the communication between the circuit board
and the computer, setting up the baud rate and bit parity. Once communication has
been established, the algorithm searches for the message starting with $C (Table 4.2)
and finishing with a break line. The tokens are then split and an option for a Kalman
filter is available to reduce the jerk. The results are demonstrated on §5.3.1 and the

158
Kalman filter dataflow illustrating how it is implemented is shown on Appendix
B.2.3.

TCM2Publisher.java (Appendix B.2.2) receives the data and requests an update it

in Coordinates data space. Integrator is triggered and requests the input of the new
data in the CVE World data base. MASSIVE-3 is then advised about the new data. It
renders and updates the visualisation of the CARS window. The contents of the tilt
sensor and the magnetic compass messages are described in §4.3.1. The process
involved in handling the data coming from TCM2 are the same presented for the data
coming from the GPS receiver. The difference relies only in the classes TCM2 and
TCM2Publisher.

Consideration also had to be taken for the TCM2-50. The default communication
port used was COM1. However, this port was already in use by the RKT-GPS
receiver. A communication port conversion between USB and the serial port was
integrated into the system. Such port emulation makes the circuit board operational
on a different address of communication port.

The system at this stage is enabled to capture the head movements of the user and
his positioning over the ground. The system is then configured to work as a field user
in standalone mode where the first experiments merging real and virtual where taken.

Keyboard Connection
The keystroke capturing in the office user side has the same principles already
introduce when new data is coming from RTK-GPS or inclinometer and magnetic
compass. Every time that a key is stroked, the data is logged in the data base
controlled by Equip. From there to the update of the CVE World and the CARS
window the process is the same as explained before. The classes created to read and
publish the data are: ReadyKeyStroke (Appendix B.5.1) and KeyStroke (Appendix
B.5.2).

Integrator
The Java class that bridges the Coordinates data base with the CVE was named
Integrator. This class was inherited from the Augurscope project (Koleva et al., 2001,

159
Benford et al., 2003a) and several modifications were made. This class was designed
to attend to the following tasks:

 It creates a set of variables that fits in MASSIVE-3 structure.

 It creates a link between the data space Coordinates and MASSIVE-3.
 It creates a control for the data coming from keyboard, tilt sensor and
electronic compass. The algorithm is aware when a new data is logged into the
data space, requesting the update of the virtual camera inside the CVE.
 It configures the embodiment in the VE overwriting the configuration file
available in MASSIVE-3. The VFoV (see §4.3.1) is set also in this class.
 All digital objects requiring dynamic update are also created and controlled by
this class. The objects are created and placed in the CVE World.

Integrator is in a very broad view the middleware between the Coordinates data base
and the CVE World data base. This integration only happens because:

 Every new data from external devices triggers Integrator.

 The data leaves the Coordinate data base through:
o equip_data_server
o equip_net_trader
o Integrator
 Integrator processes the data if required – keyboard for example.
 Integrator updates CVE World data base.
 equip_mas3_mgmserver receives the new data and renders the image in
CARS window.

The high level algorithm covering this class is given in Appendix B.2.3.

5.3.1 Integrating Software, Hardware, Field and Office Users

The first step to start the system is to setup a wireless communication between the
computers. The procedure of how to setup a wave-LAN is not explained here. But
both computer (office and the field user) should be able to identify each other through

160
Packed Internet Gopher (PING) (Wikipedia, 2006) command for example, proving
that they can establish the exchange of packages.

The following paragraphs present the necessary command lines to run in order the
field user. It is not the intention to describe in depth each of the components. Instead,
the text aims to be useful as a base reference on “how to start it” in future projects.
Commands like: trader and the cveworld that were already introduced in §5.2 and are
not reviewed in this section again.

Field User Setup

The order in which to start the system for the field user is as presented below:

 trader
 cveworld
 equip_net_trader
 equip_data_server
 equip_mas3_mgmtserver
 equip_mas3_entitygenerator
 TCM2Publisher
 GPSPublisher
 Integrator
 audio_server
 audio_test

equip_net_trader
In the field user’s context, after the trader and cveworld have been started, the
equip_net_trader opens a communication port where the requests from the clients are
managed and controlled.

The equip_net_trader process is initialised in the context of this thesis using the
command line:

bamboo ./equip_net_trader

161
Bamboo is the platform used to give EQUIP flexibility to operate with both C++ and
Java program languages. Further details about Bamboo can be found in (Bamboo
Open Source Community (BOSC), 1999).

equip_data_server
The data space is opened by equip_data_server who deals with the requests from
clients to the data spaces running in the server. All the requests arrive through
equip_net_trader. The word space was adopted here to name the data_space_name
where the VE was created.

This data_space_name was named app.

equip_data_server process is initialised in the context of this thesis using the

command line:

bamboo ./equip_data_server -equip://<IP_address>/<data_space_name>

equip_mas3_mgmtserver
As indicated in (Greenhalgh, 2001f) the “management server creates and renders a
3D graphical view of a MASSIVE-3 virtual world”. This window is linked with the
data_space_name created for the VE and identified by the word space.

The equip_mas3_mgmtserver process is initialised in the context of this thesis

using the command line:

bamboo ./equip_mas3_mgmtserver -<config_file>

-equip://<IP_address>/<data_space_name>

Where:

 config_file – this configuration file is used by MASSIVE-3 to setup the

embodiment and the virtual camera inside the VE. This file is generally saved

162
 in c:\Massive\rev-4c\etc (assuming that Massive is installed in C:\ directory
and the version is rev-4c).

equip_mas3_entitygenerator
All non static objects are controlled by this component. They are linked with a textual
file that addresses the same data space used for the VE. Integrator interacts with this
process receiving the new coordinates of the object in a thread. Every time that the
thread updates the coordinates of an object Integrator updates the CVE World. All
clients receive this update and the CVE and CARS windows are refreshed.

equip_mas3_entitygenerator process is initialised in the context of this thesis

using the command line:

bamboo ./equip_mas3_entitygenerator -<file_name.txt>

The file with the address and the name of the space is exemplified below:

Name: entgen1
DefaultDataspace: equip://<IP_address>/<data_space_name>

The world entgen1 identifies the area where the coordinates (that makes an object
move inside the VE, for example) of the object are logged in the data space used for
the VE.

The world space was substituted by data_space_name.

TCM2Publisher
TCM2Publisher instantiates TCM2 class to establish the communication with the tilt
sensor and magnetic compass. It also creates an identifier inside of the Coordinates
data base to log the data in the right place.

The coordinates are read by TCM2 and processed by a Kalman Filter (see below).
After this process the data is then sent to be logged through the following processes:

163
  equip_net_trader
 equip_data_server

For this class, the javax communication libraries (Sun Microsystems, 2004) must be
present in the Java lib directory.

TCM2Publisher process is initialised in the context of this thesis using the

command line:

java TCM2Publisher equip://<IP_address>/

<data_space_name> <COM port>

The world app was substituted by data_space_name.

GPSPublisher
The GPS data logging is similar to the TCM2Publisher, but for the fact that it is
designed for the GPS receiver. It instantiates GPSReceiver class to establish the
connection and process the data coming form the GPS receiver. As in the previous
class it also requires the javax communication libraries.

The GPSPublisher process is initialised in the context of this thesis using the
command line:

java GPSPublisher equip://<IP_address>/<data_space_name> <COM port>

The world app was substituted by data_space_name.

Integrator
The Java class Integrator makes the link between Coordinates data base with the data
space allocated for the VE – CVE World. The class also controls the dynamic objects
updating the VE when necessary. It configures and creates a link with the
management server that is used to render and control the actions inside of the VE –
equip_mas3_mgmtserver.

164
 The Integrator Java class is initialised in the context of this thesis using the
command line:

java Integrator equip:// <IP_address>/<data_space_name> equip://

<IP_address>/<data_space_name> <config_file> <IP_address>/<ID>

Where:

 data_space_name – the first data_space_name refers to the data space

allocated to the VE. The second one is the space allocated for the data coming
from the GPS receiver and the TCM2-50.
 config_file – this configuration file is used by MASSIVE-3 to setup the
embodiment and the virtual camera inside the VE. This file is generally saved
in c:\Massive\rev-4c\etc (assuming that Massive was installed in C:\ directory
and the version is rev-4c).

audio_server
The audio channel is setup after the audio server opens a space where the computers
involved in the communication are able to connect. This program should be run in all
computers that want to use the audio channel.

The audio_server process is initialised in the context of this thesis using the
command line:

audio_server

audio_test
This program establishes the link between the two machines through the audio server
making the transmission of the audio stream. The command must be run on all
machines involved on the communication.

165
 The audio_test process is initialised in the context of this thesis using the
command line:

audio_test -h <IP_address machine_1> accept < IP_address machine_2>

and
audio_test -h <IP_address machine_1> send <IP_address machine_2>

Office User Setup – following Figure 5.8

Commands like: trader and the bodyexampleclient, previously introduced in §5.2,
will not be described again. The same applies to equip_net_trader, audio_server and
audio_test – see §5.2.

The order in which the office user should start the system is as presented below:

 trader
 bodyexampleclient
 equip_net_trader
 KeyStroke
 audio_server
 audio_test

KeyStroke
This Java program class reads from the keyboard and logs the key pressed in
Coordinates database. The process involved in this action is similar to the one used to
describe the logging of new GPS coordinates in the beginning of this sub-chapter.

The KeyStroke is initialised in the context of this thesis using the command line:

java KeyStroke equip://<IP_address>/ <data_space_name>

The word app was substituted by data_space_name.

166
Kalman Filter
The class that implements the Kalman filter is called internally by those classes that
communicate with the GPS Receiver and with the tilt sensor and the magnetic
compass.

The implementation of a Kalman filter was necessary after some tests showed that
the magnetic compass suffered a constant jerk. Such movement happened even when
the TCM2-50 was completely stable. For users, the jerk provoked the uncomfortable
sensation that the VE was always “shaking”. Even with constant calibration
(recommended by the manufacturer), the tracker’s distortion was aggravated by the
soft magnetic field created by the backpack frame plus the usual magnetic distortions
present in the RE.

Based on (Hide, 2003) this filter is well suited to navigation applications, which
contains states that are time-variant, even in real-time. It is a recursive algorithm,
which filters measurements using knowledge of the system dynamics and the
statistical properties of the system measurement errors. Further details of how the
filter was implemented are showed in Appendix B.2.3.

The same filter but with different weights was also be applied to the GPS
coordinates even though the receivers already come with a Kalman Filter algorithm
implemented at hardware level.

As reported in (Capra et al., 2003b) the results achieved showed a reduction on

the jerking, resulting in a smoother view of the VE for the field users when moving
their heads. Figures 5.10 to 5.12 show 60 samples of the data related with magnetic
compass, pitch and roll. The blue line shows the raw data from the tilt sensor and the
pink line the same data after being processed with the Kalman filter. In all cases, it is
noticeable that the lines of the graphs are smoothed. The sharp edges on the blue line
were replaced by a bend curve. The result for the field user is an image stabilised with
a small delay, from the stop of the head’s movement until the update of the latest
frames.

167
 Magnetic Com pass

300

Degrees
200
100
0
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61
Samples

Without Kalman Filter With Kalman Filter

Figure 5.50 Magnetic Compass data

Pitch

50
Degrees

0
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61
-50
Sam ples

Without Kalman Filter With Kalman Filter

Figure 5.51 Pitch Data

Roll

40
20
Degrees

0
-20 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61
-40
-60
Samples

Without Kalman Filter With Kalman Filter

Figure 5.52 Roll Data

Calibration
Even though calibration is not part of this thesis, at the early stages of the project
some tests were performed to identify how much error could be observed visually. To
implement the experiment, four stacks with approximately half a metre high and few
meters far from each other were surveyed and set as targets in RE. Each target was
superimposed by a digital object (a cylinder for each stack). The results showed that

168
visually the digital cylinders were never projected further than a few centimetres.
This experiment perhaps was not deeply technical and as accurate as it should have
been, especially for an AR system. However, it showed that the projection, as it was,
would be good enough for the desired aim of this thesis.

Following Azuma (Azuma, 1995) it is possible to list the main sources of static
errors such as: distortions in the HMD, mechanical misalignments in the HMD or
incorrect viewing parameters and errors in the tracker. This reference is a good source
of information if the reader would like to know more about processes of calibration
for AR systems.

5.3.2 Notes of How to Use the System

Complementary notice: as soon as the Integrator class starts to run in the field user’s
machine, one of processes will ask for a major share of the CPU. The users are
advised to access the Windows Task Manager (this thesis was performed using
Windows 2000 as operational system) and then Process menu option. Sort the
processes by CPU usage. One of the tasks (one of the Bamboo tasks) shows a very
high percentage of CPU usage. To reduce it, click with the right button of mouse in
the task and change the priority to low. This procedure helps to avoid the freeze of the
system by CPU overload.

169
Chapter 6

Evaluation and Results

Once the configurations and settings established in the laboratory have already been
explored / taken into consideration, the next stage is to evaluate and analyse the
results achieved within this project. Perhaps the most difficult part in the evaluation
process of an AR system based outdoors is the fact that it should be performed
outside the laboratory. The logistics and the orchestration of all parts involved are
extensive, requiring attention to small details. A missing cable or battery that was not
charged properly can put at risk the entire operation.

At this stage, the integration of the system was concluded giving field and office
users the opportunity to share the same CVE. The embodiments helped users with
feelings of presence and collaboration and thanks to an audio channel, working
similar to a telephone line, field user could always have someone placed remotely to
guide and help inside of the CVE.

This chapter introduces three examples created to demonstrate the use of this
project in the RE. The first two examples are based on real situations. The third
example is a fictional construction created to show other potentialities of the tool
during the evaluation process.
The examples are:

170
  A tram station – based on a real tram station in Nottingham, United Kingdom.
 A house – based on a real construction in São Francisco de Paula, Brazil.
 A sketch of a construction – showing some extra features of CARS to users.

The evaluation process was performed with a small group representing potential final
users for the project. Those users were choose because their expertise in the areas
where they work. They are civil engineers, researchers in computer science,
researcher in fluid dynamics, researchers in high accuracy GPS, researchers in
photogrammetry, researchers in VR, and a person with background in computer
science and state-of-art technology.

For the evaluation process it is followed two main techniques to extract

information from users when exploring the CARS developed:

 Think Aloud (Someren et al., 1994, Mountain and Liarokapis, 2007,

Wikipedia, 2008f) – in this technique users are requested to say loudly
whatever they are experiencing during the evaluation. A camera man is hired
to stay with the users all the time, recording everything that he is doing and
saying. The office user (during the test it was performed by the author of the
thesis) also put questions, interacting with the field user (through the voice
channel), taking notes of some of the key points. Users in the field kept
continuously interacting, mentioning positive and negative aspects about the
system.
 Cognitive Walkthrough (Angewandte und Sozialorientierte Informatik -
Universität Hamburg, , Zhang, 2005, Wikipedia, 2008a, Abowd, 1995) – users
are instructed to what to do, how to do, what to see, how to communicate with
office user, etc. During the presentation of the exercises users are reminded
where they could find the VE in the RE and what they would able to see there.
Although there is not formal script for the exercises performed, they run in
very similar way to each other. All the processes are recorded to be analysed
after finished the trials. The results then shows where the project must be
better designed, especially where users could not reach the end of the tasks or
where the solution was not obvious to be achieved.

171
Historically, MRL uses a formative approach (Abele et al., 2005) to evaluate its
projects. Once a laboratory where technology has been developed as research and not
as final product, the usual evaluation process is focused on identification of points
that require improvement during the development process.

Formative approach is mentioned in (Vicente, 1999) as:

Formative approaches focus on identifying requirements – both technological and

organizational – that need to be satisfied if a device is going to support work effectively.
Although these requirements do not uniquely specify a new design, they are still highly
informative and thus very valuable because they can be used to rule out many design
alternatives.

The evaluation process followed by this thesis did not concern in how long it took for
a user to complete a task or how many tasks were completed. The criteria used are
focused in questions like: can the user complete a task; can the user interact with the
system at all; is the hardware working as desired; is the VE fulfilling users’
expectations? These are broad questions that were answered when users were
interacting with the system during the trial and what they answered along the
interview.

This chapter contains excerpts from interviews recorded when users when testing
the CARS created. The interviews were transcribed in Appendix C.

6.1 Three Exercises for the Evaluation

The way choose to demonstrate a CARS project for CEn purposes was to make it
happen with examples that had been or could be implemented in the RE.

The first example is the tram system that was constructed throughout the city of
Nottingham – UK, with the supervision of the Nottingham Express Transit (NET) and
implemented by Carillion.

The public tram services started in November 2003, removing the need of two million car
journeys a year and helping to make the city cleaner, less congested and even more
attractive. It is estimated that around 11 million passengers will be transported every year

172
 representing up to 30,000 people in a single day. The first line, named Line 1, has 23
tram stations – including 5 Park & Ride stations and it extends for 14Km. The users have
a faster and safer alternative for going through the city, in comparison to another type of
transport (NET, 2001).

In this example, a scene involving one of the stations and also an object representing
the tram was implemented. The reasons to implement this exercise are:

 It is a place where most of the interviewed users saw or visited before. Users
already arrived with some previous knowledge of what was being showed
before their eyes when exploring CARS.
 In a potential real scenario AEC actors could take advantage of CARS to work
over the necessities faced when in design and construction phases (see
Chapter 2).
 The use of CARS to enhance communication, collaboration and mutual
understanding in the place where the construction was set.
 Field and office users working remotely without the necessity of staying in
office.

To enrich the scene it was implemented a tram that approaches the virtual tram
station, stops for a little while, and moves away again. The benefits to add a dynamic
object in the scene are:

 The scene is closed to reality – the virtual tram is also following the scale
of a real tram, making users feeling more like in a tram station.
 The potential use of the tool – the dynamic object (the tram) could be
substituted by any other object in a different context. For instance it could
indicate wind corridors in a city or the way that water would flows in a case of
torrential rain.

The second example is based in a real construction. Although the example looks
simple in a first instance, this exercise is created to demonstrate:

 Exploring previous knowledge about a specific scene – users already arrived

with a previous knowledge of what is a house. Even though houses can have

173
 its rooms arranged differently, they have some common areas like: kitchen,
bedroom, bathroom, living room, etc.
 Checking details of the project – it explores the possibility of using CARS as a
tool to keep building projects on schedule, checking details like layers
collision, etc. This is an important topic that was already demonstrated in
§2.1.2 and a real necessity for ACE actors during the design and construction
phases of a CEn project.
 Control of what was being analysed – this example stresses the collaboration
between office and field users. Office users can add and take out layers of
what was being demonstrated at same time that communication and mutual
understanding happen.
 Collaboration between AEC – as mentioned in (Khanzode et al., 2008) actors
(in special engineers and architects) got surprised by the number of hours
spent with constructors inside of the meeting room just to clear obscured
points. Once again the exercise aims to demonstrate how practical it is for
those doing inspections on site to interact with people in the office.

The final example is another object representing a house. At this time it is not based
in any real situation. This example aims to achieve:

 Drilling information – instead of only show the shape of a house and its
phases of construction users are able to drill different level of information
like:
o Displacement of windows, doors, and stairs – actors could check for
instance if the windows are facing the desired direction.
o Water, electricity, gas, etc. – the scene is composed by pipes
representing some of the supplies of the house. Users could check if
the supplies are well distributed in the house for instance. This idea
could be extended if the VE contains house’s furniture. It could help
users to make sure that no supply (electrical or water) would get
hidden behind a piece of furniture for instance. Users could also check
if the furniture will not block a view from a specific angle in RE.

174
For general information, exercises 1 and 2 are based on real blue print specifications.
The measures are as correct as possible. However the design of the VE was created
using none CAD software by the author – that is not an architect or engineer. The
third example was created for illustration proposes only, fowling none blueprints
neither CAD software.

Tram Station – Exercise 1 – Animation Feature

The second phase of this thesis started with the design of the furniture of a tram
station. Based on the bi-dimensional plan (Figure 6.1) the objects were designed with
the three-dimensional editor AC3D (AC3D, 2004), the same used to draw the IESSG
building example. In the plan, one can see the dimensions of the platform and the
furniture of this tram stop. The following items were designed: sign, bin, lamp post,
tram shelter, and platform – Figures 6.2 to 6.6.

Figure 6.53 Blueprint of Phoenix Tram Station (Carillion, 2000)

175
 Figure 6.55 Sign and Lamp Post
Figure 6.54 Sign and Bin

Figure 6.56 Lamp Post

Figure 6.57 Tram Shelter

Figure 6.58 Platform

The scale and colours of each of the furniture are closely related to the specification
given by Carillion, apart from the platform that is not blue as shown in Figure 6.6.
The displacement of the objects in the VE follows the blue print. The design and

176
creation of the .cve file is facilitated by the know-how of what had previously been
done to IESSG VE. Figure 6.7 shows how the model looked when all features were
placed together.

Figure 6.59 Tram Station

Figures 6.8 and 6.9 are showing users (field and office users) sharing the same CVE.
Figure 6.8 shows the bird’s eye view of office user.

Figure 6.60 Tram Station – Top View Figure 6.61 Tram Station – CVE

Figure 6.10 shows a model of a tram approaching the station. The tram is a simplified
version of a model found on (Gilbert et al., 2002). The model is simplified due to
limitations of the number of facets imposed by the program dgl_convert (see §5.1.1).
The final model of the VE is a reasonable copy of how the RE is. The animation is a
key feature of this example stressing the potentiality of the visual augmentation for
applications that could require the simulation results in animated form together with
the VE.

177
 Figure 6.62 Virtual Train Approaching the Station

Figure 6.11 shows how the platform looks in the RE.

Figure 6.63 Platform in the Physical World (Nottingham Tram Consortium, 2004)

The picture of the actual site where the platform works as a final certification that
what had been evaluated by the users during the exercises reasonably corresponds to
RE.

House1 – Exercise 2 – Phases of an Engineering Project

The second example is also based on a real construction. This model was created to
show AEC and customer the construction on different phases during the design and
construction phases.

The blue print chart – Figures 6.12 – shows the internal plant of the house.

178
 Figure 6.64 Internal Division (Lucena, 1986)

Figure 6.13 shows the right side of the house following the blue chart print.

Figure 6.65 Right Side Scheme (Lucena, 1986)

This example shows how the house should look and also highlights the expected
commitment of those involved in an engineering project. Figure 6.14 is a snap shot of
the project with four stages of the construction implemented. Each stage is
represented by a different colour. During the process of visualisation users could see
the stages being placed on the top of each other or just the required layers. The
advantage is to illustrate every single phase and analyse it in a clear way.

179
Figure 6.66 Colours Representing Different Stages of the Engineering Project
(Design and Construction Phases)

Figure 6.15 shows the house built with different coloured layers. Each layer can have
a specific meaning (that can be stipulate by AEC) during the design and construction
phases.

Figure 6.67 The Complete Construction (Design and Construction Phases)

Figure 6.16 shows how the house looked at the end of the project – final construction
phase on the RE.

Figure 6.68 The House as it Looks in the Physical World

180
House2 – Exercise 3 – Drilling Information
The final example introduced in this thesis is a house where the users could drill
through information such as: the structure, the pipes (gas, electricity, etc.) and the
distribution of windows.

Figure 6.17 shows the house from outside to give an idea of how the shape of it.
Figure 6.18 shows where the windows of the house are as well as the positioning of
the stairs. In a project of a country house, for example, CARS would show the owners
of the property if they would have the best views when examining the displacement
of the windows.

The final layer of information in this example was the pipes that surround the
structure of the house as well as the underground. Figure 6.19 shows different pipes
with different colours. Each pipe could represent different meanings and follow
specifications.

Figure 6.69 The Structure of the House Figure 6.70 Windows and Stairs

Figure 6.71 Different Colours of the Pipes Indicating Different Uses

181
6.2 Evaluation

The evaluation process was performed several times during the existence of the
project. In the very early stages, most of the evaluation was just to confirm that the
system was working reliable when all the hardware and software were running at the
same time. Eventually some users were invited to check again the evolution of the
project. Throughout the entire process the project had three formal demonstrations
and several informal ones. During the first two presentations the evaluation was more
concerned with the points on which the tool should be improved. No interviews were
done and the users just expressed their comments to the author during conversation.

In order to give more flexibility to setup and demonstrate the project in both
campuses two GPS references base points were surveyed. The first was taken inside
of The University of Nottingham Main Campus close to the Sports Centre. The
second was in The University of Nottingham Jubilee Campus, on the island in front
of The Atrium building.

The coordinates (in WGS84) of the base point used close to the Sports Centre are:

 Latitude North: 52° 56’ 26.32994”.

 Longitude West: 01° 12’ 22.07388”.
 High: 88.7259m.

The coordinates (in WGS84) of the base point used on the island are:

 Latitude North: 52° 57’ 10.74172”.

 Longitude West: 01° 11’ 15.04220”.
 High: 78.271m.

These sites had advantages and disadvantages to work in. The signal captured in the
Sports Centre did not suffer as much multi-path as the experiment that took place in
the Jubilee Campus.

182
 A multi-path effect can be noticed when the GPS signal reflects on buildings, the
terrain or any other object between the satellite and the receiver. The reflected signal
arrives with some delay at the receiver causing an inaccurate coordinate (see §3.9).
This can be confirmed by the constant automatic changes of the GPS receiver from
RTK-GPS to DGPS. Both receivers (reference station and rover) were eventually too
close to a number of three-store buildings and trees in Jubilee Campus. Another
problem is the water surrounding the island. At least once in early stages, a user was
reminded that she was walking too close to the border of the island.

However in the Sport Centre the challenge is to keep the demonstration running
for long periods of time only using batteries as power supply. There is no electrical
point and everything has to be running at the moment that users arrive. In the island
on Jubilee Campus this is not a real problem. The place is not far from an electrical
socket, making the process free of the stress caused by batteries running flat. Any
required modification in the configurations or code that required debugging could be
easily done as longer as necessary once the computers are connected to a power line.

For the final evaluation the green area behind Willoughby Hall inside The
University of Nottingham Main Campus was chosen for the performance and testing.
This place has the following advantages:

 As the other areas, this place is flat. The evaluation is unlikely to run the risk
of users going up and down small hills or stairs. The area is also not
surrounded by water as in Jubilee Campus.
 There are no buildings or trees too close to where receivers would be used,
reducing the chance of multi-path.
 Electrical sockets are less then 50 metres (close enough for an extension).
 Willoughby Hall also offered the possibility to store the equipment safely in
one of the rooms, reducing drastically the time for setting up the equipment.

The coordinates (in WGS84) of the base point used in Willoughby Hall are:

 Latitude North: 52° 56' 05.87635".

183
  Longitude West: 1° 12' 20.57715".
 High: 77.7971m.

6.2.1 Pre-Test before the Final Evaluation – Final Adjustments

Just before the process of evaluation started, a pre-test was performed to ensure that
everything was running as expected. As any other project, there are always some last
minute minor adjustments to be done. This final pre-test is valuable because it should
be performed not by the author of the project but by an outsider. The advantage of the
input of a “fresh mind” that does not know how to use the system is to show to the
evaluator some missing points of how the system is going to be used.

This final adjustment is probably one of the most important points through the
perspective of final evaluation. If the developer and the evaluator are the same
person, for sure, he had to undergo a kind of self-adaptation to the system. This
adaptation could be the skills developed to type in the handheld keyboard or the best
speed to walk with the equipment, for example.

Another point is the way that the user would wear the equipment. Not all users
know how to wear it and what looks trivial for the developer could not be obvious for
users. The process of equipping the user should be as delicate as possible. Not just
because of the number of cables and fragile equipment that are involved, but also
because the user is receiving an extra layer of “content” over his body. The more the
user feel confident the more he will have a good time and his concerns will be
focused in the evaluation and not on whether he is looking good or doing well.

Positive Points of the Pre-Test

Previous tests showed a number of details that could make some difference in a real
performance. The required adjustments were implemented to optimise the experience
of the field user.

A line (a white long tape was used) in the grass marking where the VE should be
projected helped the user to identify which direction he should go in the RE. This
eliminated the need to tell constantly where users should walk. By following the line,

184
the user would automatically go to the right direction and would not be too far from
there. In previous experiences, when users does not have the facility of the white
mark they would spend more time wondering around rather than focusing their
attention in the CARS.

A wave-LAN router was used to create the link between the computers rather than
using only simple peer-to-peer configuration with computers linked purely by their
network cards. The advantages are: the signal is stronger and the router is placed half
way through the computers, enabling users to move further away from each other.

This system is more stable when compared with the previous versions. The
jerking was drastically reduced by the Kalman filter (see §5.3.1) implementation and
the installation of a faster hard disk in the notebook. The GPS signal is not affected
by multi-path as much as it is in the Jubilee Campus. This is attested by the
significant reduction of the VE drift (visual perception only). The receivers usually
locked the signal faster in the chosen green area and kept it for most of the time.

Designing more than one exercise can enrich the project significantly. In previous
performances the users were introduced only to a virtual tram station. During the final
evaluation users had a better overview of the issues covered by the project with the
three exercises created.

Learning Points from the Pre-Test

The wave-LAN router should be placed as close as possible to the area where the VE
is being projected as well as far from the ground. It was noticed that the system
stopped working – freezes – when the connection between users is lost – around 50
metres from the router.

At this stage field users could interact with the VE through a handheld keyboard.
This device is not suitable for a user without previous training. The device is helpful
if users know how to work with it but it becomes unfriendly for a user that is not used
to handle it and is already caring all the equipment in the backpack and wearing the
HMD. Instruction “on the fly” is impossible and most certainly the person will not

185
understand how it should work, getting frustrated very quickly. As the user said
during the pre-test:

“It is phenomenally difficult, unless you know what you are doing”.

6.2.2 The Final Evaluation

The final evaluation of this project is directed by the following steps:

 Embedded Test – all users were invited to test the system. All actions were
recorded for later analyses.
 Post Test with Interview – an interview just after the embedded test to extract
more information from users. Users were questioned not only about the
system but about themselves when using the system. Most of the questions
were repeatedly made to users but some of them were tailored to particular
points noticed during the trial.

It was invited for the final evaluation a small but well representative number of
people to interact with CARS. In total, eight people got involved during the
evaluation. These people were chosen based their backgrounds and their previous
experience on evaluate projects.

 Four civil engineers – with background in space geodesy, fluid simulation,

photogrammetry, AR systems, high accuracy GPS, etc.
 One architect – with experience in AR systems and VE design, also closely
linked with computer sciences projects.
 Two computer scientists – with knowledge in simulation, AR systems, VE,
GPS positioning, CE and etc.
 One ordinary person – a young male, qualified to evaluate AR systems and
computer games.

The users were invited to come for the evaluation at different times and days. This
procedure prevented their opinion being influenced by any discussion between them.

186
The exercises for each individual, three in total, took around fifteen minutes each to
be completed and the interviews after the trial around thirty minutes.

The exercises are not guided by an exact path where field users should go or by a
time frame to complete one or more tasks. However there is a pattern in the exercises
covering the following points:

Tram Station – exercise 1

 VE Identification – field users should be able to understand and recognise the
tram station associating it to their previous background. They should also be
able to identify the displacement of the furniture or the colour of it.
 Scale – users are invited to stay on the virtual tram shelter and have the
perception of scale.
 Small Details – users are also motivated to find an object representing a
Close-Circuit Television (CCTV) in the top of a virtual lamp post. This task is
particular interesting because it required users looking up trying to find a
small virtual object.
 Animation – users are also invited to stay over the platform to wait for the
tram to approach and depart.
 Broad View – field users are requested to walk away from the virtual tram
station to see the entire VE with the tram approaching and departing.

House 1 – exercise 2
 VE Identification – field users should be able to identify the house and the
rooms that shaped it based on previous knowledge of a house.
 Layer by Layer – field users are exposed to different segments of the house
construction. With only few layers with distinctive colours users are invited to
explore the rooms, find the doors and windows, stay in the veranda, etc.

House 2 – exercise 3
 Change of the VE – field users are exposed to a complete changed of layer of
information although all of them have the same reference (a house). Users are
again asked to go and explore the VE.

187
  Identification of Particular Points of Interest – users are requested to look and
walk towards the stairs. When they are closed to stairs the VE is switched
again. Users are requested to look up to find a whole in the ceiling where the
stairs climbs to.
 Be surrounded by the VE – this VE is designed larger than the other two
examples. Users should have the feeling to be surrounded by the VE where
they need to walk around 25 meters to get away from the house.

The evaluation was created to cover the topics shown on Figure 6.20.

Figure 6.72 Topics Covered by the Evaluation

 Technical – regarding to technical aspect of software and hardware used

during the exercises.
o Software: the system presented in this thesis does still a prototype. The
research is focused on the validation of CARS in CEn projects (design
and construction phases). Even though the CARS is not a final
product, the evaluation covered how reliable the system is during the
exercises.
o Technical – Hardware: GPS receiver and quality of output, HMD,
wave LAN, and inclinometer and magnetic compass are also
evaluated. The output of these devices highly affected the field users’
experience.
 Tasks – what users did during the exercises?

188
 o Instructions: how instructions are passed to field users. The role of the
office user as a collaborator in the VE.
o Reaching specific goals: how the tasks are performed by users
(although the evaluation run over a formative approach), if they could
perform what is requested, the difficulties that they faced, etc.
 Interacting with the System – how users bound with the system.
o Behaviour: How users behave when exploring the CARS.
o Immersion: Is it an AR or a VE experience?
 VE Quality of the Scene – could users identify virtual objects/spaces and
associate them with real ones? What users expect to see when exploring a
VE?
o Texture and colours: could users indentify where they are inside of the
virtual house (bathroom, living room, etc.)?
o Dynamic Objects: Does dynamic object bring any improvement to the
scene?
o Reference Points: Are users orientated when walking in the RE with
the see-through HMD on?
o Avatar: The visual aspect of the avatar and how much it represents
during the experiment.
 Office User’s Control – the aspects of the remote control of the VE by the
office user.

Communication, collaboration and mutual understanding are not evaluated in this

subchapter. The three keywords are commented in the next subchapter (see §6.3)

6.2.3 Outcome from the Exercises

The following paragraphs are structured as the evaluated topics introduced on Figure
6.20. They summarise the main points of the video material recorded by the
cameraman and the author – as office user – during the exercises and interviews. The
topics are commented along the excerpts.

189
 For the sake of anonymity field users are identified as: Engineer1 (E1), Engineer2
(E2), Engineer3 (E3), Engineer4 (E4), Architect1 (A1), Computer Scientist1 (C1),
Computer Scientist2 (C2) and Ordinary user (O1).
The exercises are identified as: tram station (exercise 1), house 1 (exercise 2) and
house 2 (exercise 3).

Technical – Software Reliability

As mentioned in §5.3.1 the computer carried by field user was running eleven parallel
processes with two data bases at same time. The system is constantly reliable until
some external factor affects its behaviour. Mostly this is an issue regarding to the
wave-LAN. The router is not strong enough to send the signal further then forty to
fifty metres. If eventually the field user is out of the range, the audio channel stopped
work as well as the other process making the system crash.

The lost of connection makes field user completely disorientated without the
power to restart the system by himself.

E1: I do not know where I am going. I am picking up the dialog screen at the
moment that it is picking up some of the environment. The brightness is also
wrong and I can see only the physical world and not the VE. If you can get rid of
these dialog screens it would be helpful. You cannot probably do it from there.

In a second occasion during a different exercise E1 faced the same problem again.

E1: Oops, it has failed and I have an error screen now.

This is the only moment that a field user went through a problem like this. The
solution is to place the wave-LAN router closer to the area where field users are
performing the test. The best solution would be to place a powerful antenna to cover
all the area plus some hidden points where a standard wave-LAN router could not
reach.

Technical – Hardware RTK-GPS

The RTK-GPS receiver used is not as small as the models used for car navigation for
instance. It requires heavy duty battery, cables, radio transmitters and a pole with a

190
specific antenna. The antenna by default should be above the user’s head (around two
meters) to reduce multipath. The pole is part of the backpack frame carried by field
user.
Ergonomic – In overall the system is not ergonomic. It is heavy specially because the
weigh and size of RTK-GPS receiver. The number of cable also makes the system
complex to setup and to carry.

Along the exercises it is noticed that a number of users getting stuck with the GPS
antenna in the branches of a tree.

E1: I got caught in the tree with the GPS receiver because I could not see the
tree!

E2: I think that I am catching a tree with the antenna.

E3: You have asked how immersed I was and that was how immersed I was in a
tree.

A1: I need to make sure that I will not run into that tree.

For the evaluation point of view there is a mix of conclusions regarding to hardware
design and user immersion in the VE. The high of the antenna can make the
difference for the ergonomic perspective. Field users like E2 and A1 are no more than
175cm height. It is possible to imagine that they did not think in giving some extra
room to pass below the obstacle. However E1 and E3 have something close to 200cm
height. In any circumstance they would not go towards the tree direction. This can
drive to the HMD setup hypothesis (see below). If the device is too dark the user will
not see through the lenses. But the tree was no more then 10 meters far from the users
during the briefing of the project. They saw the tree and they knew that it is the only
tree around. Users can get too immersed in the VE forgetting completely about the
RE and the hazards that this can offer (see below).

Accuracy – The default output rate of RTK-GPS receiver is one sample per second.
This is the fastest speed that the receiver can work when calculating RTK-GPS
coordinate in real time in the receiver. When users were in the field, they needed to
bear in mind that they should walk slower than usual. The rendering of the CARS is

191
affected if users speed up the walking, making the coordinates far from each other.
The excerpts bellow show users having the sensation that the VE jumping from point
to point.

E1: The environment is jumping a lot at the moment but if I steady on, it gets
better.

E2: The image is jumping a little bit.

A1: It is difficult to track, it is very jumpy.

A1: The frame rate makes it sometimes difficult to orientate yourself. I guess you
learn to live with that. It is sometimes difficult to use because sometimes you
cannot necessarily fix your view to a digital object. It just jumps over that point.

C2: It is jerking a little bit as I move but I still have the impression that it has a
consistent position in front of me.

The only way to go around this problem was walking slower than usual. It is
suggested something between 0.5 to 1 meter per second. However there is no rule for
this and users should find the speed where they would feel comfortable with the
visualisation. MASSIVE-3 already renders extra frames between two coordinates but
this also find a limit by the speed of the person in the RE.

The Kalman Filter implemented does not bring too much for the coordinates. In
fact, GPS receivers already contain a Kalman Filter to reduce high discrepancies
between coordinates.

Setting Up – every new exercise requirs users coming back to a place where the
system should be setup again with the new VE. This place is very close to a building
where the GPS signal is mostly blocked. After the setup of the VE and the restarting
of all process, GPS receiver must to stay exposed to clear sky to fix the GPS signal.
Finally the process could start again. However eventually the receiver does not fix
GPS signal long enough, loosing precision after a short period of time.

E1: The main problem is the perspective. I do not feel that I am walking towards
or through it.

192
By default CARS is set to be at (0, 0, 0) position when the GPS receiver is not
updating the data space with new coordinates. This became very disturbing for the
user because the person could walk in the RE but the VE is fixed to one position.
Meanwhile inclinometer and magnetic compass are working meaning that the image
is following the movement of user head. The solution relies in leave the RTK-GPS
receiver fixing GPS signal for longer.

Technical – Hardware Inclinometer and Magnetic Compass

The inclinometer and the magnetic compass attached in the crash helmet are
responsive to the iron in the frame of the backpack. This caused a jerking due to this
soft magnetic field. The jerking was considerably reduced by the implementation of a
Kalman Filter. The reduction of jerking can be seen in Figures 5.10 to 5.12.

Some of the reactions from field users are presented below:

A1: When I stand still it is ok but when I move it jumps a lot. I think that it is the
frame rate rather than anything else. It is a very low frame rate.

C1: Because my head…oh god, this is really creepy… because my head keeps
moving… I feel like I’m in the tram. Because my head keeps moving this jumping
drives me really really mad.

C1: If I stand still it is fine. If I turn my head really slowly it is also fine. I need to
restrict my behaviour and movements to suit the VE, what is not natural for me.

C2: There is a bit of delay when moving the head.

The market already offers some other devices that are not so affected by magnetic
fields – (Xsens, 2008) for instance. The reduction of jerk through Kalman Filter
brought another problem: the small delay in the movement when users turn their
heads too fast. Figure 5.10 to 5.12 already shown this delay when the graph lines are
not very close together. This delay can be perceptible by users causing some
discomfort.

Technical – HMD
The HMD used was not designed for AR systems. It is a personal wearable screen
used to watch movies for example. The size and the weight of the device are

193
reasonable good as well as the quality of image. However this device was designed
to work in controlled environments like inside of a room for example. The light of the
room can be controlled making the environment stable. The Sun light and the
shadows of clouds makes the setup or contrast and bright barely impossible when
using this equipment outdoors. If too dark, users could not see the RE, affecting then
the AR experience. Users cannot see the VE if the environment is too bright. There is
no auto adjustment for the device for variable conditions.

Another problem in the design is the light coming from around the frame of the
goggles. The external light is usually much brighter than the one filtered by the
lenses. This light became inevitable uncomfortable for users.

E1: I can see the tram and the VE but I cannot see the physical world so well
behind it.

E2: I can see something! It looks like street lights!

E2: The lenses are a bit dark but I know where the cameraman is (pointing with
his hand) and I can see grass. I have to say that, the part where I see the tram, I
do not see-through. I can see the bottom part and sides. I think that there is quite
a lot of light coming from the side of the HMD.

E2: I trust the cameraman because I do not know where I am going.

E2: When I am looking to the direction of the embodiment, there is a lot of light
coming from the Sun and I cannot see properly.

A1: It is the usual thing; I cannot see anything through the glasses. I have to look
under the glasses to not fall over. It is just the VE in the glasses. It is too dark, the
HMD set up.

A1: I am looking right into the Sun so I cannot see anything at all.

O1: I can see the physical environment below but not in front.

The HMD shows to be a critical device on the whole experiment. This is in part not a
surprise once the device is not designed for this purpose. However this is the only
device available and the design architecture created for this project requests a see-
through HMD.

194
 Some of the users put their hand in front of the device, blocking the view of the
RE to be able to see the VE. There are even users that walked completely “blind” in
the RE for long time just relying in what they are seeing in the VE. It is important to
mention that none real object was mapped in the RE to populate the VE.

The problem of the adjustment of see-through is so critical that during the

evaluation E2 is requested to look up to see some details in the VE like CCTVs
(exercise 1) or a hole in the first floor of the house (exercise 2). Eventually his vision
is blocked by the light from the Sun that came from the opposite direction making
him temporarily blind. The opposite happened when he looks at the same direction
where the Sun is pointing to. The HMD became too dark to see the RE.

When doing the evaluation, one of the concerns is to understand what sort of
immersion the user could experience. Users were asked if they could see VE and RE
seamless. Eventually, the recorded videos show users totally immersed in the VE
without noticing the RE (see more about this topic in the following paragraphs).

Technical – Wave-LAN
The router used for the wave-LAN connection was designed for domestic use only
with a very limited range. As already described there is problem when a user moves
over the router’s range causing the crash of the system. For a real application, the
solution relies over wireless Internet access or a powerful antenna.

Tasks – Instructions
One way to make sure that the field user is going in the direction suggested by the
office user in the RE is to ask him to look for the office user’s embodiment and to
walk towards it. The aim is not only to show the presence of a second user in a
collaborative way but also to pass instructions from the right position in the VE.

The excerpts and the comments below exemplify the way that the instructions
were eventually passed to and received by field users:

E1: I can see your embodiment and when you move.

195
Each user has a different approach when receiving a command or orientation. There
are people to whom just a request like: “-please turn right and look for the CCTV in
the lamp post” is enough as a command. They know what to look for and even if they
could not find the target immediately they would glance over the VE on their right
side that could look like a lamp post and then a CCTV. For E2 this command for
example had to be slightly different. When he is requested to turn right he replied
with the question:

E2: how many degrees?

After that, every time that E2 is requested to turn left or right, the number of degrees
has also to be included in the command.

During the exercises, E2 could not find the CCTVs and dropped the idea of
looking for them. Not letting the user down is one of the points to ensure users
engaged in the system. In case they cannot find a particular point, the task is moved to
an easier that is prepared to keep the user motivated. In this case, the user is requested
to look for a bin very close to the lamp post. The user replied that he could not find it.
However, because the office user is following the embodiment of the field user, he
knew that from that position it is possible to find the bin.

E2: I was looking too high. I have the bin right in front of me.

At this point E2 is better understanding the technology and he learns that the
movements with his head should be slower and smoother then the usual. By doing
this, E2 manages to find the lamp post and the CCTVs.

E2: Yeah, I can see two yellow CCTVs!

Interacting with the System – Behaviour / Interaction

The evaluation shows that users have different behaviour when exploring the CVE.
Users that are used to CVE and CARS experiments can just walk, exploring and
interacting with the system in a very natural and loose way. There is a feeling of no
surprise and they almost can predict what they are due to see and how the system
supposes to work. On the opposite way, it is noticed a mixture of surprise and

196
exhilarate for non experienced users since they started to wear the equipment.
Everything is a big novelty making them react completely different from those with
some background.

Some of the excerpts show how field users behaved when exploring the CARS.
Some of the users for instance does not like the idea of go through the objects that
compose the VE:

There is a moment during the exercises that field user is guided through the VE
where key points are demonstrated as well as the digital tram (exercise 1) that runs as
an animation. E2 walks around the green area following the guidance indications
from the office user until the digital tram is in front of him.

E2: Do I cross the tram?

In the physical world it is not natural to walk through objects. It is noticed, in a

number of occasions, that users can became afraid of walking through the elements
inside of the VE. They tend to avoid them by walking around or asking for a way in.

E2 is them invited to explore the other side of the tram station, walking over the
digital platform stopping under the tram shelter. To move from one side to another
the user should have walked through some of the digital objects.

E2: Can I go through the post? Can I walk around it? I will try to walk around it.

E2 walks sideways and very slowly avoiding virtual objects. He eventually manages
to find his way around without being run over by the virtual tram.

The same behaviour is noticed during the first exercise with E3. He gets surprised
to be run over by the virtual tram more than once. The video also shows that when he
is exploring the VE, he tries to avoid the objects by walking around them.

C1 has difficulties in understanding that it is possible to walk through digital

objects.

197
 C1: And the fact that I can walk through all these objects is very disturbing.

As with other users, C2 “jumps” when he is run over by the digital tram. When
comparing the RE with the VE, C2 states that the RE should be marked, indicating
the “dangerous areas” like the tram rail. Such boundaries could increase the sense of
realism in the scene.

Even though field users could not move the arms of the avatar the video shows
that gestures representes an important way of communication. Users tried to explain
situations or point to places when interacting with the office user. For the prototype
presented the only way to point is moving to the place and stop facing to the object.
The office user should follow the field user avatar and stop in the same position
looking to the same place. This is not a real option especially for complex VE or
places where the RE is in constant changes like in a CEn site.

Some examples of the body gestures during the exercises are:

In the beginning of the second example E3 waves for the office user’s
embodiment in the VE saying: “- Hello!”. The video shows that E3 is really talking
and waving to the office user in the VE and not for the camera.

A1 points the necessity of gestures as way to interact with other users sharing the
CARS. Throughout the time that A1 is exploring the first exercise (tram station) he is
asked about the effect caused by the office user’s embodiment in the scene. There are
advantages and disadvantages and the avatar complements the audio channel
communication and collaboration in the VE. The embodiment without the audio
would not make too much sense. It would become rather difficult without audio
commands like: “Please look to where the embodiment is looking at”. At same time
the audio without the embodiment could also make the exercise very difficult. A
request to go to a particular place to check a specific detail in the VE could become a
hard task, especially if the VE is designed without virtual landmarks. The users could
be completely disorientated and frustrated if they could not find the places where they
are being requested to go to. In the third exercise, for example, details like showing a

198
hole in the ceiling are better performed when the office user asks the field user to
move towards and look to the same direction as him.

A1: I like the idea that I can talk with somebody, as an embodiment, and follow it
and look to roughly same direction that it is looking at. You can think in the
embodiment in the level of expressiveness that it’s got. And currently it does not
have that much of it. At the moment, the office user can say “come to where my
embodiment is”. And probably you can just about say “look to where my
embodiment looks”. But that’s where it ends. So, the principle makes sense but I
guess that you cannot do really much with it. You cannot really point things with
it. You need the audio. If there is something on my left for example you cannot
point and I cannot point either because there is no interface to capture the
movement of my arms.

C1 also uses his hands to indicate things inside the VE even knowing that his gestures
could not be seen. The video shows C1 talking with the office user at the same time
that he gesticulates to confirm that he is going towards the right direction.

Interacting with the System – Immersion

The immersion of the user in the CVE is much higher as desired. Most of the users
could not see or have the feeling of VE and RE merged. The problem could rely in
several aspects like: the low quality of HMD for the task, the low quality and lack of
texture of the VE and the excitement of users in exploring the VE and not the AR
concept. This immersion is partially over taken after the first exercise. Users starts to
understand how the system works and become more independent of the field user.
The paragraphs below show some examples of this point:

During the first exercises the video shows that E2 is completely immersed, not
taking time to see the VE from some distance. He does not ask if he can take any
decision, instead, he just follows the office user. E2 stopped walking when he is sure
that the other embodiment is big enough to block his view.

E2: I think that I am close to you. You are quite big now.

During the first exercise, E4 is so impressed by the office user’s embodiment that he
forgets to explore the VE during the initial walk. For a considerable period of time he
is fixed with the idea that he should follow the embodiment in order to see what it is

199
doing. The office user has to make sure that E4 understands that he can explore the
VE by his own. Inside the VE, the field user is totally independent from where the
other embodiment is or what he is looking at. As with E2, E4 understood that if he
stays too close to the field user’s embodiment he will not be able to see the VE
properly. He decides then to open more space to see the VE better. He is requested to
look at the lamp post on his right, once he was over the platform. Only then he
notices the lamp post and walked backwards to have a better view and from where he
could also see the CCTVs.

Some of the users lost the sense of direction when surrounded by a virtual object.
This happened when they are going through an object or when the VE is swapped to
another one during the exercise 3. Most of them just immediately stopped as they are
inside of a virtual object and could not go through it.

A1 comments during one of the exercises the lack of sense of direction:

A1: I do not have a physical sense that this was the direction that I came from.
My physical orientation is completely gone because it is too immersive.

Throughout the tour E3 easily identifies all the key points of the VE like the stairs or
the gap in the roof of the first floor showing some parts of the second floor of the
house (exercise 3). The user is immersed and amused by the system until he lifts up
the HMD and says:

E3: Oops, I did not realize that I was over here! If you had asked “where are
you?”, I would not know. I could see the white line there in the ground, that was
quite useful.

VE – Quality of the Scene – Texture and Colour

One of the points rose by users is the lack of textures in the VE. Exercise one is well
accepted because in RE, the tram platform is all painted in solid colours and the VE is
created using the same colours tones. However users request more realistic textures
like brick walls for example in exercises two and three.

Another unexpected problem is regarding to one of the field users (E3) be colour
blind. The embodiment in the VE is red with white arms and E3 had some difficulties

200
to find it. The problem increases when the office user embodiment passes in front of
an object where the combination of colours made it disappear to E3’s sight.

O1 for instance has some trouble in understanding and navigating the CARS with
flat colours.

O1: It is lot harder to find my way in this one than in the tram station. The colour
scheme has some part in it because most of the structure has the same colour.
They can look like the same objects. Also the field user’s embodiment does not
stand out as it did in the tram station.

Even with the lack of texture the system provides a fairly realistic view of virtual
objects in the RE. During the third exercise, the office user’s embodiment stopped in
front of the stairs indicating where C2 should stop too. The user liked the idea that he
could look through the gap in the ceiling to the second floor of the virtual house.
From his position he could see a window in the second floor in the VE, which for him
it is fairly realistic.

VE – Quality of the Scene – Dynamic Objects

Users are also questioned about the dynamic object in the scene, in this case the
digital tram.

A1: I do not think that the tram adds anything in the scene more than the same
tram static in that positioning. If you have a static tram there I will understand
that there is a tram that passes there. It is actually confusing. Now I am so close
and if it moves past I could not say what it is. I cannot recognize anything I can
just see polygons. If I have an overview then I can see it. I imagine that, for an
example where you could see the flow of air, it will make more sense. With the
tram I am not so sure. Because of the field of view, I have the impression that I
am standing too close to the train where in the physical world I would never be so
close.

Perhaps the digital tram made the scene too complex to be seen. The tram is higher
that the tram shelter blocking the user’s view temporarily, until it moves away again.
There are two other points that could make users also disturbed. The first could be the
fact that the tram does not stay away from the digital station for a while, in order to
open the view. It keep coming and going constantly not giving proper time for the

201
tram station stays for a little while without the moving digital object (5 minutes away
for example). The second problem could be the way that the tram moves. The
coordinates should be closer to each other to maintain the smoothness of movement.

During the exercises O1 put in a question a good idea that could be possibly become a
future implementation of the system.

O1: If I go through the tram will I move with it?

The user could virtually travel in digital objects each time he approaches one. If he
moves again his view would come back to where he was previously in the CVE.

VE – Quality of the Scene – Reference Points

A1: It is really difficult to see what is what now. I understand that you gave me
different phases and I’m just about to understand that they are at the top of each
other but basically they just look like a lot of polygons.

The field user is lost due to the lack of a clear indication of the rooms that he is
visiting in the VE. The physical world contributes to the loss of navigation, once it is
flat with no reference points to aim at, such as the tape marking the boundaries in the
ground (shaping rooms for instance) or a landmark to indicate where the front and the
back of the house are. Users also suggest that the VE could have something like
furniture to indicate the rooms of the house at the same time that it would help with
the navigation.

As other users, A1 thought that the VE should be aligned with the white line in
the ground.

A1: In my idea if I want to explore the VE I should walk over this line. If I do it
now there is nothing to see. It is just a comment it does not stop me from using it.

The white tape placed in the ground is just roughly delimiting the position of the
virtual tram station and up to where users could explore the VE. This information is
passed to the users during the instructions prior to the exercises. However, most of the
users argued that the VE (in this case the platform of the exercise 1) is not being
projected exactly over the tape.

202
 C2 felt that he is being unfairly disconnected with the physical world around him,
claiming that there are no landmarks that could serve as reference points, neither in
the VE nor in the physical world. Like other users, he questions the relation of the
white line in the ground and the VE, believing that the white mark is the limit of the
house (exercises 2 and 3).

VE – Quality of the Scene – Avatar

The field user is then asked about what he thinks of the field user’s embodiment in
the VE and whether it makes sense to him.

C2: The embodiment as I see is not human enough for me to feel like I am
actually “seeing” someone standing in front of me. I have a bit of difficulty with
it. It does seem to move around and that does make me feel more like there is
actually something there as such, rather then there being nothing there. As I said
it is not particularly human enough for me to think that there is anything other
than an object in the environment.

C2 as well as other users mention there is a lack of empathy in the avatar. As a

general conclusion both field and office user’s avatars should look more like a human
being as well as be able to move their arms in order to make gestures. More about
empathy can be found in the following subchapter.

Office User’s Control

The office user has not only the important duty to guide and collaborate with the field
user when in exploring CARS. The manipulation of the objects is considered a
complex task and could not be performed by field user without a previous training.
The office user help becomes even more important once none of the invited users
could manipulate CARS using the handheld keyboard or understand the logic of
commands.

Field users could see along some of the exercises the demonstration of how the
VE could be rearranged while users are interacting in the CARS. The virtual objects
could be dragged and dropped by the office user’s embodiment (one of the features of
MASSIVE-3). The field user could then take a better position in the physical world in
order to see some of the changes that are being made by the office user.

203
 The stages of the construction are also demonstrated with CARS being
manipulated remotely the office user. Each of the phases is demonstrated
progressively, adding each new layer on the top of the other. After the repetition of
this procedure for a few more times, the phases are shown in an alternate order. All
these procedures are enriched by the possibility of communication between users.

6.3 Final Points over the Evaluation

The evaluation process helped identify a number of points that could not be predicted
during the many phases of the project. This was the moment when all that was
developed in the first few years was finally put together. The outdoor evaluation was
fairly different from those that were tested in a controlled place.

One important decision taken during the evaluation was to invite only one person
each day. All the difficulties and unexpected errors were better solved and it was
possible to learn from them. Users also did not get influenced by other users during
the exercises.

The evaluation process was not limited in observe how users interacted and
explored the system during the exercises. An interview was performed just after the
third exercise in order to complement the users’ feedbacks. Most of users answered
the same questions. Only few questions were tailored in order to cover particular
points experienced by the users during the exercises.

The questions and answers are fully transcript on the documentation provided in
Appendix C. The same nomenclature for the field users is used to indicate who is
answering when there is an excerpt. Figure 6.21 shows the areas covered in these
final points over the evaluation.

Figure 6.73 Grouping Points of Interest for the Complementary Outcome

204
Using the System – Self adaptation
Users had to adapt themselves to the system. The update of the coordinates was not
fast enough to make the calculated movement inside of the VE smooth enough. All
the users were reminded to walk slower than the usual, but not all followed the
instructions. GPS coordinates were being updated each second and the tilt sensor had
its output processed by the Kalman Filter to reduce jerk helped to make the refresh
rate of the view slower than usual. The system was stable when users stood still and
smooth when users made slow movements. Only two users tested the system before.
Users that followed the instructions commented that they had enjoyed the view,
especially those that had previously tested the system in the early stages remarking
that when standing still, the image was perfect.

Using the System – Immersion

During the exercises it was noticed that users were experiencing a total immersion in
the system instead of having an AR experience.

There are key factors that could have contributed to this happening:

 Regarding to the VE perspective users felt more familiarised with the tram
station than any other example. The colours and the shapes were very close to
the ones in the physical world. All users that evaluated the system once saw
how the tram station looked. They already had a previous knowledge about
the scenario, which made them more comfortable with what was seen. It was
noticed that users with previous knowledge of what they were due to see
would have a better experience when exploring the VE. Users put their focus
in the VE not giving chance for merging real and virtual.
 There was clearly a process of selective attention when users explored the VE.
As mentioned in (Edgar, 2002) “physiologists often regard attention as acting
like a filter”. Edgar quotes (Posner, 2002):

This “attentional spotlight” that illuminates only a small part of the visual field (i.e.
only a small proportion of everything that is registered by the eyes) and any items
that lie within that spotlight receive priority for processing. Any information lying
outside the “beam” of the attentional spotlight receives far less processing.

205
 In this case the RE is “out of the attentional spotlight” making users more
focused on what was being projected before their eyes – the VE.
 There was a disconnection between VE and RE. Landmarks (like tapes
drawing the shape of a house for instance) would help users to fix with the
idea of an AR system aiming to project a virtual object representing a house or
a tram station in that particular place.
 The see-through HMD cannot adjust contrast and brightness itself. If a cloud
covers the Sun light the image becomes too dark. If the cloud goes away, the
projection of the VE becomes very fade. Another problem was the lack of
stereoscopy. The images projected in the HMD are not three dimensional.
 Some of the users talked about the lack of texture, which could bring more
realism to the scene. Users were looking at objects in the VE, but not
connecting them to real objects (i.e. a house did not look as a real house).
 One user said that the dynamic object (the tram) had a negative impact during
the experiment. Perhaps the object could be better explored in the scene. The
tram could run smoother, not so often and with some sound effects helping to
pass a better sense of realism. It was noticed that the virtual tram often
obscured users’ view of the VE.

Virtual Environment - Overview

Texture and colours were one of the points that raised more comments from users.
Flat colours were not what users were expecting, especially when exploring a VE
representing a house. Just one of the users who were interviewed said that plain
colours did not affect him. High resolution texture would not be a requirement but
something intermediary that could pass the idea of a brick wall in the VE and not flat
brick colour only. The outcome from users just confirmed what they expected from
an AR system: the VE should be presented as close as possible of reality.

Originally the colours of the houses created for the exercise were choose to give
as much contrast as possible (to show different phases of a construction) making it
easy for users to see it.

206
 The tram station in the RE was painted in plain colours and users could easily
identify the tram shelters, lamp post, bins, etc. Two of the users shared their view
about the tram station:

E3: It was clear that it was a tram station and it really helps the tram going in
and out. It seems that inside the tram it was very detailed. That was quite
realistic. The outside of the tram was also good. I liked the lights as well and
when I could see the CCTV cameras up in the pole…and the bins. Yeah,
everything was fine.

O1: The tram station was really good because it has the same curves and same
objects that the real one. So I could say straight away what it was.

Users were then questioned about how they imagine that different phases of a project
could be presented once texture was a requirement. The suggestion was to use the
same texture but with slightly different colours. However, the issue of lack of realism
still remains when not using a “realistic” texture. As E3 stated:

E3: You’re gone decide what you try to represent. From this point of view you try
to represent the reality (pointing to the blueprint) and you see the final thing. So
texture helps there. If not, then it becomes close to a graph. It is a three-
dimensional graph in which case putting colour in, highlight in, putting different
colours for different phases makes sense.

Another point raised by users was the fact that walls (especially in the second
exercises) were represented by thin facets. Users claimed that a wall representation
should have at least the same thickness as a brick. A window should also look like
one and not like a blue rectangle. Users could not associate the virtual object with the
real one. During one of the trials E2 asked:

E2: Should I go towards the blue rectangle? If I go back I can see the yellow roof.

Some other users commented about the lack of virtual furniture in the houses. The
furniture for them could help to better understand a couple of points:

 To indicate an approximate size of the room. The previous knowledge of the

user about the dimension of a double bed for example, could give an
indication of the size of the room.

207
  To make it clear that the visited place was a potential bedroom because there
was a virtual bed in that place.

To use or not of furniture would depend of what the VE was proposed to represent
and in each phase of the CE cycle. During the design phase virtual sofas, beds,
kitchen, etc. could be added to enrich the scene with details helping architects to give
a final touch in the project. However, if the system is being used along the
construction phase it would not make sense to add such representations. More
valuable information would be required like technical information over the
construction specifications.

It was noticeable, both from previous experiences and also in the video of the
current evaluation that most of the users tried to avoid the digital objects. Some users
got frightened, for instance when something suddenly crossed in front of them, like
the tram for example. The answers show that there was a mix of feelings about going
through the digital objects. It is not natural for humans to walk through walls but this
is one of the features of the technology. The object was not there physically and the
field user can go through it because it was a shortcut or it was the best way to avoid a
hazard in the physical world (especially when considering a construction site). The
design of a realistic VE should not create barriers but extend the experience and
create possibilities for users to go beyond their natural limitations. The field user for
example could fly over the scene – although this feature was not explored. Such
extension of capabilities (to fly for instance) would become very useful if an architect
would like to show the design of a house with the gardens and the swimming pool.

Landmarks could be better explored during the exercises. A single line was not
enough to only indicate where the VE would be projected. The shape of a tram
station, house or any other VE would make users better connect what they were
exploring together with the RE. This would also demonstrate the level of accuracy
when using the RTK-GPS receiver.

Regarding the objects that could have dynamic behaviour in the scene, E1 made
the following comment that summarises the majority of the user’s views:

208
 E1: I really like the ability to have the moving train going through. Not only on
the conceptual point of view of being able to do that for example for wind flowing
or things that are usefully scientific, but actually I think it helps give reality to the
environment.

Hardware – Overview
For an outdoor AR system, RTK-GPS receivers are usually the most expensive part
of the hardware involved. It is especially true when there is the necessity of more then
one equipment. The price becomes one of the key points for researchers when
choosing which GPS solution they want to add to the design of the project. Apart of
the costs, RTK-GPS receivers also have issues regarding to size and weight. The
receiver does still big and heavy when comparable to other GPS solutions. The GPS
receiver used in this project is not the kind of equipment that is broadly accessible.
But as mentioned by A1, costs are decreasing since the number of companies in the
market offering high-quality products is increasing. Another limitation is the output
frequency of the receiver (1 MHz). However this is temporary obstacle that will be
surpassed with the natural enhancement of technology in a near future.

The area choose for the demonstration was wide open, with buildings at enough
distance of did not block the GPS signal or create too much multi-path error. All the
coordinates from the GPS receiver and tilt sensors went through a Kalman filter that
reduced the jerk considerably. These precautions resulted in a more stable
visualisation when compared to the very first trials. However, if field users walked at
their normal speed, the refresh of the frames was not fast enough giving them the
sensation of a “jumping” VE. The head movements should be done slower then usual.

As a prototype, it was not primary concern how ergonomic the introduced set of
equipment was. It was a fact that all parts together made the backpack carried by the
user not light. Batteries and GPS receivers, as well as the frame of the backpack,
made users tired after some time. However the weight was not the only part that
should be reviewed. Throughout the evaluation, at least one user had difficulties to
wear the crash helmet. For him the helmet kept loose even with the strap that keeps it
stable to the head adjusted as tight as possible. Another point was the number of
cables to connect GPS antenna to receiver; receiver to external battery; receiver to

209
computer; etc. During the exercises the cable could get eventual twists making very
difficult to handle.

There were several fragments of comments about the HMD in the text. One of the
questions for the users was: “Was it the correct way to display the VE? What about
handheld screens like PDAs and tablet PC?” Some of the users said that it was the
right choice for engineering purposes but the wrong one for other applications. At
least two of them said that they would prefer to have it as a visor in a helmet like the
ones used by jet fight pilots. A1 gave an interesting answer:

A1: I do not know if it will work better or not. What is it for? If it is for a client of
an architect, I think that a hand held will work better. Because then both will
work on it. But if it is a troubleshooting application with someone back in the
base and another person in the field this is much better. But to design something
for a client I will use something else. I think that one issue with the HMD is the
FoV. You just cannot see enough.

A1 statement can be complemented by another issue raised by (R.T.Azuma et al.,

2001) regarding to the social acceptance and the user’s limitation of interaction.
These two points could become a real dilemma when asking people to wear the
equipment to explore the VE. The social acceptance dilemma becomes real to this
project in the very early stages when people felt reluctant to wear it. The equipment is
very different from a normal outfit, making them reluctant to wear/carry something
like that in public. The equipment, due to its weight and the cables, also limit the
movement. A user with movement disability or not strong enough to carry the
backpack would find difficulties to test the experiment. The advantage to use a
handheld screen, as mentioned by A1, is the portability and the way that it can be
shared with other users.

Office and Field User – Communication, Collaboration and Mutual

Understanding
The evaluation makes it clear that for the AR system proposed on this thesis two
people were necessary to orchestrate the work. The introduction of complex tasks
made the field user even more dependent on the person in the office. On the other
hand, office users placed anywhere around the world would had difficulties to obtain
a clear interpretation of their questions if there was no field user to investigate. The

210
collaboration pushed the control of the VE to the office user, but still giving field
users autonomy and power. Field users also have some control and responsibility over
what both were sharing. The tasks performed by the person in the field could be
exemplified/enhanced to: show particular areas where the construction could face
temporary delays, point out good and bad aspects in the construction during the
investigation, scale and measure objects, measurement of the physical world, identify
features, update information, surveying, etc.

Communication – The exercises showed how important the office user was to guide
the person around the field in RE. The audio channel was essential to complement the
experience of collaboration. When field users could not find a particular point in the
VE during the exercises, the conversation proved to be fluent and useful.

The audio channel enhanced the collaboration and made easier for users to
understand each other with the help of the embodiment. From the office user’s
perspective to give directions was easy because it was possible to know what digital
object was on the left or right of the field user. The indication to where the field user
should go were all based in his position related to the VE and if he was turning to the
correct direction. E4 contributed with the following phrase that summarises the
feeling of the audio and the collaboration:

E4: The audio is critical because you are the designer. I did not know until I
arrived there what I was going to see. You gave me the instructions before we
started explaining the procedures to explore the scenarios. But only when I went
to the field I realised in reality what you wanted to achieve. I think that
communication is critical for people to understand.

Collaboration – The system proved itself that without a second person sharing the
VE, field users would easily get lost and unmotivated to keep exploring it. Users were
fairly busy trying to understand what was being presented before their eyes. They
could perform the control of very simple tasks like to type one of two buttons in a
personalised keypad (with only two keys perhaps and not with something like the
handheld keyboard introduced). However, more complex tasks would become
considerably difficult for field users to perform alone.

211
 There are strong reasons for the argument of the need of collaboration, especially
after the videos showing some of the field users walking towards a tree in the RE.
Field users in this evaluation were just walking. There was no equipment in their
hands that could distract their attention. All users that hit the tree were looking
towards it. As mentioned before, field user’s “attentional spotlight” was in the VE
and not in both VE and RE. To give more control to field user would also mean to put
in risk even more the user’s attention. The “attentional spotlight” would change more
then ever its focus, switching between: VE, RE, tasks, etc. One of these spotlights can
require more attention (resources from the person to focus in what he is doing) from
the user, cutting more and more the AR experience. The collaborative process helps
field users to be free and keep the focus in what they should do.

Mutual Understanding – The commands given by the office user through audio
channel were based on the field user’s movements observed in the VE. For example:
“Now you should turn to your left until you see a virtual tram shelter”. Commands
like this were widely used to show key points in the VE. Such interaction could only
be possible because the VE was being shared by both users. The audio channel alone
would not be sufficient to transmit all the instructions to the field user. The
instructions can only be completed by a combination of audio and the office user’s
embodiment in the VE. Even with almost no body language, the embodiment
facilitated the indication to field users of where to go and what to see.

For some users the embodiment could have been more similar to a human being
in order to create empathy, especially if in the future tools like this are introduced to
general public. Preece (Preece, 1999) discusses the empathy in symmetric and
asymmetric ways of online communication. It is possible to create a parallel between
Preece’s study and what could become CARS in the future:

Research in psychotherapy supports the assumption that empathy is

communicated better face-to-face. Facial expressions, eye gaze, tone of voice,
body language and touch all have a significant role in communicating empathy
(Eisenberg et al., 1999, Etchegoyen, 1991, Lanzetta J.T. and Englis B.G., 1989).
Nonverbal communication can be so subtle yet powerful that the communicators
may not even be aware of the messages that they convey via their body
language(Ickes, 1997).

212
The experiments showed how common understanding between users was enhanced
when they can “see each other” in the VE even with an embodiment shaped by
geometric primitives. Field users tend to copy the embodiment’s movement of the
office user either walking to the same direction or by looking at the same place. Even
with the embodiment’s restricted movements and body expression, the possibility to
move through the space and move its head was highly important to enrich the
demonstration of some places inside of the VE. The drag and drop feature available
on MASSIVE-3 was also helped to point to virtual object even in a far distance. This
feature showed a ray coming from the middle of the office user’s embodiment’s
forehead towards the aimed object. Users reported that this was a very good example
to demonstrate which direction they should look to.

Choosing the Right Approach to Demonstrate Design and Construction

Engineers and architects where questioned if the AR system presented was the best
approach to visualize a construction during the design and construction phases.
Mostly they prefer blueprints once they were trained to visualize the third dimension
from the printed data. For the non-technical audience, 2D charts are more difficult to
understand. In any case, the main issue was the low level of detail of the VE. All
participants agreed that a more natural representation would make lots of difference
in the way that the information would be interpreted.

Engineers mentioned that the tool was useful when applied in the right building
project. In small projects like a house extension, for example, it would not make as
much difference as if applied to a new residential area. Apart from the
aforementioned limitations of the system, they could see the potentiality of this
project, especially for underground pipes, health and safety training and
Computational Fluid Dynamics (CFD) (CFD Online, 1994).

E3 mentioned the idea about collaboration could be arranged between more than
one field users. This idea would be particularly interesting because as he affirmed:
“site inspectors could be back in a month to see the progress, for example”.

213
Chapter 7

Conclusions and Future Works

The design of the presented research goes back to 1999 when the author saw for the
first time the project of Silva (Silva, 1998) where two military tanks reported their
coordinates to a database at the same time that a computer displayed their positioning
over a 2D map. The tanks at that time are not more than two little squares in the
computer screen with all system working over DGPS corrections. This project is
inspired in idea of allowing people to be tracked with high accuracy GPS in real time,
feeding and sharing a real collaborative 3D environment. This supposes to make users
enhance collaboration, communication and exchange information, placed not only
outdoors but whenever Internet connection is possible.

The results of the work reached during the elaboration, implementation and
evaluation of this thesis are very prominent for coming researches. Even though it
does still a prototype and subject to constant improvements regarding to technology
and new requirements the achievements are very positive.

As a first point in the conclusion it is possible to affirm that the research achieved
its objective introduced on Chapter 1:

214
 The objective of this thesis is to investigate, create and evaluate a CARS able to explore
communication, collaboration and mutual understanding on CEn environments.

The following paragraphs describe the achievements reached by this thesis with a
brief summary over each of the chapters as well as the contributions for the
knowledge. During the research period some papers were published as well as a
television interview, collaborating with the technology dissemination. Finally a
number of future works are introduced aiming to contribute to new ideas or to create
a direct evolution from the design of the current research.

7.1 Summary and Reflections

In general the research helped to certify that a second person is necessary when an
CARS is used on CEn environments. In order to facilitate the work and the
availability of this second person an asymmetric remote allocate user (office user) is
suggested to share the CVE. So this research concludes that:

If people want to do immersive head mounted augmented reality for civil

engineering purposes, users will need a network solution of at least two people,
as applied in this project. In the virtual environment, the audio and the
embodiment of those users in the office will work as a help resource, interacting
with the person in the field. While one person has the immersion and the freedom
to explore the virtual environment seamless with the physical world, complex
tasks must be performed by a second networked person in the office. This control
coming from indoors will ensure that outdoor users are safe as well as looking to
the right things while collaborating.

To achieve such conclusion the work here presented starts introducing the three key
worlds of this project: communication, collaboration and mutual understanding –
Chapter 1. The key worlds are presented already towards the potential gaps that can
be found in CEn environments. A CEn project is complex and requires several
multidisciplinary teams that nevertheless can face difficulties over its ontology. It is
essential that team should work collaboratively, cutting edges, helping to reduce time
to achieve milestones, and reducing costs.

Once the project investigates the use of a CARS in CEn environment such
knowledge is introduced together with its several phases over its life cycle. Between

215
many, only two of then are choose as target of the research – design and construction.
Although the design of the current project has already some years it can be also
certified on an extensive research presented by Shin and Dunston (Shin and Dunston,
2008).

Once the initial gaps on CEn are presented Chapter 1 introduces with the help of a
simplified diagram (see Figure 1.2) the proposed solution. Office and Field users are
introduced for the first time together with some basic concepts of the technology that
surrounds and link them.

Chapter 1 finishes with the steps followed to achieve the thesis objectives and its
structure.

Chapter 2 introduces the challenges faced by AEC during a CEn project. The idea
of an augmented architect is not new. Brate (Engelbart, 1962) had coined the idea
almost fifty years ago. However, just now technology is good enough for such
experiments. The text goes over the challenges faced by commercial companies eager
to create standards in formats and software to be used by other CEn companies. The
solutions provided already prove to worth the high investment – especially for small
CEn companies – reducing costs and time during the projects. Researchers are more
focused on what to retrieve from data bases, which level of filtering, at what time, etc.
They are looking for ways for AEC to collaborate each time more, exchanging
information, pursing understanding when working.

Chapter 2 makes clear that there are gaps in CEn environment regarding to
communication, collaboration and mutual understanding. It also helps to indentify the
place for this project not as an overall solution but a part of a solution of how to
visualise data on CEn context.

Chapter 3 reviews the concepts around the Virtuality Continuum first introduced
by Milgram and Kishino (Milgram and Kishino, 1994). However, the core of this sub-
chapter is the long review over CE once this is the focus of the research. This review
is created specially for those not so familiarised with the topics covered by this area
of Computer Science.

216
 An extensive survey of researches is the next topic covered by Chapter 3. The
survey covers projects with direct or indirect application over CEn visualisation of
information. The projects are selected by: how information is being explored by
users; the way that information is being displayed; where information is available to
be explored; and the positioning system used to explore the information when
outdoors. It is possible to create a link of the necessities on CEn and what is being
proposed on this thesis even though some projects have no apparently direct
connection with the research. The survey once again makes clear that there is space
for the proposed research. Most of the work done so far uses limited resources of AR
systems, mostly no CVE, and limited use of positioning system – when in comparison
with the freedom required by users when exploring outdoors.

Chapter 3 finishes with a well informative sub-chapter covering GPS. This sub-
chapter aims to give an “extra mile” of knowledge for those not so familiarised with
this positioning system and makes a balance of the amount of information gave on
MR systems. The goals of this sub-chapter are the demonstration of different
methodologies applied over GPS and different results achieved when using such
methodologies. The results prove that RTK-GPS – between GPS methodologies – is
the best solution for outdoors positioning.

Once the theories and projects surveyed are reviewed Chapter 4 introduces the
design for the solution proposed. A diagram illustrates how the system should work
(see Figure 4.1) with Field and Office users sharing the same CARS. During the
description of the design overview a more complete diagram introduces all parts
involved when the project is fully operational. Such description does still in high level
but already makes clear that a single keystroke has a complex and long way to travel
from one computer to another passing through device manager, data base, and VE
manager. Chapter 4 is complemented by Chapter 5 due to the complexity of the
system involved.

Hardware, software and platforms used to make the system run are revised also on
Chapter 4. Readers are presented to the kind of message coming from GPS receiver,
tilt sensor and magnetic compass.

217
 Before the initial implementation of the CARS the research puts its focus on
surveying and creation of a VE – Chapter 5. The area around the IESSG building in
the main campus of the University of Nottingham is mapped together with the
features of the area.

The project finally enters to a next phase when CVE starts to be implemented
putting office users to share the same VE, exploring the area just surveyed in the RE.
A diagram (see Figure 5.4) shows how the configuration is when two or more office
users are sharing the CVE. This diagram introduces the processes involved to make
the system run as a complement of the information already presented on Chapter 4.
The processes are then more in deep described becoming an important source of
quick information for future researchers aiming to implement the system with some
extra functionalities.

Chapter 5 also demonstrates the results reached when the Kalman filter is applied
over the incoming data from the inclinometer and the magnetic compass (see Figures
5.10 to 5.12). The discomfort of the visualisation due to the jerking are at the time
one of the main complains from users during some of the demonstrations. The
implementation of the filter reduces considerably the discomfort for users when
wearing the see-through HMD.

The final stages of implementation and evaluation are demonstrated on Chapter 6.

For the evaluation three examples are created aiming to show the potentialities of the
tool created, as well as the investigation of how users work over the three key worlds.
Users are invited based on background and skills on different subjects. They are
researchers, technologists, engineers, etc. The evaluation is taken using a formative
approach where users are also motivated to think aloud to a hired cameraman. All
users received previous explanation of how to explore and collaborate when using the
CARS – cognitive walkthrough. The exercises are not rigid performed but mostly
users are invited to perform the same tasks. Along the exercises users are already
questioned about what they are experiencing. Answers are recorded for pos-analyses.
At the end of the third exercise users are invited to an interview. Most of the
questions are the same for users and some are tailored based on their experience
during the exercises.

218
 One of the major aspects learnt during the evaluation is the real necessity of a
remote person to perform complex tasks while there is someone exploring the CARS
outdoors. This is especially true for field users that never received a training of how
to explore the VE or how to handle a handheld keyboard. Nevertheless to reaffirm
that the project is a prototype and it is not only the case of the hardware interface but
also the software interface. To change from one VE to another or to move objects,
users should know what screens to put the right commands – once there are more than
10 processes/screens running in parallel. All these screens can be displayed at the
same time that they walk outdoors trying to understand the information that is being
presented before their eyes. The complexity of how to handle the system can grow
exponentially and users will get frustrate with the experience. During the evaluation,
users are not exposed to such kind of complexity. The result is that no one mentioned
about this issue during the interviews. Field users could see that the CVE is being
changed, object could be moved but they are far from the complexity handled by the
office user that has the facility/flexibility to work on it from his computer’s keyboard.

The process of evaluation shows that users are eventually more focused on what
they are seeing than with how they are seeing. One of the major points of discussion
is the quality of the VE like: lack of texture, colours, shapes, dimensions, etc. The
see-through HMD is perhaps the second factor that creates more negative feedback.
There is not possibility for auto-adjustment in the device as bright and contrast.
Because the system is performed outdoors the luminosity is subject to constant
changes due to clouds and sun position in the sky.

Another lesson learnt is performance of the evaluation with the right users with
the right use case. The exercises create for the evaluation are more focused on
construction than design phase. Users liked more the first exercise (tram station –
more focused on design) than the other two when comparing the answers. This is
because the tram station VE is close to what it is in RE. Users could easily indentify
the objects and colours. The same does not happen with the other two examples. The
second example showed the different phases of the project of a house – that is not too
far from the example shown by Woodward (Woodward et al., 2007a) (see Figure
3.21). Perhaps the right approach to evaluate the project is to take a real use case of a
construction and show work over the result of those professionals responsible for CEn

219
project – AEC. Another point is to make sure that the right information is displayed
for the right user. High definition textures and virtual furniture is not relevant to all
users. More technical specification like pipe dimensions, angles of the structures, etc
can be directed to those involved more in planning and construction.

7.2 Thesis Contribution

The development of this project aims to contribute to those developing AR systems

for CEn application and have low background knowledge either in CARS or GPS.
The thesis shows the several steps regarding to the design, implementation and
evaluation of such system. The main contributions can be summarised as follow:

 Identification of gaps over communication, collaboration and mutual

understanding in multidisciplinary teams on CEn projects. The bibliography
review introduces a number of issues regarding to how ontology can become
an issue when contributors are not working in the same channel of exchange
of information.
 A good effort has being done for the creation of a standard in order to bridge
knowledge but very little regarding to outdoors activities. AR systems and
specially CARS for CEn it does still in its principle specially because:
positioning system (lack of accuracy indoors) and HMDs (semi-transparent or
not).
 CARS should be treated as one of the branches for the output/input of
information and not as a total solution. Information should be handled for an
external system created to merge, retrieve, delete, add, etc.
 An ergonomic system is far for ideal. High accuracy GPS receivers are heavy.
The heavy duty batteries are also not compact and light. Computers are
getting lighter and powerful but there are still problems over the HMD. There
is no genuinely equipment designed for CEn application outdoors. Everything
runs over an adaptation and naturally susceptible to downsides. The number of
cables is also high making the system very fragile. A system with a label
“handle with care” does not fit very well in a construction site.

220
 For a tool like the one proposed it is natural to think that more features can be
add (see Future Ideas below). Such features would require a better
hardware/software interfaces and an even closer relation between field and
office users. For the field user perspective the increase of complexity in the
tasks performed does not mean the increase of how it is performed. Interfaces
must be as simple as possible, using natural body movements, voice
commands, and even eyes commands. In any case, complex tasks should be
performed with the support of a second person. User can not handle a complex
interface hardware/software while in the construction site. The environment
itself has potential hazards like moving objects, holes, construction material,
etc.
 Chapter 2 demonstrates the number of hours that office workers need to spend
with managers in order to fill up the gaps and resolve issues. This can be done
remotely and almost instantaneously with a system like this. The system
proves to be useful when used in the right place at the right time.
 The elaboration of examples should be based on real projects. Better if the
CEn project is under construction. AEC professionals will be able to see and
say the positive and negative points in a solution like this.
 The results achieved already shown the potentiality of the tool although there
is no evaluation before the project be proposed of how communication,
collaboration and mutual understanding work. All users that invited for the
evaluation did not receive any previous training of what to see, how to
explore, how to behave, etc. Users arrived and received some information but
even blue prints are not displayed to everyone. The communication during the
exercises are really high with users putting questions and being instructed of
where to go, what the features represent, how the system works. This high
interaction between office and field users creates a collaborative environment
making those outdoors motivated to keep exploring even when they are
eventually loss. The collaboration does not happen in this case on the CEn
level but in how everything works together and where it could be improved.
Finally mutual understanding is clearly achieved when users can identify the
explored places. From the first exercises (tram station) they almost
immediately get familiarise with the VE. It comes from a previous

221
 background and they just need to apply this knowledge to what is being
projected before their eyes. On the second exercises (the house) they can
recognise the places when looking for the blue print. The video shows users
pointing to the blue print showing and talking about where the living room is,
the veranda, and the other rooms of the house. From the third exercise (second
house) users can see the potentiality of the tool when exploring the CVE
specially when looking through the gaps of virtual objects or exploring the
scene between the virtual pipes. The thesis contribution from this level is the
confirmation that communication, collaboration and mutual understanding are
enhanced when using an application like this.

7.3 Dissemination

Along the years this project produces the following scholarship, poster, articles and
television interview:

The idea of the project is first presented in to Engineering and Physical Science
Research Council. The project receives a grand under the number GR/R42696/01.

The poster “Combined use of Virtual Scale Models and Space Geodesy” is
presented in Brazil (Capra et al., 2003a) during the VI Symposium of Virtual Reality.
The authors are: Capra, M.; Aquino, M. H. O; Dodson, A.; Benford, S.

The paper “Civil Engineering Application for Virtual Collaborative Environment”

is presented in Japan (Capra et al., 2003b) during the International Conference on
Artificial Reality and Telexistence. The authors are: Capra, M.; Aquino, M. H. O;
Dodson, A.; Benford, S.; Koleva-Hopkin, B.

The paper “Mixed Reality on Civil Engineering” is presented in United Kingdom

(Capra et al., 2004a) during the Ubicomp. The authors are: Capra, M.; Aquino, M. H.
O; Dodson, A.; Benford, S.

222
 The paper “A visualisation Tool for Civil Engineering Using Virtual Reality” is
presented in United Kingdom (Capra et al., 2004b) during the 1st FIG International
Symposium. The authors are: Capra, M.; Aquino, M. H. O; Dodson, A.; Benford, S.

The project is also interviewed during the program Dispatches – “Who’s Digging
our Roads?” (Channel 4, 2006)

7.4 Future Work

The following paragraphs propose some topics for future work for CARS on CEn
environments.

7.4.1 Future on Hardware

Hardware does still an issue on outdoors AR system. Although there is not too much
to do once most of the equipment came from commercial products a list of desirable
hardware is presented below:

 See-through HMD: it should be light as a pair of glasses. No wires attached

and very light batteries. High resolution and stereoscopic images. Auto
adjustable for contrast and bright. Weather proof. Retina tracking.
 Positioning system: it should be as light as single frequency receivers are
nowadays (the size of a match box) with the same low battery consumption.
The output should be ten times faster or more – actual frequency is 1MHz.
Imbibed antenna and weather proof. High accuracy 100% of time with
secondary and tertiary solutions to supplement the accuracy – like cell signal,
wireless network signal, etc. It should be reliable both indoors as outdoors. No
cables at all.
 Body movements track: through a kind of fabric (using some kind of nano
technology) the movements of users’ body could be tracked with maximum
resolution. This information would be useful to feedback the avatars creating
more empathy.

223
  Data communication: although some cities are already covered by fast
wireless network transmission this does still an issue. Fast internet should be
available anywhere at any time.

7.4.2 Future on Software

Software is more feasible to expect to change over the coming years. More oriented
platforms should extract the maximum from hardware without compromise
processing and memory. However this can only be achieved if the right hardware is in
place and fully available. One of the common issues over software is the limitation
imposed by hardware manufactories to fully access the resources of the devices.

7.4.3 Future Applications

During the development of this research a number of new applications were created.
This new ideas could become an extension of the current project or a completely new
research.

Input Interface for Field Users

The work of the Field users could be better explored when in the field. They could
collect information in the area where the construction is taking place/will take place,
feeding the DB in real time. Examples of valuable data that could be collected in real
time:

 Information about the terrain – especially after torrential showers with

surface runoff. Irregularities in the terrain like excavations or erosion.
 Construction material – what construction material is available, quantity and
position of where they are stored in the construction site.
 Technical specification on construction material – relevant information about
what is being installed and the consequences of it. Information about the
diameter of pipes, capacity of pressure, different kinds of wires, gas valves,
steel specifications, etc.
 Identification and resolution of conflicts – any conflict not detected during
the design can be discussed in real time with other AEC team members.

224
  Survey with video/photography registered by GPS coordinates – the images
can be added to audio documents reporting any issue during the inspection of
the site.

Potential Hazard and Lack of Navigation

The evaluation shows that Field users can put far more attention in the VE then in the
RE – “attentional spotlight”. It is a serious issue and can increase the chances for
Field users to expose unnecessarily to hazards like: uneven terrain, excavations,
moving vehicles and objects, loose construction material, etc. All hazards can be
monitored through constant surveys of the area. Every time that Field user is in
potential dangerous an alarm could sound and some extra information would be
displayed in the CVE. Drivers and operators would also be able to receive
information indicating the presence of a person in the area of manoeuvre in case of
moving vehicles and cranes.

The navigation through the construction area can also be performed by the
system. Indications of how to go from one point to other avoiding different hazards
can help Field users to keep safe.

Annotation Tool on CVE

A possible extension of the CVE is a 3D annotation tool. Office users would be able
to create 3D marks in different colours that would be visualised by office users. Such
indications can be helpful to indicate places and points of disagreements/agreements.
The annotation tool could also place icons and any other form to indicate a point in
the CVE. The 3D annotation tool can also be linked with audio, videos and pictures
annotation.

Non Immersive Hardware Interface

A study comparing the acceptance of non immersive hardware interface (PDAs, cell
phones, see-through screens, etc.) against immersive interfaces (HMD). The
evaluation should be aware of the discrepancy in technology between Liquid Cristal
Displays (LCD) and HMD creating an even way to compare both approaches when
exploring the same CARS. The evaluation should also compare the level of freedom
required by different users when working with the equipment. Question like: Do users

225
need both hands free all the time? Which equipment is more appropriate to perform
complex tasks? are some of the examples to be explored.

7.4.4 Further Evaluation

The thesis uses the formative approach to perform the evaluation. This approach is
based on the scale of the project and the culture of MRL and becomes appropriated
for an initial outcome from the kind of technology proposed. However a more
quantitative and comparative work is required in the future. Exercises can be better
designed following a more realistic example like a real construction taking place.
Users can also be choose and arranged in subjects of knowledge. Professionals in
AEC that are current working in a project would be invited to compare their working
methods with the CARS. This evaluation should be done in different construction
sites with approximately same scale. The time frame of the evaluation should cover
fully the design and the construction phases.

7.4.5 Concluding Remarks

The work of the presented thesis consolidates the design, development and evaluation
of a CARS used on CEn – design and construction phases. The works starts with the
identification of potential gaps over communication, collaboration and mutual
understanding on groups shaped by AEC. The extensive survey done in both
Engineering and Computer Science projects reaffirms that there are spaces for a
research like the one proposed. There are a considerable number of work over the
theme but very few using subjects like high accuracy GPS and CVE. The advantage
to use RTK-GPS is the assurance that any surveyed feature of the terrain has no more
than few centimetres of error. Users have their real position distribute to other users
in real time when walking in the field with the equipment. The error on positioning is
minimum for the trials that users are invited to perform.

The tool enhances collaboration, communication and mutual understanding as

demonstrated during the evaluation. Users demonstrate fluency after a short period of
time even though when receiving very few amount of information of how the system
works or what to do inside of the CVE. This can only be possible with the
asynchronous collaboration of a second person inside of the VE. Through the avatar

226
and the voice channel users are able to find objects, recognise places, and express
their point of view. The second person inside of the CVE works almost as a facilitator
between field users and information. It is easy for the field user to follow an avatar. It
is also easy for the office user to explain something once he is exploring the same
contend in the CVE.

The evaluation shows the necessity of a better understanding of what users expect
to see in a VE. This topic is strongly linked with who the users are: architects,
engineers, or constructors. They have different ways to see the same information. The
way that visual information is passed to them should be also different – coming from
textures, virtual furniture, etc. going to more technical and time schedule information.

This project aims to give to future researchers a solid view of from where to start
on CARS although it does not exhausts the technology neither has an absolute
statement. Hopefully this work can contribute further in the conceptual understanding
of the requirements on CARS in CEn projects.

227
228
Chapter 8

Parameter & Implementation Reference

As introduced in §5.1, Appendix A demonstrates the structures of the files that are
used in the creation of the VE. Such file is passed as second parameter in the call:

MIX006 <grid>.grd <feature>.txt <outfile>.wrl

This file contains the objects (features) found in the terrain that feed the VE, as well
as the coordinates surveyed for each one of them. When a road is present, the
algorithm paints the altimetric mesh with a different colour to indicate where it is.

In §A.1 there is a description of how to create the files that shape the VE.

The algorithm created to process the files and create the VE is explained on §A.1.

 The generated output consists of:

 A VRML file with only the landscape of the area that was surveyed.
 A second VRML file with the CVE as it will be explored in MASSIVE-3.
 A .cve file that contains the CVE description to be used by MASSIVE-3.
The code is created using the C language and the §A.2 is structured as:

 A description of the global variables.

 The documentation for each function created.
 A high level algorithm.

8.1 Features File

The second parameter when MIX006 is called is <features>.txt. This text file has the
positions and objects (features) surveyed on the area that VE was based. Each object
that is indicated there corresponds to a VRML file and the coordinates follows the OS
grid. The facility is to use the positions saved on the receiver without the need of
conversions for Cartesian coordinates. The software does it based on the minimums
and maximums of coordinates read on the square mash created by the Surfer
software.

Structure of <feature>.txt
In this example the following features were surveyed in the data acquisition step:

 Roads.
 Trees.
 Buildings.

Apart from the road, each feature had a VRML file with its own design. If a road is
included in the VE, it must to be done as the first line of the file. The road is the
unique feature that does not require any coordinate below its name. The algorithm
follows the file indicated and adds the road to the VE (A.2.1). Table A.1 exemplifies
the structure of the file that contains the features.

Table A.6 Features’ File Structure

road.txt
tree.wrl
454279 338391 31.9924
454279 338391 31.9924
building.wrl
 454299 338399 31.9924
454239 338351 31.9924

For all other features, their coordinates must be specified by a couple of control
points. There are some cases, such as lamp posts and trees, in which both control
points are the same. Buildings and any other shape that requires a positioning in the
VE have their coordinates specified by two control points, which could be placed
depending on the user criteria. Each line contains the position of the features specified
as three numbers representing: East, North, and High. With these values it is possible
to calculate the necessary rotation to place the object in the correct position.

The road file (first line of Table A.1) contains only the East and North values,
ignoring height. Usually, a road surveys in this project takes both sides of the road
instead of only the middle. When interpolation process (see §5.1) takes place the
coordinates surveyed becomes the vertices of the polygons that will shape the road.

A final modification is required in the VRML files that shape the features.
Usually the design of a new object on a three dimensional editor is done using the
coordinates (0, 0, 0) for X, Y, and Z. Even though, it is a difficult task to design the
object exactly on the position that it should be placed on the VE. Firstly, the same
corners chosen to be the control points of the features are identified on the object.
These coordinates are then explicitly at the third line of VRML file. The solution is
then given by the calculation of the distance between the minimum coordinates of the
square mesh and the place where the object must be placed. An object’s translation is
applied and finally a rotation. Table A.2 exemplifies a header of a VRML file. The
first line is standard for VRML 1.0. The following line consists of a blank line and the
control points on Cartesian terms.

Table A.2 shows an illustrative example of control points.

Table A.7 VRML Header’s Modified

#VRML V1.0 ascii
#
#20 10 1, 38.5 6 0.8
8.2 Source Code Reference

The source code was written in C language. It is expected that the reader has some
knowledge in structured programming and C syntax. The following paragraphs
describe:

 The global variables used

 The functions programmed
 A high level algorithm

Global Variables
The global variables on the code are related with the input and output files that are
handled. All of them are pointers to a file. Table A3 introduce these variables:

Table A.8 Global Variable

entrada It is a pointer to a file that contains the Surfer output altimetric file.
entrada2 It is a pointer to each one of the files that contains an object that will
feed the VE.
saida It is a pointer to a file that contains the VRML file with all objects
inserted.
saida2 It is a pointer to a file that contains the VRML file with only the terrain
surveyed.
mundo It is a pointer to a file that contains the CVE description.
Road It is a pointer to a textual file that contains the coordinates of the road
that was surveyed.

Functions Description
The Tables A.4 to A.12 describe the functions created by the following steps:

 Name.
 Description.
 Prototype.
 Parameters passed when the function is started.

Returns given by the function when the process is finished.

Table A.9 Function main

Name: main
Description: This function is called when the program starts. From the
main function all other functions are called.
Prototype: void main (int argc, char *argv[])
Parameter Passed: int argc – number of arguments passed when the program
starts.
char *argv[] – the arguments passed when the program starts.
Return: There is no return.

Table A.10 Function open_arquivo

Name: open_arquivo
Description: This function opens the files passed as parameters when the
program is started. The pointers for a file: entrada, entrada2,
saida and saida2 are started on this function.
Prototype: void open_arquivo(int argc, char *argv[])
Parameter Passed: int argc – number of arguments passed when the program
starts.
char *argv[] – the arguments passed when the program starts.
Return: There is no return.

Table A.11 Function take_information

Name: take_information
Description: This function reads the square mesh file created by Surfer
software extracting the minimum and maximum values of the
area. It also creates the header of VRML files generated as
output of the system. The function reads from the pointer
named entrada and write on saida and saida2.
Prototype: void take_information()
Parameter Passed: There is no parameter passed on this function.
Return: There is no return.

Table A.12 Function vrml_generator

Name: vrml_generator
Description: This function reads the altimetric data from the file pointed
by entrada and places the coordinates using the VRML syntax
on saida and saida2. For each feature read on entrada2 an
action is taken. If a road that is being added, the function
input_road is called. Otherwise, it is if a feature, the
algorithm calls input_feature function.
Prototype: void vrml_generator(int ln, int cl, float min_h, long int x_mn,
long int y_mn, long int x_mx, long int y_mx, float h_mn)
Parameter Passed: int ln – number of lines contained on the square mesh.
int cl – number of columns contained on the square mesh.
float min_h – minimum value for high.
long int x_mn – minimum value for East.
long int y_mn – minimum value for North.
 long int x_mx – maximum value for East.
long int y_mx – maximum value for North.
float h_mn – maximum value for high.
Return: There is no return.

Table A.13 Function input_feature

Name: input_feature
Description: It extracts the features from the file pointed by entrada2 and
places the objects on the output file respecting the translation
and rotation.
Prototype: void input_feature(long int x_min, long int y_min, float
h_min)
Parameter Passed: long int x_min – minimum value for East.
long int y_min – minimum value for North.
float h_min – minimum value for high.
Return: There is no return.

Table A.14 Function build_cve

Name: build_cve
Description: This function creates the CVE file to be used further on
MASSIVE-3.
Prototype: void build_cve(float tra_x, float tra_y, float tra_z, double rot,
int flag, ch ar feature[13])
Parameter Passed: float tra_x – Cartesian coordinate X of the object.
float tra_y – Cartesian coordinate Y of the object.
float tra_z – Cartesian coordinate Z of the object.
double rot – Rotation angle in radians.
int flag – Flag used to indicate what action the algorithm need
to follow.
char feature[13] – Feature’s name.
Return: There is no return.

Table A.15 Function tabela_poligono

Name: tabela_poligono
Description: It fills a table with the coordinates of the road surveyed.
Prototype: long int tabela_poligono(float mat[256][2])
Parameter Passed: float mat[256][2] – matrix with the road coordinates.
Return: It returns the number of polygon vertices created by the
surveyed road.

Table A.16 Function fbguPnpoly

Name: fbguPnpoly
Description: This function verifies if a coordinate is in, out or in the
polygon border.
Prototype: long fbguPnpoly (long Npoly, float *Poly, float *Pto)
Parameter Passed: long Npoly – number of polygon vertices.
float *Poly – array of polygon vertices.
float *Pto – coordinate that will be checked.
Return: -1 if the coordinate is out of polygon boundaries.
0 if the coordinate is out the boundaries of polygon.
1 if the coordinate is inside of polygon boundaries.
OBS: This function was consented by Luiz Eduardo Seabra Varella.

Table A.17 Function input_road

Name: input_road
Description: This function sets the colour for each node of the square
mesh. If the node is part of the road the colour is grey
otherwise green.
Prototype: void input_road(long int x_min, long int y_min, long int
x_max, long int y_max)
Parameter Passed: long int x_min – minimum value for East.
long int y_min – minimum value for North.
long int x_mix – maximum value for East.
long int y_mix – maximum value for North.
Return: There is no return.

Algorithm
The high level algorithm presented below offers for those who would like to use the
code, an easy understanding of how it works. The following steps documents only
what was judged as important for the understanding of the algorithm. Table A.13
shows the high level algorithm. The highlighted worlds identify the functions and file
pointers.

Table A.18 High Level Algorithm

void main (int argc, char *argv[])
{
open_arquivo(argc,argv);
take_information();
fclose(entrada);
fclose(saida);
fclose(entrada2);
}

void open_arquivo(int argc, char *argv[])

{
if number of parametres wrong exit;
open entrada;
open entrada2;
open saida;
open saida2;
open mundo;
}
void take_information()
{
it reads minimums and maximums from the square mesh;
It adds the headers of saida and saida2;
vrml_generator(lin, col, h_min, (long int)x_min, (long int)y_min,(long int)
x_max, (long int) y_max, h_min);
}

void vrml_generator(int ln, int cl, float min_h, long int x_mn, long int y_mn, long
int x_mx, long int y_mx, float h_mn)
{
It completes the headers of saida and saida2;
The coordinates from the .grd file is transferred to .wrl file format;

if (file name road.txt is found in entrada2)

input_road(x_mn,y_mn,x_mx,y_mx);

input_feature(x_mn, y_mn, h_mn);

}

void input_road(long int x_min, long int y_min, long int x_max, long int y_max)
{
tabela_poligono(matriz);

for (each row and line)

fbguPnpoly(vertices, Poly, Pto);

The altimetric model is painted with the right colour;

}

void input_feature(long int x_min, long int y_min, float h_min)

{
build_cve(0, 0, 0, 0, 10, "land");

while(It is not end of file (entrada2))

{
The control points of the object are read;
The translation and rotation are calculated and written on the output file;
build_cve(ponto[0].x, ponto[0].y, ponto[0].z, alpha, flag, nome);
}
}

long int tabela_poligono(float mat[256][2])

{
It reads from the road.txt file and logs the coordinates in a table;
The return is the number of the vertices read;
}

long fbguPnpoly (long Npoly, float *Poly, float *Pto)

{
 It checks if the coordinate is inside, outside or in the polygon boundary;
}

void build_cve(float tra_x, float tra_y, float tra_z, double rot, int flag, char
feature[13])
{
An header describing the name of the object, positioning and rotation is created;
}
Chapter 9

Implementation Reference
Appendix B brings relevant details about the source code implemented for this
project. Some knowledge of Object Oriented (OO) structure in Java language is
expected from the reader. Each sub-item on this Appendix refers to a class that
explores the most important methods in order to facilitate the comprehension of those
that could use this work as a base for future research.

The classes developed for this research complement the facilities provided by
EQUIP’s to work with external devices connected to the computer. As soon as the
data is received and filtered, there are EQUIP methods to publish the new information
on the data space, as it will be demonstrated below.

The sub-items in this appendix as follows:

 Overview – summarises what the class does.

 High Level Algorithm – as a complement to the overview.
 Class Constructor – how the class is inherited.
In the high level algorithm the Begin and End indicate where the class starts and
finishes. However, one or more classes can be inside of the main class. Even though
Java is an OO language, the algorithm was written to be the most linear possible for
the sake of clarity.

9.1 RTK-GPS Class

As mentioned in §5.2 the communication with the RTK-GPS receiver and the
publication of the data on the data space are shared in two classes named:
GPSReceiver.java and GPSPublisher.java.

9.1.1 GPSReveiver.java
In the class GPSReceiver.java, the receiver transmits all data using as bytes via
COM1 port. The data is analysed and if it starts with the characters “GPLLQ” it is
split into tokens where only the significant part is retained, as east, north, and height.
The data is then normalised for the Cartesian coordinates followed by MASSIVE-3
and finally published on a data space. There is the option to process the data using a
Kalman Filter. The processing of the data to a Kalman Filter class is done before the
data is published. Table B.1 shows the high level algorithm. In the source code the
variables are set as:

 x = east.
 y = high.
 z = north.

Table B.19 GPSReceiver.java Algorithm

Begin
initCommPort()
initiation of communication port
setup of communication port
initiation of variables that do the in/out interface with the receiver
Begin
run()
While (there is a byte coming)
read byte
convert to a character type and add it on a string
If (the character received is equal to a brake line)
decode(String with character received)
 clean String
EndIf
EndWhile
End
decode(String with data received from RTK-GPS receiver)
initiation of variable that receives the String in tokens
For (each token)
add it on a array of tokens
EndFor
associate to a variable token number 3 and subtracting it from the OS
value of the base point
associate to a variable token number 5 and subtracting it from the OS
value of the base point
associate to a variable token number 10 and subtracting it from the OS
value of the base point
Kalman(x, 2.0, (-)z, 1)
updateCoordinates(z, x, 2.0)
End

Class Constructor
The GPSReceiver class constructor is:

 GPSReceiver(GPSPublisher, String)

Where:

GPSPublisher – is an instance of the class GPSPublisher

String – is communication port identifier

9.1.2 GPSPublisher.java
For each new set of data received and processed from the RTK-GPS receiver, this
class makes it available on the data space. This data space is created at the beginning
of this class and identified for further utilisation. Table B.2 shows the high level
algorithm.

Table B.20 GPSPublisher.java Algorithm

Begin
checks if the URL received is valid
checks if the data service can be activated
starts the data space with an identification
creates an instance of the GPSReceiver Java class
 updateCoordinates add a new 3D position on the data space
End

Class Constructor
The GPSPublisher class constructor is:

 GPSReceiver(String, String)

Where:

String – is the IP address followed by the data space identification

String – is communication port identifier

9.2 Tilt-Sensor and Magnetic Compass Class

As introduced in §5.2 the communication with the tilt sensor and magnetic compass
circuit board and the publication of the data on the data space were shared in two
classes named: TCM2.java and TCM2Publisher.java.

9.2.1 TCM2.java
This class is similar to the previous class showed in B.1.1. The connection is opened
in the early stages of the algorithm. If successfully opened, the class starts to listen to
the port collecting the data until it finds a line break. The beginning of the line is then
analysed. If the line started with the characters “$C” it is known that the message is
not broken. The message is then split into tokens and only the values of pitch, row
and yaw are retained. An option for a Kalman filter is available that can be used or
not. The data is then sent to TCMPublisher class to be stored on the data space. Table
B.3 shows the high level algorithm. The values are converted to two digits of
precision in order to reduce the small drift.

Table B.21 TCM2.java Algorithm

Begin
initCommPort()
initiation of communication port
setup of communication port
initiation of variables that do the in/out interface with the receiver
Begin
run()
While (there is a byte coming)
read byte
convert to a character type and add it on a string
If (the character received is equal to a brake line)
decode(String with character received)
clean String
EndIf
EndWhile
End
decode(String with data received from tilt sensor and magnetic compass)
initiation of variable that receives the String in tokens
pass the values of the correct tokens into the correct variable
Kalman(x, 2.0, (-)z, 0)
updateCoordinates(xtcm, ytcm, ztcm)
End

Class Constructor
The TCM2 class constructor is:

 TCM2(TCM2Publisher, String)

Where:

TCM2Publisher – is an instance of the class TCM2Publisher

String – is communication port identification

9.2.2 TCM2Publisher.java
This class has the same function introduced in B.1.2. It creates another reference in
the data space with an identity and logs the data received from TCM2.java. Table B.4
shows the high level algorithm.

Table B.22 TCM2Publisher.java Algorithm

Begin
checks if the URL received is valid
checks if the data service can be activated
starts the data space with an identification
creates an instance of the TCM2 Java class
updateCoordinates add a new 3D position on the data space
End
Class Constructor
The TCM2Publihser class constructor is:

 TCM2Publisher(String, String)

Where:

String – is the IP address followed by the data space identification

String – is communication port identification

9.3 Reading from the Database

The following class is designed to check any new data that was published in the data
space such as: GPS coordinates, tilt sensor and compass data or any key command
pressed by the office user. Each new data is a trigger for an action performed by the
Integrator class.

9.3.1 Integrator.java
The Integrator class is used by the field user when all devices are connected to the
computer carried on the backpack. GPS, tilt sensor and magnetic compass data, when
available, are read from the data space and pass to the variables into MASSIVE-3.
The update corrects the users’ visualisation of the VE in relation to the real
environment. Because the office user can control the VE the database can de updated
each time that a command key is pressed in his keyboard. This means that field and
office user receive a new object or even a new a VE depending on the exercise being
performed when each new income data from this particular area of the database is
read. Table B.5 shows the high level algorithm. As mentioned previously the
Integrator class inherited its structure from AugurscopeControl class created for the
Augurscope project (Koleva et al., 2001, Benford et al., 2003a).
Table B.23 Integrator.java Algorithm
Begin
constructor initialisation
GUID factory
dataspaces
activate management server
key stroke input session
compass input session
gps input session
management server initialisation
dynamic objects initialisation

a thread controls the multiple process running in parallel as dynamic objects

and positions update

update dynamic objects

update positions

Begin
management server control class
End

Begin
Key stroke event handler class
End

Begin
GPS event handler class
End

Begin
GPS event handler class
End

End

Class Constructor
The Integrator class constructor is:

 private Integrator(String, String, String, String)

Where:

String – is the IP address followed by the VE identification

String – is the IP address followed by the data space identification
String – configuration file used by MASSIVE-3
String – is the same name used to start the cveworld command

9.4 Kalman Filter

The filter is implemented to reduce the jerk of: GPS coordinates, tilt sensor and
compass.

9.4.1 Matriz.java
This class implements a Kalman filter to be used on the data received from the
magnetic compass, the tilt sensor and the RTK-GPS receiver. The implementation of
the filter is necessary in order to reduce jerk in the data received.

Two matrixes supply the models for the error and noise processed by the Kalman
filter. The values of such matrixes are set up in accordance with the equipment in use.
Instead of an algorithm, a diagram is chosen to introduce the concept due to the
simplified way it demonstrates the process. Figure B.1 shows the diagram.

There is a set of important variables that contribute to the final result. These variables
are:

 x – matrix with the predicted coordinates

 q – matrix of noise - covariance
 p – matrix with estimated error
 r – matrix with the noise from TCM2
 k – Kalman gain number weight
 z – matrix with the data from TCM2 – innovation vector
 phi – matrix identity

The filter is well suited to navigation applications, which contain states that are time-
variant, even in real-time. This is a recursive algorithm, which filters measurements
using knowledge of the system measurement error (Hide, 2003). Even being
recursive, the algorithm is based on the last estimate and current measurements. There
is no necessity to log all data to process again. The result is a computationally
efficient algorithm that allows real-time processing, preventing the swing of the HMD
image and smoothing the movements registered by the tilt sensor and the RTK-GPS
receiver. It is not within the scope of this thesis to describe the Kalman filter
equations. The reader is invited to read (Welch and Bishop, 2001, Hide, 2003) for a
better description of the equations. The following dataflow (Figure B.1) describes in a
few steps the Kalman filter that is implemented.

The first step of the filter is the calculation of the estimated error (p). This matrix
is the result of the sum of (p) and the matrix of noise - covariance (q) that is
propagated to the next epoch (Figure B.1 – step 1). It is convenient to mention that (q)
is static and contains an approximate model of the signal that is being filtered.

Figure B.74Kalman Filter Dataflow

The second step (Figure B.1 – step 2), (k) (Kalman gain number weight) receives the
matrix with the estimated error (p) multiplied by the inverse of the result of (p) added
to the matrix with the noise from TCM2 (r). In this step, (k) is receiving the
adjustments based in the last sample of data that will be applied in the next epoch.

On the third step the algorithm combines the data from the sensor (z) (innovation
vector) with (x) (the predicted coordinates) through a subtraction (Figure B.1 – step
3). The result is then multiplied by (k) (with the adjustments based on the previous
samples) and added to (x).
 Finally (Figure B.1 – step 4), (p) receives the new Kalman values (k) after a
subtraction with a (phi) matrix identity.

Class Constructor
The Matriz class constructor is:

 Matriz()

9.5 Office User Key Stroke Command

The Office user can control the VE from his keyboard. Each time that a new
command is typed a new object can be added to the VE or the entire scene can be
changed. This flexibility gives freedom for the Field user.

9.5.1 ReadKeyStroke.java
This class is used by the Office user to type any command in the keyboard that will
be logged in the database by the KeyStroke class. The commands are at the moment
the letters from “a” to “f”. Each letter when read by the Integrator (see §B.3.1) is
interpreted as a command. At the moment the VE can be changed completely or
objects can be put in or out of the visualisation. Table B.6 shows the high level
algorithm.

Table B.24 ReadKeyStroke.java Algorithm

Begin
run()
While (true)
Thread (it sleeps each 1000 milliseconds)
read byte
If (character typed)
update database with new character
EndIf
EndWhile
End

Class Constructor
The ReadKeyStroke class constructor is:
  ReadKeyStroke (KeyStroke)

Where:

KeyStroke – is an instance of the class KeyStroke

9.5.2 KeyStroke.java
This class has the same function introduced on B.1.2. It creates another reference in
the data space with an identity and logs the data received from ReadKeyStroke.java.
Table B.7 shows the high level algorithm.

Table B.25 KeyStroke.java Algorithm

Begin
checks if the URL received is valid
checks if the data service can be activated
starts the data space with an identification
creates an instance of the ReadKeyStroke Java class
updateCoordinates the key stroke is translated to a number to be logged in
the database
End

Class Constructor
The KeyStroke class constructor is:

 KeyStroke(String)

Where:

String – is the IP address followed by the data space identification

Chapter 10
User Interviews – Questions and Answers

The questions and answers made throughout the interview during the evaluation
process (Chapter 7) are shown here. All the comments can be found in §6.2.3. The
codes used to identify the users were maintained: Engineer1 (E1), Engineer2 (E2),
Engineer3 (E3), Engineer4 (E4), Architect1 (A1), Computer Scientist1 (C1),
Computer Scientist2 (C2) and ordinary user (O1).

The area covered by the questions are as follows:

 Merging real and digital environments.

 Hardware.
 Digital environment.
 Usability.
 Different/complementary approaches.
 Office and Field Users.
10.1 Merging Real and Digital Environment
The question involves the main topic of this thesis: AR. The users were questioned if
they needed to adapt themselves to the VE and if the scale and distances walked were
the same that they were used to see in the physical world.

Self adaptation
Did you have to adapt the way that you move in order to have a better perception of
the VE?

E2: Actually the turning of my head is much better. I think that the difficulty is when
you come closer to an obstacle and the program decides, I guess, whether you are
half way through it, through it or just in front of it. This is the most, I would say,
annoying. But moving my head or walking towards when I was quite far was not a
problem. Quite far is something around two metres.

E3: Yes, but I was ok with that. It was a bit juggling but you explained to me that it
was the GPS. I was not expecting it to be so “shaking”. Then I learned that I have to
move my body much slower to look around.

O1: Yes, I had to. I noticed that I moved my head a lot less.

Immersion
Did you feel immersed in the VE or did you have an AR experience?

E1: I did it in the tram station. I did not so much in the buildings. I think that is to do
with the texture and the reality.

E2: In the beginning when the see-through of the HMD was not set as clear, maybe
because we have a sunny day today, I did not realise that I went so far. For the tram
station I was immersed completely in the VE. When you adjusted the see-through then
I was a bit more conscious of where I was. For my eyesight I have two layers so if I
want to see the grass I change my eyes focus and it takes a second for me to see-
through. If I focus on my goggles alone then I am immersed in the VE.

E3: Yes, you saw me walking through the tree (a tree in the physical world). I was
quite happy to concentrate on that. Because of the brightness and contrast I could not
see much of the background. I could mostly see the VE and I was quite comfortable.

A1: I think so. You cannot get any sense of distance because you cannot see the
physical world. I was often amazed when I looked around to see where I was in the
physical world. The augmented process did not work really. For me there was not a
relation between the physical and the digital environments. I was immersed. It was
not mixed reality but only the VE there to be seen. And I think that part of the
problem is that there are not physical features to relate to it.

C1: After a while I was totally looking into the VE. I was trying to find things in
there, trying to find what was going on. I think that the physical world went to the
back of my mind. But then when you said: “-Turn around to my physical location”
the two worlds just swapped their positions. I was again able to see-through the
goggles. But while in the VE I forgot sometimes that I could see through the HMD.

C2: The tram driven through me took away the sensation of immersion because it
was difficult to work out how to avoid the tram. However, I think that there is some
relation about this and how dark the glasses are. As you make the glasses darker you
feel more involved in the digital environment aspect of the mixed reality stuff and
obviously less involved in the physical world.

O1: Partially. From the local view yes but at the same time you know that it will not
be there. Why? The floor. When you walk you can feel the grass.

Scale
What can you say about the scale? Was it coherent or not? Did you have any
difficulty to know how high a wall or a tram shelter was?

A2: I can imagine but it is difficult to know without a proper scale. Is the Office
User’s embodiment to scale? It has about two metres high. It is possible because I
could still see it after the first phase. I have no idea how tall the house is in reality
but I would say three and a half metres tall or something like this. The first phase I
would have said was…I could still see the top of the embodiment’s head, one metre
sixty or something like that, then one third plus one third.

A3: One of the difficulties is to get scale. It would be nice if you put some image that
we could recognize. Like you, it was useful to have you wondering around that gave
me something to compare with scale. From the perspective point of view it will be
nice if there is something I could make some comparison.
E4: If the pipes were not in the right scale (but) the idea of different colours was
good. You just need to make sure that each colour is representing the right thing in
the proper diameter.

C1: The scale was not very precise. Some more information is necessary to know how
far you are from the house or how high is a wall. When I was walking towards the
house I did not have the feeling that I was getting close to the house. I just kept
walking until I eventually got there. When I walked around the house the scale looked
ok. Another point is how high you put the “eyes of the Field User’s Embodiment”. I
could misunderstand a view because I think that an object is too high or too low
based in the perception of my own body.
10.2 Usability
Did you feel annoyed to walk through digital objects?

E1: No.

E2: No, but each time I asked you if I could go through them. It also means that you
need to re-collect what you have in front of you. If I cannot find the Office User’s
embodiment I need to imagine where was it at the last time or any other digital object
that could help me in the navigation. I keep going through until I find it again. But it
means in a complex environment if you need to use another application where people
may be moving or things may be moving, it is a bit annoying, but other than this,
there is no problem.

E3: It was a little perplexing to sort it out. I was quite happy to walk through a wall
but for a long time you could just see blue for example (when going through the
windows in the third exercise). I was not sure there if I was getting any closer or any
further away.

C1: Extremely! It throws the user off, because the user is not expecting to go through
it. I think that for me it reduces the experience. I started to disbelieve in a lot of other
things within that environment and that represented what it should be. Scale, how big
things are, movements or if I go through this wall maybe it is not even positioned
there.

O1: I did it first, but once you got used to it, it becomes fairly easily. Why did you
feel annoyed? Partially because the frame rate that was hard to judge how is your
movement. Because when you move in the physical world you have your view
changing straight away. The system has a delay that you have to anticipate the
movement to make it stop in the right position.

10.3 Hardware
What could you say about the hardware used? Did the GPS receiver and the tilt
sensor work adequately?

E1: I think that when you stand still or walk really slowly is much better. It is better
than in the previous prototypes. The positioning and orientation are quite good. The
orientation changes quite quickly, as you turn round it follows you quite quickly. I
think the problem is more in the position of the GPS when you move. For GPS
equipment is coming down in price. I think in overall the problem is the weight and
the number of cables.

E2: The only jumping action was when you become close to something. Let’s say that
I want to touch these pipes then I want to be close to them. Maybe I want to touch
them physically and in the VE. I noticed when I was coming close to the Office User
in the tram station the issue of the positioning that made me go through the wall due
to some drifting and also the frame rate.

E3: I think so. It helped me when you told “do not look around so fast”. When I felt
more in control of that I did look around much more slowly and waited for the thing
to catch up. That was much better and gave me a much better smoother view.

A1: I know that the GPS was being updated each second but it still is very jumpy. It
does not matter when you are outside of the digital house for example, but when you
are inside the house where it is pretty narrow it becomes a problem.

C2: It seems to not have too much drifting at all. The whole scene does not seem to
drift from the physical world. It was jerking a little bit but for me it was not a real
problem. As soon as I stopped moving everything seemed to settle down.

10.4 Digital Environment

Texture and Colours
What can you say about the VE? Where the colours used ok? How could the design
be more appealing, attracting more users to it? Should texture make more sense than
only colour?

E1: Yes. So it is to make clear that you are looking at bricks, wood, glass or
something else. It helps you to visualise what you’re looking at. I think that it is quite
difficult to visualise what it is, with only solid colours. And it is not only shadows but
it does not have any point of realism to relate to. If you look at a wall and you have
bricks you know that it is a wall. If you look through glass you know that you have a
window. The perspective of what are you looking at, how far away it is, is quite
difficult again when you have not got the texture. But when you have not got texture it
is really difficult to visualise the size of what you are looking at. In a blue print, there
are the dimensions or something indicating what that room is. When you look at the
elevation on a blue print you have a brick texture there. It is not always coloured but
there is some attempt to show what is a brick, what is a window and what is a roof. It
just helps your conceptualisation of what you are looking at.

E2: It does not bother me if the wall in one of the phases was blue. But for somebody
maybe less able to think in space maybe they will want to see something that could
help in the visualisation, like bricks or cement textures. But it is not a big deal if you
can imagine that it is a new geometry. If you imagine that it is a house and you
cannot imagine the geometry, maybe you want to see more.

E3: The colours sometimes confuse, I had to turn my head to get a view of what was
going on and what the colours represented. Because there is no stereo you need to
move your head to actually see it. For the external wall I suggest a brick texture
making it more photorealistic and perhaps the internal walls as well.

E4: You could use different or higher resolution to present a house. So now, because
you assembled everything together the viewer has a rough idea about how the house
will be and how it will be integrated with the environment. That is a good idea
particularly for the environmentalists and planners. Its assisting software is good.

A1: It is not the problem with the concept but the VE for example could be more
realistic. I would like to see more textures that could help you to distinguish between
the walls from the floor. Of course that there are different colours but it is more
difficult to see. Is that right that parts of the walls have different colours? Yes. Of
course you do not get there in a real building. Usually the walls have the same colour
from bottom to top. Sometimes you do not know if there is a real window or
something else. It is because each colour represents a different phase in the project.
Ok, but for orientation it does not help. The windows in the third example could also
be improved if a more realistic representation is put there. I did not have any sense of
wall thickness, what was odd. Are there any? Or the walls are just polygons? They
are all polygons. This is also weird because if you go around the corner it is just flat.
It becomes more difficult to know if you come through something or not. It is difficult
to say if you are in one side or in the other side of the polygon. And you should not be
able to go through walls. How do you imagine it working? You could make the
animation stop for example. I do not know if something like it will work. Probably
not. But it makes it weird because we all know that that we can go through them, but
because of the rendering you cannot really see what side you are. You never know
when you are in it, or walking through it or just stuff inside of it.
C1: Most of our video processing uses colours. It is natural to relate things to
colours like when you think in a brick you think red. If you think a tyre you think
black. There are lots of ways that you can make it help to visually stimulate the user
adding a bit more of details like in the wall, the floor and the ceiling. This sort of
texture helps to emphasise things like: that is a wall and that is a wall further away
from you because it is slightly darker for example. Lightening is another thing. If I
have a blue wall a few centimetres from my face the colour is different from if I had
the same wall a few metres away. At the moment the system does not support this kind
of lightening process. The user automatically loses the sense of depth perception.

C2: The fine details should be right to become a real good tool. The objects do not
really look like in the physical world. Like texture for example. A window is not just a
square or a rectangle shape with glass that allows him to see through it with some
particular properties. For me to see a blue block is a bit confusing.

O1: Yes you can tell which is which part of the project. Did you get confused with the
colours? I did with the pipes (third example) because the floor and the walls are very
similar colours. Perhaps because the colours are flat shaded. When the surface
merges with each other, each time that he moves because there is no difference in
contrast between the two polygons, that represents different walls.
10.5 Office User and Field User
Collaboration

How did the collaboration between Field and Office User work? Did you feel the
necessity to have two people to guide you (Office User’s embodiment) and control
the VE on your behalf? Even with very limited gestures and movements the Office
User’s embodiment could point or turn its head. What do you think about it?

E1: What it did help me to do, is if you said to me to look to a certain direction it is
easy to say: “-Look where my embodiment is”. It gives me direction to look at. I think
that having not necessarily being you but having something as a pointer and then you
say: “-Look where this pointer is” then it gives you something to relate to. I think this
is useful and also the collaborative discussion is useful as well. Not only because of
the complexity of controlling the environment as a user, but also once the user is in
there they may want to ask questions about what is he looking at which in a
collaborative environment someone can talk to him and discuss about it and ask for
changes. Even if it was a man user in the field, if he’s got a question, how does he
ask? Because the Office User knows where I am in the VE and what I am looking at
and then you can answer questions about what I am looking at.

E2: Well, the Office User pointed out things with the embodiment’s head to me. So I
used that. There were instances that the embodiment did a movement up and down
with its head to show me something I was able to associate this as a human
interaction. Walking towards me also worked well to indicate direction. When I was
on my own I was also able to explore things like in the second example. The only
doubt that I had was when I was exploring the bathroom because in my country that
place does not have big windows. Does it make a difference to have the embodiment?
Yes it does if the Office User is pointing things or like when you are moving thinks
about I can see the string coming from your head when it moves its head I could
associate. But if there is no embodiment there and the explanation is verbally clear
enough. I think that I managed to find my way around it.

E3: For me it was useful to have you to follow as a guide. I felt less secure when I
could not see you. When you passed close to an object where I could not see you I did
not know where I was going. When I could see you I was happy to follow you around
and look to what you were looking. When I did lose myself I found myself wanting to
see you. In relation to gestures, it was really good when I looked to the stairs in the
third exercise. When you stood in the stairs and looked up that helped me to look in
the right direction because you have something specific to look out there.

E4: In the beginning I did not understand your “role in the game”. And then you told
me follow me and I did looking at the same places that the Office User’s embodiment
was looking at. You were there to guide the other users to see for example the future
of the city. The embodiment is really necessary. I was never lost I just followed you
and your instructions. I think that you could implement a kind of laser ray coming
from the embodiment’s eyes to point at things in the VE. It should be easier than to
make the embodiment gesticulate to the right position or digital object. It could not
make a big difference if the VE was controlled by you or by me. The result will be the
same but the viewer is who will make the decision if it is good or not. To have
someone working in front of the computer to relieve the hassle of the Field User, to
press buttons is really good. If you have asked me if I would like to control the VE by
my own I would say no. I would certainly refuse. Along the exercise we were
continually exchanging information and you gave me the guidance of what I should
do and what action I should take. That was good.

A1: I think that it was useful. You could say something like: “-come over here”. That
was useful. Or “look where I’m looking”, that also worked. And to be able to talk
with each other is useful as well. I think that the Field Users could have a kind of
keyboard with only two keys and flip through them. Like go up and go down.
C1: Definitely! I would be lost on that VE where I was. The most that I could do was
to walk blindly just going through different rooms. The overall structure that the
Office User explained while I walked, I could not understand if alone. After all I was
able to explain where things were event though there was no feature like living room
sofas or tables to indicate that I was in the living room. But because the Office User
could explain the layout of the house I was able to create a mental map of how the
house was spatially distributed. It worked almost as a tour guide. The embodiment
made me feel like having a tour guide. And to see you move away from me and move
towards me is quite reassuring because besides being lost in that environment I
would not know where I started from. When you stood in a particular point and you
said to me: “-Look at where I am looking” and I can see where you are facing and I
know where to go. It also helps me to explore the VE in a more secure perspective.

C2: There is a lot of cognitive effort to working your way in the environment.
Because you are missing other senses, information, or because you are just getting
visual information, you have to think harder how to navigate around. And because it
is jerking, you need to think slightly more and memorise how things are before you
start to move and go back to this. For your brain is working quite hard to navigate
around the scene without having someone else showing you around and changing the
VE for you. The control in the Office User’s hands takes one task away from your
brain allowing you to focus more on the building.

O1: It helped me to see what you tried to show me. If I just had the sound I would
find it really hard to find the things. Wherever the Office User was in the VE, I could
spot him and go towards his direction. What about the head movements? Did it make
any difference when I gesticulated with my head looking up or down? Yes, it helped
me to know where I should look.
10.6 Different/Complementary Approaches
Merging blue prints, scale models and AR

When you look at the blue print could you associate what you saw, with the VE that
you walked through? What about the picture of the house in the second exercise?
Could you make any association?

E2: I could imagine the house made of bricks as you showed in the picture but I
thought that the pillars were made of wood.

E3: Ah, yeah, yeah, yeah! Most of the time I stood over here (pointing to where he
stood in the blue print). I definitely saw that in there. I can recognise it because
mainly I stood here (pointing to the place in the blue print). This is the elevation of
the house. I did not really see the external, I was standing there when we lost contact.
I think if I stood there in the balcony (in the physical world) rather than this distant
view here (the picture) it would be closer to what I saw (in the VE). This pillars
where not there (pointing to the picture). They were there. I saw these two in one side
but not on the other side.

A1: Yes and no. I mean most half of it. I understood that there was a veranda. I
understood that part. I have no concept of this. I remember that you told me what is
what, but I think that these walls were not there. Partially I think…looking at the
elevation in the blue print I can bring all the view together but I cannot remember to
see this view at all in the VE. And I also did not have an idea of how the shape of the
roof is.

Can you imagine the house in the physical world (questioning A1)? I can imagine
very bright colours because this is what sticks in my mind. I know that there was
some windows but I could not say where is where. I can relate this picture to the plan
but it is because I used to read plans. I can relate the veranda that I saw in the VE. I
cannot relate this window…oh I remember now that I saw this window when I was
inside the house. Although that on, in the VE, looks completely different, much wider.
If you asked me how the window looks like on that side I would draw a window that
goes around the corner very thin and long. Maybe you are confusing with the
kitchen’s window because that one is like this. There is also a door not far from the
window at the corner of the house. Ah…but that is weird because I am pretty sure
that I turned round. Do you remember when we talked about a very small window
here (pointing in the blue print to a small window between the living room and the
veranda). But I thought that it was inside the house between the kitchen and the
bedroom. I did not realise that it was in the outside wall of the house.

So you cannot imagine the house based in the VE (questioning A1)?

I would not go that far. I could imagine part of it as I also found partly confusing as
well. It is not necessary the principle of it. But of course the model that you have is
extremely simple and the colours which ever direction you look you have the same
colours. You could not say from the colours that I was looking this way or that way.
You have to take all the other features. And that is probably where the confusion
comes in. I also did not have any physical orientation. Obviously when you are in the
physical world you can easily say that that tree is there and that other tree is there
without even looking at. And if I turn around I still know where the trees are. But in
the VE I could not tell you if when I turned left, I would see something that I want to
see. I will have to turn around to find it again. I do not think that necessary you have
a mental map or it is very abstract. I could not just say that the kitchen was in my left.

C1: It definitely did, I could visualise the house although I may have a different
colour in my head. But it was very similar. So if you show me a picture of the house I
can tell you that it looks something that I could visualise. On paper, on two
dimensional, it is hard to tell heights or the third dimension. This is basically what
you can get from VE. You can have a better perspective and distances and how things
overall look. With the blue print it is very difficult.

O1: Yes, but it is difficult to say the size of the bathroom. You can tell for example
that in a direction there is a bedroom but without furniture you cannot tell how big it
will be.

Best approach to explain a new development

What do you prefer to see if someone comes to your house with a new project that
will be implemented in few days time: a blue print, a scale mode, an artistic drawing
or the tool in this project?

E1: For someone that is an engineer and used to look to blue prints and is not used to
look at VE, it is a bit difficult comparison to give. I think that the VE, the rendering,
is quite simplistic, so realising what it is you’re looking at is more difficult. As soon
as you put the roof on, because of the shape of the roof it is obviously that it is a roof.
But because all the other squares and blocks are solid coloured it is more difficult to
understand. So I think the important development will be to get the rendering more
appropriate to what are you looking at and then it will make it easy to understand.

C1: I think because an artist can paint in tree dimensions like in a picture it will give
more information than a blue print. In the VE you can experience, you can walk
through, you can feel the surroundings. You are immersed. For me the VE as shown
is more helpful than the other options.

O1: I will prefer the blue print. Between the scale model and the tool presented I will
prefer the second option because it is possible to move around. It is more useful.
What about the level of realism of the model? It did not make that much difference.
You do not need fantastic graphic to know the shapes, you still can find your way
around.
Real potentiality of the tool
In a real situation do you think that a system like this would be worth?

E2: If there is a need for it or a use for it is quite easily to visualise. For a house I do
not think that you need it but for a big project I think that could be useful.

E3: I think that the level of detail in the tram station was quite impressive. The
buildings with the colours you got to have to sell, people understand what it is. When
people want to see how high the construction could go, you need to relate it with
some sort of scale and for me this is what was missing in the buildings.

E4: The quality of the VE in the first exercise and the LoD like the CCTV were
amazing. There is potential use for security issues.

A1: I think that it is going to be useful. I have issues with the HMD. This area is flat
there is nothing to fall from, but it is just a dangerous thing to have. Because you
cannot really see where you are going to. It can be useful for the construction
application as you suggested.

Do you have something else to add to your comments?

E2: We were quite unlucky to have too much light coming from the Sun and shades
from the clouds intercalated. You asked me to look at things in the roof, because it is
in the VE, I do not have the benefit of the shade of the physical roof, I was looking
direct to the Sun or to the sky. It is impossible to see even when the see-through is in
its darkest.

I found it very easy to use. I am not good in visualising things, but I found it
particularly easy to use.

Usefulness yes, the pipes and things that you told me. All the underground networks,
all sorts of pipes, cables, telephone, gas. I worked on this when I was in the army
looking at underground systems and very often the only map that you got is somebody
taking the measurement with the reference to a wall or to a curb. Of course that if
somebody moved the curb or the wall you lost the reference. Very often what you do
is to take one metre and a half between the curb and the access. You marked it. And
this is not very accurate. You may be ending digging some area that it is not suitable.
Maybe with GPS system and the see-through you could visualise things better. For
building houses I would say that it is suitable for bigger projects. Maybe also for
people who is buying on plants that could be useful. Either for a promoter or a buyer
to see how it will look like. Maybe with a better rendering you could make it more
appealing.
I cold see this idea working indoors in the future for heath and safety where you
could even couple with other things. You could create a digital training environment
for the fire brigade without having to burn something. You could put them in a
situation, or to police forces too. We could bring some physical modelling or CFD to
the research.
E3: I will be interested to find how you work with a group of people as Field Users.
Let’s say that if you want to sell the idea of design of a tram station or a building to a
group rather than an individual. I can imagine that they will want to interact and
they could individually go to explore the VE. I am not quite sure how could they make
it as a group in the VE. I could imagine site inspectors coming and saying that they
will be back in a moth to see the progress for example. The same could happen if you
are an architect and want to sell something to a panel and not to an individual.
Bibliography

Abele, E., Anderl, R. and Birkhofer, H. (2005) Springer, pp. 282.

Abowd, G. (1995), Vol. 2008.
AC3D (2004), Vol. 2004.
AceCad (2004), Vol. 2008 AceCad.
AEC Design Group (2008), Vol. 2008 AEC Design Group,.
Ammar-Khodja, S., Perry, N. and Bernard, A. (2008) Concurrent Engineering, 16,
89-101.
Angewandte und Sozialorientierte Informatik - Universität Hamburg.
Aquino, M. H. O. d. (1988) In Institute of Engineering Surveying and Space
GeodesyUnivesity of Nottingham, Nottingham, pp. 155.
Aquino, M. H. O. d. (1992) Petrobras, Rio de Janeiro - Brasil.
Aquino, M. H. O. d. (1999) In Institute of Engineering Surveying and Space
GeodesyUniversity of Nottingham, Nottingham - UK.
Askew, B. (2007) The University of Nottingham - School of Civil Engineering,
Nottingham, pp. 39.
Augenbroe, G., Verheij, H. and Schwarzmüller, G. (2002) In Engineering,
Construction and Architectural Management, Vol. 3 Blackwell Science Ltd,
pp. 259-271.
Autodesk Inc. (2008a), Vol. 2008 San Rafael, CA, USA.
Autodesk Inc. (2008b), Vol. 2008 San Rafael, CA, USA.
Azuma, R., T. (1995) In Department of Computer ScienceUniversity of North
Carolina, Chapel Hill - USA, pp. 262 pages.
Bamboo Open Source Community (BOSC) (1999), Vol. 2004.
Bell, B., Feiner, S., Höllerer, T. (2001) In ACM Symposium on User Interface
Software and TechnologyOrlando, Florida, USA.
Benford, S., Schnädelbach, H., Koleva, B., Gaver, B., Schmidt, A., Boucher, A.,
Steed, A., Anastasi, R., Greenhalgh, C., Rodden, T. and Gellersen, H. (2003a)
University of Nottingham - Mixed Reality Laboratory, Nottingham - UK, pp.
19.
Benford, S., Schnädelbach, H., Koleva, B., Gaver, B., Schmidt, A., Boucher, A.,
Steed, A., Anastasi, R., Greenhalgh, C., Rodden, T. and Gellersen, H.
(2003b), Vol. 373x503 (Ed, augurscope2.jpg) Nottingham - UK.
Benford, S., Seagar, W., Flintham, M., Anastasi, R., Rowland, D., Humble, J.,
Stanton, D., Bowers, J., Tandavanitj, N., Adams, M., Farr, J. R., Oldroyd, A.
and Sutton, J. (2004) In UbiComp 2004(Eds, Davies, N., Mynatt, E. and Siio,
I.) Springer, Nottingham - UK, pp. 70-87.
Bentley Systems Incorporated (2008) In Buildings, Vol. 2008 Bentley Systems
Incorporated,.
Bingley, R., M., Roberts, G., W. (2001) Institute of Engineering Surveying and Space
Geodesy - University of Nottingham, Nottingham - UK.
Bingley, R. M. (2003), Vol. 2 Institute of Engineering Surveying and Space Geodesy.
Bouchlaghem, D., Shang, H., Whyte, J. and Ganah, A. (2004) Automation in
Construction, 287-295.
Brate, A. (2002) Technomanifestos: Visions of the Information Revolutionaries,
Texere; 1 edition (July 4, 2002).
Brooks, F. P. (1999) IEEE Computer Graphics and Applications, 19, 16-27.
Brousseau, É. and Rallet, A. (1995) Revue d’Économie Industrielle,, 74, 1-20.
Bruce, V., Green, P. R. and Georgeson, M. A. (1996) In Visualization in Landscape
and Environmental Planning: Technology and Application(Eds, Lange, E. and
Bishop, I. D.) Taylor & Francis, pp. 296.
Building Integration Software Company Ltd. (2008), Vol. 2008 (Ed, Building
Integration Software Company Ltd.).
BulidingExplorer.com (2007), Vol. 2008 BulidingExplorer.com,.
Bureau, U. S. C. (2001) In GIS Frequently Asked Questions and General Info List
Index, Vol. 2004.
Capra, M. (2000) In Departamento de Engenharia CartográficaInstituto Militar de
Engenharia, Rio de Janeiro - Brasil, pp. 144.
Capra, M., Aquino, M., Benford, S. and Dodson, A. (2004a) In Ubicomp, Vol. 1
Nottingham, UK.
Capra, M., Aquino, M., Dodson, A. and Benford, S. (2004b) In 1st FIG International
Symposium - NottinghamNottingham, pp. 3.
Capra, M., Aquino, M. H. O., Dodson, A. and Benford, S. (2003a) In VI Symposium
on Virtual RealityBrazil.
Capra, M., Aquino, M. H. O., Dodson, A., Benford, S. and Koleva-Hopkin, B.
(2003b) In International Conference on Artificial Reality and Telexistence,
Vol. 1 Tokyo - Japan, pp. 314.
Carillion (2000) Nottingham.
CFD Online (1994), Vol. 2006 http://www.cfd-online.com/Links.
Channel 4 (2006) In DispatchesChannel 4, UK, pp. 60 min.
Churchill, E. F. and Snowdon, D. N. (1998) Virtual Reality: Research, Development
and Applications, 3, 3-15.
Churchill, E. F., Snowdon, D. N. and Munro, A. J. (2001) Collaborative virtual
environments: digital places and spaces for interaction, Springer, London -
UK.
CityScape (2004), Vol. 800x600 (Ed, Olympia311.jpg) CityScape.
Columbia University Computer Graphics and User Interfaces Laboratory (2004a),
Vol. 2004 New York - USA.
Columbia University Computer Graphics and User Interfaces Laboratory (2004b),
Vol. 2004 New York - USA.
Delaney, B. (1999) IEEE Computer Graphics and Applications, Mar/Apr Edition.
Delaney, B. (2000) IEEE Computer Graphics and Applications, May/Jun Edition.
D-Link Systems Inc. (2006) D-Link Systems.
Edgar, G. (2002) In Mapping Psycology 2(Eds, Miell, D., Phoenix, A. and Thomas,
K.), pp. 3-56.
(Eds.) Simpson J. A. and Weiner E. S. C. (1989) In Collaboration(Ed, Public Domain
/ Wikipedia).
Eisenberg, N., Fabes, R. A., Schaller, M. and Miller, P. A. (1999) In Empathy online,
Vol. 4 (Ed, Preece, J.) Springer London, London, pp. 74-84.
Encyclopedia Brittanica Online (2007) In Collaboration, Vol. 2008 (Ed, Public
Domain / Wikipedia) Wikipedia.
Engelbart, D. (1962) In Technomanifestos(Ed, Brate, A.) Texere LCC, New York and
London.
Equator (2005) In www.equator.ac.uk, Vol. January 2005.
ESRI (2001) In Your Internet Guide to Geographic Information Systems, Vol. 2004
California.
ESRI (2003), Vol. 2004 ESRI.
ESRI (2004a), Vol. 1280x995 (Ed, 3dextensionsshotgreen3_lg.jpg) ESRI, Redlands -
California - USA.
ESRI (2004b), Vol. 2004 ESRI, Redlands - California - USA.
Etchegoyen, R. H. (1991) In Empathy online, Vol. 4 (Ed, Preece, J.) Springer
London, London, pp. 74-84.
European Space Agency (2005) In ESA - Navigation, Vol. 2005 Paris - France.
Evans, A., J., Roberts, G., W., Dodson, A., H., Denby, B., Hollands, R., Cooper, S.
(2001), Vol. 2001.
Federal Aviation Administration (2005a) In Programs - Wide Area Augmentation
System, Vol. 2005 U.S. Department of Transportation, Washington.
Federal Aviation Administration (2005b) In Programs - Wide Area Augmentation
System, Vol. 2005 U.S. Department of Transportation, Washington - USA.
Feiner, S., MacIntyre, B., Höllerer, T., Webster, A. (1997) In International
Symposium on Wearable Computing'97Cambridge, Manchester, UK, pp. 74-
81.
Fiksel, J. and Hayes-Roth, F. (1993) In Establishing and processing client
requirements-a key aspect of concurrent engineering in construction, Vol. 1
(Eds, Kamara J. M., Anumba C. J. and O., E. N. F.) Blackwell Science Ltd.
Gilbert, S., Mitchell-Jones, I. and Vardy, M. (2002) In Virtual Reality for Engineers -
J1C VRE, Vol. 2004 (Ed, Schofield, D.) Nottingham.
Godfried Augenbroe, Hans Verheij and Schwarzmüller, G. (2002) In Engineering,
Construction and Architectural Management, Vol. 3 Blackwell Science Ltd,
pp. 259-271.
Golden Software (2004), Vol. 2004 Golden, Colorado, USA.
Gomez M., Sun D., Waelchli C., Balsiger P. and F., P. (2002), Vol. 2006 (Ed,
Neuchâtel, I. o. M. U. o.) Neuchâtel.
Google (2006).
Graphisoft R&D Zrt. (2007), Vol. 2008 (Ed, Graphisoft R&D Zrt.) Budapest,
Hungary.
Gray, B. (1989) In Engineering, Construction and Architectural Management, Vol. 3
(Eds, Godfried Augenbroe, Hans Verheij and Schwarzmüller, G.) Blackwell
Science Ltd, pp. 259-271.
Greene, J. H. (1984) In Civil Engineering Practice in the Twenty-first Century:
Knowledge and Skills(Ed, Grigg, N. S.) Reston Pub. Co.
Greenhalgh, C. (1997) (Ed, 7) University of Nottingham, Nottingham, pp. 19.
Greenhalgh, C. (1999a) Large scale collaborative virtual environments, Springer,
London - UK.
Greenhalgh, C. (1999b), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2000a) (Ed, 3) University of Nottingham, Nottingham, pp. 8.
Greenhalgh, C. (2000b), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2000c), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2000d), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2001a), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2001b), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2001c), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C. (2001d), Vol. 2001 Mixed Reality Laboratory - University of
Nottingham - UK.
Greenhalgh, C., Benford, S. (2001e), Vol. 2001 Mixed Reality Laboratory -
University of Nottingham - UK.
Greenhalgh, C., Izadi, S., Rodden, T. and Benford, S. (2001) Mixed Reality
Laboratory - University of Nottingham - UK.
Greenhalgh, C., Izadi, S., Rodden, T., Benford, S. (2001f) Mixed Reality Laboratory -
University of Nottingham - UK.
Greenhalgh, C., Purbrick, J. and Snowdon, D. (2000) In CVE2000San Francisco
USA, pp. 8.
Handykey Corporation (2006), Vol. 2007 Denver, CO, USA.
Hartshorn, J., Maher, M., Crooks, J., Stahl, R. and Bond, Z. (2005) Canadian Journal
of Civil Engineering, 32, 170-180.
Haymaker, J., Fischer, M., Kunz, J. and Suter, B. (2004) ITcon, 9, 419-441.
Heike, F. (2002) In Lehrstuhl für Realzeit - ComputersystemeTechische Universität
München, München - Germany, pp. 163.
Heintz, J. L. (2002) In Engineering, Construction and Architectural Management
2002, Vol. 3 Blackwell Science Ltd, pp. 181-191.
Hide, C. D. (2003) In IESSGUniversity of Nottingham, Nottingham.
Höllerer, T., Feiner, S., Bell, B., Hallaway, D. (2001a) IEEE Computer Graphics and
Applications.
Höllerer, T., Feiner, S., Terauchi, T., Hallaway, D., Rashid, G (1999) IEEE Computer
Graphics and Applications, August Edition.
Höllerer, T., Feiner, S., Tinna, N., Hallaway, D. (2001b) In Second International
Workshop on Artificial Intelligence in mobile SystemsSeattle, Washington,
USA.
Human Interface Technology Laboratory at the University of Washington (2004)
Washington - USA.
Huss, E. (1997), Vol. 2008.
Ickes, W. E. (1997) In Empathy Online, Vol. 4 (Ed, Preece, J.) Springer London,
London, pp. 74-84.
IESSG (2002), Vol. 2004 IESSG, Nottingham - UK.
IESSG (2008) In www.nottingham.ac.uk/iessg, Vol. 2008 IESSG, Nottingham - UK.
Interactive Media Systems Group Institute for Software Technology and Interactive
Systems Vienna University of Technology (2004) Vienna - Austria.
Jonsson, B., Hedling, G. and Wiklund, P. (2003) In ION GPS/GNSSPortland, Oregon,
U.S., pp. 10.
Kamara J. M., Anumba C. J. and Evbuomwan N. F. O. (2000a) In Engineering,
Construction and Architectural Management 2000, Vol. 1 Blackwell Science
Ltd, pp. 15-28.
Kamara J. M., Anumba C. J. and Evbuomwan N. F. O. (2000b) In Concurrent
Engineering: Research and Applications, Vol. 1 (Eds, Fiksel, J. and Hayes-
Roth, F.) Blackwell Science Ltd.
Kamara J. M., Anumba C. J. and Evbuomwan N. F. O. (2000c) In Establishing and
processing client requirements—a key aspect of concurrent engineering in
construction, Vol. 1 (Eds, J. M. Kamara, C. J. Anumba and Evbuomwan, N.
F. O.) Blackwell Science Ltd, Engineering, Construction and Architectural
Management 2000, pp. 15-28.
Kaplan, E. D. (1996) In Understanding GPS principles and applicationsArtech
House Publishers, Norwood, pp. 237-383.
Khanzode, A., Fischer, M. and Reed, D. (2008) ITcon, 13, 324-342.
Koleva, B. (2001) In Computer ScienceThe University of Nottingham, Nottingham,
pp. 239.
Koleva, B., Adams, M., Taylor, I., Benford, S., Fraser, M., Greenhalgh, C.,
Schnädelbach, H., Lehn, D. v., Heath, C. and Row-Farr, J. (2001) In SIGCHI
conference on Human factors in computing systemsACM Press, Seattle,
Washington, United States.
Ku, K., Pollalis, S. N., Fischer, M. A. and Shelden, D. R. (2008) ITcon, 13, 458-485.
Kubicki, S., Bignon, J. C., Halin, G. and Humbert, P. (2006) ITcon, 11, 566-586.
Lange, E. and Bishop, I. D. (2005) Visualization in Landscape and Environmental
Planning: Technology and Application, Taylor & Francis.
Lanzetta J.T. and Englis B.G. (1989) In Empathy online, Vol. 4 (Ed, Preece, J.)
Springer London, London, pp. 74-84.
Leclerc, Y., Reddy, M., Eriksen, M., Brecht, J. and Colleen, D. (2002) In SRI
InternationalSRI, Menlo Park, pp. 45.
Leica Geosystems LLC. (Ed.) (1999) Technical Reference Manual Version 2.0,
Printed in Switzerland, Heerbrugg, Switzerland.
Leica Geosystems LLC. (2001), Vol. 2004 Leica, Atlanta - Georgia - USA.
Leica Geosystems LLC. (2003), Vol. 2004 (Ed, 3.0) Heerbrugg Switzerland.
Leica Geosystems LLC. (2005) In Product News, Vol. 2006.
Leica Geosystems LLC. (2006) In TS 3 – Forum for Providers and Users of Real
Time Correction ServicesMunich Germany.
Liston, K., Fischer, M. and Winograd, T. (2001) ITcon, 6, 69-82.
Lucena, V. (1986).
Lucent Technologies - Bell Labs Innovations (2000).
Maldovan, K. (2007) CONVR 2007.
Masanori, N. S. and Sawada, K. K. (2004) In Technical Report: Matsushita Electric
Works, Vol. 2004 (Ed, Matsushita Electrocnic Works Ltd) 81, Tokyo.
Mathieson, K. and Keil, M. (1998) Information & Management, 34, 221-230.
Matsushita Electronic Works Ltd (2003), Vol. 1054x464 (Ed,
shiodome_cyberdome_2.jpg) MEW, Tokyo.
Matsushita Electronic Works Ltd. (2003), Vol. 675x269 (Ed,
shiodome_cyberdome_1.jpg) MEW, Tokyo.
Merrian-Webster (2007) In Collaboration, Vol. 2008 (Ed, Public Domain /
Wikipedia) Wikipedia.
Merrian-Webster (2008) In Merrian-Webster Online Dictionary, Vol. 2008 (Ed,
Merrian-Webster) Merrian-Webster, Springfield, MA, USA.
Milgram, P. and Kishino, F. (1994) In IEICE Transactions on Information Systems,
Vol. E77-D (Ed, IEICE) IEICE.
Mokhtar, A. H. M. (2002) In Engineering, Construction and Architectural
Management 2002, Vol. 3 Blackwell Science Ltd, pp. 111-231.
Mountain, D. and Liarokapis, F. (2007) Aslib Proceedings: New Information
Perspectives, 59, 422-436.
Neil S. Grigg, Marvin E. Criswell, Darrell G. Fontane and Thomas J. Siller (2001) In
Civil Engineering Practice in the Twenty-first CenturyASCE Publications, pp.
83-108.
Nemetschek North America (2008) In Vectorworks, Vol. 2008 Nemetschek North
America,.
NET (2001), Vol. 2001 Nottingham Express Transit.
Nitithamyong, P., Skibniewski, M. J. and Endowed, A. J. C. (2007) ITcon, 12, 39-59.
Nottingham Tram Consortium (2004), Vol. 2006 Nottingham Tram Consortium,
Nottingham.
Novak, J. (1999) In Electrical Engineering and ComputingFaculty of Electrical
Engineering and Computing, Zagreb - Croatia, pp. 110.
Office of Superintendent of Public Instruction In Communication, Vol. 2008.
Origin Instruments (2002) Origin Instruments, Grand Prairie - USA, pp. 6.
OSG Community (2004) In OpenFlight, Vol. 2008.
Peansupap, V. and Walker, D. H. T. (2005) ITcon, 10, 193-218.
Pendleton, G. (2002) In Directions Magazine.
Petrovski, I. (2003), Vol. 2004 (Ed, Communications, A.) GPS World.
Petrovski, I., Kawaguchi, S., ISHII, M., Torimoto, H., Fujii, K., Sasano, K., Kondo,
M., Shoji, K., Hada, H., Uehara, K., Kawakita, Y., Murai, J., Townsend, B.,
Cannon, M. E. and Lachapelle, G. (2000) In GPS2000Salt Lake City.
Piekarski, W. (2002) In Proceedings of IEEE Virtual Reality'2002IEEE, Orlando,
Florida, USA.
Piekarski, W., Bernard, G., Thomas, B. (2001a) In 2nd International Symposium on
Augmented RealitySan Francisco, USA.
Piekarski, W. and Thomas, B. (2002), Vol. 664x577 (Ed, tinmith.jpg) Adelaide -
Australia.
Piekarski, W., Thomas, B. (2001b) In Linux Conference, Vol. 2001.
Piekarski, W., Thomas, B. (2001c) In 2nd International Symposium on Augmented
RealityNew York, USA.
Piekarski, W., Thomas, B. (2001d) In International Symposium on Wearable
ComputersZurich, Switzerland.
Piekarski, W., Thomas, B., Gunther, B. (1999) In DCNET'99Sidney, Australia.
Pinho, M., S. (1999), Vol. 2002 Pontifícia Universidade Católica - RS - Brasil.
PNI Corporation (2004) Santa Rosa, CA, USA.
Posner, M. (2002) In Perception and Attention(Ed, Edgar, G.) Open University, pp.
15.
Preece, J. J. (1999) Virtual Reality, 4, 74-84.
Public Domain / Wikipedia, Vol. 2008 Wikipedia.
Public Domain / Wikipedia (2008) In Wikipedia - The Free Encyclopedia, Vol. 2008
Wikipedia.
Purbrick, J. C. R. (2001) In Computer ScienceUniversity of Nottingham, Nottingham,
pp. 176.
Qiu, Y., Ge, P. and Yim, S. C. (2007a) Concurrent Engineering, 15, 357-368.
Qiu, Y., Ge, P. and Yim, S. C. (2007b) Concurrent Engineering, 15, 369-380.
R.T.Azuma, Baillot, Y., Behringer, R., Feiner, S., Julier, S. and Mac-Intyre, B. (2001)
In Augmented reality navigation systems, Vol. 4 (Eds, Narzt, W., Pomberger,
G., Ferscha, A., Kolb, D., Muller, R., Wieghardt, J., Hortner, H. and
Lindinger, C.) Springer Berlin / Heidelberg, pp. 177-187.
Reddy, M., Leclerc, Y., Iverson, L., Bletter, N. (1999) IEEE Computer Graphics and
Applications, Mar/Apr Edition.
Reitmayr, G. and Schmalstieg, D. (2001) In ISCW 2001 Zurich, Switzerland.
Reitmayr, G. and Schmalstieg, D. (2004a) In Symposium Location Based Services
and TeleCartography 2004 Vienna - Austria.
Reitmayr, G. and Schmalstieg, D. (2004b), Vol. 681x494 (Ed, reitmayrlbs2004.pdf)
Vienna - Austria.
Reitmayr, G. and Schmalstieg, D. (2004c), Vol. 686x513 (Ed, reitmayrlbs2004.pdf)
Vienna - Austria.
Roberts, G., Evans, A., Dodson, A., Denby, B., Cooper, S. and Hollands, R. (2001) In
ION DPS 2001Salt Lake City, USA.
Roberts, G., Evans, A., Dodson, A., Denby, B., Cooper, S. and Hollands, R. (2004)
Nottingham - UK.
Roberts, G., W. (1997) In Institute of Engineering Surveying and Space
GeodesyUniversity of Nottingham, Nottingham - UK.
Roberts, G., W., Evans, A., Dodson, A., Denby, B., Cooper, S., Hollands, R. (2002)
In ION DPS 2001, GPS WorldUK.
Rotating Tower Technology International Limited (2008), Vol. 2008 Florence, Italy.
Salazar, G., Mokbel, H. and Aboulezz, M. (2006) In ASEE New England Section
2006American Society for Engineering Education, New England, pp. 10.
Sapeta, K. (2003) In Photogrammetry & Remote Sensing, Vol. 69 ASPRS, Bethesda -
Maryland - USA, pp. 114-115.
Sarshar, M., Tanyer, A. M., Aouad, G. and Underwood, J. (2002) In Engineering,
Construction and Architectural Management 2002, Vol. 2 Blackwell Science
Ltd, pp. 152-160.
Satoh, K., Hara, K., Anabuki, M., Yamamoto, H. and Tamura, H. (2001) In ISMR
2001(Ed, ISMR) Yokohama - Japan, pp. 2.
SC3000.COM (1998) In SimCity 3000 Resource Center, Vol. 2004 Wren M. Weburg.
Scheer, S., Junior, R. M., Quevedo, J. R. S., Jr, J. M. and Fontoura, P. S. (2007)
ITcon, 12, 221-230.
Schildt, H. (2001) Java™ 2: The Complete Reference, The McGraw-Hill Companies,
Berkeley - USA.
Schnädelbach, H., Koleva, K., Flintham, M., Fraser, M., Chandler, P., Foster, M.,
Benford, S., Greenhalgh, C., Izadi, S., Rodden, T. (2001), Vol. 2001 Mixed
Reality Laboratory - University of Nottingham - UK.
Shelbourn, M. A., Bouchlaghem, D. M., Anumba, C. J., Carillo, P. M., Khalfan, M.
M. K. and Glass, J. (2006) ITcon, 11, 57-71.
Shin, D. H. and Dunston, P. S. (2008) Automation in Construction, 17, 882-894.
Silva, L. F. C. F. d. (1998) In Engenharia de TransportesUniversidade de São Paulo,
São Paulo, Brasil.
Snowdon, D. and Greenhalgh, C. (1996) University of Nottingham, Nottingham, pp.
10.
Someren, M. W. v., Barnard, Y. F. and Sandberg, J. A. C. (1994) The Think Aloud
Method - A practical guide to modelling cognitive processes, Academic Press,
London, UK.
SONY (1998).
SRI International (2000a), Vol. 654x543 (Ed, campus.gif) Menlo Park - CA - USA.
SRI International (2000b), Vol. 579x519 (Ed, win.gif) Menlo Park - CA - USA.
SRI International (2002), Vol. 2004 Menlo Park - CA - USA.
Sun Microsystems (2004), Vol. 2001.
Sun Microsystems (2008) In Java 3D API, Vol. 2008.
SWEPOS Control Centre (2005), Vol. 2005 Gävle - Sweden.
Tamura, H. (2000) In Imagina 2000.
Tamura, H. (2001a) Japan, pp. 97-104.
Tamura, H., Yamamoto, H., Katayama, A. (2001b) IEEE Computer Graphics and
Applications, Nov/Dec Edition.
Tekla (2008), Vol. 2008 Tekla.
The University of Nottingham (2008) In www.nottingham.ac.uk, Vol. 2008.
Thesaurus (2008) In The Free Dictionary, Vol. 2008 Farlex.
Tilley, P. and Barton, R. (1997) In Engineering, Construction and Architectural
Management 2002, Vol. 3 (Ed, Mokhtar, A. H. M.) Blackwell Science Ltd,
pp. 111-231.
Torán-Marti, F., Ventura-Traveset, J. and Chen, R. (2003) In GNSS2003, pp. 15.
TOSHIBA (Ed.) (2002) TOSHIBA Satellite Pro 6100 Portable Personal Computer
User's Manual.
Trimble Navigation (2002), Vol. 2005 Sunnyvale - USA.
Tuzmen, A. (2002) In Engineering, Construction and Architectural Management,
Vol. 3, pp. 209–221.
Ubisense (2008), Vol. 2008.
Ugwu O. O., Anumba C. J. and Thorpe A. (2001) In Engineering, Construction and
Architectural Management, Vol. 3 Blackwell Science Ltd, pp. 211-214.
United States Naval Observatory (2005), Vol. 2005.
University of Loughborough, University of Salford and London School of Economics
(2001), Vol. 2008.
Uschold, M. and Gruninger, M. (1996) In Knowledge Engineering Review, Vol. 2
Cambridge Journals.
UST (1998), Vol. 1024x610 (Ed, lax_05_big.jpeg) UST website, Los Angeles.
UST (2003), Vol. 1024x610 (Ed, la_08_big.jpeg) UST web-page, Los Angeles.
UST (2004), Vol. 2004 Los Angeles.
Vicente, K. J. (1999) Lawrence Erlbaum Associates.
Virtools (2004) In Applications > Industry, Vol. 2004 Montreal - Canada.
VTT (2008), Vol. 2008 VTT.
W3C (1995), Vol. 2008.
Webster, A., Feiner, S., MacIntyre, B., Massie, W. and Krueger, T. (1996) In
Computing in Civil Engineering California - USA, pp. 1090.
Weimer, N. (1998), Vol. 2005 (Ed, Canada, U. o. A.-.).
Welch, G. and Bishop, G. (2001) In SIGGRAPH(Ed, ACM, I.) Los Angeles,
California, USA, pp. 46.
White, C. (2006) In BIM Panel Discussion Denver, CO, USA.
Wikipedia (2006), Vol. 2006.
Wikipedia (2008a), Vol. 2008 (Ed, Public Domain).
Wikipedia (2008b), Vol. 2008 (Ed, public Domain).
Wikipedia (2008c), Vol. 2008 (Ed, Public Domain).
Wikipedia (2008d), Vol. 2008 (Ed, Public Domain).
Wikipedia (2008e), Vol. 2008 (Ed, Public Domain).
Wikipedia (2008f), Vol. 2008 (Ed, Public Domain).
Wilkinson, S. (2001) In Engineering, Construction and Architectural Management
2001, Vol. 3 Blackwell Science Ltd, pp. 160-170.
Winner, R. I., Pennell, J. P., Bertrend, H. E. and Slusarczuk, M. M. G. (1998) In
Establishing and processing client requirements-a key aspect of concurrent
engineering in construction, Vol. 1 (Eds, Kamara J. M., Anumba C. J. and
Evbuomwan N. F. O.) Blackwell Science Ltd, pp. 15-28.
Woksepp, S. and Olofsson, T. (2006) ITcon, 11, 267-240.
Woodward, C. (2007) In IBM iForum 2007 Zurich, Switzerland.
Woodward, C., Lahti, J., Rönkkö, J., Honkamaa, P., Hakkarainen, M., Jäppinen, J.,
Rainio, K., Siltanen, S. and Hyväkkä, J. (2007a) In W78 Conference Maribor.
Woodward, C., Lahti, J., Rönkkö, J., Honkamaa, P., Hakkarainen, M., Jäppinen, J.,
Rainio, K., Siltanen, S. and Hyväkkä, J. (2007b) In W78 Conference Maribor,
Vol. 309x233 (Ed, ARMobile.jpg) Finland.
Woodward, C., Lahti, J., Rönkkö, J., Honkamaa, P., Hakkarainen, M., Jäppinen, J.,
Rainio, K., Siltanen, S. and Hyväkkä, J. (2007c) In W78 Conference Maribor,
Vol. 432x343 (Ed, ARWebCam.jpg) Finland.
Xsens (2008), Vol. 2008.
Yamamoto, H. (1999) In 1999 IEEE Systems, Man and Cybernetics SocietyTokyo,
Japan.
Yunchun Yang, Richard T. Sharpe and Hatch, R. R. (2002) In ION 2002Portland,
Oregon, USA, pp. 953-961.
Zhang, Z. (2005), Vol. 2008.