Learning With Mobile Device

learning with mobile devices research anddevelopment
a book of papers edited by Jill Attewell and Carol Savill-Smith
learning with mobile devices research anddevelopment

a book of papers edited by Jill Attewell and Carol Savill-Smith
Published by the Learning and Skills Development Agency www.LSDA .org.uk Feedback should be sent to: Information Services Learning and Skills Development Agency Regent Arcade House 1925 Argyll Street London W1F 7LS . Tel 020 7297 9144 Fax 020 7297 9242 enquiries@LSDA .org.uk Registered with the Charity Commissioners Designer: Dave Shaw Cover illustrator: Joel Quartey Cover photographs used with the permission of Hewlett-Packard Italiana Srl, IPM Group Spa and Fujitsu Personal Systems Printer: Blackmore Ltd, Shaftesbury, Dorset
1440/05/04/2500 ISBN 1 85338 833 5
Learning and Skills Development Agency 2004 Unless otherwise stated, the copyright for individual papers is retained by the author(s). You are welcome to copy this publication for internal use within your organisation. Otherwise, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, electrical, chemical, optical, photocopying, recording or otherwise, without prior written permission of the copyright owner. The views expressed in this publication are those of the authors and are not necessarily held by the Learning and Skills Development Agency.
Further information For further information about the issues discussed in this publication please contact: Jill Attewell M-learning Project coordinator Learning and Skills Development Agency. Tel 020 7297 9100 jattewell@LSDA .org.uk
This publication was supported by the Learning and Skills Council as part of a grant to the Learning and Skills Development Agency for a programme of research and development, and by the European Commissions Information Society Technologies programme.
Contents
Foreword Introduction Mobile learning and social inclusion: focusing on learners and learning Jill Attewell and Carol Savill-Smith Evaluating non-functional requirements in mobile learning contents and multimedia educational software Gianna Avellis, Antonio Scaramuzzi and Anthony Finkelstein A learning space model to examine the suitability of different technologies for mobile learning Chris Baber, Mike Sharples, Giasemi Vavoula and Philip Glew 1 3
13
21
Self-produced video to augment peer-to-peer learning 27 Eva Brandt, Per-Anders Hillgren and Erling Bjarki Bjrgvinsson Individualised revision material for use on a handheld computer Susan Bull and Eileen Reid Take a bite: producing accessible learning materials for mobile devices Jo Colley and Geoff Stead 35
43
Participatory design in the development of 47 mobile learning environments Karin Danielsson, Ulf Hedestig, Maria Juslin and Carl Johan Orre Using mobile devices for the classroom of the future Peter Dawabi, Martin Wessner and Erich Neuhold Adaptive navigation for mobile devices Paul Graham, Chris Bowerman and Albert Bokma Collaboration and roles in remote field trips Nick Hine, Rosaleen Rentoul and Marcus Specht Learning can happen anywhere: a mobile system for language learning Vaida Kadyte A context-awareness architecture for facilitating mobile learning Peter Lonsdale, Chris Baber, Mike Sharples and Theodoros N Arvanitis
SMILE : the creation of space for interaction through blended digital technology Rose Luckin, Diane Brewster, Darren Pearce, Richard Siddons-Corby and Benedict du Boulay
55
61
69
73
79
87
Learning tools for Java-enabled phones: 95 an application for actuarial studies Mara Cruz Mayorga-Toledano and Antonio Fernndez-Morales Learning 2 go: making reality of the scenarios? Louise Mifsud Exploring the potential of a games-oriented implementation for m-portal Alice Mitchell Mobile learning as a service offering with near-term technologies Chris Noessel Tate Modern Multimedia Tour pilots 20022003 Nancy Proctor and Jane Burton Usability and accessibility of personal digital assistants as assistive technologies in education Peter Rainger Evaluation of a mobile learning organiser and concept-mapping tools Mike Sharples, Tony Chan, Paul Rudman and Susan Bull Designing scalable, effective mobile learning for multiple technologies Andy Stone Fragmentation in mobile learning Antti Syvnen, Marika Pehkonen and Hanne Turunen A task-centred approach to evaluating a mobile learning environment for pedagogical soundness Josie Taylor Designing for learning or designing for fun? Setting usability guidelines for mobile educational games Siobhan Thomas, Gareth Schott and Maria Kambouri Mobile learning evaluating the effectiveness and the cost John Traxler A critical approach to an adaptive user interface design Teija Vainio and Mikko Ahonen Mobile cinematic presentations in a museum guide Massimo Zancanaro, Oliviero Stock and Ivana Alfaro Author index 99 105
117
127
131
139
145
155
167
173
183
189
193
197
Acknowledgements
MLEARN 2003 conference chairs
Jill Attewell, m-learning Project coordinator Giorgio Da Bormida, MOBI learn Project coordinator Professor Mike Sharples, University of Birmingham
MLEARN 2003 conference programme committee
Dr Theodoros N Arvanitis, MOBI learn project, University of Birmingham, UK Jill Attewell, m-learning Project coordinator, Learning and Skills Development Agency (LSDA ), UK Giancarlo Bo, MOBI learn project, GIUNTI Ricerca, Italy Giorgio Da Bormida, MOBI learn Project coordinator,
GIUNTI Ricerca, Italy
Dr Ingo Dahn, Department of Computer Science, University of Koblenz-Landau, Germany Kevin Donovan, Development adviser, Learning and Skills Development Agency (LSDA ), UK Dr Michael Gardner, Institute for Socio-Technical Innovation and Research (Chimera), University of Essex, UK Terry Keefe, ICT Research and Development, U f I Learndirect, UK Dr Paul Lefrere, Institute of Educational Technology (IET ), Open University, UK John ODonoghue, the Learning Lab, University of Wolverhampton, UK Dr Vanessa Pittard, Department for Education and Skills (D f ES ), UK Government Dr Carol Savill-Smith, m-learning project, Learning and Skills Development Agency (LSDA ), UK Professor Mike Sharples, MOBI learn project, University of Birmingham, UK David Traub, Epiphany Partners, USA
Foreword
I am delighted to provide a foreword for this publication as it brings together such an interesting and varied collection of articles describing the latest practical and theoretical work in the exciting new area of mobile learning. Advances in technologies inevitably bring with them additional opportunities to facilitate and enrich the experiences of individual learners. Sometimes, we in education are slow to grasp those opportunities but I hope that this publication will stimulate thought about how the explosion of mobile phone and handheld technology can be exploited to reach out to more and more learners. In recent years the Learning and Skills Council (LSC ) has invested millions of pounds in promoting and facilitating the use of information and learning technologies (ILT ) to support and enhance teaching and learning in the UK . In the late 1990 s the Further Education Funding Council (FEFC ), British Educational Communications and Technology Agency (Becta) and the Further Education Development Agency (FEDA , which subsequently became LSDA ) worked together to improve the ILT literacy and skills of teachers, support staff and managers in further education and sixth form colleges. The National Learning Network (NLN ) has built upon this platform to maximise the availability and effectiveness of e-learning in colleges and in the wider post-16 community. The Learning and Skills Council (LSC ) and the Department for Education and Skills (D f ES ) are working together to give learners of all ages e-learning of the highest calibre to support their lives, extend their choices, enrich their competences and strengthen their autonomy at home, work and in the community. We intend progressively to remove barriers to access to lifelong learning, to enable learners to take full advantage of learning opportunities regardless of mode and place of study and to promote experimentation and innovative development. In 2002 we published the influential report of the Distributed and Electronic Learning Group (DELG ). In the introduction to this report the chair of DELG , Professor Bob Fryer, observed: Already many young people are growing up familiar with digital gadgetry and computerised processes and are skilful in their application, as a normal part of their lives. It is certainly true that many people are familiar and skilful with mobile phones. In supporting the work of the m-learning project, which is exploring how mobile learning might attract reluctant young learners and help improve their life skills, we hope to move a step closer to the dream of providing any time, anywhere learning which is consistent with learners lifestyles. I hope you find this publication informative and inspiring.
Keith Duckitt Head of ICT Learning and Skills Council, UK
Introduction
The authors who have contributed to this book are researchers, developers and practitioners in both educational and commercial organisations from a number of different countries. They all have in common an interest in the new and quickly evolving field of mobile learning. The papers are based on presentations given at a very successful and enjoyable international conference, MLEARN 2003 , which was hosted in London in May 2003 by the Learning and Skills Development Agency (LSDA ). The conference was organised by LSDA , as the coordinating partner of the m-learning project, in collaboration with our sister project MOBI learn. Some of the papers detail the findings of mobile learning projects, some are based on desk research and the authors attempts to identify and further develop theory relevant to mobile learning, and some report work in progress. Work in progress includes both research and the development of mobile learning materials and systems. Many of the papers have been written by colleagues who are partners in one of the two large mobile learning projects supported by the European Commissions Information Society Technologies programme m-learning and MOBI learn. Further information about the projects can be found on the project websites at www.mlearning.org and www.mobilearn.org. Further information on the progress of these projects will be reported at the MLEARN 2004 conference in Rome in July 2004 , and the conference website can be found at www.mobilearn.org/mlearn2004 / Some presenters at the MLEARN 2003 conference were not able to write a full paper for this book, but a brief summary of their work can be found in the MLEARN 2003 Book of abstracts which can be downloaded from the LSDA website at www.LSDA .org.uk/events/mlearn2003 I hope you enjoy reading the papers in this book and learning about the innovative and exciting work of the projects described.
Kate Anderson Director, Research Learning and Skills Development Agency, UK
Attewell, Savill-Smith
Mobile learning and social inclusion: focusing on learners and learning

Jill Attewell Learning and Skills Development Agency Regent Arcade House, 1925 Argyll Street, London W1F 7LS E-mail: jattewell@LSDA.org.uk Abstract
This paper describes how the m-learning project is investigating whether the use of mobile technologies in the hands of young adults (aged 1624) might engage them in learning activities, start to change their attitudes to learning and contribute towards improving their literacy, numeracy and life chances. The findings of the research carried out by the Learning and Skills Development Agency are outlined, together with its plans for the major research activity in phase 2 of the project involving 200 learners in three European countries. Keywords: m-learning, mobile phones, handheld devices, social inclusion, basic skills
Carol Savill-Smith Learning and Skills Development Agency Regent Arcade House, 1925 Argyll Street, London W1F 7LS E-mail: csavill-smith@LSDA.org.uk
580 000 or so 16-year-olds who leave school each year, around 150 000 are below Level 1 in both Maths and English, where Level 1 is the level of attainment school pupils are expected to achieve by age 11. Furthermore, 22% of these young people do not go on to training or work after they leave school (DfEE 2001). The statistics illustrate that this is an intractable problem and we believe that imaginative and innovative approaches are needed to bring about improvements in learning such basic skills. The approach of the mlearning project is to offer small sets of learning experiences on mobile devices similar to the mobile phones which many young people are comfortable using and enthusiastic about.
1. An important issue for educators

The 1997 International Adult Literacy Survey (OECD 1997) found that many developed countries had functional illiteracy rates of 20% or more and worse innumeracy. An example of functional illiteracy is that if given the alphabetical index to the Yellow pages (a directory where local traders can advertise their services to the public), some 7m adults could not locate the page reference for plumbers. An example of innumeracy is that one in four adults could not calculate the change they would get from 2 if they purchased a loaf of bread costing 68p and two cans of beans at 45p each (Moser 1999). In 1999 in the UK one in five adults was found to have less literacy than is expected of an 11year-old child (Moser 1999). Two years later UK government figures revealed that of the
2. Background to the m-learning project

The m-learning project is a 3-year, panEuropean research and development study with partners in Italy, Sweden and the UK. Its aim is to use portable technologies to provide literacy and numeracy learning experiences for young adults (aged 1624) who are not in a full-time education environment, and to promote the development and achievement of lifelong learning objectives. The m-learning project is coordinated by the Learning and Skills Development Agency (LSDA) and project partners include two commercial companies (Cambridge Training and Development Limited in the UK, and Lecando AB in Sweden) and two university-based research units (Centro di Ricerca in Matematica Pura ed Applicata at the University of Salerno in Italy and Ultralab at Anglia Polytechnic University in the UK).
Learning with mobile devices: a book of papers
3. Investigating the context of mobile learning

The m-learning project has several research strands. The objectives of phase-one research activities were to assist the project partners in selecting appropriate technologies in the rapidly evolving field of personal information and communications technologies (ICTs) and to inform the design and development of learning materials. Three literature reviews were carried out focusing on different, but complementary, domains of mobile learning research, relating to: mobile phones, their uses and users the use of palmtop computers for learning the use of computer/video games for learning.
Table 1 Young adults owning a mobile phone in Italy, Sweden and the UK Country Age group Percentage owning a mobile phone 1519 2.91 85% 2024 3.40 94% 1519 0.53 91% 2024 0.51 92% 1519 3.77 90% 2024 3.57 81% Based on Brown and Dhaliwal 2002 Population (millions)
Italy Sweden UK
At the same time LSDAs survey of 746 young adult mobile phone users across the UK explored users attitudes towards their mobiles and their initial reactions to the idea of a phonebased game that might help them with their reading, spelling or maths. Other research activities by project partners have included: identifying relevant technologies and their functionality; standards for learning materials development and interoperability; learner and knowledge modelling; small-scale user trials of learning materials and user interfaces or microportals.
Ownership statistics for other countries can sometimes be surprising. For example, in the USA, the take-up of mobile phones has been much slower and only 40% of 1519 year olds, and 61% of 2024 year olds, own mobiles (Brown and Dhaliwal 2002). On the other hand, in some developing countries, like Botswana, US $5 million per month (4 million per month) is spent on mobiles, raising concern that this may divert spending from necessities such as food and clothing, and also from entertainment (Mogapi 2000; quoted in Mutula 2002). Townsend (2000) states that mobile phones are now appearing widely in the squatter communities that surround the cities of developing countries, places where conventional fixed-line telephony has never existed. It is further suggested by Townsend that in future the use of smart phones, which have voice recognition, will benefit members of the world population who have no ability to read and write. The personal nature of the mobile phone, together with its constant presence on or about the users person, the types of communication it enables and its importance to teenage identity and friendships (Ling and Yttri 1999; Eldridge and Grinter 2001) all support our belief that its popularity is not just a short-term fad. The role of phone calls and messaging in friendship rituals such as gift giving and sharing (Taylor and Harper 2002; Bauman 2003) suggests the mobile phone has potential as a collaborative learning platform. Ichinohe and Suzuki (2002) comment that there are many Japanese websites offering learning materials to i-mode users (i-mode is a Japanese mobile internet service). In December 2001, they estimated there were 30 million imode users. Outside Japan, however, it would appear that the use of mobiles in learning is still
4. The findings of the research reviews

4.1.1 Past research into the impact of mobile phones and their possible use for learning The research review highlighted the universal spread of the mobile phone. The overwhelming majority of young adults in our target age group in our project partner countries (UK, Italy and Sweden) own mobiles, as can be seen in Table 1 (based on data from 2002). Of course, many young adults share mobiles, perhaps among family members or with friends, and so the actual number of users may be higher than Table 1 suggests.
relatively rare and usually occurs as part of short-term or pilot projects. There have been a number of instances of the use of SMS (text messaging) soap operas to encourage pupils to revise early for examinations, eg WAN2lrn (see www.interactivesolutions.co.uk/sms/sms.htm) and, more recently, the BBCs GCSE Bitesize games as examination preparation that can be downloaded to mobile phones (see www.bbc.co.uk/schools/gcsebitesize). Prototypes have been developed for learning languages which use quizzes, word and phrase translations, working with a coach and vocabulary work (Regan 2000). Mobile phones have also been used as the subject of teaching eg teaching A-level Physics students about how mobiles work (Edwards 2000) and as a way of encouraging the creativity of young pupils in art lessons, eg through designing phone fantasy sculptures because of their relevance to their life outside school (Szkely 2001). It is frequently suggested that the use of SMS may inhibit the learning of correct spelling and grammar. However, many young people who would not normally pick up a pen and write messages are enthusiastic texters, and there are suggestions that the verbal skills of some usually reticent teenage boys are improving as they chat on their mobiles (Plant 2001). Thus they are uniquely placed to contribute to improving young peoples literacy especially as mobile phones are increasingly being designed with the extra facilities commonly found on palmtop computers as well as cameras and picture messaging. This provides more opportunities for visual and literary expression. 4.1.2 Past research into the use of palmtop computers for learning The distinction between mobile phones and palmtop computers is becoming less and less obvious. While more mobile phones now have Palm-like functionality (eg contacts database, a calendar, etc), palmtop computers now include, or can be upgraded with, mobile phone functionality. Also a number of hybrid phonepalm devices are now available which combine typical phone and palmtop functionality. One result of these developments is that published research about the use of palmtop computers is increasingly relevant to a project primarily focused on the potential of mobile phones. It has been found by Savill-Smith and Kent (2003), in a review of the published literature
about the use of palmtop computers for learning that palmtop computers can: assist students motivation help organisational skills encourage a sense of responsibility help support both independent collaborative learning act as reference tools track students progress deliver assessment.
and
This review highlights a number of learning games that have been designed specifically for palmtops, including the Cooties and Geney simulation games. The Cooties game is a virus simulation game for learning science (see www.goknow.com/Products/Cooties.html), where students and the teacher use infra-red beaming to infect their personalised coodles (or pets, akin to the Tamagotchi concept). The Cooties game has been found to encourage collaborative and group working and increase the amount of writing produced (Shields and Poftak 2002). The Geney game (see http://geney.juxta.com/chi2001_handheld.pdf) simulates a population of fish representing a gene pool, and the goal is for students to work together to produce a fish with certain genetic characteristics (Danesh et al. 2001). According to Mandryk et al. (2001), this produces a rich social interaction which is found to excite and motivate learners to interact, including those who were less inclined to do so. Other uses of palmtops for learning have been: to increase the amount of childrens reading and writing (eg the Docklands Learning Acceleration Project (McTaggart 1997); to help with the collection and analysis of data for science fieldwork (Graham 1997); in sports and physical education; and the use of reflective logs (often used in medical education). However, as was found with the review of the mobile phone literature, there has been a lack of comparative research studies, and studies that relate their work and outcomes to theories of learning. 4.1.3 Past research into the use of computer games for learning Playing computer games is a popular activity for many young adults and, increasingly, mobile phones are incorporating more games. Such games are often based on early computer games, eg Snake, which is available on Nokia phones. It is projected that the value of the mobile games market in Europe, the USA and Japan will grow from 73m (104m) in 2001 to
over 1.4 bn (2bn) by 2005 (Spectrum Strategy Consultants/DTI 2002). To-date, computer games have been commonly played on the following delivery platforms: portable, handheld games devices, such as the Game Boy personal computers (ordinary desktop computers) specialised games consoles (powerful computers with high-specification graphics capability for use in homes, eg the Sony PlayStation2 or Microsoft Xbox) games machines located in amusement arcades. Computer games are also played online via the internet, on an individual or networked basis, and on interactive TV (iTV) platforms. The m-learning project intends to develop some of its learning materials using a gaming philosophy to make their use attractive to young adults. It has been suggested in a study carried out by Becta (2001) that the benefits of using games software in education are: ICT skills development (but this is time-consuming) increased motivation (but games can be too complex for classroom context); encouragement of collaborative learning (but games can be so engaging that the educational focus is lost); and the development of thinking skills. Positive side effects have included increased library use among the learners (but learners may find that they suffer from inappropriate vocabulary or reading level to take full advantage of this), and increased self-esteem (although technical problems with using computer games can militate against this) and better engagement with the content. Thus, gaming can encourage thinking, reflecting and creativity. Becta suggests areas of further research into the use of computer games, including their use to motivate disengaged pupils. Prensky (2001) has identified a combination of 12 elements that make computer games engaging. These are summarised in Table 2.
Table 2 Engaging characteristics of computer games Characteristic of the computer game How they contribute to players engagement enjoyment and pleasure intense and passionate involvement structure motivation doing (ie the activity) learning
Fun Play
Rules Goals Interaction
Outcomes and feedback Adaptive flow Winning ego gratification adrenaline Conflict/competition/ challenge and opposition Problem-solving sparks creativity Interaction social groups Representation and a emotion story Adapted from Prensky 2001, pages 1067 In any game focused on learning, as can be seen above, Prensky considers that it is the feedback that encourages learning. Such feedback is a complex issue to game designers, as too little or too much can quickly lead to a players frustration. Examples of feedback in commercial games are where the player is constantly learning how the game works, what the designers underlying model is, how to succeed, and how to get to the next level and win (eg to get rewards for mastering something, a prompt to try again, seek help, etc, until it is mastered). In the m-learning projects early trials of learning materials, it was generally found that the young adults who took part had high expectations concerning the content and quality of the computer games. They were enthusiastic about the use of games but wanted tougher, more dramatic, storylines. This was characterised by one young person saying he wouldnt have died if he knew his sums. These findings appear to support Prenskys explanation above of why games engage players. They also highlight one of the dilemmas facing the designers of such computer games, ie the need to balance young peoples desire for excitement with producing effective and ethically sound learning materials. The other major problem faced by anyone
interested in developing computer games for learning is that users will always compare these with the entertainment games produced by commercial companies, with their advantage of multi-million Euro budgets. Two examples that give some idea of the scale of costs and manpower involved are: first Fox (2003) who states that the average development cost of a Nintendo GameCube game is 72m yen which equates to approximately 560 000; second a Spectrum Strategy Consultants suggestion that the average development for a premium console title is in the range of 12m (1.52.8m) and requires a team of at least 20 people working for 1824 months (Spectrum Strategy Consultants/DTI 2002).
How do young adults use their mobile phones?
Most young adults used their phone for telephone calls for between 5 and 60 minutes per day, they sent and received 2 to 10 text messages per day, and played games for 5 to 30 minutes per day. What might be the future take-up of new services and facilities on mobile phones and other technology devices?
5.
Our findings the survey
The most wished-for mobile phone functionality was music, followed by the radio. There was a general reluctance to consider a future purchase of a palmtop computer and the main explanations included a perception of these as tools for business people and concerns about cost. Are mobile phones likely to be used beyond a short-term fad?
The survey questionnaire was developed by LSDA to discover: how young adults use their mobile phones what the future take-up of new services and facilities on mobile phones and other technology devices might be whether mobile phones were likely to be used beyond a short-term fad whether young adults would be willing to use their phones for literacy and numeracy learning.
This survey was not targeted specifically at the m-learning projects target audience (ie disengaged young adults). Instead it tried to capture a broad range of responses from young adults in the target age range of 1624. The research took place in seven different UK locations with sizeable populations of young adults. It was operationalised in December 2001 and January 2002 at three different venues per location (shopping, leisure and job/employment centres) that young people were observed to frequent (the latter venue to capture the responses of those young adults who were not currently employed). The survey included both open and closed questions designed to capture participants quick responses and also their further views when appropriate. All questionnaires were completed by the interviewers to ensure that completing the questionnaire would not be an inhibiting factor to their participation. This resulted in 746 completed questionnaires for analysis. Some of the key findings relating to the research questions are summarised below:
More than three-quarters of the young adults were not worried about their health and safety when using a mobile phone, and more than half considered that having one had changed their life (with most of the reasons related to allowing them to stay in contact with others). An open response question allowed participants to offer other comments. Most of these related to respondents perceptions of mobile phones as handy or convenient. Other issues were also raised including the cost of ownership, the wider societal impact of use, users reliance or dependence on their mobiles, and security issues. Two examples which indicate the depth of feeling about using their mobile phones were: o I couldnt live without my phone, its a part of my life
and o Phones are bit like a soap opera you get addicted and you cant wait for the next soap you cant wait for the next call. Would young adults be willing to use their phones for literacy and numeracy learning?
Almost half of respondents expressed an interest in using phone-based games to improve their spelling and reading (49%) and maths (44%). The greatest interest was expressed by 1619 year olds and young adults educated to Levels 2 and 3 (GCSE and A-level, or the equivalent). Females were more interested in a game to learn reading and spelling, but no gender difference was noted for a game to learn maths. However, many young adults stressed that learning games must be appealing, relevant and fun (even addictive in the case of maths) if they were to sustain interest. This response suggests that getting the design and content of learning games right is crucial. A few also expressed an interest in using phone-based games for learning a foreign language or for English as a foreign language.
6.
Further exploration
In the current phase, phase 2 of the project, we are involving community, voluntary and education organisations, and the groups of young adults they support, in further research. This research will explore: how different groups of young adults interact with, and experience, the learning materials and systems designed by the project whether their enthusiasm for learning in general appears to be improved by their mobile learning experiences whether learning gains are made, including young adults perceptions of their own progress different models of learning and support, eg collaborative learning, individual learning (with or without peer support), online tutoring, blended learning and stand-alone units of learning, etc how mobile learning might contribute to government targets for improving basic skills and engagement in education and training.
Figure 1 is a diagrammatic representation of the phase 2 learner research.
Figure 1 Inter-related aspects to the research activities of the m-learning project in phase 2 It will be seen that in Figure 1 we have placed the learner at the centre of the diagram, as their learning experiences form the core of this research (together with the experiences and perceptions of their tutors, mentors or others who support them). The three concentric circles represent the three different stages of the research. The inner circle relates to trialling the mobile learning materials and researching responses to these. The middle circle describes research into the use of the learning materials together with a special user interface layer we call the microportal, or m-portal. The outer circle includes research involving the integrated mlearning environment including an intelligent tutor and a learning management system. Depending on our development experience, and the decisions taken on the specific equipment to be used, the second and third stages in this process may be merged. The diagram also indicates that learning materials are being developed that are designed to assist with improving literacy and numeracy. This research activity is being led by LSDA and will involve 200 learners based in the UK, Italy and Sweden. In the UK we expect to involve about 150 learners. In carrying out the research we will be collaborating with organisations who have established supporting relationships with groups of young adults in our target audience of 1624 year olds who are disengaged from education and training. The duration of the involvement of each organisation and each group of learners in the research will vary within an anticipated range of 112 weeks between December 2003 and June 2004. It is hoped that by taking part in the mlearning project young adults will engage in interesting and stimulating mobile learning activities that will start to change their attitudes to learning. Our learning experiences are also designed to contribute towards improving their literacy, numeracy and life skills and thereby their life chances.
7.
References
Bauman Z (2003). Liquid love: on the frailty of human bonds. Cambridge, UK: Polity Press. Becta (2001). Computer games in education project. At www.becta.org.uk/research/research.cf m?section=1&id=2826, accessed December 2003.
10
Learning with mobile devices: book of papers
Brown G and Dhaliwal J (2002). Mobile youth: 2002 report. London: w2f Limited. Danesh A, Inkpen K, Lau F, Shu K and Booth K (2001). Geney: designing a collaborative activity for the Palm handheld computer. Paper presented to the Human Factors in Computing Systems Conference, Seattle, USA. DfEE (2001). Skills for life: the national strategy for improving adult literacy and numeracy skills. Nottingham: Department for Education and Employment. At www.dfes.gov.uk/readwriteplus/bank/AB S_Strategy_Doc_Final.pdf, accessed December 2003. Edwards C (2000). Physics learning through a telecommunications context. Physics Education 35 (4), 240244. Eldridge M and Grinter R (2001). Studying text messaging in teenagers. Paper presented to the Conference on Human Factors in Computing Systems (CHI), Seattle, USA. At www.cs.colorado.edu/~palen/chi_works hop/papers/EldridgeGrinter.pdf, accessed December 2003. Fox F (2003). CESA 2003 White Paper. At www.videofenky.com/archives/000050.html, accessed December 2003. Graham B (1997).The world in your pocket: using pocket book computers for IT. School Science Review 79 (287), 4548. Ichinohe, A and Suzuki, K (2002). Expansion of the i-mode drill The world of Kanji with review functions for m-learning. Paper presented to the International Conference on Computers in Education (ICCE 2002), Auckland, New Zealand. At www.computer.org/proceedings/icce/15 09/15091472.pdf, accessed December 2003. Ling R and Yttri B (1999). Nobody sits at home and waits for the telephone to ring: Micro and hyper-coordination through the use of the mobile telephone. Oslo: Telnor research and development.
Mandryk RL, Inkpen KM, Bilezikjian M, Klemmer, SR and Landay JA (2001). Supporting children's collaboration across handheld computers. Conference on Human Factors in Computing Systems (CHI), Seattle, USA. At http://geney.juxta.com/chi2001_handhe ld.pdf, accessed December 2003. McTaggart M (1997). Palms take root in East London. The Times Educational Supplement. 20 June 1997, page 23. Mogapi S (2000). Cell phones hit beer industry hard. Botswana Gazette. 29 November 2000, page B7. Moser C (1999). Improving literacy and numeracy: a fresh start. Sudbury: Department for Education and Employment. Mutula SM (2002). The cellular phone economy in the SADC region: implications for libraries. Online Information Review 26 (2), 7991. OECD (1997). Literacy skills for the knowledge society: further results from the International Adult Literacy Survey. Canada: Organisation for Economic Co-operation and Development. Plant, S (2001). On the mobile: the effects of mobile telephones on social and individual life. Motorola. Prensky, M (2001). Digital game-based learning. New York: McGraw-Hill. Regan, M (2000). Mobile learning. At http://sll.stanford.edu/projects/mobile learning/, accessed on 8 October 2001. Savill-Smith CA and Kent P (2003). The use of palmtop computers for learning: a review of the literature. Learning and Skills Development Agency. At www.mlearning.org/docs/the_use_of_palmtop_ computers_for_learning_sept03.pdf, accessed December 2003. Shields J and Poftak A (2002). A report card on handheld computing. Technology and Learning, 22 (7), 2536.
11
Spectrum Strategy Consultants/DTI (2002). From exuberant youth to sustainable maturity: competitiveness analysis of the UK games software sector. Department of Trade and Industry. At www.dti.gov.uk/industry_files/pdf/compu tergames_main.pdf, accessed December 2003. Szkely G (2001). Phone artists. Arts & Activities, 129 (4), 3441. Taylor AS and Harper R (2002). Age-old practices in the 'New World': a study of gift-giving between teenage mobile phone users. Paper presented to the Conference on Human Factors in Computing Systems (CHI), Minneapolis, USA. Townsend AM (2000). Life in the real-time city: mobile telephones and urban metabolism. Journal of Urban Technology, 7(2), 85104.
The m-learning project is supported by the EC Information Society Technologies Programme. See website at www.m-learning.org Copies of the three literature reviews noted above, ie the use of mobile phones, palmtop computers and computer games for learning, will be published by LSDA in 2003 and 2004, and available from this website.
Avellis et al.
13
Evaluating non-functional requirements in mobile learning contents and multimedia educational software
Gianna Avellis, Antonio Scaramuzzi TECNOPOLIS CSATA SP.per Casamassima Km.3 70100 Valenzano/Bari/Italy E-mail:g.avellis@tno.it, a.scaramuzzi@tno.it Abstract
Constructing, evaluating and evolving wireless and e-learning content to meet users requirements are some of the challenges faced by developers of current e-learning and mobile learning systems. Key users requirements are non-functional requirements (NFRs). This is because functional requirements set out services expected by the system user, whereas NFRs set out the constraints of the system and the product and process standards to be followed. As such, they play a central role in evaluating the quality of wireless and e-learning modules. We have developed a scheme for representing critical NFRs, and applied it to the domains of mobile e-learning contents (MLC) and multimedia educational software (MES) for validation. Our approach extends the model for the representation of design rationale by making the evaluation goals explicit and providing the means to improve the quality of e-learning content (especially mobile learning content). Finally, further issues for research are highlighted, including the need to relate NFRs to system architectures.
Anthony Finkelstein University College London, Department of Computer Science Gower Street, London WC1E 6BT, UK E-mail: a.finkelstein@cs.ucl.ac.uk
Keywords: non-functional requirements (NFRs), mobile learning contents (MLC), multimedia educational software (MES), system architectures
1. Introduction
It is widely recognised that non-functional requirements (NFRs) are crucial in the development of software and that different architectural choices can have a different impact on the quality of the final system (Arango and Prieto-Diaz 1991; Devanbu et al., 1991; Avellis 2000). However, there is a perceived gap in the way that current software
development methods build on, and keep track of, the links between such requirements, especially NFRs, and system architectures in constructing and evolving complex systems. In this paper, we will provide a map to help identify the explicit links between the NFRs and mobile learning systems, and use this map to consider the value of the system and incrementally evaluate the NFRs during software development. We focus on the analysis of, and reasoning about, the process of building a value model of a software system by explicitly representing NFRs. The techniques and representations in the paper are then demonstrated on two application domains. Mobile learning systems represent a broad class of software systems with complex characteristics that tend to make evaluation difficult, because there are no existing comprehensive frameworks for formative evaluation in the mobile environment. The effectiveness and pedagogical soundness are, for example, very important to evaluate in mobile contents. Quite apart from the intrinsic difficulty in assessing these characteristics, the novelty of mobile application makes this a very hard task. The educational potential of mobile learning contents, both as a learning and teaching tool, is widely acknowledged, and various initiatives have been undertaken to encourage the integration of educational multimedia resources in school practice (Avellis and Capurso 1999a). The aim of this paper is to address the main issues of evaluation of mobile learning content and multimedia educational software and to tackle the problem of evaluating NFRs by developing a scheme for annotating NFRs to the architectures. Section 2 describes the context of the problem. Section 3 identifies the features of the software domain and points out the needs in evaluating mobile learning contents. The evaluation criteria are developed in the framework of ERMES (EuRopean Multimedia
14
Educational Software network). This is an ESPRIT project (Avellis and Ulloa 1997), which is the integrated programme of information technologies managed by the Directorate General for Industry of the European Commission. Section 4 introduces the annotation scheme to represent NFRs. This is the selected scheme of NFR representation, which is a process-oriented, rather than product-oriented, representation. Finally, the conclusions identify further research issues in building links between NFRs and system architectures.
2. Background
The functional viewpoint is not the only design dictum in engineering. Petroskis 1994 refutation of the design dictum form follows function applies to software systems as well as any other complex systems. The main developments in software engineering have centred on the functional and object-oriented perspective. This is mainly because the functionality of the system offers an explicit level of representation of system capabilities, and the object-oriented representations provide a suitable basis for understanding the application concepts as the represented objects, which can be easily mapped with the real world objects. This perspective has been pursued for many years. One of its main advantages is that it provides the means to localise the effects of functional changes in system architecture. It also restricts the impact and propagation of changes, so that the changes which take place in an aspect of the system are mapped to the changes to other aspects of the system (Avellis 1992; Avellis et al. 1991). We use the terms aspect or view of a system to mean a set of abstractions that provides us with one of many possible characterisations of a software system. A model of view captures the semantics used by that view (Avellis 1990; Avellis and Borzacchini 1994). In the literature on reverse engineering, a view is often a structural view that contains information about the structure of the product. For instance, the Software Reengineering Environment (SRE) of CSTaRArthur Andersen (Kozaczynsky and Ning 1989) stops at the level of identifying generic programming plans well before identifying application-specific knowledge. One of the consequences of not having applicationspecific views of the system is that the
maintainer has to compute their own complex mapping between the description of a change and the part of the system to be changed, where most changes are expressed in terms of the vocabulary of application domains (Arango and Prieto-Diaz 1991). The understandability issue (Corbi 1989) that is grasping the relationships between the different views of a software system and their interconnections relates to the built-in limitation of human beings in dealing with large-scale complex objects. The structure of software, unlike that of buildings or automobiles, is hidden and the only external evidence we have of the software is related to its behaviour. This phenomenon of invisibility has been highly emphasised for many systems in the research literature (Devanbu et al. 1991). There is, thus, a need to develop richer models for capturing and analysing NFRs in software engineering. However, this is not a simple enterprise, as examples of difficult tasks include: choosing an architecture to satisfy some NFRs evaluating the impact of a change of NFRs on the system structure modifying the architecture evaluating NFRs during system development.
One open problem in our research is to map the NFRs to architectures to analyse the impact of changing the NFRs on the architecture. Another open issue concerns understanding how the prioritisation and evolution of NFRs affect the requirements traceability problem and choices of software architecture. Requirements traceability (Finkelstein 1991) refers to the ability to describe and follow the life of a requirement, both forwards and backwards through the design process. A lack of a common definition of requirements traceability (purpose-driven versus solution-driven versus informationdriven versus direction-driven) has been detected by Gotel and Finkelstein (1996); the requirements traceability problem was perceived not to be uniform due to the diverse definitions and several fundamental conflicts. The need for improved requirements specification traceability is evident from the literature (Harandi and Ning 1988). NFRs have yet to be incorporated at the core of product and process specification, design, implementation techniques and tools. So progress in this area has been limited.
Avellis et al.
15
Software quality is attracting more and more attention in software engineering for two reasons: on the technical side, it is usually not clear to those involved in the development how to measure the various quality criteria on a day-to-day basis (ie formative analysis), nor how to achieve them and measure them on completion (summative analysis). On the customers side, the issue is simply not knowing what to ask for. To this end, a distinction has been made between basic quality factors, such as functionality, reliability, ease of use, economy, safety, and extra quality factors, such as flexibility, repairability, adaptability, understandability, documentation and enhanceability. The latter are quality factors related to the external, or observable, quality of a piece of software and are particularly important in the world of e-learning where technical strategies are emerging in parallel with educational and pedagogical strategies. They are also important in the framework of mobile learning, where the constraints of mobile devices and the supported software are very important for delivering effective contents, in addition to mobile quality factors identified so far such as accessibility, navigation, presentation and system user operation. However, it is important to grasp the internal quality of a system. Ultimately, the external quality of a system depends on its internal quality. For example, the enhanceability of a system is directly related to how well structured the internal design is, ie the size, definition and relationships between modules and subsystems. Internal quality factors include completeness, consistency, parsimony, traceability, rationality, structure, paradigm, and quality of algorithms and representations, as well as understandability and documentation. The nature of these factors is not well understood, which is why we propose to research how to evaluate quality factors in wireless and e-learning modules, and apply the research results to several domains and scenarios to validate the scheme. This will produce an integrated set of mobile learning training modules, and an analysis and assessment of evaluation criteria to understand their requirements for advanced mobile and wireless technologies. To this end, we will collaborate with current standardisation working groups, especially the evaluation and assessment of NFRs of mobile learning and elearning modules. The current industry standards such as Aviation Industry CBT Committee (AICC), Instructional Management System (IMS),
Dublin Core Metadata Initiative (DCMI), Institute for Electrical and Electronic Engineering/Learning Technology Standardization Committee (IEEE/LTSC), Information Society Standardization System/Learning Technology Workshop (CEN/ISSS LTWS), Alliance of Remote Instructional Authoring and Distribution Network for Europe (ARIADNE), PROmoting Multimedia access to Education and Training in EUropean Society (PROMETEUS) have already addressed the problem of metadata tagging of educational resources to allow easier access and retrieval through e-learning systems. Further improvements in standardisation could be achieved by extending the NFRs (eg target delivery device) to include the set of characters currently adopted to describe and classify learning modules. This will result in an increased capability of the user to assess the suitability of selected educational material for a specific application environment (eg mobile learning).
3. Evaluation of mobile learning contents and multimedia educational software

Mobile e-learning is relatively new, so we are only beginning to see the potential of mobile devices in training and performance support. Mobile devices are small, portable and compact. They can often fit in a pocket or purse. Unlike laptop computers, which are expensive, heavy and power-hungry, mobile devices are relatively low-cost, lightweight, and some work for a long time on an electrical charge or using a couple of standard disposable or rechargeable batteries. The small screen size of mobile devices (an NFR) makes some people question their worth as e-learning delivery tools. Some of these devices have good audio capability, allowing students to listen to a narrated lecture, rather than read material on a small screen. However, some critics do point to the restricted input capabilities (another NFR) of some of these devices, questioning students ability to enter large amounts of text into a device to take notes or answer an essay-type question. Many of these devices are, however, extremely adaptable (again, an NFR) and can be attached to a full-size folding keyboard that makes entering large amounts of information every bit as fast (another NFR) as it is with a conventional computer.
16
3.1 Mobile e-learning in practice

Mobile e-learning is currently in its infancy. Although many experts in the field see great potential for the use of mobile devices in e-learning, there are presently very few successful implementations on which to base a study of best practice. Because of this, and the fact that some mobile devices are similar in functionality to conventional computers, it is only natural that the first generation of mobile e-learning content will closely resemble conventional e-learning, presented on a smaller screen. As mobile devices evolve and people discover new ways in which the functionality of mobile devices can be applied to training, mobile e-learning will probably become increasingly different from conventional elearning; no longer a miniaturised version of it. Internet-connected phones may be applied to mentoring and used to register students on courses and pay their fees, as well as present training content through the use of audio. Another development may be content development tools that will provide the ability to publish learning content adaptively to a wide range of mobile devices. In addition, the student may well have control over reading or listening to the content using voicesynthesised XML technologies. Since mobile e-learning technology is so immature, there are presently more possibilities relating to what could be done with this technology than concrete examples. But with the number of mobile devices predicted to surpass the number of conventional computers for web access in the near future and with bandwidth for mobile devices predicted to increase dramatically in the short term, mobile e-learning appears certain to become an important part of training in the future. Many national and international activities in mobile learning contents (MLC) and in multimedia educational software (MES) in general are currently partially funded by the European Commission, involving private and public sector organisations (Avellis and Fresa 1999). In this context, the need for educational multimedia for vocational training purposes is widely recognised. However, users of educational multimedia cannot appraise educational resources because they are not able to evaluate their characteristics, potentialities and limits (Avellis and Capurso 1999a).
3.2 Evaluation issues

The reason it is not easy to carry out a critical evaluation of mobile educational multimedia is that these resources are relatively new compared to traditional printbased learning materials. Most people are still not used to handling them nor aware of their educational potential. Educational multimedia software has an additional intrinsic complexity because it is a type of software that runs on a computer and also an educational resource. Evaluating both these aspects is very different from evaluating a book or any traditional educational resource because of the interleaving of the two aspects: software and learning resource. The distinction between software and supporting learning is blurred because of the way the application runs, which affects its educational effectiveness, and the educational purpose, which underlies the design of the software. Therefore, both aspects must be carefully considered during the evaluation. However, it is difficult to develop a pre-defined set of standards against which the educational value of the software can be defined, because it is not possible to define a unique and general instructional approach. Thus the mobile educational value of a piece of software is very difficult to define in practice (Avellis and Capurso 1999b). The evaluation methodology adopted in the ESPRIT project ERMES (Avellis and Ulloa 1997) consists of identifying aspects of the object under evaluation, and then defining quality indicators in relation to these aspects. Defining the object of evaluation is a key step, because it suggests the evaluation criteria to be used (Ulloa 1998). We group the characteristics of multimedia educational software under the following four evaluation categories: educational features technical features aspects relating to the ease of use (usability) aspects relating to the content.
Each one of these categories has been further divided into sub-categories. For example, educational features can be divided into target users, pedagogical characteristics, instructional support materials, and so on. That means that when evaluating the educational features of an MLC or MES, the aspects relating to the target users, the pedagogical characteristics, the instructional support materials, and so on, all have to be taken into account.
Avellis et al.
17
MES is a computer program, which performs a specific educational task. The multimedia component can be identified in the use of a variety of media to deliver instruction or support for the learning activities. MES is also characterised by the presence of interactive components, which should enable the user to control the learning environment. Features of mobile learning contents (MLC) include the: content to be taught delivery media used to provide information user interface the way the educational software presents itself to the user; interaction devices by which the user interacts with the computer, making choices, answering questions or performing activities, and is provided with feedback to each response instructional strategy adopted access which refers to the navigational paths available to the user to reach the needed content navigation allowing the user to go from one piece of content to another presentation which can provide guidelines for defining the visual communication strategies or presenting the content, navigation strategies and operation to the user user operation those operations that are visible to the users and the only ones the user must be aware of system operation that are not visible to the users, but are essential in building user operation (Avellis and Capurso 1999a; UWA Consortium 2002).
4. A scheme for critical NFR representation

Techniques are needed to express NFRs, which include quality requirements (Finkelstein 1994). This underlines the centre of the development process, the generation of a value model such as is used in classical engineering disciplines (Finkelstein and Finkelstein 1983). A key component of the system development process is achieving a model of what is valued in the resulting system. Using this view, quality characteristics are not externally imposed on a development process but constructed within it. The scheme developed to express NFRs is based on the work done by Kunz and Rittel (1970), particularly in the area of design rationale (Potts and Bruns 1988). We also take into account the issue-position-arguments model (Conklin and Begeman 1988). In our scheme, an issue, that is a problem to solve, is an NFR, or quality characteristics/subcharacteristics to evaluate. An argument, that is, a supporting justification of the issue, is a procedure that helps to determine which design alternative to choose to implement in the related NFR. Finally, a position that is a solution to the problem, is either a statement of the NFR, which gives a quality goal to be supported by the final design, or design alternatives. A statement is an ascertainable property (possibly measurable) characterising NFRs. The set of links is given in Figure 1.
18
Figure 1 Non-functional requirements representation scheme
It is important to underline that the statement contains measurable elements by which the NFR can be constructed in software systems. It is a procedure that applies to different architectural choices. In this way we relate NFRs to architectures, by linking statements and different system architectural choices. We have enhanced the representation of NFR with quality function deployment (QFD) features. Since the late 1960s Mizuno and Akao (1978) have established a new systematic method of design-oriented approaches to ensure that customer needs drive the product design and production process. They developed a method called quality deployment and/or quality function deployment (QD/QFD). We have enhanced the scheme of NFR representation by introducing the context of evaluation and weights to the links as follows. To be assured that we will achieve a particular software quality characteristic it is helpful to associate it with some activities within the software evaluation and development process. Activity is the evaluation and/or implementation activity of the quality characteristic that provides the context of evaluation. A quality characteristic is obtained in a strong/medium/ weak/negative way as a result of performing an activity. In a quality-function-deployment (QFD) style we attach some weights strong/ medium/weak/negative to this link, to let the end users (teacher, trainers, students, administrators) assign a weighted value to the characteristic of the system under evaluation. Although a quality characteristic can be constructed independently of the description of the development process of a product, it is useful to link the product and process
descriptions to the quality characteristics. Avellis (2000) provides insights into how to relate this process view to a product view, by introducing the role played by the architecture of a software system and relating it to the NFRs. Here are two examples of the application of the scheme above to MLC and MES. An NFR related to a MLC could be: the MLC should fit the subject/topics and learning objectives of my course. The activity related to this example is to: evaluate the educational aim of the MLC package, which strongly achieves the quality characteristics educational features. Educational features quality characteristics have several sub-characteristics to be taken into account, such as instructional characteristics, which suggest by their requirement statement that appropriateness of learning objectives are suitable for the age and competence of target users and this is measured by a procedure to verify that the content and learning objectives are consistent with the national curricula requirements. The second example is the NFR the MES package should be easy to operate. The activity related to this is understanding the usage of a MES package, which achieves in medium form the quality characteristics of usability. This in turn can be further specialised into the sub-characteristics ease of use, which is suggested by the requirements statement the way software operates and several procedures are used to measure usability: What are the IT skills required to operate the software? Is onscreen help available? Are directions clear and accurate? Are directions available at all times? Is the management of assessment instruments easy?
Avellis et al.
19
5. Conclusions and further research

This paper presents work in progress to improve the current Evaluation Tool based on the framework of the ESPRIT project ERMES. The key issue is how to incorporate in the tool the scheme to annotate NFRs to MLC and MES. Further research is needed in this context on, for example, how to annotate NFRs to architectures. A system quality attribute (ie the NFR) is largely permitted or precluded by its architecture. The motivation for software architecture is to have a basis for understanding and standardising systems and their components. Software has yet to achieve the level of reuse realised by hardware disciplines. Although software is easy to reproduce, its variations are much more difficult to standardise, identify and control. Although a universal reuse solution remains elusive, great improvements have been made by focusing on well-defined areas of knowledge or activity domains (Arango and Prieto-Diaz 1991). Architectures provide a means for structuring knowledge of the system within a domain, including their requirements. The possibilities for reuse are greatest when the specifications are the least constrained at the architectural level. Reuse is normally considered only at the implementation phase. This practice limits reuse to fine-grained modules at best, and fails to allow for broader use of assets at a subsystem or higher level, by neglecting to plan at the early stages of development. In this paper, we have focused on setting down a process where argument on the quality of an MLC and MES is considered on the basis of identified NFRs, and have developed some case studies to evaluate the process critically. A follow-up research result project will develop an evaluation tool to help not only MES users but also MLC users to choose educational software of high quality, suitable for their needs and valuable as an educational resource to integrate into their own courses or current curriculum based on the selection of NFRs. A further aim is to research and demonstrate innovative mobile contents for training in the IT and education sectors, and to evaluate the requirements, especially NFR, of e-learning modules for mobile applications and services. The wireless e-learning solution will focus on the representation of mobile learning
objects that suit the mobile delivery media, and on methodologies for adapting MES to mobile learning environments.
Acknowledgements
This work has been partly financially supported by the European Commission under the Human Capital and Mobility scheme and the ESPRIT projects ERMES (EuRopean Multimedia Educational Software) and MACS (Maintenance Assistance Capabilities for Software) (Avellis 1990), and finally the ETI action MOBITECH (European SMEs Challenge in Mobile Communication Technologies).
References
Arango G and Prieto-Diaz R (1991). Domain analysis and software system modelling. California: IEEE CS Press. Avellis G (1990). MACS ESPRIT project and the software evolution expert system. In Annual Conference of Associazione Italiana per lInformatica e il Calcolo Automatico. Bari: Laterza Publishing Company, (1), 37. Avellis G (1992). CASE support for software evolution: a dependency approach to control the change process, In G Forte, N Madhavji, HA Muller (eds), Proceedings of the Fifth International Workshop on Computer-Aided Software Engineering. Montreal: IEEE CS Press, 2673. Avellis G (2000). Annotating multimedia educational software with non functional requirements: an approach to evaluate educational multimedia. UCL internal report. Avellis G and Borzacchini L (1994). A blackboard model to design integrated intelligent software maintenance environment, In Proceedings of the Fourth International Conference on Software Engineering and Knowledge Engineering. Capri: IEEE CS Press, 325332. Avellis G and Capurso M (1999a). ERMES evaluation methodology to support teachers in skills development. In E Tuomi, M Salonen, P Saarinen, M Sinko (eds), Proceedings of IFIP WG3.1 and 3.5 Open Conference on Communications and Networking in Education: Learning in a Networking society. Hameelinna: Painopaikka Yliopistopaino, 1219. Avellis G and Capurso M (1999b). ERMES Services for Education, In J Roger, B
20
Stanford-Smith, PT Kidd (eds), Business and work in the information society: new technologies and applications. Stockholm: IOS Press, 520527. Avellis G and Fresa A (1999). Education and training: a challenge between industries and cultures, In J Roger, B Stanford-Smith, PT Kidd (eds), Business and work in the information society: new technologies and a p p l i c a t i o n s . Stockholm: IOS Press, 401407. Avellis G, Iacobbe A, Palmisano D, Semeraro G, Tinelli C (1991). An analysis of incremental capabilities of a software evolution expert system. In Proceedings of Conference on Software Maintenance, 1991. Sorrento: IEEE CS Press, 220227. Avellis G and Ulloa AS (1997). ERMES technical annex. ESPRIT 24111 Project, ERMES Consortium (eds). Conklin J, Begeman ML (1988). gIBIS: a hypertext tool for exploratory policy discussion. ACM Transactions on Office Information Systems, 6(4), 303331. Corbi T, (1989). Program understanding: challenge for the 1990s. IBM System Journal, 28 (2). Devanbu P, Brachman R, Selfridge P, Ballard B (1991). LaSSIE: a knowledge-based software information system, In Communications of the ACM, 34(5), 3449. Finkelstein A (1991). Tracing back from requirements, In IEE Colloquium on Tools and Techniques for Maintaining Traceability During Design, Computing and Control Division, Professional Group C1, Digest No 1 9 9 1 / 1 8 0 , London: The Institution of Electrical Engineers. Finkelstein A (1994). Quality arguments: product and process interaction. In Proceedings of Software Quality Management 94. Finkelstein A, Finkelstein L (1983). Review of design methodology. The Institution of Electrical Engineers, 212222. Gotel O, Finkelstein A (1996). An analysis of the requirements traceability problem. In R Arnold R and S Bohner (eds), Software change impact analysis. IEEE CS Press. Harandi M, Ning J (1988). PAT: a knowledgebased program analysis tool. In Proceedings of Conference on Software Maintenance 88. Phoenix: IEEE CS Press, 312319. Kozaczynsky W, Ning J (1989). SRE: a knowledge based environment for large scale software re-engineering activities. Communications of the ACM. Kunz W, Rittel H (1970). Issues as elements of information systems. Technical Report S-
78-2, Institut fur Grundlagen der Plammung. Stuttgart: Universitat Stuttgart. Mizuno S and Akao Y (1978). Quality function deployment. Nikkagiren-Syuppan (ed) Tokyo, Japan. Petroski E (1994). Design paradigms: case histories of error and judgement in e n g i n e e r i n g . Press Syndicate of the University of Cambridge. Potts C, Bruns G (1988). Recording the reasons for design decisions. Proceedings of the Tenth International Conference on Software Engineering. Singapore, IEEE CS Press. Ulloa A (1998). ERMES evaluation guidelines for MES, ERMES ESPRIT Project Del.5.1. Valenzano, Bari, Italy. UWA Consortium (2002). The UWA approach to modeling ubiquitous web applications. Proceedings of IST Mobile and Wireless Telecommunications Summit 2002: Towards Ubiquitous Communications, Thessaloniki, Greece.
Baber et al.
21
A learning space model to examine the suitability of different technologies for mobile learning
Chris Baber, Mike Sharples, Giasemi Vavoula and Phillip Glew University of Birmingham Electronic, Electrical, and Computer Engineering, Birmingham, B15 2TT E-mail: {c.baber, m.sharples, g.vavoula, p.glew}@bham.ac.uk
Abstract
Mobile learning is an emerging paradigm that has yet to be clearly defined. To better understand the nature of mobile learning we propose a multi-dimensional learning space model that can be extended as new technologies and new ways of using existing technologies are developed. The potential use of a specific technology for a particular type of learning within the space is evaluated by using the quality function deployment (QFD) set of tools. It is hoped that the model will help to identify and classify types of learning facilitated by mobile environments and to explore particular combinations of learning and technology. Keywords: learning space, mobile technology, quality function deployment, m-learning, e-learning
Traditionally this question has been answered in one of two ways: either by asking potential users to indicate their requirements and seeking to select technology that can match these requirements or by developing or modifying technology and then using this to run user trials. Both approaches have proved successful, but both suffer from problems. The latter requires the development of kit, which means that certain design decisions need to have been made to support the development process; which could mean that design decisions have been motivated, or at least influenced by, the capability of the available technology rather than the requirements of potential users. The former raises the problem of relating user requirement to system specification. For example, how do we know that a given set of user requirements will be met by the functions of particular products? Of course the answer to this question is simply that we know whether requirements and functions match and that, consequently, we can perform a simple match between these concepts. However, simply matching a set of user requirements with a set of functions does not lead to a rigorous and auditable process. Furthermore, it is likely that the relationship between products, requirements and functions will be complex and that this complexity might be distorted by using so straightforward a subjective matching.
1. Defining the problem

One of the motivating factors for this paper is the observation that there are many possible technologies that can be readily adapted to support mobile learning. This raises the question of how we can select the technology and whether the selected technology is sufficient for the purpose to which it will be put. If the technology is not sufficient, do we need to design new technologies?
22
Type of learning X Project Different technologies Z
Episode
Activity Context Presentation Management Communications Y
Supporting functionality
Figure 1: Learning space model showing relationship between type of learning, possible technologies and supporting functionality The z axis illustrates specific instances of technology that could be employed in mobile learning, ie, PDA = personal digital assistant; wearables = wearable computers; MMS = multimedia messaging service; SMS = short message service; laptop = laptop computer.
Figure 1 presents a schematic of the possible relationship between a particular set of requirements, ie type of learning, a set of functions that could be supported by technology, and a set of technologies. This figure is not a graph showing real values; rather it is intended to indicate that the relationship between requirement and function is probably mediated by the technological platform used. The idea is that a change in one aspect of Figure 1 will deform the shape not only locally but also globally. Changing the applicability of context for the Personal Digital Assistant (PDA) to type of learning, for example from curriculum-supported to serendipitous, will not simply result in a lowering of all scores for the PDA; rather other components might increase or simply remain at the same level. The point is that we can assume non-linear relationships between functions and devices. The x and y axes of Figure 1 represent types of learning and supporting functionality. Previous work into everyday learning proposed that learning could be divided into projects (long-term learning
behaviour, eg completing a module or course in a particular topic); episodes (medium-term learning behaviour that may be a subset of a project, eg attending a lesson on a particular topic); activities (short-term learning behaviour, eg acquire information to find ones way from hotel to conference hall). It should be apparent that the requirements to support each type of learning behaviour would differ. The functionality in Figure 1 is based on some of the key issues that are important for mobile learning: the device must be able to: respond appropriately to changes in the learners context manage presentation of information effectively assist the learning in the management of learning support communications while the learner is mobile and when the learner is in different locations. The z axis of Figure 1 represents a set of possible technologies that might be employed for mobile learning.
Baber et al.
23
2. Quality function deployment

A challenge, therefore, is how to capture the essentially non-linear relationship between functions offered by devices and the applicability of those devices to specific types of learning. In this paper we explore the potential of quality function deployment as a mechanism by which the relationships can be captured and examined. According to Zultner (1993), the main purpose of QFD is as an aid to preventing dissatisfaction by having a deeper understanding of stated requirements and implied customer needs, and then deploying these expectations downstream in order to design value into the system. In broad terms, QFD can be thought of as a collection of methods that allows analysts to link the functionality of a particular product (or class of products) to customer requirements. The approach is typically used in industries that are seeking to reduce leadtime and create rapid response to changing market demands, such as the automotive industry. However, there is a growing interest in the use of QFD as a method for requirements engineering (MacCaulay 1996). There are several approaches to QFD, but it is generally applied in one of four phases: product planning, parts deployment, process and control planning, production planning. Of particular relevance to this paper is the set of techniques that are applicable to product planning; in particular, the use of the House of Quality (see Figure 2) as a means of representing the relationship between product functionality and user requirements.
Exciting requirements: There are some requirements that may neither be stated by customers nor defined as expected requirements, but which represent that wow factor in product design. Often these requirements are created by the design team and then assessed through presenting the design back to potential customers. From initial work into mobile learning, we propose the following shortlist of normal and expected requirements. This set of requirements has been elicited from informal discussions with people involved in education and technology development. It is not intended to be exhaustive but provides a useful starting point for discussion and demonstration of the QFD method: adapt functionality for learner characteristics and learning context discover, access, evaluate, store, retrieve learning objects monitor, utilise, evaluate learning outcomes assist in the recovery of breakdowns and errors during and due to learning support the learners mobility.
4. Defining functionality
In Figure 1 four aspects of supporting functionality were presented: context; presentation; management; communications. These aspects reflect groups of functions that are important for mobile learning. Each aspect can be broken down further into specific components, as described below. Context: It is anticipated that an effective mobile learning application will be able to model and respond to changes in the learners context (both in terms of changes in situation, location, time, etc and in terms of changes in the learner, ie developments of the user model). In addition to allowing changes in response, context can also be used to manage the presentation of information and can support memory, through the recall of context. Finally, context can be used to provide suggestions for activity to the learner. Presentation: As mobile learning might involve a combination of devices, for example a mobile telephone and a PDA, so the manner in which information is prepared for presentation to the user will need to adapt. The notion of an adaptive interface also calls to mind the question of modifying information presentation on the basis of context. Finally, we have been exploring the use of overlaying
3. Defining user requirements

For QFD, one can consider user requirements to take three forms, as described below. Normal requirements: These are elicited from the customer, perhaps through market survey techniques such as interviews or focus groups. They are normal in the sense that they represent requirements about which the majority of customers express an opinion. Expected requirements: There are some requirements that might go without saying; they seem obvious or can be carried forward from previous studies. However, it is important to ensure that these are represented in the analysis.
24
computer-generated images, ie augmented reality, as an aid to exploring and receiving information about exhibits in galleries. Management: The manner in which a person is expected to learn can be influenced by the type of learning they are following, ie project, episode or activity, as well as by the curriculum that is relevant to learning about specific subjects. Furthermore, it is necessary to check that learning is being effective, for example through testing learned information. Communications: With the rise of 3G and related mobile multimedia communications technology, it is possible to send all manner of data to handheld devices. Thus, one aspect of communications relates to whether the content will need to be sent as text, picture or video. Another aspect is whether the learner requires two-way communication, for example in the form of a voice or video link with a teacher.
Stage 3: The weighting for each function is calculated. Again, calculation is typically performed statistically. However, we use a simple manual technique (reported by Bergquist and Abeysekara 1996) which allows a reasonable calculation of weighting to be performed quickly. For each cell in the table ie each value from the mapping, multiply the rank of the user requirement for that row with the mapping score. Repeat this calculation for each cell in a column until you have produced a sum for the column. Thus for column one, the weighting is: (8x3)+(4x1)+(3x1)+(7x0)+(6x8)+(1x1)+(2x1)+( 3x5) = 97. Stage 4: Creating the roof of the House of Quality, ie the triangular apex on Figure 2. The roof represents the associations between each function, for example perhaps all functions associated with communication can be linked. The idea is that if one makes a change to one function, then this change will also affect associated functions. Stage 5: The final stage is to define evaluation criteria. In this stage, one can define criteria for evaluating the functions perhaps by defining appropriate technical tests, and for evaluating user requirements perhaps by defining appropriate user trials. This is useful in that it allows the design team to focus on evaluation at an early stage in the design process and provides an opportunity to determine acceptable targets for performance. Table 1: Key to function numbers Context aware 1 User model 2 Recall context 3 Augmented reality 4 Adaptive interface 5 Rendering x device 6 Content management 7 Annotation 8 Creation 9 Curriculum management 10 Text 11 Audio 12 Picture 13 Moderation 14 Peer-to-peer 15 Broadcast 16
5. Building a House of Quality

Figure 2 illustrates the House of Quality that can be developed using the requirements and functionality reported in Sections 3 and 4. There are five stages to producing the House of Quality. Stage 1: The user requirements are ranked (with the highest number reflecting the highest rank). The ranking involved six people, working in the Educational Technology Research Group, ranking the requirements. The median ranks were then taken (in the case of a tie, we placed the requirement with the smallest spread on a higher ranking). Stage 2: The requirements are mapped onto each function. The scoring system uses a 9point scale. Thus, 0 means that there is no possible association between a function and a requirement. A score of 1 indicates a possible association, while 3 indicates a definite but weak association, and 8 indicates a strong association. The mapping of requirement to function was performed by two of the authors. It was felt that while we could have called on the people who had ranked the requirements, for illustration purposes two people could produce a reasonable pattern of data. Typically, mapping is performed by multivariate statistical analysis and relies on data generated by several judges. However, mapping can be a useful exercise in itself, particularly if most of the design team is involved; the exercise of mapping requirement to function allows design assumptions to be brought out into the open and discussed.
Baber et al.
25
Function # (see Table 1) Requirements Activities Episodes Projects Adapt Learning objects Learning outcomes Breakdown Mobility Rank 8 4 3 7 6 1 2 5 Weight 1 3 1 1 0 8 1 1 3 2 8 8 3 3 0 1 3 8 3 8 3 1 0 1 3 8 1 4 3 1 1 1 3 1 1 3 5 3 1 1 1 3 1 1 3 6 3 1 1 1 3 1 1 3 7 1 3 8 8 3 8 3 1 8 8 3 1 1 1 0 0 8 9 8 3 1 1 8 0 0 8 10 11 12 13 14 15 16 3 1 3 8 8 8 1 8 3 1 1 8 3 1 1 8 8 3 1 8 8 3 8 8 1 1 1 1 3 1 1 3 8 8 3 1 3 8 8 3 3 3 3 8 8 3 8 3 8 8 3 8 8 8 8 8 Evaluation criteria for user requirements
97 173 109 74 74 7413713217419114824258169183273 Evaluation criteria for functions
Figure 2: House of Quality for learning technologies
6. Conclusions
From the analysis using QFD, shown in Figure 2, three levels of attention can be proposed to be given to the design project, that is, the effort needed to develop a particular function given these user requirements: primary attention: peer-to-peer, annotation, curriculum management, content management, rendering the content to fit the constraints of a specific device, adaptive interface, context, user model secondary attention: moderation, creation tertiary attention: broadcast, text, audio, picture, augmented reality.
possible to consider which functions are essential to support user requirements, and which are less important. Such an evaluation can be very useful in the design process in that it can helps to determine the relative effort needed to support and develop each function.
Function Annotation Peer-to-peer Curriculum/content management Rendering/adaptive interface Context User model Requirement Write/draw onto content Communicate between users Select content for user Adapt content to display Sense and adapt to context Maintain model of specific user
Taking the primary level of attention, a new set of user requirements can be proposed for this set of functions. In this paper, we have demonstrated the potential use of quality function deployment as a method to aid in the capture and evaluation of user requirements. The intention is to demonstrate the application of this approach for the design of learning technologies. It provides a means of comparing user requirements against the assumed functionality of the technology that is being developed. By exploring this relationship, it is
References Bergquist K, Abeysekara J (1996). Quality function deployment (QFD): a means for developing usable products. International Journal of Industrial Ergonomics, 18, 269275. MacCaulay LA (1996). Requirements engineering. Berlin: Springer-Verlag. Zultner RE (1993). TQM for technical teams. Communications of the ACM, 36.
Brandt et al.
27
Self-produced video to augment peer-to-peer learning

Eva Brandt Space Studio The Interactive Institute Beijerskajen 8, S-20506 Malm Sweden E-mail: eva.brandt@tii.se Per-Anders Hillgren Erling Bjarki Bjrgvinsson School of Art and Communication, Malm University Beijerskajen 8, S-20506 Malm, Sweden E-mail: per-anders.hillgren@tii.se, erling.bjorgvinsson@k3.mah.se
Abstract
This paper describes how self-produced videos, made with a digital video camera and later viewed on handheld mobile computers, support informal learning at an Intensive Care Unit. The learning process supported is peer-topeer learning, where colleagues use mobile IT to communicate and learn from each other. The handheld computers are equipped with barcode readers, which give easy access to the learning videos. These mobile computers also make it possible to configure where and when the learning is going to take place. The staff themselves decide the content and how to produce the videos. Examples are given of how the spatial and social work environment is important in facilitating both the production and use of the videos. The success of the peer-topeer learning process seems to lie in the fact that the person on the video and the colleagues watching it all share the same social and cultural community of practice. Keywords: peer-to-peer learning, selfproduced learning material, video-films, handheld computers, barcodes, context, health care.
the staff as peers in a learning process where many communicate with many, and where they produce learning material (short videos) for each other. The paper has four sections. First, various views on learning are described to frame our research. Second, the research project, the resulting mobile IT concept and the design process that brought it about are described. Third, two examples are discussed. One shows how the production of learning videos can result in learning for the people involved; the other shows how the use of mobile computers with self-produced videos augments peer-to-peer learning. In the last section the results are summarised.
2. Peer-to-peer learning
The term peer-to-peer arrived in the computing field during 2000, as the musicsharing application called Napster changed the network model of the internet. Instead of using home computers to browse the web and exchange e-mails, computers could connect to each other directly, form groups and collaborate to become user-created search engines and file-systems. Content, choice and controls given to the users are important in peer-to-peer applications. In peer-to-peer web applications, the computers are viewed as peers in a larger network of cooperating computers (Oram et al. 2001). Our research is based upon viewing the staff in a work-setting as peers in a learning process where many communicate with many. Research about learning has traditionally focused on issues related to schools and other
1. Introduction
This paper discusses peer-to-peer learning augmented by mobile IT. The IT concept has been developed and implemented to support informal learning between colleagues at an Intensive Care Unit (ICU). It is based on viewing
28
educational institutions. Learning is primarily seen as a cognitive process within the individual pupil, initiated by the teacher who decides which learning materials and didactical means to use. The teacher is in charge of the learning situation and is the mediator of pre-set content. In short, the traditional learning scheme focuses on oneto-many communication and views learning as knowledge or information processing that can be decontextualised and explained using, for instance, words and images (Meier 1997). Ingela Josefson (1988, 1995) has conducted research on the training of nurses. Two irreconcilable traditions of learning are found to dominate. On the one hand, there is the scientific model, where the theories of learning are based on cognitive science. On the other hand, there is the apprenticeship model, which focuses on qualities that cannot be expressed in language. In the development of the nurses' profession the latter tradition is attacked while the first is pleaded for. This is explained by the aim of giving care a higher status in the society. Josefson emphasises that basing the development of IT-technology on the scientific model might have unfortunate consequences as it undermines knowledge gained from experience. Our research contradicts the traditional learning scheme and emphasises, more in line with Josefson, practice-based learning. We focus on informal learning, where colleagues communicate with each other, and what is to be learned is not set in advance. Instead, the learning material is produced when needed by the staff themselves. Schn's (1987) understanding of learning also inspires us, as it takes into consideration how professionals actually work. Schn has investigated the professional work practices of architects, lawyers and medical doctors. He argues that the most important aspect in professional competence and action is the ability to 'reflect-in-action' and 'reflect-on-action'. The first is essential when handling situations that are puzzling, troubling and uncertain. Reflectionin-action means reflecting in the midst of the action while handling the situation and material at hand. Practitioners make on-the-spot experiments that talk back to them, and this in turn becomes the basis for new on-the-spot experiments. Reflection-in-action is often stimulated by a surprise, meaning that the outcome of an action (or part of it) is unexpected. 'Reflecting-on-action' takes place after the action is performed and involves
distancing oneself and reflecting on what happened. Schn emphasises that professionals knowledge is embedded within a shared community of practice and that we are usually unable to make it verbally explicit. This is in line with another important source of inspiration, namely Lave and Wenger (1991) and their understanding of learning as a situated activity taking place within communities of practice. Atlhough Schn does not describe learning as a social process, Lave and Wenger have studied the relationship between people, activities and situations in several communities of practice. They argue that learning takes place while performing concrete activities and that all kinds of activities can be learning; not only those explicitly defined as such. Lave and Wenger have developed a notion of learning that they call legitimate peripheral participation. Legitimate peripheral participation describes how newcomers take part in concrete activities with more established practitioners and each of them has a different view of the actions. Both Schn and Lave and Wenger stress that learning depends on context and that professional practitioners are in constant negotiation with the situation at hand, changing the course of action as needed.
3. Mobile computers with video

For two and a half years the KLIV research project has studied how learning at an Intensive Care Unit can be supported with mobile devices (Bjrgvinsson and Hillgren 2002; Brandt et al. 2002). KLIV is the Swedish acronym for 'continuous learning within healthcare'. The project was carried out with close collaboration between the Intensive Care Unit at the University Hospital in Malm and the Interactive Institute. The design process was iterative, interdisciplinary and user-centered focusing on collaborative inquiry and participatory design. We were present at the unit during their everyday work and the staff have participated in various kinds of workshops both inside and outside their work environment. We often call this approach an event-driven design process (for a further development of this notion see Brandt 2001). The KLIV project found that the Intensive Care Units daily oral learning was a vast resource in the development of the staff's professional competences. This is in line with Orrs (1996) observation of the importance of technicians sharing oral stories to sustain and
Brandt et al.
29
develop their community memory. The staff now augment this oral resource by producing short videos around patient care and medical devices with a digital video camera. Passarge and Binder (1996), whose study involved spring machine setters in documenting their best practice on video, have influenced our work. Their videos were used for learning out in the workplace among the spring-setting machines. The videos were viewed on a stationary computer with a laser disc placed on a mobile table. In the KLIV-project, the staff themselves produced the videos and there were no professional cameramen involved. Another difference is that the videos are viewed on handheld computers by scanning barcode cards placed out in the work environment. This gives more flexibility and the chance to configure the learning situation according to different needs. In the video a colleague shows how the task is carried out including practical tips gained from experience. The process of making the videos is a collaborative learning process including several steps. It starts when the films are to be recorded and the people involved reflect on how the task is best carried out. The reflection continues when they and colleagues informally review the videos. This might result in a new film being made. Later there is a more formal review session where staff members with different responsibilities and backgrounds examine the content closely. Finally the videos are used and reflected on during daily work. The reflections might initiate the production of a new video to develop their work practice further.
Margareta (a nurse) wanted to experience the actual filming. She asked Bengt Gran (the nurse's aide) and Lena (another nurse) to help explore how to produce the films. They decided that Bengt Gran should be the patient and Lena should demonstrate how to carry out bladder scanning. They recorded the video in a vacant patient room. Bengt Gran is lying in a patient bed and Lena and Margareta stand by the bladder scanner placed on a mobile table at the foot of the bed. Margareta holds the video camera in her hand. She asks Lena: Margareta: How do you think we should start? Lena: I thought by starting it [the bladder scanner] up so you can see where it all starts. Margareta: Yes, that is what you are going to show, yes. Lena: Should I do that in connection with the whole thing? Margareta: Yes, you can just start Bengt Gran (interrupts): We first do a sequence where you prepare the equipment, then Margareta (interrupts): Yes, precisely. Yes. And when you feel that its time I can stop [filming], you know. Lena: I dont know what the buttons are called, but I know where to push. Margareta: Yes, but then I will film that, you understand. Lena: Shall I do that first? Margareta: Yes, let me see [looks at the camera to find the record button]. Margareta, Lena and Bengt Gran discuss how they should go about filming: whether to film in one continuous sequence or several short sequences. Confronted with the concrete situation of bladder scanning, the situation at hand speaks back to them and they discuss how they should give form to the film session and the bladder-scanning video itself. Faced with the task of showing how to scan Lena reflects on the fact that she does not know the names of the buttons but she knows which
4.
Self-produced learning material
This section discusses the production of selfproduced learning material. The example is taken from early in the design process where the focus was on investigating how the personnel from the Intensive Care Unit could produce the learning videos themselves. The specific case we discuss below is the production of a video on how to bladder scan that is, scanning to see how much fluid the urine bladder contains. In the process three videos were made. We would argue that the physical situation facilitates the production. More importantly, the construction of the videos prompts an exchange of views and negotiation, which influences the video recordings. The process of making the videos gives a voice to various individuals. Each has a different perspective on the community of practice, which is essential for learning and developing the practice.
30
buttons to press and why. Margareta thinks this is fine and that naming is unimportant in this case. It seems that Lenas knowledge of scanning relies on having the equipment in front of her. Our point is that the physical environment, and especially the bladder scanner in this case, are used, as suggested by Papert (1980) as 'things-to-think with'. This implies that knowledge cannot be separated from the context or the medium that makes the expression possible, and that objects expand our minds and bodies.
Lena: But usually, after narcoses, it can be up to 600ml as far as I have understood from the new PM. Margareta: Mmm, I was thinking even about this one [lifting the probe] when one has to use the setting man when women have been operated upon. Lena: Okay, thats what you mean. Margareta: Hysterectomy, yes. Lena: Yes, yes. A few minutes later: Lena: Then some can have air in their bladder also from the examination, as I understand. While making the video, Lena and Margareta dive into the situation, and become one with it. Ackerman (1994) has argued that moments of separation from the situation are also important in relation to learning. After the recording they need to step back or distance themselves to impose momentary order on the situation. They reflect on how the recording session went and on the content of the video. This is in line with Schn's notion of reflection-on-action. In the example Lena misunderstands Margareta and explains the new scanning procedures, thinking that this is what Margareta misses from the video account. What Margareta really misses, however, is how hysterectomies are scanned. Discussing unusual cases Lena explains that she has heard that scanning can be tricky after certain examinations since the bladder can contain a residue of air. Their dialogue shows that they have different experiences of scanning and therefore a different knowledge of what it means to bladder scan. Schn (1987) explains that practitioners construct artefacts as a way to discuss with themselves how to proceed with the problem they are facing. In such an instance the artefact talks back to the practitioner and is actively part of the practitioners inquiry. An artefact, according to Schn, is not necessarily physical: it can, for example, be a physicians diagnosis, a discussion or more generally the way a practitioner frames a problem. In making the video Margareta and Lena have, in Schns terms, framed or imposed a momentary order on what it can mean to show how to bladder scan. When viewing the video, the artefact being constructed informs their ideas of what the video should contain and what is missing
Figure 1: Margareta, Lena and Bengt Gran produce a video about bladder scanning. While filming,
Margareta edits the film on location in the camera. They produce the video by filming it in three short sequences retaking the first sequence once. Directly after the filming they watch the video in the camera, and reflect on the filming and the content. Margareta states that Lena did well but that she spoke softly. Lena agrees. They also agree that the video shows that it is not always easy get a good picture of the bladder. After the video is finished they continue discussing it. Margareta: One thing you did not say is, well, how you actually operate it. Lena: We have that picture in there, I had to leave it behind but a new PM has just been written for it. [A PM is a written description of how a specific task is carried out.] For one minute Lena tells the others what the PM contains. She goes into detail about how many millilitres are standard and what to do if the scanning differs from that. In the end Lena says:
Brandt et al.
31
from the account. Ackerman, in line with Schn, makes the point that we construct, or in her terms, build forms of, or understanding of, a situation. We do this to share our understanding of the situation and to negotiate its meaning (Ackerman 1994). The same week an informal review of the video was held with other people from the staff. It became apparent that the video needed to depict better how the scanner probe is handled. There was too much focus on what went on in the bladder scanner display. The video also lacked information on how to set up the scanner for children or for patients who had had hysterectomies. The following week a new bladder scanner film was produced. As we wanted more people to gain experience with the film process a new temporary film group was formed. Peter was the cameraman, Gran the person to be filmed, and Bengt Gran again acted as patient. All of them are nurses aides. We, the researchers, acted as the intermediaries, explaining what had been said at the informal reviewing session. Bengt Gran confirms that it is difficult to handle the probe. When they hear that they need to include information on the scanning of hysterectomies and children they start asking each other if anyone knows how to do that. Suddenly it becomes evident that all of them are unsure about what to do. Britt, a nurses aide, passes by and joins the discussion. She offers to go and find out what the procedures are. In the mean time they record a new video focusing on how to use the probe. While reviewing the resulting video Britt returns and explains that women who have undergone a hysterectomy, and children under the age of 12 of either sex, are scanned as men.
The new version of the video was shown at an informal review. The video was considered to be good, but the reviewers wanted added to the video information on how to discover and handle sources of error. For example, patients who have undergone laparoscopic surgery can be full of gas; other patients have fluid in the abdomen. Two weeks later Gran and Peter appended the bladder scanner film with information on how unusual scanning cases are handled. Two months later the videos are formally reviewed by the people who are responsible for the quality within various areas at the unit. The last videos were judged to be very good and accepted for use. Tracing the path of producing the learning material reveals that multiple voices within the intensive care community of practice shaped the videos. Each person contributed their knowledge and experience, which at times resulted in negotiations about the content and structure of the films. It became evident when knowledge was lacking, and which issues had to be investigated. Learning happens by discovery, while using the video medium to give form to their ideas of how to carry out the task, were which then discussed, argued for and justified. Learning happens on various levels. On one level it gives them a chance to discover and reflect on their work practice and, with this, how to make descriptions that can be understood by their colleagues. On another level, it gives insight into different experiences and how these can be given a form that helps develop the knowledge that exists at the unit. Making the videos was, therefore, both a dialogue with the physical situation of scanning, and a social peer-to-peer dialogue between colleagues. The production and final acceptance of the bladder scanner video ran into several months. This was because more than one film had to be made and we were exploring what kind of video production process was needed. There are examples of much shorter processes. Still, common to all of them is that they are iterative processes involving several staff members. The advantage of producing their own learning material is that the content reflects experiences of working at the unit: for example what to be observant about and what can be difficult. The production process being carried out in-house allows the video to be formed and transformed reflecting the experience of the staff and therefore what knowledge is needed in their communities of practice. Many of the staff
Figure 2: Based on the comments from reviewing the first bladder-scanning video Peter, Gran and Bengt Gran produce a new film focusing more on how the probe is handled.
32
have also reported that it is important for them to know the people in the videos, as it gives them confidence in the way things are carried out in their work environment.
5. Using mobile video

Viewing video on handheld computers makes it possible to configure where and when the learning is to take place. The following is an example where it is important to view a video close to the medical equipment that is to be assembled. It also illustrates that the staff, working together when using the video, draw on each others experiences and complement the video. In the example below Ulrika, a newly employed physiotherapist, is involved in a training sessions where she and Jenny, the senior physiotherapist, attach a Fisher & Paykel (a humidifier apparatus) to a ventilator.
procedure. They know that their colleague, Anna, who knows the equipment well, has made a video about it. The video is accessible both via a handheld computer and a stationary computer but the latter is not located in the medical equipment room. At the time both Jenny and Ulrika were unfamiliar with the handheld computers, having missed the introduction at the unit. Therefore they decided to watch the video on the stationary computer. Afterwards they go to the equipment room and begin assembling the Fisher & Paykel. Having a problem mounting the expiration valve, they decided to bring the valve with them to the stationary computer so that they can easily compare it with the video. They watch their colleague in the video pointing at two small holes on the expiration valve. Holding the valve close to the screen they alternate between looking at the video and examining the physical valve. In the video Anna emphasises that one should avoid letting the holes come close to each other because then the air will leak out. In the video Anna says: Im putting in the expiration valve so that you can hear a click. At the same time you see her performing the action and hear the click in the video. Jenny: Now we know how we will mount it, like this. Simultaneously Jenny shows how, by moving the valve in a similar way to the video. Ulrika: Yes. Ulrika points at the flow sensor in the video and asks: Ulrika: Then you dont need to take that part away or? Jenny: Yes I think you do, Im not sure but I think I have seen people doing that. They return to the room where the ventilator is and mount the expiration valve. They proceed with mounting the Fisher & Paykel by examining the equipment to figure out which parts fit together. After some time they get stuck. Ulrika: Where did I put the short one Jenny? Jenny: The short one you put on well yes, we should have a look on the handheld computer now.
Figure 3 a,b: As part of a training session Jenny (senior) and Ulrika (newcomer) watch a video on how to mount a Fisher & Paykel to a ventilator. They have brought part of the equipment (the valve) into the room with the stationary computer to be able to compare the valve with the video.
Jenny has been away from work for two years and is therefore a bit uncertain about the
Brandt et al.
33
One of the researchers gives them a short introduction to the handheld computer. Even though they already have managed to mount the expiration valve they decide to see this part of the video to doublecheck if it was correctly done. Watching the sequence Jenny says: Jenny: There you can see that she [Anna in the video] doesnt have the flow sensor there [mounted at the expiration valve]. A little later Ulrika points at the video and says: Ulrika: She [Anna] already has that one in place [meaning that the flow sensor is mounted to the ventilator]. Jenny: Hmm, yes probably she has, but I cant see it. A little later Jenny says: Yes she puts it in place in the little rubber shall I move on?
equipment was how the expiration valve connects to the flow sensor and the ventilator. They want to be sure that they have done this right. In the video Anna moves the flow sensor to the left on the ventilator after she mounts the expiration valve. By watching the video and discussing it with each other they realise that the flow sensor is not connected to the valve before connecting the valve to the ventilator. Ulrika sees on the video that the sensor is mounted to the ventilator before the expiration valve. First Jenny cannot see that, but she soon realises that this is the case when watching Anna move the sensor to the left. When handling the situation Jenny and Ulrika reflectin-action while having a continuous dialogue with each other, the video and the equipment. The situation is made up of their activity, the different kinds of equipment, their colleague on the video and their previous experience of and intentions with the activity. An important concern in the project was whether the use of learning videos would prevent the staff from reflecting on their practice as they would just follow and trust an experienced colleague on the video. The example illustrates that the video augments interpretation, reflection and learning. Jenny holds the handheld computer and they go on watching the video and mounting the equipment. Jenny: And then it is the apparatus filter. Anna in the video: And this filter you may need to change two times a day. Jenny: There is quite a lot of condensation in the tubes. Sometimes you may open up and empty the tubes if youre allowed to interrupt the respirator circle. From her experience Jenny adds information that is not in the video but which she thinks is important for Ulrika. For some of the procedures, the videos only explain how they should be performed and not why. Unnecessary reasons for things that are familiar to most staff can be excluded since the videos are produced by people in the context where they are to be used. However, it is a tricky balance where no common rules can be applied. Still, when the videos are used, as in the example, colleagues help each other by adding knowledge.
Figure 4 a, b: The small mobile computer with the learning video were found more handy than the stationary computer.
Their reason for doublechecking was that their main concern when mounting the
The learning situations with Jenny and Ulrika resemble legitimate peripheral participation, where the newcomer, although on the periphery, is actively taking part in the concrete activity. Looking back on the situation, Jenny
34
said that it was obvious to her that she was learning from Ulrika as well as the other way around.
l e a r n i n g . Presented at the National Educational Computing Conference (NECC). Boston, Massachusetts, 1994. Bjrgvinsson EB and Hillgren Per-Anders (2002). Readymade design at an Intensive Care Unit. P a r t i c i p a t o r y Design Conference, Malm. Brandt E (2001). Event-driven product development: collaboration and learning. PhD thesis: department of manufacturing engineering and management, Technical University of Denmark. Brandt E, Bjrgvinsson E, Hillgren P-A, Bergqvist V and Emilson M (2002). PDAs, barcodes and video-films for continuous learning at an Intensive Care Unit. NordiCHI conference, rhus, 2002. Josefson I (1988). The nurse as engineer: the theory of knowledge in research in the care sector. In B Granzon and I Josefson (eds), Knowledge, skill and artificial intelligence. Berlin, Heidelberg: SpringerVerlag 1988. Josefson I (1995). A confrontation between different traditions of knowledge. In B G r a n z o n Skill, technology and enlightment on practical philosophy. Berlin, Heidelberg: Springer-Verlag. Lave J and Wenger E (1991). Situated learning: legitimate peripheral participation. Cambridge (England), Cambridge University Press. Oram A (ed) (2001). Peer-to-peer: harnessing the power of disruptive technologies. USA: O'Reilly & Associates, Inc. Orr, J (1996) Talking about machines: an ethnography of a modern job. Ithaca, NY: ILR Press. Papert S 1980. Mindstorms: children, computers and powerful ideas. New York: Basic Books, Inc. Passarge L and Binder T (1996). Supporting reflection and dialogue in a community of machine setters: lessons learned from design and use of a hypermedia type training material. AI & Society, 10: 7988. London: Springer Verlag. Schn DA (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass Publishers.
6. Conclusions
The KLIV project has shown that selfproduced learning materials in the form of short videos augment peer-to-peer learning. It was found that the use of video was excellent as a common reference point for learning. The collaborative production and use of the video films has made the work practice visible for more colleagues and this gives them opportunities to reflect on, learn from, and develop their work practice. The highly collaborative making of the videos helps to make the content particularly relevant. It is essential that the videos are contextual; meaning that the staff produce the videos in their own work environment and they are later used in the same environment. The spatial work environment facilitates the production and use of the videos. Most importantly, what makes the videos relevant seems to be the shared social and culture community of practice. Viewing colleagues on film gave confidence about how things are to be solved here and simultaneously the opportunity to reflect on the work practice and discuss it with colleagues. Furthermore, it was found that learning takes place throughout the whole process of making, reviewing and using the video. Watching the videos on small mobile computers provides the possibility of configuring the learning situation independent of a specific place. This was highly valued by the staff. Using cards with barcodes gives easy access to a specific video, which was also appreciated.
7. Acknowledgements
We would like to thank all the staff at the Intensive Care Unit at the University Hospital in Malm. A special thank you goes to Sue Harden and Else-Maj Rosenlf for initiating, supporting and providing means to accomplish the research project. Thanks also go to Thomas Binder at the Interactive Institute for helpful advice and valuable discussions.
References
Ackerman E (1994). Perspective-taking and object construction: two keys to learning. Y Kafai and M Resnick (eds), Construction in practice: rethinking the roles of technology in
Bull, Reid
35
Individualised revision material for use on a handheld computer

Susan Bull and Eileen Reid University of Birmingham Educational Technology Research Group, Electronic, Electrical and Computer Engineering, University of Birmingham, Edgbaston, Birmingham B15 2TT E-mail: s.bull@bham.ac.uk
Abstract
This paper introduces an adaptive learning environment for use on a desktop PC and a handheld computer. At the end of the desktop PC tutorial session, revision material tailored to the needs of the individual and appropriate for viewing on a handheld computer is recommended for synchronisation to the handheld device. Thus the student has access to additional individualised mobile revision material for use at times and locations where it would not normally be possible or convenient for them to study, but where they might nevertheless welcome this opportunity. Keywords: learner model, revision material, handheld computer
1. Introduction
Adaptive learning environments provide the opportunity for computer-based educational interactions to be tailored to the specific needs of the individual student, by taking into account their existing knowledge and misconceptions (ie incorrect facts or incorrect concepts believed by the student), as revealed during the interaction. Because of the advantages of such individualised approaches to computer-based learning, adaptive tutoring systems are becoming more common in the desktop PC and web-based context. However, despite the recent interest in possibilities for learning with handheld computers, little attention has been directed at the potential for individualised mobile learning.
The Adaptive Geometry Game (Ketamo 2002) offers one example, where a childs accuracy and speed of response to polygon recognition questions is modelled so that the system may adapt questions as appropriate for the child's skill level. This is a simple adaptive system where children are shown a target polygon and required to identify which of the additional polygons (which may be rotated) are identical to the target shape. Depending on the time taken to answer and the number of errors in the responses, the system will apply one of the following conditions: remain at the current level of difficulty; offer an easier task; or offer a more difficult task. A second individualised approach is suggested by Malliou et al. (2002), who propose creating courses by combining modules into a personalised virtual document according to a user profile. Their system will be able to recognise where students lack skills, and then proactively recommend appropriate content to the learner. Approaches such as the above could be extended to allow more complex learner modelling to include representations of misconceptions, as is common in desktop PC intelligent learning environments, in situations where individualised interactions based on (correct and incorrect) learner beliefs could be a useful way to support learning in a mobile context. In cases where a mobile educational environment is not designed for use in a specific context only (for example inside a museum where the interaction is related to the particular exhibits available or on field trips where the interaction relates to specific features of the trip), a learning environment could
36
usefully span both the desktop PC and the handheld computer. Indeed, uniting the desktop and handheld computer has been suggested for synchronous and asynchronous collaborative learning (Farooq et al. 2002), to allow, for example, students in the field to communicate with peers in the classroom. In one-to-one adaptive learning environments, combining the use of desktop PCs and handheld computers provides the benefit of adaptive tutoring on the desktop computer when the learner is in a location where a standard PC or laptop is available, but also allows the interaction to continue, albeit with some restrictions dictated by the limitations of the handheld device, when such facilities are not present. Although the Adaptive Geometry Game described above has versions for the laptop and handheld computer, interactions on each device are separate, that is a learner will use either a laptop or a handheld computer. An example of a fully combined desktop PC/mobile environment is C-POLMILE (Bull and McEvoy 2003), where an individualised tutoring session for C programming can take place on either the desktop PC or handheld computer, as is most convenient for the learner at the time. Apart from some presentation issues due to the size of the screen, interactions on the two devices in C-POLMILE are identical. The results of a session can be synchronised to the other device for seamless continuation of the interaction. Another approach to joint desktop PC/mobile learning environments is to offer different interactions on the different devices, each interaction option being suitable for the particular device, combining to form the complete computer-based educational interaction. This assumes that the learner will have frequent access to a standard PC or laptop, but is also a regular user of a handheld computer. This paper introduces the MoreMaths (mobile revision for maths) environment, implemented in Java and designed to support a university mathematics course. The main tutoring interaction takes place on a desktop PC, where the learner can review and practise material taught in lectures (or catch up if they missed lectures), and receive individualised feedback on their responses, much as in other adaptive environments. The advantage of MoreMaths over other intelligent learning environments, however, is that after the desktop PC-based interaction, students may take away tailored revision material on their handheld computer for later consultation at their convenience. This revision content is generated for the individual according to their specific learning needs as revealed during their interaction with the desktop PC component of the environment. It is
constructed from a library of short excerpts of static content, composed and presented via text templates.
2. Learner modelling
As stated above, there are two components to MoreMaths. The main interactive session takes place on a standard PC, where the learner can view tutorial materials and answer multiple choice and text entry questions to practise and test their understanding of lecture content (Figure 1). A model of the students knowledge and misconceptions is automatically built, depending on their answers to these questions. This learner model is used in part in the conventional manner, as information to allow system adaptation to the needs of the individual. For example, if the student responds correctly to questions in MoreMaths, they are offered the option to increase the difficulty level of content and questions presented; whereas if some of their responses are incorrect, they are offered the possibility to decrease the current difficulty level. If the students answers reveal a likely misconception at any level, subsequent learning content addresses this misconception as appropriate for their skill level.
Figure 1. The desktop PC tutoring system In addition to the above, the learner model is used at the end of the session to suggest suitable revision material that the student can synchronise to their handheld computer: material that is tailored to their specific current learning requirements. This enables the student to continue learning away from the PC, at times and locations where individualised interactions would not normally be possible, but where the student might nevertheless welcome the opportunity for further study. Roy et al. (2002) also argue for mobile learning materials based
Bull, Reid
37
on student responses to questions. In addition to dividing the interaction between the desktop PC and handheld computer to make the most of the potential offered by each device, our system extends this notion by offering tailored revision materials according to a students learner model, the materials aimed not only at helping the learner understand areas of difficulty, but also explicitly addressing their misconceptions. In addition to revision materials, the learner model can also be synchronised for viewing. This is because as well as enabling tailored interactions, an open learner model (ie one that is inspectable by the student) can also be a useful learning resource as it can help the learner to assess their knowledge and promote reflection on their learning (Bull and Nghiem 2002; Mitrovic and Martin 2002).
3. Mobile revision material

The attributes included in the learner model and revision material are based on a study exploring the contents students would find helpful in an educational system with a mobile open learner model (Bull 2003), namely: a statement of known topics a statement of problematic topics a statement of likely reasons for difficulties (eg misconceptions) a comparison of student beliefs and correct concepts a statement of revision requirements appropriate revision material. In the synchronised material the student may see a statistical overview of their responses to 'end of topic' questions in the form of a graph illustrating their performance (Figure 2), a slightly more detailed textual equivalent (Figure 3), and the learner model: specific textual descriptions of topics known, problematic areas, and explanations of likely misconceptions (Figure 4). Showing the learner model to the learner as part of the material created for synchronisation to the handheld computer is designed to help the student better understand where their difficulties lie, what their specific problems are, plan their learning and reflect on the learning process, as their beliefs (knowledge, difficulties and misconceptions) are made explicit. Additional revision material is tailored to the individual according to the contents of their learner model. If a student appears to be having only minor difficulties, the mobile revision material for that area is quite brief, mainly having the function of reminding the learner of
information. This is illustrated in Figure 5 with the example of introductory material on adding fractions, where the learner is reminded of the steps involved. However, more detail is provided if the learner model contents indicate more serious problems or misconceptions. This can be seen in Figure 6, where explanations are also given. Where possible, descriptions in the revision material refer to concepts already known, to support explanations of difficult areas. In the example in Figure 5, the learner already knows the terms numerator and denominator, therefore these can be used without explanation. In more advanced materials, more complex known concepts can be used to help explain advanced content. Misconceptions may also be dealt with openly in the learner model as illustrated in Figure 4, where the learner believes that to add two fractions, the numerators and denominators are added together.
Figure 2. Mobile graphical performance overview
Figure 3. Mobile textual performance overview
38
not available. It is intended as a supplement to the PC-based interaction, to extend individualised learning opportunities to other contexts. When a student returns to the desktop computer having reviewed the mobile material, they are given a brief test on the revision content to update their learner model to ensure that the new interaction will be adapted appropriately for their current understanding. At the end of this new interaction, further individualised mobile materials are created for synchronisation, as is appropriate for the student's new knowledge state. Figure 4. Mobile open learner model
4. Initial pen and paper evaluation

Following the study described earlier, to determine the attributes students might find useful in a mobile open learner model, a second pen and paper study was undertaken after the MoreMaths system had been implemented based on these findings. There are many studies on the effectiveness of desktop computer-based adaptive learning environments. The aim of this study, therefore, was to evaluate the potential utility of tailored mobile revision material, such as presented in MoreMaths, created according to the results of the desktop PC interaction, and to identify any aspects of the environment that might require further consideration before a more detailed evaluation is undertaken of the MoreMaths system in use. This initial evaluation consisted simply of obtaining student reactions to screen shots, but was nevertheless useful because of the unusual combination of the desktop and handheld computer interaction: it was not obvious whether students would welcome this type of learning support, or if they did, what form this should take. The intermediate study sought to determine whether the current implementation is suitable for a full evaluation across several weeks, where actual and natural use can be observed. Apart from the likely utility of the different components of the mobile materials, a major general question for investigation was whether students believed there to be a role for static content on the handheld computer, since reading on the small screen forces different reading strategies to those generally used (Waycott 2002). Would students be prepared to adapt?
Figure 5. Tailored mobile revision material: less detailed
Figure 6. Tailored mobile revision material: more detailed The tailored mobile revision and other material is designed to be consulted at a time and place that is convenient for the user, when other individualised learning opportunities are
4.1. Subjects
The subjects of the study were nine volunteers taking an MSc in human-centred
Bull, Reid
39
systems at the University of Birmingham. All subjects had been loaned a Compaq iPAQ Pocket PC for the duration of their course. The study took place seven months after the students had received their iPAQs. These students had previously taken part in a logbook study of their general iPAQ use (Bull 2003), and additional questionnaire studies on their use of specific applications on their iPAQs (Sharples et al. 2003). All subjects therefore had a good awareness of their general usage patterns and were in a position to judge whether they would be likely to consider using a system such as MoreMaths as part of their routine. All subjects had also completed an MSc module and associated assignments in educational technology, which included components on mobile learning, adaptive learning environments and evaluation of educational applications. Subjects were therefore well able to appreciate the aims of MoreMaths and the questionnaire study from the information provided.
4.3. Results
Table 1 and Figure 7 show the results for students' beliefs about the likely utility of the various components of the mobile part of MoreMaths, in table and graphical format. The results were generally positive. The likely most useful component of mobile MoreMaths is the individualised revision material, with eight users expecting this to be useful (2) or very useful (6), and one unsure. The possibility to access the learner model data was also felt to be potentially useful by most users, with six subjects selecting useful (3) or very useful (3), one unsure and two believing that viewing the learner model would probably not be useful. Seven subjects also felt that the graphical statistical overview of their performance would be useful (2) or very useful (5), with two being unsure. The textual statistical overview was the least popular, though nevertheless potentially useful to just over half of respondents. Five subjects expected the textual overview of performance to be useful, three were unsure and one felt that it probably would not be useful. Table 1. Expected utility of mobile MoreMaths components (textual)
More Maths graphical overview (Fig 2) textual overview (Fig 3) learner model (Fig 4) revision material (Fig 5,6) very useful 2 useful poss useful 2 prob not useful 0 not useful 0
4.2. Materials and method

A questionnaire with short descriptions and screen shots of the various components of the desktop PC and mobile MoreMaths environment was sent by email to the 17 MSc students who had been loaned iPAQs. The earlier studies mentioned above had been compulsory for those borrowing iPAQs. The present study was optional. Students had not been expecting the questionnaire: nine of the seventeen questionnaires were returned a 53% return rate. The questions required answers on the following scale: very useful/useful/possibly useful/probably not useful/not useful. These questions related to the various mobile components of MoreMaths. As not all students would need information, practice and revision material in mathematics, they were asked to consider their responses with reference to the possibility of similar materials relating to their own courses. It is, of course, not possible to determine the likely utility or uptake of software based only on viewing screen shots, but this initial evaluation does provide an indicator of whether students would welcome the approach as presented, and whether they feel that there is an obvious need to improve or change any aspects of the environment to better suit their requirements before undertaking a full evaluation of the system in use. Obtaining such information was the aim of the evaluation.
Figure 7. Expected utility of mobile MoreMaths components (graphical) Table 2 provides the responses for each volunteer. breakdown of
40
Table 2. Expected utility of mobile MoreMaths components by subject

More Maths graphical overview (Fig 2) textual overview (Fig 3) learner model (Fig 4) revision material (Fig 5,6) very useful S2 S7 useful S1 S3 S4 S6 S9 S1 S2 S5 S8 S9 S6 S7 S9 S4 S6 poss useful S5 S8 S4 S6 S7 S3 prob not useful not useful
4.4. Discussion
The results presented above show that there are no obvious patterns to students' beliefs about the likely utility of systems such as MoreMaths. The strongest tendency was for some students to prefer a graphical over a textual statistical overview of their performance, or vice versa, with a greater number of students preferring the graphical option. Nevertheless, some students expected that both would be useful. There were no other general tendencies. However, it is interesting to observe that S5 and S8 gave identical responses (uncertain about the graphical overview but in favour of the other components), and S4, S6 and S7 gave similar, though not identical responses (uncertain about the textual overview, but positive about everything else). Nonetheless, this does not suggest anything other than the fact that the subjects believed MoreMaths could be useful in general, but they would probably prefer the overview statistics in one format over the other. Overall the textual overview was the least popular component of mobile MoreMaths. This may be because the graphical overview is easier to interpret quickly. However, additional detail in the textual version includes information about which questions were answered correctly and which incorrectly, that is not available in the graphical overview. Nevertheless, students may have preferred to be informed what the correct answers were (in cases of incorrect responses), rather than just an indication of the incorrectness of their answer (this was indicated by one student who provided extra information on the questionnaire). However, since the aim of the open learner model and revision material is to help students overcome problems and misconceptions, at the design stage it was decided not to provide precise details about answers in the overview material. However, it appears worth investigating whether this might be done in the main study, for example, by providing some learners with the possibility of viewing this information with such additional detail, while others receive the current implementation. Differences in how users interact with the other materials depending on which version of the textual overview they have available, may then be observable. For example, will students try to work out their difficulties by using the correct answers in the overview, requiring less from the learner model and individualised revision material? Or will the provision of correct answers distract students, and not adequately support their needs? For instance, providing the correct answer will not necessarily help a
S3
S4 S5 S8 S1 S3 S5 S7 S8 S9
S1 S2
S2
Nearly all subjects felt that some of the mobile MoreMaths components would be useful, but were also unsure about other components, or felt that they would probably not be useful. Only S9 had a positive response for all components. Some individuals expected to find either the graphical or the textual performance overview helpful. This situation was apparent with two subjects (S5, S8) who preferred the textual alternative, and were unsure about the graphical version; and four subjects (S3, S4, S6, S7) who preferred the graphical, but had lower expectations of the textual description. The remaining three subjects (S1, S2, S9) thought that both approaches would be useful. It is harder to look for patterns between preference for viewing the learner model or revision materials, since eight of the nine subjects felt that individualised revision content would be helpful. The one student who was unsure thought that the learner model would probably not be useful (S2). However, the reverse was not true: S3 who was unsure about the learner model, and S1 who thought that it would probably not be useful, were both very confident about the likely utility of the revision materials. There was no clear split between subjects who would prefer only one of performance overview (in textual and/or graphical form) or additional individualised descriptions (learner model and/or revision material). The only example was S2, who would use both forms of overview, but was less interested in the more detailed individualised explanations. There were few patterns to responses. Only S5 and S8 had identical responses unsure about the graphical overview, but positive about everything else. S4, S6 and S7 had generally quite similar expectations, being unsure about the textual overview, but positive about everything else.
Bull, Reid
41
learner to understand that they hold a certain misconception. However, if students use the learner model and revision content as intended, and then use the correct answers provided as a check to their understanding after reattempting the questions, this could be a powerful learning support. This will be a useful question to investigate in the main study. The main concern about the textual mobile materials in general was whether, despite the fact that they are tailored to the individual's needs, students would regard the materials in mobile form as redundant. For example, instead of reading static information on a small screen, they may prefer to print it. The individualised revision materials are usually the longest and most detailed, therefore these were of particular concern. In future work we plan to investigate the potential of interactive tailored revision materials, where this question would be less applicable. However, even then, some of the materials might still be most usefully available in static form. Therefore it was important to find out whether students would be likely to accept this, as there would be little point in running a larger study of the system in use if the main component appeared unwelcome to users. The results showed, however, that users were keen to have individualised, static mobile revision materials. In fact, this was the most popular component of mobile MoreMaths. This suggests the hypothesis that users who regularly use handheld computers will welcome (static) mobile individualised revision materials for use in conjunction with their courses. This hypothesis now needs to be tested, to ascertain whether such materials really are found to be useful in practice, or whether the small screen does, in fact, impose too many restrictions on the learner. Two-thirds of subjects also thought that the mobile open learner model would be useful. This suggests that although it is an unusual method of feedback for most of the target users, they would find explicit reference to their misconceptions and identification of areas of difficulty helpful. Again, this now needs to be evaluated in practice. Only S2 had a preference for overview material over the more detailed open learner model and revision material. There were no preferences the other way around. Therefore, it appears that most learners would appreciate both kinds of information about their progress and learning, and there is no indication that either kind of information should be removed from the implementation before the main study. It is, of course, probable that students taking a degree in human-centred systems who have taken the optional educational technology
module, and who regularly use handheld computers, are more likely to be open to using a system such as MoreMaths. However, we do not claim that the system should be suitable for all students or for all subject areas: work on individualised mobile material is worthwhile if sufficient students find the approach a useful additional support. From a sample of 9 it is not possible to make any strong claims, especially since the study was based on viewing screen shots rather than actually undertaking a series of interactions. As stated in Section 4.2, the usefulness of software cannot be ascertained simply by having potential users look at screens. The most that can be determined is whether users are interested and likely to be open to trying the approach essential if they are to go on to use the system. Such information is a useful intermediate step as it can highlight potential problems that can be given attention before the main study is undertaken. This willingness and interest in trying the approach has been demonstrated in the study the generally positive replies do indicate that a full study of the system in use is worthwhile at this stage. It also suggests that similar approaches might be welcomed by students. An obvious disadvantage of MoreMaths and similar systems is that students need to have a handheld computer in addition to access to a desktop PC or laptop, and this is not yet the case for most university students. Over the coming years, the Electronic, Electrical and Computer Engineering Department at the University of Birmingham will see an increase in the number of iPAQs, some of which will be available for loan to students. It is therefore expected that full-scale studies of MoreMaths and associated systems will be possible in the context of the courses to which they relate.
5. Summary and further work

This paper has presented the MoreMaths system: a combined desktop PC/mobile adaptive learning environment. In-depth maths tutoring takes place on the desktop PC, where it is easier to interact with information and obtain a well-structured overview of each topic, and practise the target material. Mobile revision material based on the learner model, created according to the student's performance in the desktop PC environment, and the mobile version of the learner model, are intended as an additional interaction to the main computer session. These can be reviewed at a convenient time and place on a handheld computer, when a desktop PC is unavailable.
42
All mobile learning content is tailored to the specific needs of the individual. An initial pen and paper evaluation has been completed, suggesting the potential of the approach. Future work will involve detailed evaluation of the desktop PC system, the mobile revision materials, the open learner model, and the effectiveness of uniting the two components of the learning environment in a single system. The potential for interactive mobile revision materials will also be investigated.
6. References
Bull S (2003). User modelling and mobile learning. In P Brusilovsky, A Corbett and F de Rosis (eds) User modeling 2003: 9th International Conference. Berlin Heidleberg Springer-Verlag, 383387. Bull S and McEvoy AT (2003). An intelligent learning environment with an open learner model for the desktop PC and pocket PC. In U Hoppe, F Verdejo and J Kay (eds) Artificial intelligence in education. Amsterdam: IOS Press, 389391. Bull S and Nghiem T (2002). Helping learners to understand themselves with a learner model open to students, peers and instructors. In P Brna and V Dimitrova (eds) Proceedings of Workshop on Individual and Group Modelling Methods that Help Learners Understand Themselves, International Conference on Intelligent Tutoring Systems. San Sebastian, 513. Farooq U, Schafer W, Rosson MB and Carroll JM (2002). M-Education: bridging the gap of mobile and desktop computing. In M Milrad, U Hoppe and Kinshuk (eds), Proceedings of the IEEE International Workshop on Wireless and Mobile Technologies in Education. IEEE Computer Society, 9194. Ketamo H (2002). MLearning for kindergarten's mathematics teaching. In M Milrad, U Hoppe and Kinshuk (eds), Proceedings of the IEEE International Workshop on Wireless and Mobile Technologies in Education. IEEE Computer Society, 167168. Malliou E, Stavros S, Sotiriou SA. Miliarakis A and Stratakis M (2002). The AD-HOC Project: eLearning anywhere, anytime. In A Anastopoulou, M Sharples and G Vavoula (eds) Proceedings of the European Workshop on Mobile and Contextual L e a r n i n g . Birmingham: University of Birmingham, 4750. Mitrovic A and Martin B (2002). Evaluating the effects of open student models on learning, in P De Bra, P Brusilovsky and R Conejo (eds). Proceedings of Adaptive Hypermedia
and Adaptive Web-Based Systems. Berlin Heidelberg: Springer Verlag, 296305. Roy S, Trinder J and Magill J. (2002). Portable learning and assessment: towards ubiquitous education. In S Anastopoulou, M Sharples and G Vavoula (eds), Proceedings of the European Workshop on Mobile and Contextual Learning, Birmingham: University of Birmingham, 70. Sharples M, Chan T, Rudman P and Bull S (2003). Evaluation of a mobile learning organiser and concept mapping tools. In J Attewell, G Da Bormida, M Sharples and C. Savill-Smith (eds), MLEARN 2003 book of abstracts. London: Learning and Skills Development Agency, 6061. Waycott J (2002). An evaluation of the use of PDAs for reading course materials. In M Milrad, U Hoppe and Kinshuk (eds) Proceedings of the IEEE International Workshop on Wireless and Mobile Technologies in Education. IEEE Computer Society, 177178.
Colley, Stead
43
Take a bite: producing accessible learning materials for mobile devices

Jo Colley and Geoff Stead Cambridge Training and Development (CTAD) Lincoln House, The Paddocks 347 Cherry Hinton Road, Cambridge CB1 8DH E-mail: joc@ctad.co.uk, geoffs@ctad.co.uk Abstract
Writing basic skills materials for use on a handheld device is challenging. The m-learning project is attempting to meet the challenge by producing a set of innovative games, materials and activities which will not only motivate reluctant young learners but also give them an opportunity to improve their basic maths and English skills in a way that complements their disconnected, mobile lifestyles. Evaluation is another important area, especially relating to cognitive, meta-cognitive and affective changes which may come about in learners as a result of using their materials and systems. Various techniques and activities, eg VXML and SMS, are being used to discover how the target group approach and experience learning using mobile devices in terms of developing their skills and motivation for learning in general. Keywords: basic skills, games, learning materials, evaluation comfortably and accessibly in a handheld device. There are also opportunities for the technology to affect the pedagogy, particularly through the use of collaborative activities that incorporate elements of mobile learning in new and effective ways. The projects aim is to capitalise on the obvious benefits of using the technology while developing innovative materials that maintain a clear perspective on the learning goal. Materials produced by the m-learning project partners combine sound basic skills pedagogy with ground-breaking use of new technologies and devices. We are currently in phase 2 of the project. In phase 1 we trialled several different approaches with a range of learners. Now, with many lessons learned (and fed into our development) and remarkable improvements in technology, we are busy developing a new generation of materials and templates to help us take these lessons one step further.
2. 1. Developing content
Platforms
The aim of this paper is to discuss the pedagogical issues affecting the creation of basic skills materials to be used on an electronic device and to explore ways in which learning materials can be tailored to make the most of the mobile learning opportunity. There has been considerable research into the positive benefits of improving basic skills using a computer: issues such as maintaining privacy, avoiding stigma, working at your own pace have been well rehearsed. The challenge that the m-learning project sets itself is attempting to maintain these benefits while shrinking the materials themselves to fit
Mobile phones were selected as one platform as they are the communication tool of choice for the m-learning projects target group of learners (young adults aged 1624 not in fulltime education or training) and a relatively inexpensive hardware option. Personal digital assistants (PDAs) were the second platform since they provided greater computing power to support multimedia applications in a small device, and it was anticipated that the technologies of PDAs and mobile phones would soon converge. Indeed this is happening during the course of the project. Specifically we are using the iPAQ, but most content is being developed to run in a browser on the iPAQ so that it is platform
44
independent and thus able to run on any palmtop and many of the Smartphones.
3.
Working with palmtops
designing content that can either provide or stimulate learning requires ingenuity, knowledge of the technological constraints and the ability to think inside a fairly small box. As well as devising a series of themed quizzes that are also linked to the basic skills curricula, the m-learning partners are working on materials that permit Figure 2: interaction between learners. Sample SMS activity It may also be possible to use mobile technologies to assess attitude to learning in more innovative ways, making use of VoiceXML (voice recognition software) and SMS (text messaging).
5.
Figure 1: IPAQ showing m-learning material Palmtops have the advantage of a relatively large screen so that using graphics, animations and video-clips is feasible. In the first phase of the project, literacy and numeracy games and activities were developed to engage learners with the technology. More complex learning objects were also created (see Section 5), including an animated video soap opera which was devised on the theme of housing, exploring life skills such as dealing with landlords but also providing a context for the practice of basic skills, for example working out area of walls to calculate amount and cost of paint required to decorate. Scaled-down quizzes on sports themes were devised and proved popular in trials, particularly a football referee quiz allowing learners to assess their knowledge with a view to further training. Palmtops also link the learner with the internet and therefore with a potentially rich field of other learning materials. However, they are expensive: more executive toy than simple, practical communication device. They do not feature large in the landscape of a mobile youth lifestyle.
Themed learning
How can we make these tiny bits of content part of a larger whole? In phase one we developed themes of content (in conjunction with trial sites and project partners), spanning all the available technologies. These themes were: an urban soap opera involving two young characters moving into a new flat. The characters were introduced using a Flash movie (iPAQ), and daily updates to the story were available via the phone (VoiceXML). Matching learning activities were delivered as described above via the iPAQs football refereeing timed to coincide with the World Cup. Animated quizzes on the iPAQs were used, as was a daily quiz of five questions via the phone (VoiceXML).
In the second phase we are extending this to cover driving, a broader range of sport and some aspects of health education, as well as incorporating a wider range of technologies.
4.
Working with mobile phones

Figure 3: Sample from an iPAQ quiz, part of the urban survival theme
Mobile phones are used by an enormous number of young people as part of their social and cultural life. They are relatively cheap and easy to use. They can be used as a hook to draw in non-traditional learners, particularly those in the 1624 age bracket. However,
Colley, Stead
45
6.
Learners working together
beneficial experience for even the most resistant learner.
Although the use of a mobile or palmtop computer might seem, on the surface, to be an extremely isolated, individualistic activity, research has shown that young people can communicate with each other quite successfully in the process of using the devices. Learning itself can be enhanced by peer collaboration, as suggested by theorists such as Vygotsky (1978). Educational research into situated learning has also pointed out the importance of giving learning a context. In the situated learning approach, knowledge and skills are learned in the contexts that reflect how knowledge is obtained and applied in everyday situations (Lave and Wenger 2001). Situated cognition theory conceives of learning as a socio-cultural phenomenon rather than the action of an individual acquiring general information from a decontextualised body of knowledge (Kirschner and Whitson 1997). Thus, materials, especially games (see the work of Prensky 2001), can be developed that are usable by groups as well as by individuals. Mobile devices are a key feature in many activities carried out by young people: making arrangements, passing on information, passing on gifts in the form of jokes or graphics, sharing and comparing ring tones, texting each other using a still developing new language. This desire to work collaboratively and share information has been built into a group activity which encourages learners to develop a virtual map by attaching photos, text and audio clips they have gathered during a mobile activity exploring, for example, their geographical area.
8.
Evaluating progress
Being able to evaluate progress made in learning via a mobile device has its own set of challenges, especially when the devices themselves are used in the evaluation process. The project has looked at which other aspects of the learning process also need to be evaluated (see below). How do mobile devices motivate learners to progress to other learning opportunities? Do materials presented on mobile devices attract a different type of learner, one who is less likely to go for a more conventional approach? Is it possible to identify changes in attitude to learning when mobile devices are used, and can the devices themselves be used in the evaluation process?
In phase 1 the user trials were very limited. They were conducted in four separate centres and involved the presentation of materials to 34 learners (19 males, 15 females). Limited data was collected through observation of the learners using the equipment, short interviews with the learners and questionnaires that were completed by the learners and their tutors. However, this data was purely exploratory, and used to inform the development of the learning materials rather than for formal evaluation of the learners experiences and possible measurement of learning gains. As such, the findings are not reported further here. For phase 2 we are trialling learning materials with a much larger group of learners, in a formal manner, which will be fully evaluated and the results published accordingly. We are building into the learning materials various tools and back-end systems to allow us to collect data on how the learners use these as well as some feedback on their experiences. Associated with this, we will also be researching different models of learning and the support required by learners, including collaborative learning, individual learning (with or without peer support), online tutoring, blended learning and stand-alone units of learning, etc.
7.
Technologies
We have also reviewed the technologies for which we are developing content, and are now using: SMS text messages M M S multimedia messages, including camera phones V o i c e X M L dialogues over the phone J2ME small games on mobile phones WAP, MiniBrowser a collection of technologies letting you browse websites from small screen devices.
None of these technologies is particularly rich by itself, but we suggest that combined appropriately they can provide an engaging and
46
9.
References
Kirschner D and Whitson J (eds) (1997). Situated cognition: social, semiotic, and psychological perspectives. Mahwah, NJ: Lawrence Erlbaum Associates. Lave J and Wenger E (2001). Situated learning. New York: Cambridge University Press. Prensky M (2001). Digital game-based learning New York: McGraw-Hill. Vygotsky, L (1978). Mind in society. Cambridge MA: Harvard University Press.
Danielsson et al.
47
Participatory design in the development of mobile learning environments

Karin Danielsson, Ulf Hedestig, Maria Juslin and Carl Johan Orre Department of Informatics Ume University, Sweden E-mail: kdson/uhstig/mjuslin/cjorre@informatik.umu.se
Abstract
This paper explores the concept of participatory design in the development of mobile learning environments. The study has approached the design of these environments from a learner-centred design perspective (LCD), since traditional methods in user-centred design (UCD) are too limited to support the development of educational technologies. By combining LCD scenarios with learners knowledge of the use of personal technologies, we propose a new design perspective for learning environments. In our study LCD provided us with a deeper level of contextual understanding of the students interaction in mobile learning settings. Keywords: participatory design, personal technologies, learner-centred design, mobile learning
1. Introduction
Since the late 1990s, there has been increased use of mobile phones among all age groups (Ling and Vaage 2000). A growing interest in the use of mobile technology in education has emerged, and a number of pilot projects have tried to find out how these technologies can be integrated into learning settings (Chen, Myers and Yaron 2002; Roschelle and Pea 2002; Lundby 2002). The increased use of different artefacts in private and public situations makes it necessary to broaden the perspective of the use of these technologies. The concept of personal technologies, coined by Sharples (2000), where mobile technology and other internet
technologies are defined as subsets of technology used in private and public situations, provides a promising approach. This concept helps to illuminate both the structures of interaction and the relations between technologies and the situations in which they are used. This concept also stresses the importance of reconsidering the design models for learning environments. Traditionally, the development of learning environments is characterised by a teacher-centred perspective (Carroll et al. 2002). The models are designed according to rules imposed by teacher practice or technical constraints. This discussion implies a shift to how we can include a learner-centred perspective. Such a perspective would focus on design of learning environments that support an understanding of the social context of learning and capitalise on the use of personal technologies based on how students interact and communicate. This paper presents an example of how to approach the problem so that user-centred design (UCD) is used to inform the design and to gain an understanding of the mechanisms of the students learning context. A model that can be aligned to the UCD approach is participatory design (PD) (Carroll et al. 1998). However, both UCD and PD have had limited impact on the design of educational technologies. One reason for this is that learning settings have not yet been able to develop strategies for technology use and pedagogical models for such use.
2. Towards learner-centred design

According to PD, users are seen as experts in a specific context of development and,
48
therefore, PD involves users and designers working together during an extended period. The advantages of PD are that users and designers may exchange perspectives; learning about each others skills and values, and jointly identifying appropriating sets of requirements (Carroll et al. 1998; Ehn 1988; Kensing and Munk-Madsen 1995; Kyng 1994). An alternative to UCD is learner-centred design (LCD). This approach moves beyond usability issues to the challenge of developing computer systems that support people in a learning environment, ie in developing expertise in work practices that are new and unknown (Soloway et al. 1996). It was developed as an alternative to the UCD framework (Norman and Draper 1986) and addresses some of the problems not supported by UCD. An LCD perspective sees the participants as heterogenous learners, and not as homogenous users within a work context. The UCD perspective addresses the needs of particular users with expertise in the target work practice, and the main goal is to develop tools that support that work practice. Within LCD, the goal is instead to help learners (novices in a given work practice) to learn new practices. The unique characteristics in LCD identified by Soloway et al. (1996) and further developed by Quintana et al. (2002) are listed below. Growth: The development of expertise must be the primary goal of educational software. It supports the learner to learn by doing rather than just do tasks, eg be more efficient (as in UCD). Diversity: Learners are heterogeneous in contrast to users within UCD who can be seen as homogenous. Users within UCD share a common work culture but the learners might not share a common culture or level of expertise in the work practice. Motivation: The learners motivation and engagement cannot be taken for granted throughout the design process, in contrast to professionals who, by the nature of their involvement with their work, have an intrinsic incentive to contribute.
novices in their work practice and the tools to be developed are not explicitly to support task performance. Instead, they need to address the learners lack of experience and support them while they engage in the new learning practice, to reduce the risk of a misunderstood or incorrect work model. According to Soloway and Pryor (1996) learners can be characterised as follows:
Learners do not possess a significant amount of expertise in the work practice. Learners (for example, the business student or the new consultant) do not share an understanding of the activities, terminology, and so forth of the work practice with their professional counterparts (like financial analyst or the corporate manager). More specifically, learners have an incomplete or nave mental model of the work they are trying to perform. It may be the case that learners have an empty model of the work practice, not having any idea about what is involved in the work. It may also be the case (and more often than not) that learners have some model of the work practice, albeit a misunderstood or incorrect work model. Regardless, Learner-Centred tools need to take this lack of experience into consideration and address the corresponding learner needs. This way, learners can engage in the work to form a more correct and appropriate model of the work practice that they are engaging in. Quintana et al. 2002, page 607
2.1. The learner: expert or novice?

As computer users in UCD are considered to have expertise in their work practice and mainly need tools developed to support their implementation of work, they contribute to the design process with their expertise and knowledge of work and task performance. The learners knowledge of personal technologies qualifies them as experts in their use of technology but, on the other hand, they are
The LCD definition coined by Quintana et al. (2002) is considered in three different dimensions. First, the identification of the audience addressed by LCD learners; second, the LCD problem which is described as the conceptual gap between the learner and their work; and third, the underlying approach taken by LCD to address the central design problem. Rather than addressing the gulf between execution and evaluation (Norman 1989, pp4952), which is central to traditional user-centred design, the issue is rather the conceptual gap between the learner and their work. The gap can be described as a gulf of expertise between the learner and the model of expertise, embodied by an expert in the work practice (Quintana et al. 2002). This identifies the distance between the novice, who arrives in a new community of practice, and the full participants in that particular practice. To be able to carry out their daily practice, the learner needs to develop an appropriate and correct conceptual model of the work involved.
2.2. Towards a conceptual model the use of scenarios

One approach to applying LCD is to use scenarios to encourage learners participation
Danielsson et al.
49
and to explore and explain their behaviour. Scenarios can be used throughout the development, as it is an iterative process. The assumption is that designers will begin at a fairly coarse-grained level, adding more detail as they better understand the users needs and the opportunities offered by the system (Kyng 1994; Winograd 2002). Designers may choose to work only with scenarios when reasoning about, and elaborating on, a systems functionality. However, they may also elect to make their reasoning more explicit by analysing the design rationale claims associated with user scenarios (Carroll et al. 1994). According to Quintana et al. (2002), scenarios can help us analyse and articulate the overall structure and components of work practice to be mediated to learners. The use of scenarios can promote an exploration of future work context and the development of a conceptual model among learners. Thus, learner-centred tools should embody the work culture in a manner that the learner can understand. Through observation, the designer can develop the experience and expertise needed to shape a work model of the target work practice. This model needs to be conceptualised in a way that the learner can understand. By using work experts (professionals), to create a good work model, and educational experts (such as teachers and educational researchers) to support the communication with the learners and to guide the learners in their learning process, the LCD team might promote the shift from learner to expert (Quintana et al. 2002). In our study, we have adopted a LCD perspective, which involves learner involvement through various techniques and workshops, such as interviews and development of scenarios. Our application of PD strives towards a sequential development aiming to give designers a structure for handling the complex interdependency between learning, users and tools.
assistants (PDA) and mobile phones. Thus, our aim has been to understand the existing communication patterns and use of mobile artefacts among undergraduate students, and to use this knowledge to inform the design of learning environments for distance and decentralised education. Our point of departure has been a bottom-up approach that views learning from a students perspective and so gets close to how students handle their everyday life and studies (Hedestig et al. 2002). The third phase, discussed in this paper, contains a user-centred approach to the design of a learning environment, which gives us the advantages of making a distinction between the design of new educational technologies and the design of learning environments. Instead of a new artefact design, we propose to use existing artefacts. To accomplish this we have applied two different design methods, ie PD and LCD. The students in this study have freely participated in the project and consist of 24 offcampus learners in northern Sweden. They have good or extensive experience of the use of personal technologies in private and more public situations. In the preliminary design process, we took into account the difficulty of retaining the learners motivation and engagement throughout the design process (Soloway et al. 1994). Thus, previous meetings with the students have illuminated and investigated the students intrinsic motivation to participate and contribute. The students stated that their motivation depended on their present knowledge of technology and future work practice. Our design team consisted of work experts (to conceptualise the work model to the learners), and educational experts (teachers and educational researchers). The study applied ethnography as the main data collection method. We have had continuous meetings with the students and during the meetings project participants have taken field notes. The observations have been complemented with video and audio recordings. Furthermore, all field notes and recordings have been transcribed. Each group has been analysed and compared, with the aim of identifying key passages in the empirical material.
3. The object of study and method

The data presented in this paper is generated from a longitudinal case study. Since 2000 our work has been directed towards the understanding of mobile technologies and their use in the social context of distance learning. The study can be divided into three stages. The first phase of the study focused on the communications patterns among the learners (Hedestig and Orre 2002). The second phase of the study focused on the learners exploration and practice of personal technologies, such as personal digital
4. Scenarios in practice
This case study was conducted in spring 2003, with students participating in a decentralised educational programme 140km from the campus. Eight researchers and
50
teachers participated from the Department of Informatics, together with 24 students studying informatics. The main activities in the spring concentrated on user meetings and the construction of scenarios.
4.1 Designing scenarios

PD scenarios can be used to characterise workflow and breakdowns, and can be used as conversational props in userdeveloper workshops. With scenarios, users do not need to understand the underlying design model or the implementation to provide highly specific change requests. In the work with scenarios, designers have the opportunity to articulate and confront their design suggestions and models with real users, and so test and evaluate their relevance for the specific context. Humanverbal interactions consist largely of exchanging stories (Carroll et al. 1998). The use of scenarios can give opportunities to generate a vocabulary that includes both a description of current use of technology in a learning situation, and future use based on the integration of personal and educational technologies. The objective in our study was to accumulate learners attitudes to the learning practice and possible future settings. The scenarios were characterised as a catalyst, and generated and developed visions of how new software could affect their studying and learning practice. Our project started with an opening session that explained the idea of scenarios as a technique for requirements analysis in system design. Thereafter we scheduled four usermeetings with the students, approximately once each month. In each session, two scenarios were discussed, with narratives that focused either on individual use of personal technology or on collaboration in a student group or class. Both scenarios highlighted learners reflections about the individual learning setting combined with their private situation, and coordination, cooperation and collaboration within the group. In each session, we divided the class into small groups of three to six members. In the first session, the groups were divided into sets of six students and thereafter we carried out the meetings in smaller sets of three to four members. The size of the groups was designed to capture the participants' experiences and reflections both as a member of a larger group and as a legitimate member of a small group. The length of the session was approximately three hours and was conducted in the afternoon, after the students ordinary lecture time. The students usually discussed each scenario for one hour. One member from the project team was present in each group although their role was only to clarify if there was a
misunderstanding connected with the scenarios. After the group discussion, a summary was made with the whole group. This gave us the opportunity to compare similarities and differences within the groups and to incorporate reflections from the complete group. These discussions also contained an evaluation of the use of the scenarios.
4.2 User scenarios
interaction
and
vision
The four user meetings had different perspectives on the learning settings. The first session included scenarios that concerned personal information and communication technologies (ICT) within a learning context. Our aim was to capture the learners current communication behaviour and their use of technology, and compare those to a previous analysis of their communications patterns. These scenarios followed the key ideas of UCD, namely the construction of user interaction scenarios, and a narrative description of what people do and experience as they try to make use of computer systems and applications. User interaction scenarios can also be used as a medium for representing, analysing and planning how a computer system might affect its users activities and experiences (Carroll 1997). During the first session, the learners reflected on present performance in relation to possible interactions through new technology. Much of the discussion concentrated on their recognition of the cases and willingness to adjust to the new circumstances presented in the scenarios. Remarks such as I do not recognise myself in this scenario and Ah that is just you [Lars]! were common as were comments such as: This sounds really great, I would like to do that and This is not how we do things, we might, but then I see problems. A great part of the discussion focused on the interaction possibilities of the technology, eg notification, network, size of technology, etc: Here it would be great to get both sound and colour notification, sound to hear and colour to find the change made or To be able to check this on the bus back home, would give [me] more freedom. Thus, some learners did not recognise the potential of the technology to the same extent as their fellow participants; for them, the availability of face-to-face meetings was the ultimate solution. This gave us indications of their willingness to use new technology in their present learning setting. Specific design suggestions were also given, based on their knowledge of technology. The second session included v i s i o n scenarios that focused on the future learning
Danielsson et al.
51
setting in relation to less specific technologies. Vision scenarios can be used as a guide for development with a focus on needs and opportunities as a means to convey and develop visions of how new technology could affect interaction. Design proposals for the future system can be discussed and reflected on in an early stage of collaboration between designers and users, eg feasibility and effectiveness issues are discussed (Carroll et al. 2 0 0 2 ) . Educational experts and teachers within the project team developed the scenarios used in the second session. Our interest in vision scenarios was based on their impact on the learners' future learning practice. Here the ability of the learners to combine previous and present discussions in the scenario sessions was obvious. The core element in their discussions was the future learning setting in relation to the present. The learners reflected on their present group activities, problems and opportunities. They made interesting points about their present learning setting, both as a group and individuals. We would meet up at campus, wouldnt we? and I think I would study at home and then contact you through [some] technology if I had any questions were just two of many examples. The students also discussed ways of interacting within study groups and between learner and teacher. Typical comments were Its quite easy to share information by using e-mail and E-mail isnt useful when you need to mediate difficult information, like Java code. Even where they were used to employing a specific support, they were open to something else: It is always fun to try another learning practice. They recognised their learning practice in the narrative presented in the scenario and were therefore able to reflect on shortcomings and visions for their learning. It might happen that it is more effective learning in a bigger group, A great thing would be if communication could take place in an online forum as we comment in earlier session and A forum would open up for the possibilities for the teacher to communicate knowledge to the whole group. This illuminates their reflections on present study techniques, and their willingness to accept or dismiss the new suggestions presented by teachers in the scenarios. The third session included both a userinteraction scenario and a vision scenario. The user-interaction scenario contained more functions and communication possibilities, integrated in a new, more recently developed technology. The aim was to capture the learners first impression, and their thoughts and possible experiences, when they were presented with a new setting. As these learners will be presented
later with the technology developed within the project, an interesting aspect has been to compare their first impression from the narrative presentation with their actual, physical use of the learning system. The second scenario was a vision scenario developed to capture the learners reflections on their long-term use of the technology, in a possible future learning setting. In the third session, participants were encouraged to use a matrix presenting the individual in relation to their group, their class and their teacher, and the diversities of personal and public information (see Figure 1).
Individual MP3, Media, Address book Group Share notes, address book Class Information on class website, address book Send and receive materials from course
Private
Public
Teacher
Feedback from teacher
Feedback from teacher
Figure 1: The matrix used in the third session. This matrix contains example information from one of the groups.
During the third session students appeared to make great use of the matrix presented to them. Discussions happened according to the categorisations: Should we start by choosing category? PrivatePublic, is that one OK? Also, during the third session it was noteworthy that the students first described all the features in the first scenario as interesting, and discussed possible use situations. There was a high level of acceptance. Later, when they discussed the second scenario, they hesitated over some of the features, for example a camera, to use in a learning setting. They compared some of the features to games interesting, but with shifting use over time. Other features were seen as beneficial, and compared to those features already adopted from previous phases of the project such as the calendar. Interestingly, during all the sessions remarks were made about benefits they found which might not suit them but might help someone else. I dont need this, but I think its needed for NN, so why not! or This would actually be advantageous for the teachers as well are two examples. The fourth session has not yet been conducted but, according to our plans, we will present a mock-up prototype based on results from the previous sessions described above.
Group
MP3, Media etc. Chosen private information
Share pictures
52
5. Discussion
Using the scenarios in PD gave us the chance to capture the learners thoughts, views, and behaviours in a mobile learning setting. Using interviews and scenarios we integrated their reflections into our conceptual model of their present and future learning setting. Learners participation in the design process promoted their exploration of their future work context and the development of a conceptual model. The sessions gave them the chance to connect their present learning setting to possible future settings. They could reflect on their present performance, benefits and problems connected to their strategies to achieve knowledge, and they had the opportunity to shape new ways of performance jointly. Any changes in the scenarios were based on their remarks, and supported the development of their conceptual model of a future learning setting. An LCD approach guides the learners in making the shift from learner to expert, eg to bridge the conceptual gap (gulf of expertise learner/model of expertise expert)). In our case, did the scenario sessions promote such bridging but also question and shape our (expert) conceptual model? The learners, with their reflection, gave us insights into whether or not our assumptions were correct. The mediation of the conceptual model was a bridging from two directions, rather than a oneway street. The difficulty of measuring motivation is an aspect of concern in LCD. As previously stated, learners motivation and engagement cannot be taken for granted throughout the whole design process. The incentive to participate in our case was, according to the learners, their interest, and knowledge of personal technology. Many of their discussions concerned the interaction possibilities of technology, eg notification, network, size of technology, other choices of resemblance, etc. However, it was not only the interest in technology that affected their motivation; using scenarios gave them an insight into their contributions, as the scenarios were used as an iterative process and so they could identify all the changes made from their previous remarks. Recognising themselves in the narrative scenarios could also be a motivational factor, as it implies that they are at the centre of interest. The use of the scenarios revealed the heterogeneous character of the learners, not only to us as designers, but also to themselves. This was illuminated by their identification of use situations and their concretisation of possible solutions from the scenarios that could benefit both themselves as individuals and the group as
a whole. The participants acknowledged the value of the scenarios as the foundation for discussion and reflection. The iterative process of the scenarios also gave them the chance to make connections between different discussions and include features or situations excluded in later sessions. Mostly they saw scenarios as a tool to structure their discussions so that they did not drift from a given subject. The students knowledge of personal technologies has had an impact on the construction of the scenarios. The knowledge of their everyday practice of interaction provides design proposals that would not have been accomplished without their participation. Instead of following traditional guidelines while designing learning environments, we were able to reconsider the very foundation of using ICT tools in learning environments.
6. Conclusions
Our study shows a prominent difference between the traditional UCD approach and LCD. First, users within UCD are seen as homogenous, while learners within LCD are seen as a heterogeneous group. The learners might not share a common culture or level of expertise in the work practice and this diversity must be taken into account in the development process. Second, within LCD the goal is to help learners (novices in a given work practice) learn new practice. Thus the primary goal when we develop educational software should be to support the learning process rather than a predisposed process to accomplish a specific task as articulated by UCD. Third, the LCD approach emphasises the need to integrate private use of personal technologies into public learning settings. Therefore, we extended the focus on technology use, not only to a formal domain such as work or learning but also to informal communication patterns. Finally, our studies show that the students knowledge of personal technologies defines them as expert users, and that their participation reveals concealed use of personal technologies in the learning environment. This knowledge levels out the relation between learners and designers, and stresses a deeper level of user involvement in the design of mobile learning environments.
Acknowledgment
This project is funded by the National Agency of Higher Education and Vinnova.
Danielsson et al.
53
References
Carroll JM, Chin G, Rosson MB, Neale DC (2002). The development of cooperation: five years of participatory design in the virtual school. In JM Carroll (ed) Humancomputer interaction in the new millennium. New York: ACM Press, Addison-Wesley, 373395. Carroll JM, Rosson MB, Chin Jr G, Koenneman J (1998). Requirements developments in scenario-based design. IEEE Transactions of Software Engineering, 24(12), 11561170. Carroll JM, Mack RL, Robertson SP, Rosson MB (1994). Binding objects to scenarios of use. International Journal of HumanComputer Studies (formerly International Journal of ManMachine Studies), 41, 243276. Carroll JM (1997). Scenario-based design. Hellander, MG, Landower, TK, Prabhu PV (eds) Handbook of HumanComputer I n t e r a c t i o n , Amsterdam, Netherlands: Elsevier Science BV, 383406, second edition. Chen F, Myers B, Yaron D (2002). U s i n g handheld devices for tests in classes. HumanComputer Interaction Institute, School of Computer Science, Carnegie Mellon, University, PA. CMU-HCI-00-101. Ehn P (1988). Work-oriented design of computer artefacts. Stockholm: Arbetslivscentrum. Hedestig U, Orre CJ (2002). Personal technologies and development of learning environments. In Flckiger F et al. Proceedings of 4th International Conference on New Educational Environments, Lugano, Switzerland. Hedestig U, Kaptelinin V, Orre CJ (2002). Supporting decentralized education with personal technologies. In Proceedings of World Conference on E-learning in Corporate, Government, Healthcare, & Higher Education, E-LEARN 2002, Montreal. Kensing F , Munk-Madsen A (1995). Participatory design: structure in the toolbox. Datalogiske Skrifter, 56. Denmark: Roskilde University. Kyng M (1994). Scandinavian design: users in product development, Proceedings of the SIGCHI conference on Human Factors in Computing Systems. Boston, Massachusetts: ACM Press, 3-9. Ling R, Vaage OF (2000). Internett og mobiltelefon ikke lenger bare for de f. Samfunnspeilet, 6. Statistisk sentralbyr. Lundby K (ed) (2002). Knowmobile: knowledge access in distributed training: mobile opportunities for medical students. InterMedia, 5, University of Oslo. Norman D (1989). The design of everyday things. London: MIT Press.
Norman D, Draper S (1986). User-centered system design. Hillsdale, NJ: L Erlbaum & Associates. Quintana C, Carra, A, Krajcik J, Soloway E (2002). Learner-centered design: reflections and new directions. In JM Carroll (ed), Humancomputer interaction in the new millennium. New York: ACM Press, AddisonWesley, 605626. Roschelle J, Pea R (2002). A walk on the WILD side: how wireless handhelds may change CSCL. In G Stahl (ed) Proceedings of Computer Support for Collaborative L e a r n i n g . Hillsdale, NJ: L Erlbaum & Associates. Sharples M (2000). The design of personal mobile technologies for lifelong learning. Computer & Education, 34, 177193. Soloway E, Guzdial M, Hay KE (1994). Learnercentered design: the challenge for HCI. In st 21 Century Interactions, 1(2), 3648. Soloway E, Pryor A (1996). The next generation. In HumanComputer Interaction, Communications of the ACM, 39(4). Winograd T (2002). Interaction Spaces for Twenty-First-Century Computing. In JM Carroll (ed), Humancomputer interaction in the new millennium. New York: ACM Press, Addison-Wesley, 259279.
Dawabi et al.
55
Using mobile devices for the classroom of the future

Peter Dawabi, Martin Wessner, Erich Neuhold Fraunhofer Integrated Publication and Information Systems Institute (IPSI) Dolivostr. 15, 64293 Darmstadt E-mail: {dawabi, wessner, neuhold}@ipsi.fraunhofer.de Abstract
Face-to-face learning scenarios are characterised by rich forms of cooperation including natural speech, gesture and other visual communication. However, with a growing number of participants the cooperation needs to be coordinated and the individual participation in the cooperation decreases. Using mobile devices like personal digital assistants (PDAs), interactivity and cooperation in such scenarios can be enhanced. But the design of such a technical system must also carefully preserve the traditional advantages of face-to-face scenarios. In this paper, we present an approach to integrating mobile devices into face-to-face learning scenarios which combines the advantages and benefits of both. We describe ConcertStudeo, a platform that implements our approach. ConcertStudeo provides tools for interactions, such as brainstorming, a quiz, voting and others, by using wirelessly connected PDAs in combination with an electronic blackboard. We sketch the current implementation of ConcertStudeo and report some experiences with using the system in a university course. Face-to-face learning offers many advantages and is, in comparison to individual or distributed learning, accompanied by richer natural communication, including gestures, mimicry, body language and so on. But as the number of participants increases the efficiency and quality of communication can get worse. This can lead to fewer contributions per participant and can lower the learning results. The following approach, based on the ConcertStudeo research project carried out at Fraunhofer IPSI (Darmstadt, Germany), aims to improve face-to-face learning by supporting and enhancing interaction and cooperation between the tutor and the students.
2 Requirements for face-to-face learning support

As mentioned above, the advantages of face-to-face learning are the wide range of possibilities it provides for direct communication. The goals of ConcertStudeo are to keep as many of the advantages of direct communication as possible and to provide additional functionality to enhance interactivity and cooperation in the classroom (compare to Roschelle and Pea 2002a, 2002b). In traditional learning scenarios, for example, a blackboard or an overhead projector is used for the presentation of the learning material. If there are questions or unclear points each student can immediately ask the teacher and receive an answer verbally, as a written text or as a sketch on the board. Vice versa, a teacher can test knowledge by asking a question, which has to be answered by one or more students. In most classroom scenarios each student can express his or her opinion or contribute an idea spontaneously, even without being asked a question.
Keywords: interaction and cooperation support, computer-supported collaborative learning (CSCL), face-to-face learning, PDA
Introduction
The quality of e-learning scenarios varies in many ways including synchronicity, local distribution, group size and efficiency. In this paper we concentrate on face-to-face learning scenarios, such as workshops and classroom sessions, which are still the most common way of educating people.
56
The front-oriented presentation on the board provides a common focus and direct awareness of the current activities. With a traditional blackboard the presentation of the learning material can happen on demand and can be interrupted at any time. Using blackboards causes almost no noise or other distraction. It can provide supplementing information to the students without replacing existing communication channels like speech or gestures. Its use is an e n r i c h m e n t in communication rather than a replacement. Similar gains should be aimed at with a faceto-face learning support system. In practical terms this means a system for supporting faceto-face learning that is comparable to traditional scenarios with respect to size, noise, usability and the potential to integrate personal learning material. To display a similar amount of information, an electronic support system should ideally have a display area the same sort of size as a traditional blackboard. Furthermore, it should not cause too much distraction by noise, and its use should ideally be as intuitive as a blackboard. It should also allow the integration and navigation of personal learning material and support a process to generate or change it dynamically.
3 Interaction and cooperation design

A system for face-to-face learning support should keep the advantages of traditional faceto-face learning scenarios and provide additional benefits. Potential extra benefits include structuring the learning process, increasing the participation of the students, evaluating the students interactions, and integrating the interaction with the learning material. The learning process can be structured by assigning roles to individuals and by dividing the process into a sequence of phases. Student participation can be increased by providing anonymity (eg in anonymous voting), by providing interactive interfaces (eg in a single-choice quiz the students can select only one answer), and by involving all students in parallel, using one device per student. The system can accelerate the evaluation of an interaction by automatic aggregation of the students contributions (eg a histogram for alternatives, an overlay for sketches, statistical values for numbers, etc). Interaction can be initiated by the trainer spontaneously or, if it is logically bound to a specific part of the learning material, it can be predefined and anchored in the learning material. The result of an interaction (eg a structured collection of ideas as the outcome of
brainstorming) can be linked to the existing learning material, mirroring the specific learning process of a given class. The design of an interaction type and consequently the design of an interaction tool involves tackling a range of questions concerning the above-mentioned potential benefits. In the following we will use brainstorming as an example for an interaction in face-to-face learning and show how a system can improve the brainstorming. In general, the goal of brainstorming is the creative generation of ideas. In a learning context, brainstorming is used to collect, activate and communicate existing knowledge on a certain subject matter. The brainstorming method is based on a small set of rules: (1) no criticism of ideas, (2) free association, no idea is too silly, (3) all ideas belong to the group as a whole, (4) building on the ideas of others is explicitly encouraged, and (5) generate as many ideas as possible in a short time. Like many other group processes brainstorming suffers from some problems including productivity loss, unequal participation, fear of criticism and non-taskrelated activities (Diehl and Stroebe 1987, 1991). An electronic brainstorming system can address these problems. For example, the system can allow ideas to be entered in parallel and all contributions to be treated anonymously (Valacich et al. 1991). How can the brainstorming process be structured? How can participation be increased? How can the students interactions be evaluated? How can the interaction be integrated with the learning material? The brainstorming process can be split into two phases: a phase for generating and collecting ideas and a phase for structuring the collected ideas into clusters. The system can support different roles and activities in the brainstorm. A person taking the moderator role is allowed to delete and structure ideas in the second phase; a regular participant is allowed to contribute ideas but not to delete ideas and so on. Tightly related to the structuring of roles and activities is the structuring of the display of information in the brainstorming process. In general, information can be displayed on individual displays for individual participants or on a public display for the whole class. To promote building on the ideas of others, the public display as the common focus is used for the display of ideas. The individual displays are used to input the individual ideas in parallel. In traditional brainstorming all ideas have to be noted by a moderator or written cards have to be attached to a pin board. An electronic brainstorming can collect and display all ideas
Dawabi et al.
57
automatically which accelerates the evaluation and further processing of the collected ideas. For future use and reference the results of the brainstorming can be stored as a part of the learning material.
Implementation of ConcertStudeo
The ConcertStudeo platform (Wessner et al. 2003; ConcertStudeo 2003) provides configurable tools for spontaneous and planned interactions such as brainstorming, a quiz, voting or ranking. The system hardware consists of an electronic blackboard, a connected PC or laptop, and a number of PDAs for the students. Our current configuration is based on a SMART Board, a Windows-based laptop and six Pocket PCs (Toshiba e740). The ConcertStudeo software includes the following parts: CS Board, the software that runs on the PC of the electronic blackboard and which is mainly used by the tutor or trainer CS Control, the software that runs on each of the PDAs used by the students CS Server runs on any PC, eg the one running the CS Board software. The CS Board software allows the seamless embedding of arbitrary learning material by integrating the functionality of the Internet Explorer. This way any kind of web-based learning material like HTML files or Powerpoint slides can be integrated and navigated.
intended, ie has been prepared by the tutor in advance (see Figure 1). Alternatively, the tutor can initiate an interaction at any position in the learning material. In this case the parameters for an interaction, for example a question and answers, are not pre-defined but given by the tutor spontaneously. After initiating the interaction tool, the display area of the CS Control switches to the appropriate interaction interface. CS Board displays the collected input or answers of the students and provides dynamic interaction.
Figure 2 Screenshots of a ConcertStudeo quiz. Left: electronic board Right: PDA An example of the automatic synchronisation and information exchange between the CS Board and the CS Control software is presented in Figure 2. The CS Board software on the left displays a quiz interaction with an animated question and the results (so far received) of the students. As already mentioned, CS Board runs on the electronic blackboard and is mainly controlled by the tutor. One of the students PDAs with the synchronised user interface is displayed on the right. It shows the selection options and a Send Answer button. Taking the example we used for discussing the design for face-to-face learning support, we also sketch the brainstorming interaction. In Figure 3, the CS Board software on the right displays the current collection of ideas. On the left the interface of one students PDA is presented. ConcertStudeo is currently implemented as a clientserver architecture: CS Board and CS Control act as clients exchanging data with the CS Server. The CS Board communicates with the CS Server to start an interaction in the class and to display the interactions results. CS Control communicates with the CS Server to display the appropriate user interface according to the current interaction type, and to submit the users input, for example a text string or (multiple choice) selections.
Figure 1 Screenshots of ConcertStudeo learning content For each type of interaction a button is located at the bottom of the display of CS Board. With these buttons the interactions can be initiated when the tutor navigates to a position in the learning material, where an interaction is
58
CS Board and CS Control are part of a wireless LAN (WLAN).
Figure 3 Screenshots of a ConcertStudeo brainstorming. Left: PDA Right: electronic board The network can be set up as a so-called adhoc network based on the built-in WLAN cards of the PDAs and the server PC or the clients can connect to an access point, which connects the WLAN with the outside world. The communication itself is encapsulated as HTTP requests, which allow the server to be located behind firewalls.
Figure 4). The lecturer initiated mainly quizzes and brainstorming. The quizzes served for comprehension tests and to diagnose the students current mental concepts. Brainstorming was used to (re-)activate the students previous knowledge and to collect ideas for solving specific exercises. Voting was used rarely in this case. As no delicate questions arose, it was used as an alternative to the quiz tool when the students had to choose between two options. Real votes, for example on whether a proposed solution is correct, were cast by raising hands. An available video tool was not used because the lecturer did not have any video to be presented. In general, the lecturer and the students had very positive comments on the system; they liked the additional possibilities to design and perform the lecture. All the participants also liked the integration of the learning material and the interactions, ie that the original Powerpoint slides could easily be used with the ConcertStudeo system and that the interaction results are preserved as part of the learning material for further use. Discussions with teachers and trainers from various education and training fields demonstrated their various requirements of the desired interaction forms. To address this, we plan to turn the ConcertStudeo system into a highly configurable system. In such a toolbox, various interaction types could be selected and adapted to the needs of a specific learning context. Another problem for teachers is the limited budget for PDAs in schools. To minimise the costs of PDAs, we are working on reducing the degree of parallelism a bit by sharing PDAs between multiple users in the classroom.
Experiences and outlook
ConcertStudeo was tried out internally at Fraunhofer IPSI and in a university course. The internal evaluation by six researchers focused on the technical stability and the usability of the system.
References
Diehl M, Stroebe W (1987). Productivity loss in brainstorming groups: toward the solution of a riddle. Journal of Personality and Social Psychology, 53, 497509. Diehl M, Stroebe W (1991). Productivity loss in idea-generating groups: tracking down the blocking effect. Journal of Personality and Social Psychology, 61, 392403. Figure 4 The use of ConcertStudeo in a university course In a course on computer-supported cooperative work (CSCW), at the Darmstadt University of Technology, ConcertStudeo was used by eight students and the lecturer in two sessions (see Roschelle J, Pea R (2002a). A walk on the WILD side: How wireless handhelds may change CSCL. Proceedings of the Conference on Computer Supported Collaborative Learning (CSCL) 2002, 5160, Boulder, CO, USA.
Dawabi et al.
59
Roschelle J, Pea R (2002b). To unlock the learning value of wireless mobile devices, understanding coupling. Proceedings of the IEEE international workshop on wireless and mobile technologies in education (WMTE 02), 26, Vxj, Sweden. ConcertStudeo (2003). ConcertStudeo: interaction and cooperation in face-to-face learning. At www.ipsi.fraunhofer.de/concert/projects/studeo Accessed on 04 November 2003. Valacich J, Dennis A, Nunamaker J. (1991). Electronic meeting support: the GroupSystems concept. In Computer-supported cooperative work and groupware. London: Academic Press, 133154. Wessner M, Dawabi P, Fernandez A (2003). Supporting face-to-face learning with handheld devices. In B Wasson, S Ludvigsen and U Hoppe (eds) Designing for change in networked learning environments, Proceedings of the International Conference on Computer Support for Collaborative Learning (CSCL) 2003, 487491, Dordrecht, Kluwer, 2003.
Graham et al.
61
Adaptive navigation for mobile devices

Paul Graham University of Sunderland/ North Tyneside College Embleton Avenue, Wallsend, NE28 9NJ E-mail: Paul.Graham@ntyneside.ac.uk Abstract
This paper describes a wireless and adaptive website developed for use by information and communications technology (ICT) students at a college in the United Kingdom. It was developed so that students can look at course information any time and anywhere. The website uses adaptive navigation support (ANS) techniques usually found within adaptive hypermedia systems (AHS) that personalise the users web experience by highlighting links of interest through direct guidance, hiding irrelevant links and sorting links. Through the use of these techniques, and in particular the use of both user models and a Markov model, it was found that the number of steps used and the time it took users to navigate the site was improved significantly. It was also discovered that the users preferred an adaptive system to a nonadaptive system. Keywords: adaptive hypermedia systems (AHS), adaptive navigation support (ANS), user model, Markov model, wireless, ICT
Chris Bowerman University Of Sunderland St Peters Campus, Sunderland, SR6 0DD

E-mail: Chris.Bowerman@sunderland.ac.uk
Albert Bokma University Of Sunderland St Peters Campus, Sunderland, SR6 0DD

E-mail: Albert.Bokma@sunderland.ac.uk
communications technology (ICT) students at North Tyneside College. According to a recent survey by the Learning and Skills Development Agency (LSDA), 85% of the 1624 year olds interviewed use the internet, 54% are in education, but only 4% of them use the internet on their mobile phones (LSDA 2002). We aim to bring these three areas together and produce a personalised, adaptive website containing course information that can be accessed efficiently anywhere and at any time using a wireless device, such as a mobile phone.
Web personalisation
Introduction
Navigation on mobile devices is cumbersome and time-consuming because of their limited screen size and bandwidth. As a result, it is suggested that methods are needed to facilitate navigation, based on user models that enable users to gain fast and efficient access to relevant information. This paper discusses an adaptive site developed for a mobile device; it personalises the web experience and has been tested on course information for information and
To improve the web experience the prospective users motivation needs to be established and their interests and goals determined. For example, when a student is looking for the information about their assignment, this is referred to as goal-directed browsing. The students behaviour is directed towards a given objective; they are not interested in diversions but only in finding their goal and therefore, personalisation would be used to link directly to the information sought by predicting the user's current goal. Personalisation can potentially be used to overcome problems such as disorientation and information overload by providing relevant content and navigation through application of navigational and presentational techniques. The users experience can be personalised in respect of their knowledge, background and goals, by using adaptive techniques usually found within adaptive hypermedia systems (AHS).
62
Adaptive hypermedia systems
A number of pioneer adaptive hypermedia systems were developed between 1990 and 1996 (Brusilovsky 2003) to overcome problems with standard hypertext systems, which motivated the evolution of adaptive hypermedia systems and hypermedia research (Brusilovsky 1996). This area of research has grown further in the last seven years because of the growth of internet and www-based adaptive systems. AHS bring together ideas from hypermedia, hypertext and intelligent tutoring systems, to enable personalised access to information. The goal of this research is to improve the usability of hypermedia resources in the area of mobile devices. The aims of AHS are to build a model of the goals, preferences, and knowledge of each user, and use this model throughout the interaction with the user, in order to adapt to the needs of that user (Brusilovsky 2001). A classic hypermedia application serves the same pages and the same set of links to all users. This is true even for most applications that are built on top of systems that are capable of presenting different views to different users (De Bra et al. 1999). AHS make it possible to deliver personalised views or versions of a hypermedia document (or hyperdocument for short) without requiring any kind of programming by the author(s) as the AHS do all the adaptation automatically, simply by observing the browsing behaviour of the user. There are many adaptive systems that allow personalised views based on user-selected stereotypes like beginner, intermediate and expert, or based on interface and style preferences. Quentin-Baxter and Dewhurst (1999) deal with this issue in an educational context. The crucial differences with AHS are that in these systems the adaptation uses a much more fine-grained user model, and that the adaptation is achieved automatically instead of being selected by the user. A number of systems have been developed and the application areas for these systems range from educational hypermedia to information retrieval systems with a hypertext interface. Various research groups have developed different techniques to adapt aspects of hypermedia systems to the individual characteristics of a user. Brusilovsky (1996) gives a comprehensive review of adaptive hypermedia techniques and systems. These systems can offer the authors of hypertext flexible means to present information with resulting systems that give users greater navigational freedom.
Adaptation is a powerful way of augmenting the functionality of a hypertext system. There are two main areas that can be adapted within a system, namely the links and the information within each page (node). Adaptation of links affects navigation within the system and adaptation of nodes affects what and how the information is presented to the user. The two techniques usually used in AHS are adaptive navigation and adaptive presentation (Brusilovsky 1996), and they are described in the next two sections.
Adaptive navigation support
Navigational adaptation aims to help users find their goal without getting disorientated. It does this by adapting the navigation (links) to the goals, knowledge and other characteristics of an individual user. Adaptive navigation support can guide the user both directly and indirectly and can work with large amounts of material using simple user models. There are several ways to adapt links and the most popular adaptive navigation support (ANS) techniques are described below. Direct guidance: A next or continue link is usually shown in adaptive hypermedia systems where the destination of this link is the page (node) that the system determines to be most appropriate. Sorting links: An adaptive system that uses link sorting, displays a list of links presented in an order from most relevant to least relevant. This technique is usually found in goal-oriented educational systems (Hohl et al. 1996). Link hiding: Links leading to inappropriate or irrelevant information are hidden from the user. Link annotation: Link anchors are presented differently depending on the relevance of the destination; see systems such as ELM-ART (Brusilovsky and Weber 1996) and Interbook (Brusilovsky et al. 1998) that use coloured dots as annotations. Link disabling: This technique disables inappropriate links; whether the link anchor is visible depends on the combination of this technique with link annotation or link hiding. Link removal: Appropriate links (and anchors) are simply removed; this works well in lists, but removing the anchor text does not work for running text. ISIS-Tutor (Brusilovsky and Pesin 1998) uses link removal. Map adaptation: Some hypermedia systems provide a graphical presentation of (part of) the link structure (Mukherjea 1999;
Graham et al.
63
Benford et al. 1999); these can also be adapted. Another common set of AHS techniques is known as presentational adaptation. Here the content is changed, rather than the links.
represented in a frame, as found within Hypadapter (Hohl et al. 1996). Slots of a frame can contain several explanation variants of the concept, links to other frames, examples, and so on. In the next section the WANTIT system and the AHS techniques it uses specifically to adapt the information within the system are discussed. The way in which the system builds a model of the goals, preferences and knowledge of each individual user and the way it uses this model throughout the interaction with the user are also discussed.
Presentational adaptation
The aim of presentational adaptation is to hide information that is of no use to the user, as it may be irrelevant or too advanced. The following techniques are used to accomplish presentational adaptation. Conditional text: The conditional text technique used in ITEM/IP (Brusilovsky 1992) and C-Book (Kay and Kummerfeld 1994) divides all possible information about a concept into sections, known as chunks. Each chunk is set a condition indicating what type of user should be presented with that specific chunk of information. For example, expert and novice users may be presented with different chunks for the same concept. Stretchtext: This is a technique suggested in MetaDoc (Boyle and Encarnacion 1994) and used in KN-AHS (Kobsa et al. 1994). In regular hypertext, activation of a hot word results in moving to another page with related text, but when using stretchtext this related text simply replaces the activated hot word, extending the text of the current page. For example, a novice user with poor knowledge of a concept will always get additional explanations of this concept. Page variants: This technique can be found in Anatom-Tutor (Beaumont 1994) and CBook (Kay and Kummerfeld 1994). With this technique the system keeps several variations of the information presented, but keeps it in different ways. It could be stored as a different style or level and the system selects the most appropriate page for the user. Each variant is prepared for each type of stereotypical user. For example, a novice will be shown the information in one format, and an expert user in a different format, based on the users previous experiences and interests. Fragment variants: A good example is Anatom-Tutor (Beaumont 1994) where a page is broken into a number of fragments and a number of variants of each fragment are prepared. The system stores several different explanations and the user is shown the page corresponding to their knowledge of the concepts presented in the page. F r a m e - b a s e d : In this technique, the information about a particular concept is
6 The WANTIT system

The system developed is known as the wireless and adaptive navigation site to help IT college students (WANTIT). The WANTIT system holds information on an ICT course taught at North Tyneside College, in the United Kingdom. The system is designed to enable students to access course information, such as class notes, assignments, links to important information and news, and is viewed through a mobile (cell) phone. WANTIT uses typical AHS techniques to present its users with appropriate information. The adaptive system creates three sets of lists that the system feels are the most appropriate for that user and displays the links in a suggestion list. This suggestion list is found once the user logs into the system and can be viewed at any time while the user is using the system. The suggestion list is split into three separate areas. These areas are: 'popular links', 'predicted links' and 'previous links' and are created by using a combination of three adaptive navigation support (ANS) techniques. The three techniques are direct guidance, link sorting and link hiding which are used to display links within the suggestion list, which the system deems to be the most relevant information to the user. The suggestion list is split like this because of the limited space on the screen; it saves the user time not having to scroll through the list to find what link they are looking for. The three sections of links found within the suggestion list are ordered with the top link using the direct guidance technique. However, in the WANTIT system the next link is replaced with the name of the page that the system feels is the appropriate next link. The links in all three sections are ordered using the sorting technique; they also apply a user model and a Markov model (Cormack and Horspool 1987) to predict which links are most
64
and least relevant. Both the user model and the Markov model will be discussed in Sections 7 and 8 and the results from using these models are displayed in the suggestion list. The top three links in the suggestion list are shown to the user, while the other links that the system predicts are not so relevant are hidden. Since the system is optimised for use on a mobile phone, which has a limited screen size only allowing four lines of text to be viewed, it is not possible to use navigation techniques such as link annotation, link disabling or map adaptation. The adaptation of the content (presentation adaptation) and the techniques associated with it, such as stretchtext, are not used within the adaptive system because of the lack of space on the screen. Other presentational techniques were not used, as the adaptive system has only one set of content for each user and if the user is simply not ready to access certain content, they are not given a link to it. Within the system, a user model and a Markov model are used to determine which links to manipulate and to guide users towards interesting and relevant information. These topics are discussed in the following sections.
example, if a user with a user model identified as an intermediate came to a part of the system they had not been to before, the system could not predict their navigational behaviour. Since the user would not have any previous knowledge of that part of the system within their user model, the system would look at users with a similar user model (intermediate). These user models would be used along with a Markov model to produce a suggestion list of the predicted links that similar users used when they were at that section of the system.
The Markov model
The Markov model is derived from Markov chains, which are sequences of random variables, in which the future variable is determined based on probability. A Markov model contains a single variable, the state, and specifies the probability of each state and of transiting from one state to another (Anderson et al. 2002). An example of how a Markov model works is discussed below. Suppose we have five pages, we will use a 5 by 5 transition matrix, as seen in Figure 1. (col j) 1 (row i) 1 2 3 4 5 2 3 4 5
User modelling
The system maintains a user model for each of its users and stores information such as knowledge, preferences, background and experience about the user, by keeping attributes such as Has the user read this page? through observing and recording their actions while they are browsing the site. The information is stored in one of three stereotypical user models: beginner, intermediate or expert. This model is used to build a model of the goals and preferences of each student, throughout the interaction, to adapt it to their needs. If a user is new to the system they use a stereotypical beginners user model. As the user continues to use the system, all their browsing information is recorded so that their user model slowly evolves from a stereotypical user model to a unique user model only used by that particular user. The user model develops from being classed as a beginner through intermediate to expert as the user progressively works through the system and views more and more information. The stereotypical user models are trained by observing the users browsing patterns so that if a visitor is new to a particular part of the system, a reliable prediction about their future navigation can be made. This is done by looking at a user model of a similar type of user (either beginner, intermediate or expert model) who has visited that section of the system in the past. For
Figure 1: Sample Markov model The number in row i and column j are the probabilities that you go to page j next given that you are in page i now. For each row you need to store the total number (n) of times that this page has been visited and the numbers n1, n2 n of times that pages 1, 2 n have been visited next. At any time, the probabilities that we use for transition to page j are (in the simplest case):
Using this model, the students goal is predicted and shown within the suggestion list found within the WANTIT system. The links are predicted by looking at the user model of the current user and defining where they currently are situated within the system or by looking at the last page they viewed if they have just entered the system. The probability of the future next page (node) is then calculated in relation to where the user is currently situated within the system. The higher the probability, the more confident the system feels that the user will
Graham et al.
65
decide to follow that path and visit that predicted page (node) next. Figure 2 below illustrates how the list is dynamically produced.
Figure 2 How the list is produced By looking at Figure 2 we can describe how the WANTIT system works to produce the suggestion list. Since the list is split into three sections we shall discuss each section separately. The 'predicted links' will be derived by looking at the type of user model the current user is identified as using and the current page of the system they are viewing. The WANTIT system will use the Markov model to predict the list of links the current user will want to view, by looking at the current page the user is viewing and predict popular patterns that previous users had followed. The list is ordered with the highest probability at the top and the lowest at the bottom. The one with the highest probability is the direct guidance link and the links found after the first four shown are hidden because of the lack of screen space. The titles of the pages are displayed with the suggestion list, as illustrated in Figure 3.
The 'popular links' section will be predicted by using the Markov model to suggest the list of links the current user will want to view, by calculating the most viewed pages, by using the last pages-visited model that users with a similar user model have previously viewed. Again, the list is ordered with the highest probability at the top and the lowest at the bottom. The one with the highest probability is the direct link and the links found after the first four shown are hidden. Finally, the 'previous links' section will not use any predictions and therefore the Markov model is not used for this section. To create this list of links the last four links stored in the user model that the user viewed, will be displayed on the screen, with the latest viewed link at the top and the oldest at the bottom. This section is used in case a user would like to carry on using the system from where they had logged out of their last session, when they had used the system.
Hybrid system
As discussed by Perkowitz and Etzioni (2000), automated approaches may not always correctly predict the user's goal. They consider that the best approach, as used in our system, is a hybrid system where personalisations are generated automatically and the web author optionally provides guidance to enable the correct model to be used. The hybrid system can be useful when the models have to be slightly changed. For example, where an assignment is given out and the tutor would like it to be viewed by all the students as soon as possible. Using a hybrid system, the popularity of the page can be changed so that it is displayed in the suggestion list even if it has not been viewed before.
10
Testing and results
Figure 3 The suggestion list
An experiment was conducted to test whether an adaptive system is better than a non-adaptive system for use on a mobile phone. Subjecting the users alternately to two systems tested the benefits of adaptive navigation. One system had no adaptive techniques available and the other system contained the same content, but had the combination of all three adaptive navigation techniques. The test was controlled with the use of a pretest screening questionnaire to ensure a representative sample of 20 users of a similar age, knowledge and experience. The 20 test users were then split equally into two groups of 10. The first group tested the adaptive system first while the second group tested the non-
66
adaptive system, followed by a crossover to allow users to compare the merits of the two systems. As part of the experiment, users had 10 tasks to complete, in which they had to find 10 different pieces of information. Once the systems had been tested, the two experimental groups switched over to test the other system, keeping the experiment controlled. To make the tests fair, both sets of groups were given some different tasks to complete, in which they had to find different pieces of information. Both groups were given five different tasks and five previously seen tasks in case they had become accustomed to how the site was structured when they came to test the other system for a second time. The 10 tasks given to the 20 users when testing the adaptive and the non-adaptive systems were given in a different order so that no browsing patterns could be easily seen in the system. The experiment recorded the paths the users had taken, the directness of the paths, and how long it took them to find the goal by the number of links they visited and the time in seconds it took them to find the relevant information. The experiment also tested 'lostness' (how lost users got) using the method proposed by Smith (1996) comparing the number of information items inspected, compared with the number of items which normally needed to be inspected to locate the required information. From the results of the experiment, it was found that of the 20 users who used the adaptive system to complete the 10 tasks, only two users were measured as 'lost' while using the adaptive system, compared with four users who were measured as lost while using the nonadaptive system. Examining each group in more detail, we can see that of the first group to test the adaptive system, seven out of the 10 users thought the adaptive system was quicker than the nonadaptive system, as well as easier to use. The remaining three users thought that both systems were the same in quickness and ease of use. Examining the data collected in the user models, seven out of the 10 users were actually timed in seconds as being quicker, with an average of 11% while using the adaptive system, and with a mean saving of 1.74 links in each task. Eight of the users said they preferred the adaptive system, even though it was slower than the nonadaptive system for one of these users. Looking at the second group, who tested the adaptive system last, eight of the 10 users thought that the adaptive system was quicker than the non-adaptive system. However, when examining the data collected in the user models, only five of the 10 users were actually timed in
seconds as being quicker, while using the adaptive system, and the other users were actually timed as being slower by an average of only 1%. It was also noted that seven users thought that the adaptive system was easier to use and by looking at the data collected, a mean of 1.22 links were saved in each task while using the adaptive system. Of the 10 users, eight said they preferred the adaptive system even though the adaptive system was slower than the non-adaptive system for five of these users.
11
Conclusions
In conclusion, websites that are designed for viewing on wireless devices require careful content management as these devices are characterised by their high operating costs, limited screen size and limited navigation capability (Jari et al. 2000). With the use of a user model that contains information about the user, such as their goals, interests, preferences and knowledge, suggestions can be made with the help of a Markov model to help create a personalised and adaptive site. Using personalisation and adaptive navigation techniques within the WANTIT system students will not waste time and effort looking through course material that is inappropriate to their needs. The WANTIT system has shown that the users can reach their goal (the information they are looking for) as quickly and efficiently as possible without getting 'lost' and they can do this in an environment they prefer and find easy to use.
12
References
Anderson C, Domingos P, Weld D (2002). Rational Markov models and their application to adaptive web navigation. KDD-02 The Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 2002. Beaumont I (1994). User modeling in the interactive anatomy tutoring system ANATOM-TUTOR. User Modeling and UserAdapted Interaction, 4(1), 2145. Benford S, Taylor I, Brailsford D, Koleva B, Craven M, Fraser M, Reynard G, Greenhalgh C (1999). Three-dimensional visualisation of the World Wide Web. ACM Computing Surveys Symposium on Hypertext. Boyle C, Encarnacion AO (1994). MetaDoc: An adaptive hypertext reading system. User Modeling and User-adapted Interaction, 4, 119. Kluwer Academic Publisher. Brusilovsky P (1992). The intelligent tutor, environment and manual for introductory
Graham et al.
67
programming. Educational Technology and Training International, 29(1), 2634. Brusilovsky P (1996). Methods and techniques of adaptive hypermedia. User Modeling and User-Adapted Interaction, 6, 87129. Brusilovsky P and Weber G (1996). Elm-art: an intelligent tutoring system on worldwide web. Third International Conference on Intelligent Tutoring Systems, ITS-96, Montreal, LNCS 1086, 261269. Brusilovsky P, Eklund E, Schwarz E (1998). Web-based education for all: a tool for developing adaptive courseware. Computer Networks and ISDN Systems (Seventh International World Wide Web Conference), 30, 17, 291300. Brusilovsky P and Pesin L (1998). Adaptive navigation support in educational hypermedia: an evaluation of the ISIS-tutor. Journal of Computing and Information Technology, 6(1), 2738. Brusilovsky P (2001). Adaptive hypermedia. User modeling and user-adapted interaction, 11, 87110. Brusilovsky P (2003). Adaptive navigation support in educational hypermedia: the role of student knowledge level and the case for m e t a - a d a p t a t i o n . British Journal of Educational Technology, 34 (4), 487497. Cormack GV and Horspool RN (1987). Data compression using dynamic Markov modeling. Computer Journal, 30(6), 541550. De Bra P, Brusilovsky P and Hoube, G (1999). Adaptive hypermedia: from systems to th framework. ACM Computing Surveys 31 (4 edition), 12. Hohl H, Bcker H-D, Gunzenhuser R (1996). HYPADAPTER: an adaptive hypertext system for exploratory learning and programming, user modeling and useradapted interaction, 6(23), 131155. Jari L, Harri K, Multisilta D (2000). Wireless Application Protocol (WAP): technology platform for wireless learning environments. IASTED International Conference: Computers and Advanced Technology in Education, Cancun, Mexico, May 2000. Kay J and Kummerfeld B (1994). An individualised course for the C programming language. Proceedings of the 2nd International WWW Conference. Kobsa A, Mller D, Nill A (1994). KN-AHS: an adaptive hypertext client of the user modeling system BGP-MS. Proceedings of the Fourth International Conference on User Modeling, Hyannis, MA, 99105. Learning and Skills Development Agency (2002). A presentation for Online Educa. At www.m-learning.org/reports.html, accessed November 2003.
Mukherjea S (1999). Information visualization for hypermedia systems. ACM Computing Surveys, Symposium on Hypertext and Hypermedia. Perkowitz M and Etzioni O (2000). Towards adaptive websites: conceptual framework and case study. Artificial Intelligence Journal, 118,12. Quentin-Baxter M and Dewhurst DG (1999). Quantitative evidence for differences between learners making use of passive hypermedia learning environments. ACM Computing Surveys, 31(4). Smith P (1996). Towards a practical measure of hypertext usability. Interacting with Computers, 8(4), 365381.
Hine et al.
69
Collaboration and roles in remote field trips

Nick Hine, Rosaleen Rentoul University of Dundee Division of Applied Computing, Dundee, United Kingdom E-mail: [nhine, ros] @computing.dundee.ac.uk Abstract
To ensure that classroom-based pupils benefit from the experience of a field trip being undertaken by their peers, investigations are underway within the RAFT project to provide a cooperative learning environment spanning the field trip and the classroom. Various roles are being explored to ensure that all participants are fully engaged in the event. The background to this investigation is the need to embed the experience within the curriculum, and to explore the event within peer-based learning, situated learning and vicarious learning pedagogic principles. Keywords: field trips, cooperative learning
Marcus Specht Fraunhofer-Gesellschaft, Institute for Applied Information Technology, St Augustine, Bonn, Germany E-mail: marcus.specht@fit.fraunhofer.de
the classrooms and the field trip and the capabilities of the current wide area technologies, while anticipating the capabilities of the future infrastructures. This paper will report on this initial work and highlight the nature of the collaboration being explored and the place of roles within the mobile learning situation.
2. Curriculum and pedagogies

Along with the development of this approach to field trips, investigations into the current state of field trips and the curricula in various countries are being investigated. Despite the recognised educational worth of field trips there are many limiting factors, such as time in the curriculum, health and safety, insurance and staffing which inhibit the number of these activities (Barker et al. 2002). It is hoped that this approach to field trips will help alleviate these problems by only removing a small group of students from the school but also that it will provide an even richer educational experience for all students. Comparisons of curricula in various countries have been made. There are enough common areas in the curricula to be able to develop European and internationally relevant field trips. To ensure that the field trips are worthwhile they are designed to fit the curricula of each of the participating countries by consulting and working with teachers in each of the countries involved. The field trips that have been proposed by practising teachers at RAFT workshops are noted in a RAFT portal (www.raft-project.net) and some have been chosen to be fully developed for the project and to provide exemplar field trips for future use. The learning theories and pedagogies appropriate to RAFT include collaborative and cooperative learning, situated learning (Lave and Wenger 1991), peer-assisted learning (Topping and Ehly 1998) and vicarious learning (Lee et al. 1999). Examples of where these may occur are noted in Table 1.
1. Introduction
The Remote Accessible Field Trip (RAFT) project is implementing a method of spanning field trip and classroom locations to provide an integrated event for the set of pupils involved. This will take the practice of mobile learning into the realms of fully cooperative and collaborative learning. To ensure that all pupils are fully engaged in the event, the project team is exploring a range of roles that could be taken by the pupils. The scope of these roles is being explored, together with the qualities that will be developed in the pupils taking these roles. A variety of scenarios are being explored, focusing on the areas of biology, art and history. The curriculum is being considered within the secondary education systems of Canada, Scotland, Germany and Slovakia. Prototype systems are being constructed based on innovative classroom technologies and on mobile devices in the field. Initially a variety of simple scenarios are being enacted with school pupils to explore the nature of the roles that might be necessary and the functionality required of the systems to enable these roles to be enacted. In addition, trial field trips have taken place to explore the interactions between
70
Table 1 Learning theory examples in RAFT Learning theory Situated learning RAFT examples Field trips present many real-life practical problems to be solved and demands from several directions. Examples: visiting and interviewing a professional an artist in their studio, a journalist at work, a scientist in the laboratory. Working with peers in the field and in the classroom; distributed working with other students in different parts of the world on topics of mutual interest. Group tasks, eg on pollution in river. Within the group there are roles, eg researcher, communicator, measurer, collator and developer. Each contributes to achieve the joint result a jigsaw process. In the field, a visually impaired student may be assigned a student to be his/her visual helper the two work together gathering information for their group. A student takes the role of process observer in a group investigating the Roman Army. The student learns different techniques from observing other groups approaches to their tasks. This is meta-learning.
4. Roles identified
To make the learning experience valuable with only a few students in the field and most of the students in the classroom, the field trip, as well as having to be well organised and planned, should allow all students to be and feel involved. To achieve this a cooperative approach to learning has been adopted (Johnson and Johnson 1994). Each student has a particular role in a group each of which has a specific task. All students are working towards a common goal and each students contribution is important. The initial roles proposed include, in the classroom: classroom coordinator, classroom communicator, researchers and archivist and, in the field: field communicator, scouts, data gatherers and annotators. These are shown in Figure 2.
Collaborative learning
Cooperative learning
Peer-assisted learning (PAL)
Vicarious learning
3. Functional overview of the RAFT system

There are many facets to this proposed approach to field trips. These include planning, coordinating and evaluating the field trip as well as managing the users, their learning and the collaborative learning environment. There could be other classrooms and experts from anywhere in the world participating in the field trip in real time thus extending the collaborative learning experience. The proposed system is defined in Figure 1.
Figure 2 Some of the field-trip roles Figure 1 Functional overview of RAFT system
Hine et al.
71
7. Issues being explored

From initial trials, the most effective field trips with interaction and a feeling of involvement from all students have occurred when the classroom coordinator has shown leadership skills and has taken responsibility for driving the communication. Field trips have also worked more smoothly when the archivist has been familiar with the presentation software. It is also emerging from initial prototyping that each person in a group needs a uniquely defined role and these roles may include researcher, communicator, analyst, collator and developer. Therefore, there may be several of each of these in the classroom depending on the number of tasks and groups involved. The classroom should be busy with activity from before direct communication with the field-trip site is established to after the field-trip activity has been completed. The project is developing quickly and many issues have emerged. These include: networking: wireless technology, suitability and reliability collaborative tools: whiteboards, video conferencing, text and mobile phones as reserves learning delivery tools: LMS with RLOs, future role of metadata classroom the design of the future classroom (DfES 2002, Tinzmann et al. 1990) field trip decisions about what type of devices will be useful and will they be designed specifically for children interfaces issues such as having a common interface roles decisions on the role of the archivist and the amount of responsibility (s)he is given file transfer from field to class management setting up a collaborative learning environment methods for teachers to find out about and participate in RAFT.
5. Role of the learning management system

In addition to providing an educational experience, the data gathered and generated by the students can be stored as learning objects and re-used by others in the class for their particular task, in future years, by other classes and by others who have access to the learning management system (LMS). It may be that as the project develops the whole potential of the LMS is utilised in terms of controlling and monitoring the students educational experience and progress but at this stage it is the re-usable learning objects (RLOs) that are of interest to the RAFT project.
8. Future work
RAFT is at the prototyping stage and for the next year the data gathered will be used in the development stage. After that full field-trip trials will occur and these will be evaluated in a comprehensive manner covering many areas from learning efficiency to effectiveness of video-conferencing. At all stages communication and cooperation with schools, teachers and students will continue to ensure functionality and suitability. This should result in a system that is accepted by teachers, where they are sure that the content is curriculum-specific and the technology is used naturally and appropriately to enhance the educational experience.
6. Technology
As the nature of the approach is to have students communicate from remote places to classrooms, wireless communication is important. Various types including Bluetooth, 3G and wireless local area networks (WLAN) are being investigated for suitability. Personal digital assistants (PDAs), tablets and laptops are all being trialled for suitability of use for students on the move and in classrooms and for suitability of running available software. Customised interfaces are being developed from input from students in the classroom and prototypes are being tested. The idea of mobile computing is also being prototyped with ideas about designs and preferences for new devices being developed in discussions with students.
9. Acknowledgments
This work is part-funded by the Commission th of the European Community under the IST 5 Framework Programme. The authors gratefully acknowledge the support of the RAFT team in this work.
72
10. References
Barker S, Slingsby D and Tilling S (2002). Teaching biology outside the classroom. Is it heading for extinction? A report on biology fieldwork in the 1419 curriculum. FSC Occasional Publication 72. Shrewsbury, UK: Field Studies Council and British Ecological Society. Department for Education and Skills NGfL (2002). Transforming the way we learn: a vision for the future of ICT in schools. DfES/0008/2002. Norwich: Her Majestys Stationery Office. Johnson RT and Johnson DW (1994). An overview of cooperative learning. In J Thousand, R Villa and A Nevin (eds) Creativity and collaborative learning: the practical guide to empowering students and teachers. Baltimore, MD: Brookes Press. Lave J and Wenger E (1991). Situated learning: legitimate peripheral participation. Cambridge, UK: Cambridge University Press. Lee J, Dineen F, McKendree J and Mayes T (1999). Learning vicariously in a distributed environment. Active Learning 10 (July), 49. Tinzmann MB, Jones BF, Fennimore TF, Bakker J, Fine C and Pierce J (1990). What is the collaborative classroom? Oak Brook. IL: North Central Regional Educational Laboratory. Topping K and Ehly S (1998). Peer-assisted learning. London and Mahwah, New Jersey: Lawrence Erlbaum Associates.
Kadyte
73
Learning can happen anywhere: a mobile system for language learning

Vaida Kadyte Turku Centre for Computer Science, Abo Akademi University Institute for Advanced Management Systems Research, Mobile Commerce Laboratory, Abo Akademi University Lemminkainengatan 14 B, 20520 Abo, Finland E-mail: vaida.kadyte@abo.fi Abstract
The current social trend towards learning during leisure time, together with the rapid development of advances in mobile technology, gives rise to the vision of enabling consumers to learn any time, anywhere. However, until recently there have been relatively few attempts to explore the learning opportunities presented by mobile devices systematically. This article investigates the importance of starting from the mobile users perspective with a conceptual framework for developing mobile applications for learning. It includes the results of a mobile commerce expert survey, carried out by IAMSR Research in five different countries, to illustrate the impact of cultural differences in user behaviour on the potential success of mobile learning applications. It was found that time, information and location dimensions delineate the impact on users' contexts and the selection of innovative mobile technology, and together with our results suggest the design of a mobile system for language learning. Keywords: mobile system, overlapping contexts, leisure and learning technology often does not save any time, but erases boundaries between work and leisure activities. The new paradigm of being able to undertake all tasks, all the time, is a warning of our complicated lifestyles. Historically, at the same time as the mobile device became affordable for many people in both industrialised and developing countries, it st also turned into a mass tool of the 21 century by creating newfound freedoms in users personal lives. With the increasing level of functionalities, mobile devices are increasingly used for entertainment. Consequently, one of the core issues in this paper is how to improve the effective use of leisure time in a knowledgebased society. The rapid development of information and communications technology (ICT) has contributed to greater economic choice and higher social prestige for consumers. There is evidence that ICT is reshaping consumer behaviour in the following ways: individual preferences have shifted from mass production to mass customisation self-service allows users to respond to new business opportunities instantly and react quickly to changes consumers are becoming smart shoppers, spending their money increasingly strategically product quality is perceived in an extremely rational priceperformance trade-off (Meffert 2000). The rising level of education is another powerful variable shaping very distinct consumer and organisational behaviour (Falk and Dierking 2002). Continuous learning, learning during leisure time, grows more important, because considerable investment in personal development and education is necessary to compete in the global economy. Knowledge-intensive workers have more resources and are more aware of change and technology. However, they can suffer from information overload. Therefore, our knowledge society can be characterised as time-poor and money-rich (Lindskog and Brege 2003). Learners in a knowledge society want their leisure learning to meet their individual interests
1. Leisure and the learning society

Leggiere (2002) argues that it is both evident and ironic that progress in information technology (IT) has resulted in more work in our st society. However, in the early 21 century societal expectation might be that there would be a global increase in the amount of leisure because of the new and pervasive productivityenhancing technologies, given that the internet and mobile wireless communications have freed businesses from the constraints of working at a specific time and place. Technology does indeed simplify routine tasks, for example sending email messages instead of fax messages, linking electronic databases globally rather than using local libraries, and so on. Moreover, technology also saves us time, which people can use to work on other tasks. Thus, it can be argued that
74
and needs (Wikstrm 2000), so demand for tailored, contextualised and responsive learning is rising. Traditional institutions for continuing education can be perceived as being barely connected to their students. In fact, they are unable to establish a dynamic interaction on a real-time basis outside the classroom or to provide personalised information at the point of need. Continuous learning during leisure time already happens; so learning organisations are starting to borrow ideas from e-commerce. Another trend is the multi-ethnic nature of current society, as a result of emigration, students studying abroad and a growing number of international corporations. In developed countries this has led to a prominent number of newcomers, or foreigners, seeking to be integrated into the local society; perhaps first by learning the local language. To consider how wireless technologies are being adapted to meet changing educational needs, the authors looked at language-learning systems on mobile devices. Mobile solutions are only effective when people feel that they add value or bring new freedom. An important question is how to create a mobile system for language learning, which has added-value features for its user. The next sections note how the factors contributing to the development of a successful mobile system for language learning were investigated.
active maps that automatically change as the user moves (Schilit et al. 1994) applications whose user interfaces adapt to the users location.
Another aspect of the context in past research is related to the orientation of device position both indoors (Bahl, Padmanabhan 2000) and outdoors (Priyantha et al. 2000). A number of papers have focused on the creation and evaluation of location-aware mobile systems, called the e-guide (Cheverst et al. 2000). Here, the researchers have found a surprisingly high level of acceptability across a wide range of users. However, they also evaluated users frustration while using a system at their current location and not being able to query the system for further information. The main conclusion drawn from these studies is that context-aware systems are not affected by the design of the user interface alone. They are also governed by the design of the infrastructure that supports them. Groot and Welie (2002) analyse a mobile context of use and define it as a users ecosystem in the sense that it presents many design challenges, and that context of use is crucial for increasing service value and solving usability problems: Even though the industry likes to talk about 'virtual services', people with their millions of years of history of handling artefacts, will probably view the mobile Internet as a, what we call, 'ecosystem of connected terminals', where the interaction with each terminal is dependent on the context of use. Context of use (screen size and colour depth, input mechanisms, network latencies, etc) can be seen as the key challenge arising from the differences between traditional web design and micro design for mobile devices. These studies all point to the importance of the context of use, and this should be governed by the users overall ecosystem, rather than by the physical location alone. To develop an indepth understanding of the widespread take-up of mobile applications, we need to appreciate the conditional elements of System Theory (Bertalanffy 1962). The System Holism principle, derived from System Theory claims that it is necessary for a system to have functional parts that communicate where the sum of the parts is greater than the parts added together. In analysing the relationships between the different success factors of information system artefacts, we present a conceptual framework that can be used to
2. Conceptual framework for designing a prototype mobile learning language system

High added-value applications in mobile commerce are becoming some of the most popular topics of interest in information systems (IS) research. They are also a core activity for many businesses operating in the wireless world. Scholars exploring mobile markets in different countries are likely to have different perceptions of what kind of mobile services will be the most popular. On the other hand, they share quite similar views on the success of the design of services for mobile products, that is, designing from the customers perspective. Context-aware mobile applications, which adapt their behaviour to the environmental context, are an important class of applications in emerging mobile systems; the most commonly researched area in mobile context studies is the physical location of the mobile user. Examples include: location-aware applications that enable users to discover the resources available in their physical proximity (Harter et al. 1999; Priyantha et al. 2000)
Kadyte
75
justify the most appropriate context of mobile learning.
3. Contexts of mobile learning

Establishing the right contextual model is a starting point for the design of any mobile system for consumers. We discovered that a typical mobile user is involved in a number of different overlapping contexts (Falk and Dierking 2002), thus any activity and experience that result are influenced by the interactions between these contexts. The definition of an overlapping context is not new, though it has generally not been emphasised in IS research. The contextual model used by Falk and Dierking (2002) implies that overlapping contexts contribute to and influence the interactions and experiences that people have when performing certain activities. Our suggestion is that there are three overlapping contexts the personal mobile, the learning community (which exists in virtual, physical, or both forms, depending on where the learning interaction with other community members occurs) and the cultural. Together they contribute to the design of experiences that people have when engaging in mobile learning. This can be seen in Figure 1, which introduces the principle of System Holism in that it suggests that the existing contexts of mobile learning and the mobile system for language learning can combine for a smooth user experience.
depend on a network of mobile users. It can be useful to consider this in relation to the network economic theory (Shapiro and Varian 1999), which claims that the value of the network increases and costs decrease with a growing number of users. In other words, the common logic suggests that the value of a product within a network depends on the product adoption behaviour of other users. An important point to note is that there are the two contexts the physical and the mobile that overlap. Thus, the language learning experience, and any activity that results, is influenced by the interactions between these contexts. It is also important to consider the cultural context of use when designing a mobile system. Relationships exist between usability and culture (Hillier 2003). As a proof of reference we present the results of a mobile commerce expert survey, carried out by Carlsson and Walden (2002) at the Institute for Advanced Management Systems Research in five different countries. The results suggest that cultural differences in usage behaviour characterise the potential success of mobile learning applications. Thirty-one interviews were conducted with national mobile commerce experts in Finland and Hong Kong. It could be argued that this is a limited population from which to make comparisons between the two countries. But the business of developing mcommerce applications and services is just beginning and there are still only a few companies operating in this area in either Finland or Hong Kong countries considered to have very high-tech oriented infrastructures.
PERSONAL MOBILE
Time
VIRTUAL/ PHYSICAL COMMUNITY

Information
Location
Question: How do you evaluate the likelihood of firms acheiving a satisfactory level of turnover for the MLearning / M-Education in the next 18 months? HK 1 FIN 1.5 4.7 0 Very Good 2 4
Poor
CULTURAL
Figure 1. Overlapping contexts of mobile learning The contextual model presented in this paper emphasises both the personal attributes of the user operating in a mobile context and the community attributes that are essential for the physical or virtual learning context. Learning does not exist in isolation. Moreover, mobile systems are not simple IS products; they
Figure 2. Expert opinion on the potential success of mobile learning and mobile education in Finland and Hong Kong The respondents from Finland and Hong Kong had different perceptions of mobile learning (see Figure 2) because they originated in different cultural contexts and well-defined educational systems. The theory of cultural
76
differences published by Edward Hall in 1976 assumes a strong linkage to exist between culture and communication. He used general terms high context and low context to describe cultural differences between societies. High context refers to societies or groups where people have very few but close connections over a long period of time, with less verbally explicit communication and written/formal information involved. Many aspects of cultural behaviour are not made explicit because most members are linked from years of interaction with each other. In low-context cultures the context carries relatively little information, where people tend to have many connections but of shorter duration. In these societies, cultural behaviour and beliefs may need to be spelled out explicitly so that the listener wants to get lots of information, and to have it at once. The following ethnographic study revealed that both Finland and Hong Kong have elements of high-context cultures and both were identified as supporting innovation. However, strong differences existed in their educational culture and funding mechanisms. In Hong Kong, education is mainly privately funded; in Finland education is government funded. In Hong Kong, people prefer to learn at home in quiet surroundings because the pace of life is hectic and they already have contact with others during the day. This contrasts with Finland, where people work in a more isolated fashion and regard learning as a semi-social activity. This overlapping of contexts opens up a variety of different features and attributes that influence the correct definition of user requirements and the appropriate design of the overall mobile system.
4.
Innovation with voice technology
We have chosen to describe the freedom of learning as a process, evolving within different dimensions including time, information and location. Our basic assumption is that time, information and location have implications for the learning society, which is eager to transfer and share knowledge effectively through the use of mobile communication technology. Researchers have long recognised that media vary in terms of their information richness; that is, the way in which an environment presents information to senses (Steuer 1992). The overlapping contexts the mobile (which is highly personal) and the physical (characteristics of the features of a community the user belongs to) should be considered within the scale of time and information. A location-based service can also offer value to the users. For example, if
users want to learn about nature, they can go to a park. Such location-dependent learning can increase the sensual experience and enhance language learning. The idea behind this is that the interface designed for language learning is appropriate during the usage time; that is, it should not be too complex or too simple. The three underlying parameters of mobile communication require different ways of presenting information to the senses, thus assisting in the selection of appropriate technological solutions to create a positive experience of mobile learning. The sharing of extensive information within a non-specified time frame does not require the sender and receiver to be in the same place at the same time. Using voice technology to provide a rich media content for the user solves the current limitations of input and output mechanisms of mobile devices. Moreover it is an innovative solution in terms of mobility and contextual learning. The main results of the study can be translated into a general mobile-learning prototype for learning the Finnish language (see Figure 3). This system has multilingual content, but the information is in a single language specified by the user when first subscribing to the profile. The learner can click words (such as vocabulary, topics and milestones) on a screen and specify in which language they want to receive that information. It is suggested that the learning be divided into milestones for tracking the users personal progress. If using a mobile phone, it can deliver small lessons in Finnish using both sound and text. Through the use of mobile headphones, a Language Learning Guide is able to explain the main grammar rules within the vocabulary section, in order to introduce each of the contextual topics according to the personal profile. For example, push technology enables the user to post the most recent news to sections created for use by teachers and groups. Probably the oldest and most widely used push technology is e-mail. This is a push technology because you receive mail whether you ask for it or not that is, the sender pushes the message to the receiver. SMS and MMS alerts are expected to be used for retrieving problematic or time-critical information and can also direct the learner to tutor support. Posting an alert can catch the attention of whichever students and teacher are available at the time for a face-to-face meeting. In our prototype we address the small screen size constraints and limited amount of text information by using voice technology that can relay more information.
Kadyte
77
yet unknown m-education market is problematic and requires further in-depth research. Using simple but critical dimensions of the mobile communication and contextual learning for marketing purposes will contribute to the design of meaningful and therefore successful mobile applications and services. Being first is a challenge facing many institutions developing mobile learning products. The authors consider that the development of mobile learning is inevitable in the future: educational institutions have no choice unless they want to become an isolated, old-fashioned, elitist sector outside the mainstream of society. We consider that further research is necessary so that mobile systems with value-adding solutions will facilitate but not replace conventional learning.
6. References
Figure 3. Screenshot of the service interface (Nokia 7650 mobile device) In an ideal case, the user can move freely within the different environments in accordance with the context of the selected topic, and listen to the correct pronunciation of the language through earphones. An implementation of such a mobile system, designed to meet the particular needs of leisure students, requires further research before such freedom of learning can be realised.
Bahl P, Padmanabhan VN (2000). An in-building RFbased user location and tracking system. Proceedings of IEEE INFOCOM. Tel-Aviv. Israel. Bertalanffy LV (1962). General System Theory: a critical review. General Systems, 7, 120. Carlsson C, Walden P (2002). Mobile commerce, the hype and the first hard facts. Presentations of the th IAMSR 10 Anniversary. 24 September 2002. Cheverst K, Davies N, Mitchell K, Friday A (2000). Developing a context-aware electronic tourist guide: some issues and experiences. Lancaster, UK: Distributed Multimedia research group. Falk JH, Dierking LD (2002). Lessons without limits: how free-choice learning is transforming education. Institute for Learning Innovation. Groot B, Welie M (2002). Leveraging the context of use in mobile service design. C o n f e r e n c e th proceedings of 4 International Symposium, Mobile HCI 2002. Pisa, Italy: Springer 334338. Hall ET (1976). Beyond culture. Anchor Press, New York. Harter A, Hopper A, Steggles P, Ward A, and Webster P (1999). The anatomy of context-aware th application. Proceedings of 5 ACM MOBICOM Conference. Seattle, WA. Hillier M (2003). The role of cultural context in multilingual website usability. E l e c t r o n i c Commerce Research and Applications, 2 (1), Spring 2003, 214. Leggiere P (2002). What happened to the leisure society? Journal Across the Board, 4248. Lindskog H, Brege S (2003). The importance of timerich and time-poor consumer behaviour for the Eth commerce. Proceedings of the 24 McMaster World Congress. Canada, January, 2003. Meffert H (2000). Consumers tomorrow. In D Corsten, DT Jones (eds) ECR academic report 2000: ECR in the third millennium academic perspectives on the future of the consumer goods industry. Brussels: ECR Europe Academic Partnership, ISBN 3 905613 01 8, 6871.
5. Further research
One of the most modern, inexpensive and convenient ways of helping visitors or newcomers to learn the basics of a foreign language is being developed using mobile phones. Mobile education influences not only the students, but also the lecturers. Therefore many educational aspects and teachers usability issues need to be considered. The possibilities offered by the third-generation (3G) mobile networks are immense. They have the potential to increase the interactivity and personalisation of applications and services with video, audio and text data. Mobile solutions are only effective when people feel that they add value, are cost-effective and reliable. First, data must be gathered about students and teachers needs to generate ideas for future products and services that could fulfil their expectations. Second, managerial considerations on gaining the critical mass are necessary to make the applications financially attractive. Therefore marketing in a promising, phenomenal and as
78
Priyantha NB, Miu AKL, Balakrishnan H, Teller S (2001). The cricket compass for context-aware mobile applications. MIT Laboratory for Computer Science. Shapiro C, Varian HR (1999). Information rules: a strategic guide to the network economy. Boston: Harvard Business School Press. Schilit et al. (1994). Disseminating active map information to mobile hosts. IEEE Network. Steuer J (1992). Defining virtual reality: dimensions determining telepresence. Journal of Communication, 42(4), 7393. Wikstrm S (2000). Consumers and consumption patterns in transition. In N Corsten, G Doukidis, Huchzermer (eds) Academic perspectives on the future of the consumer goods industry. Brussels: ECR Europe Academic Partnership.
Lonsdale et al.
79
A context-awareness architecture for facilitating mobile learning

Peter Lonsdale, Chris Baber, Mike Sharples, Theodoros N. Arvanitis University of Birmingham Electronic, Electrical, and Computer Engineering, Birmingham, B15 2TT E-mail: {p.lonsdale, c.baber, m.sharples, t.arvanitis}@bham.ac.uk
Abstract
The MOBIlearn project (EU IST-2001-37187) aims to support a wide range of services and applications for learners using mobile computing devices such as phones, personal digital assistants (PDAs) and laptops. The display limitations of these devices mean that it is crucial to deliver the right content and services at the right time. One way of doing this is to use contextual information to derive content that is relevant to what the user is doing, as well as where and how they are doing it. We present an object-oriented, feature-based architecture for a context-awareness subsystem to be implemented within the MOBIlearn project, and consider the implications involved in the use of such a system for mobile learning. Keywords: context awareness, e-learning, mobile computing, m-learning
computing devices for a recent review see Chen and Kotz (2000), and some examples of current projects include Kolari (2003) and Chalmers et al. (2003). People on the move need information relevant to their location and immediate needs. The display capabilities of mobile devices are restricted when compared to desktop alternatives and the mobile communications channel may have limited bandwidth, so the sub-system must match the content to the available display and communications, and also to the learner's needs and preferences. It is also important for a mobile device to provide services (such as collaboration tools) and user options (such as interaction preferences) that are appropriate to the situation of use.
1.1. Context awareness for mobile learning

Context awareness in MOBIlearn is implemented as a context-awareness subsystem (CAS) that selects content reflecting the requirements of a specific individual and then presents this content with minimal user effort. There are two potential advantages to this approach: it reduces the need to define search terms and perform a content search the system is usable while the person is engaged in another activity.
1. Mobile learning in MOBIlearn

MOBIlearn is a worldwide, European-led research and development project exploring context-sensitive approaches to the application of mobile technology to informal, problem-based and workplace learning. The MOBIlearn system will deploy a generic mobile learning architecture based on sub-systems that interact through web protocols to provide relevant and timely learning content and services. Context management is a key sub-system that delivers content appropriate to the learner's goals, situation and resources. Context awareness is a highly desirable feature for mobile
The usefulness of this approach has already been demonstrated by Bristow et al. (2002) who showed that simple sensor input indicating user status could provide effective context-
80
dependent content provision. For example, a user walking past the library sees a link to the library homepage on a head-up display (a compact display unit that can present information to the wearer without obscuring their view of the world), and if they stand still they are presented with a brief version of the page itself. If they then sit down, they see the page in full. In broad terms, the aim of the CAS is to provide a means by which users of mobile devices can maintain their attention on the world around or the task at hand, while their mobile devices provide timely and effective computer support. The CAS provides a mechanism by which relevant content can be selected, filtered, and passed to the user. Users can then either look at the content or select other content from the filtered set. MOBIlearn aims to provide users with a rich and flexible learning experience, and therefore content includes not only learning objects or materials per se but also resources, services, and options that might be relevant to the learner in their current context. For instance, other learners themselves might constitute resources in a learning environment, and users should be offered the opportunity to make contact when appropriate. By including services, we aim to address the need to make learning content available from a variety of sources, not limited to the content set available immediately to the user in their current location. There are many examples of varied uses of different elements of contextual data but there is no over-arching architectural approach. What we are concentrating on for MOBIlearn is developing a re-usable architecture for a wide range of applications and scenarios. This is in line with the general MOBIlearn aim of producing a reference architecture for mobile learning that affords flexible re-use and application to a wide range of mobile learning scenarios. Our aim is to produce a simple, yet powerful, approach to building context-aware applications that non-expert users can easily customise for their own needs.
phone; art-history students following a study guide around a museum can be offered relevant content whenever they stop in front of specific art works; and a first-aider in the field can be given just-in-time advice by a device that is able to respond to the severity of injury, distance to the nearest medical facilities and the experience of the person administering the first aid.
1.3. Approaches to context awareness

A survey of the current literature concerning context-aware computing indicates that there are two main approaches to building contextaware artefacts. The technologically driven approach is focused on what capabilities can be provided by the available hardware and software. For context-aware applications, this is usually a case of determining what data can be obtained through sensors and what processing can be carried out on that data by the available devices see Want et al. (1992) and Abowd et al. (1997) for some examples. Conversely, the application-driven approach concentrates on what capabilities are required by a particular learning application or context of use, including the requirements of the user(s) themselves. Some examples relevant to this alternative approach can be found in Lueg (2002) and van Laerhoven (1999). We aim to reconcile these two approaches by maintaining an awareness of current technical capabilities and limitations as well as taking into account the needs of learners in the scenarios for which we are developing. This means that we are aiming to provide learners with a flexible context-awareness system that can react to their needs as anticipated by authors, publishers and developers, and also to their direct input should the need arise. One of our primary assumptions is that the system could fail or make an erroneous judgement at any time, and that users need to have the opportunity to influence and correct the system.
1.2. Scenarios of use

The development of content and delivery mechanisms within MOBIlearn is based around the development of learning scenarios in three key areas: Master of Business Administration (MBA) students, museum visits and healthcare/first-aid provision. All these areas provide rich contexts of use that can determine what content is appropriate for a given user at a given time. For example, MBA students travelling on the train can be given a short multiple-choice quiz that downloads quickly to their mobile
1.4. Describing context

Our starting point in a definition of context is to identify the purpose of the context we are interested in. For MOBIlearn, the purpose is learning, specifically learning on mobile devices, and so our approach to describing context and applying this description to produce a usable software architecture is based on this focus. Figure 1 shows the basic hierarchy for our description of context. Instead of a rigid definition, our intention is to provide a hierarchical description of context as a dynamic process with historic dependencies.
Lonsdale et al.
81
By this we mean that context is a set of changing relationships that may be shaped by the history of those relationships. For example, a learner visiting a museum for the second time could have his or her content recommendations influenced by their activities on a previous visit. Figure 1: Context hierarchy
Context
Whats going on over time
in Figure 2. A learner with a mobile device is connected to a content delivery subsystem, which in turn is linked to the context engine. The mobile device passes contextual information obtained from sensors, user input, and user profile to the context subsystem which then compares this metadata to the content metadata provided by the delivery subsystem and returns a set of content recommendations. These recommendations are used by the delivery subsystem to determine which content to deliver to the learner. Figure 2: Context awareness in action
Context State
Elements from the Learning and Setting at one particular point in time, space, or goal sequence
Contextual information
Contextual information User input
Context Substate
Elements from the Learner and Setting that are relevant to the current focus of learning and desired level of context awareness
Context subsystem Context subsystem Context engine

Context subsystem Context subsystem
Content index metadata metadata

metadata
Context
metadata metadata
metadata
Recommendations
Context Feature
Context Feature
Context Feature
Context Feature
Content delivery
Content delivery Content delivery subsystem subsystem

subsystem
Content
User profile
A snapshot of a particular point in the ongoing context process can be captured in a context state. A context state contains all the elements currently present within the ongoing context process that are relevant to a particular learning focus, such as the learners current project, episode or activity. A context substate is the set of those elements from the context state that are directly relevant to the current learning and application focus, that is to say those things that are useful and usable for the current learning system. Context features are the individual elements found within a context substate and each refers to one specific item of information about the learner or their setting (for example, current learning task or location). Implementing context awareness within our architecture is a matter of deriving a context substate and using the context features contained within it to determine what content might be appropriate.
The basic cycle of operation of our contextawareness system is as follows: 1. 2. 3. 4. 5. input of context metadata construction of context substate exclusion of unsuitable content ranking of remaining content output of ranked list of content.
2. Context-awareness architecture
The basic representation of how the contextawareness system functions as part of the MOBIlearn content delivery system is illustrated
The CAS comprises a set of software objects called context features that correspond to realworld context features relating to the learners setting, activity, device capabilities and so on to derive a context substate, as described above. Data can be acquired through either automated means (for example sensors or other software subsystems) or can be input directly by the user. This context substate is used to perform first exclusion of any unsuitable content (for example high-resolution web pages that cannot be displayed on a PDA) and then ranking of the remaining content to determine the best n options. This ranked set of options is then output to the content delivery subsystem. The following sections explain the operating principles underlying the context features, beginning with an outline of the kinds of metadata we anticipate using in the system, followed by a description of the context feature
82
software objects that perform the context processing.
2.3. Acquiring context metadata

We have identified two main aspects of the learners context, namely their setting (including physical location, objects and people in close proximity, and available resources) and the learner themselves (including their current activities, goals and learner profile). Setting metadata Any context-aware application or service depends on being able to obtain contextual information from the users environment or s e t t i n g . For the MOBIlearn system, we anticipate relying on both automated input from sensors and other software, and input from the user themselves about their state and the state of the world. Some possibilities for automated input include the use of location data derived from tracking a device within a wireless Local Area Network (LAN), and the use of infra-red or radio frequency (RF) tags to signal the proximity of nearby objects. Wireless network tracking is becoming an increasingly feasible option with the availability of software such as Ekahaus Positioning Engine (see www.ekahau.com) which can use wireless LAN signals to locate a device to within a few metres, and RF tags are also looking promising as a way of implementing cheap and robust object identifiers. Also, since many handheld devices now feature Bluetooth technology as standard, this is another way in which RF technologies could be used for identification and communication purposes within a contextaware application framework. Learner metadata Users also create their own context, and we anticipate the use of contextual metadata relating to both the user status (including their current goals and activities) and learner model. User status includes the users current goals, intentions, activities and routines. Cognitive states such as goals and intentions can only be acquired accurately by asking the user to describe them, for example through forms or checklists, but the users activities and routines could be inferred directly. Bristow et al. (2002) have used accelerometers, either on the body or on a handheld computer, to detect whether a user is walking, standing or sitting. This can be combined with location data to tailor the display of information. A learner model is a computer representation of the learners current knowledge, misunderstandings, styles and strategies of
2.1. Use of metadata

The primary purpose of CAS is to perform intelligent matching between metadata on learning materials, services and options (content metadata) and metadata on the learner and their setting (context metadata). By looking for content metadata that matches the metadata of the current context, the system can make recommendations about what content is appropriate. Figure 3: Context awareness and metadata
Metadata
Content = Learning Objects + Resources + Services
Learner + Setting Context awareness = Context Substate
This process is illustrated in Figure 3 context metadata from the learner and their setting is matched to content metadata drawn from the set of available learning materials, options and resources. There are two crucial prerequisites for the successful completion of this process. First, available content must be appropriately marked up with a suitable metadata schema. Second, the system must have access to relevant metadata about the context, ie the learner and their setting.
2.2. Acquiring content metadata

For our first prototypes we are anticipating the use of a metadata schema being developed as part of a PhD at the University of Birmingham by Chan. This schema is based on the draft IEEE learning objects metadata schema (IEEE 2002) and includes extra elements appropriate to our approach to context and our desired level of context awareness. Work is underway to build a database of content suitable for the MOBIlearn project, and it is anticipated that the learning content management subsystem will handle this content and all associated metadata, making it available to other architecture components as required.
Lonsdale et al.
83
learning. Learner modelling has formed an important part of research into the design of intelligent tutoring systems (see, for example, Anderson et al., 1995, Brusilovsky et al., 1996). The learner model can be based on the systems assessment of the learners: knowledge or misconceptions learning style motivation progress in solving a problem (see Jameson 2003 for more details). The LISP Tutor of Anderson et al. (1995) employed the technique of knowledge tracing. It represented teaching knowledge as thousands of individual goal-oriented rules. As the student worked through exercises set by the Tutor, it estimated the probability that the student had learned each of the rules and then used this knowledge to guide the teaching. The tutoring system can call on the learner model to adapt its teaching by: selecting the form of information to be presented adapting the content of problems and tasks changing the content and timing of hints and feedback (see Jameson 2003 for more details). This approach to context awareness is relevant to MOBIlearn, and we anticipate a use of learner models within the context-awareness subsystem. The underlying assumption behind the use of learner modelling is that it can make the learning more effective and learners experience more enjoyable, and this assumption appears to be borne out by the research (for example Corbett 2001, cited in Jameson 2003). A further possibility is to mirror an aspect of the learner model back to the student. The ELMART system from Brusilovsky and colleagues (Brusilovsky et al. 1996) explored open learner modelling in which the learner is shown a visualisation (in the form of a skillometer bar chart) of the systems model of their current skills. The learner can reflect on the model and adapt their learning strategy. Clearly, there are ethical issues to consider when gathering detailed information about users, and this issue is discussed further in Section 3.
2.4.
Context features
The context-awareness sub-system comprises a set of software objects called context features that respond to features of the real-world context to provide an ordered list of recommended options. Types of context features Context features are either excluders or rankers. Items of content that are deemed entirely inappropriate for the current context are excluded. That is to say they are removed from the list of recommended content and not subject to any further consideration. Content remaining in the list after the exclusion process is then ranked according to how well it matches the current context. The ranking process simply increments the score of each item of content that has metadata matching the stimulus values of any particular context feature. The size of the increment depends on the salience value of the context feature doing the ranking. Individual context features can have their salience values changed so that they exert more influence on the ranking process. Any individual context feature can be de-activated at any time so that it has no effect on the exclusion or ranking processes. A context feature has a set of possible values, and an indicator of which value is currently selected. It is also possible for context features to have multiple sets of possible values, with the current active set being determined by the current value of another linked context feature. Linked context features Each context feature responds to only one metadata tag and performs either an exclusion or ranking function. To achieve more complex filtering of content, context features can be linked together so that their function (ie their stimulus and response values, their salience, and whether they are enabled) can depend on the state of other context features. For example, we might choose to have a context feature that excludes content based on its filesize such a context feature should be active if the learner is using a low-bandwidth connection, but should remain quiescent if a high bandwidth connection is available. By creating a context feature that responds to bandwidth availability and allowing it to control the status of the context feature that responds to file-size, we can easily create a pair of context features that respond to a more complex context. The linking process is
84
transparent to the user and to individual context features, so long chains can easily be created to cope with complex situations. Output The ordered list of ranked items of content is passed to delivery subsystems for use in determining exactly what content should be made available to the user. In this way, the context-awareness sub-system has no way of specifying exactly what is made available the system is intended only to make recommendations to the system and to the user.
There are five main questions to ask when considering the ethical implications of the use of contextual data as follow. 1. 2. 3. 4. 5. What information do we obtain? How do we obtain it? What do we use it for? What risks are there in doing this? What do users think about it?
2.5. Integration with MOBIlearn system

The overall aim of the MOBIlearn project is to produce a reference architecture for delivering learning content to users of mobile devices. The CAS represents an essential part of that architecture, but ultimately it depends on other subsystems to fulfil the goal of providing the user with a rich and effective learning experience. As the CAS is responsible for recommending learning content to be delivered to users, there are clear interdependencies between the CAS and the content management system which is responsible for handling the content database, accepting recommendations, and then making appropriate content available to the user. The CAS will also have intimate links with an adaptive human interface (see Vainio and Ahonen 2003), intended to provide a usable, functional interface on a variety of devices. Users must be able to inspect, understand and modify the context model at any time. Given our aim of providing recommendations not only about content per se but also relevant services and options, the CAS will be linked to a collaborative learning system. Content delivery itself will be handled mainly by a learning content management subsystem that will maintain a repository of available learning content along with any associated metadata. The content delivery sub-system and the CAS will function in tandem to deliver contextually relevant content to the learner.
Data being gathered without users being aware of this is part of our learning solution but unfortunately also part of the larger problem of ethical gathering and application of user data. There are specific international guidelines and legislations that address concerns about the gathering and use of such data, and we will defer to the recommendations set out in such documents in our use of contextual data. The following issues are of concern. Informed user consent: users must be made aware of what data is being gathered and what it is being used for, and this consent should be ongoing in the sense that users are kept informed for the whole time they are using the system and have the right to change or withdraw their consent at any time. They should also be made fully aware of the security risks of this data being gathered, stored and used. Control: where consent has been given for the gathering and use of contextual data, users should be given information, access, and control over data. Security: it will be necessary to ensure that any gathered data is stored securely, available only to necessary parties, and to prevent the misuse of data by third parties.
3. Ethical issues
Clearly, the gathering of contextual data could involve the use of information that is personal and private to the users involved. Such information needs to be gathered with the consent of users, and must be stored securely to prevent misuse by third parties.
Particular security problems arise when information is stored on computers other than the users own. We must ensure that as little information as possible is used, ie only the essential minimum, and that it is held securely to be accessible only within the MOBIlearn system. Some useful work addressing these issues has already been described in Rainio (2000) and the intention within MOBIlearn is to follow up such work and ensure that we adhere to any relevant guidelines and legislation.
4. Current status and work in progress

The CAS is currently implemented as a prototype demonstrator in Java. The prototype illustrates the operating principles of the architecture as a stand-alone demonstrator.
Lonsdale et al.
85
The next steps for the MOBIlearn context awareness sub-system involve linking the CAS prototype to sensors that can provide real-world context features on a mobile device such as a tablet PC or PDA. As we move towards integrating the CAS with the rest of the MOBIlearn architecture we are exploring implementation methods such as web services architectures (see Booth et al. 2003) to achieve flexible integration of the relevant components. The CAS prototype will be evaluated as a stand-alone, context-aware application by asking users to perform a set of simple information retrieval tasks in mobile contexts. The results of these evaluations will inform the next phase of our design.
5. Acknowledgements
This research is supported by the European Commissions Fifth Framework Programme on Information Society Technologies, project number IST-2001-37187, MOBIlearn: Nextgeneration paradigms and interfaces for technology-supported learning in a mobile environment exploring the potential of ambient intelligence. We would like to acknowledge gratefully the contributions of non-author project partners to the work reported in this paper. More details of the MOBIlearn project can be found at www.mobilearn.org
6. References
Abowd GD, Atkeson CG, Hong J, Long S, Kooper R, Pinkerton M (1997). Cyberguide: a mobile context-aware tour guide. Wireless Networks, 3(5), 421433. Anderson JR, Corbett AT, Koedinger KR, Pelletier R (1995). Cognitive tutors: lessons learned. The Journal of the Learning Sciences, 4(2), 167207. Booth D, Champion M, Ferris C, McCabe F, Newcomer E, Orchard D (2003). Web services architecture. W3C working draft 14 May 2003. At www.w3.org/TR/2003/WD-ws-arch20030514/ Bristow HW, Baber C, Cross J, Wooley S (2002). Evaluating contextual information for wearable computing. In Proceedings of the Sixth International Symposium on Wearable Computers. New York: IEEE Computer Society, 179185. Brusilovsky P, Schwarz E, Weber G (1996). ELM-ART: an intelligent tutoring system
on the world wide web. In Proceedings of ITS'96 Conference, 261269. Chalmers D, Sloman M, Dulay N (2003). Contextual mediation enables appropriate data selection. ERCIM News, 54, 1516. Chan T, Sharples M, Vavoula G, Lonsdale P (2004). Educational metadata for mobile learning. To appear in Proceedings of Second IEEE Workshop on Wireless and Mobile Technologies in Education. Chen G, Kotz D (2000). A survey of contextaware mobile computing research. Technical report TR2000-381, Dartmouth College. Corbett A (2001). Cognitive computer tutors: solving the two-sigma problem. In A Jameson, C Paris and C Tasso (eds) User modelling: Proceedings of the Sixth International Conference, UM97, New York, Vienna: Springer Wien, 327337. IEEE (2002) Draft standard for learning objects metadata. IEEE 1484.12.1-2002. Jameson A (2003). Adaptive interfaces and agents. In JA Jacko and A Sears (eds) Handbook of human-computer interaction in interactive systems, Lawrence Erlbaum Publishers. Kolari J (2003) Kontti: Context-aware mobile portal. ERCIM News, 54, 14. Lueg C (2002). Operationalizing context in context-aware artifacts: benefits and pitfalls. Informing Science: Human Technology Interface, 5(2). Rainio A (2000). NAVI Programme 2000-2002. Technical report VTT research notes 2038, Technical Research Centre of Finland. Vainio T, Ahonen M (2003). A critical approach to an adaptive user interface design. In J Attewell, G Da Bormida, M Sharples and C Savill-Smith (eds) MLEARN 2003: learning with mobile devices: book of abstracts for the second annual MLEARN conference, London: Learning and Skills Development Agency. van Laerhoven K (1999). Online adaptive context awareness, starting with lowlevel sensors. PhD thesis. Brussels, Belgium: University of Brussels.. Want R, Hopper A, Falcao, V, Gibbons J (1992). The Active Badge Location system. ACM Transactions on Information Systems, 10(1), 91102.
Luckin et al.
87
SMILE: the creation of space for interaction through blended digital technology
Rose Luckin, Diane Brewster, Darren Pearce, Richard Siddons-Corby, Benedict du Boulay IDEAS Lab, School of Cognitive and Computer Science (COGS) University of Sussex Falmer, Brighton BN1 9QH E-mail: rosel@cogs.susx.ac.uk dianeb@cogs.susx.ac.uk darrenp@cogs.susx.ac.uk rsc@cogs.susx.ac.uk bend@cogs.susx.ac.uk
Abstract
Interactive Learning Environments is a course at Sussex University in which students are given mobile devices (XDAs) with personal digital assistant (PDA) functionality and full internet access for the duration of the term. They are challenged to design and evaluate learning experiences, both running and evaluating learning sessions that involve a blend of technologies. Data on technology usage is collected via backups, e-mail and website logging as well as video and still photography of student-led sessions. Initial analysis indicates that large amounts of technical support, solid pedagogical underpinning and a flexible approach to both delivery context and medium are essential. The project operated under the acronym SMILE Sussex Mobile Interactive Learning Environment. Keywords: XDA, pedagogy, conversational framework
planning the latest version of the course, for spring term 2003, it was decided that we should be exploring the learning possibilities offered by new mobile technologies. Eighteen mobile devices, XDAs with PDA functionality and full internet access, were used as part of the course. Students were allocated devices for the term and were expected to use them as their own. Postgraduate students had a device each, while the undergraduates had to share them in small groups of three to four students. The course itself has a wider remit than mobile technology alone, covering everything from the early development of intelligent tutoring systems to the experimental tangible and pervasive systems currently being 2 developed in COGS. One of the core issues for the course team was to ground the students understanding of educational technology within an appropriate pedagogical context. Consequently, the students were introduced to different pedagogical models that might underpin different kinds of learning systems and encouraged to use whatever technology best fitted their purpose. The course team also provided a website for information along with access to lecture slides. The website was particularly well used during a lecture on online learning which was delivered via the site (details below). Two one-hour lectures a week were used to cover the syllabus content, while the seminar time, two hours for postgraduates, one hour for undergraduates, was given over to an exploration of the issues surrounding the use of
1
1. Structure of the course

Interactive Learning Environments is the latest incarnation of a long-running course, at the School of Cognitive and Computer Sciences (COGS), that explores the use of technology in education. The course is offered to third-year undergraduate students as well as to postgraduates from a variety of masters courses. Because of the speed at which educational technology develops, the course has to be regularly rewritten and updated. In
88
mobile technology for learning. Seminars for both groups of students consisted of practical activities using the mobile device, as well as workshops on topics such as personalisation, collaboration, design and evaluation. At the start of the term all the students took part in a datagathering exercise, either on campus or in central Brighton. The exercise was designed as a familiarisation exercise and to illustrate the potential use of mobile devices within the context of Key stage 2 of the UK National Curriculum, the students acting as designers for technology to support 1011-year-old pupils. Having gathered the required data they then had to send it back to base where it was collated onto a spreadsheet and displayed on the course website for viewing with the XDA. In reflecting on the exercise, students were expected to consider practical and safety issues, as well as issues of pedagogy and appropriate use of the technology. Towards the end of the term the emphasis for postgraduates and undergraduates shifted, in that the postgraduates had to design and run a learning experience for the undergraduates using the devices the undergraduates then had to evaluate this session. Formally assessed work at this point in the course was similarly focused, with the postgraduates required to concentrate on producing their own design guidelines for developing interactive learning environments using mobile devices, while the undergraduates used multiple data sources, including video and still photography, for their evaluation of the session run by the postgraduates.
communication. When using digital technology to support learning, the artefacts in use (XDA, networked PC, paper) and the operations they provide (Powerpoint, www, e-mail, word processing, etc) are merely dialogue enablers not the focus of attention. To maintain a coherent narrative about the domain being learnt about in this case the design of interactive learning experiences the course material was structured in episodes that were specific to a particular learning goal and not tied to the technology through which they might be experienced. An example will clarify. In week 8 of the 10week course the topic was online and distance learning environments. The goal for the session was for students to gain a greater understanding of the challenges facing the designers of online and/or distance learning environments. This section of the course consisted of multiple linked elements: a Powerpoint presentation, an interactive webbased poll and a discussion forum. Online and paper-based resources were also identified. Students were encouraged to log on to the course website at the normal lecture time (9.15am on Wednesday morning) and to follow the Powerpoint presentation. Within the presentation students were asked to consider the key features required for effective face-toface learning and likewise for effective online learning. They were then asked to follow a link to the interactive polls on the website and vote for the three features they felt were the most important in each of these learning contexts. On returning to the Powerpoint presentation they were encouraged to reflect on their views and move on to the online discussion group to share and discuss their reasoning with the group. The learning context was, to an extent, within the control of the individual learner: students could choose to log on via a computer on campus or at home. Alternatively all the course elements could be accessed via the XDA, in which case students may be in bed, on the bus or in a coffee bar in town. The material in the course was designed in accordance with the session goal. The material was developed in a manner that allowed it to be accessed across multiple platforms. The Powerpoint slides were simple, with audio annotations and no images, so that if students chose to use their XDA the file could be downloaded with minimal delay and viewed easily on the small screen. During the one-hour lecture session learners were required to be discursive, adaptive, interactive and reflective with the support of multiple media and a choice of technology platform and location. The online group forum remained a repository of the
2. Pedagogical grounding for the course

The pedagogical grounding for the course itself derives from Diana Laurillards Conversational Framework (Laurillard 2002). This approach aims to clarify the mediated nature of academic learning and to define its essential components. It identifies the component activities of an effective learning experience, describes them as discursive, adaptive, interactive or reflective, and stresses that learning is an iterative dialogue within the learner and between the learner and others. This dialogue must operate both at the level of operations and at the level of conceptions. Both these levels must be interlinked so that learners engage with the concepts of the domain to be learnt about not just with the medium of their
Luckin et al.
89
discussion, as well as providing storage and exchange facilities for other student-generated data.
4. Preliminary data analysis

4.1. What did the students think of the device?
Attitudes ranged from enthusiastic to antagonistic, with most students recognising the potential of the technology but making statements such as the device isnt quite there yet. Others felt that they had not really had the opportunity to engage properly with the device, either because they had to share one: little incentive to use calendar, etc when you only have it for three non-consecutive weeks, or because handing it back at the end of the course limited how much data they were willing to put onto it: it wasnt m y device so I didnt bother putting stuff on it. Interestingly, very few students took the opportunity to synchronise the device with a home PC (the third party software we purchased for Mac synchronisation has proved to be problematic). The large number of different functions were also considered off-putting by some: you can do too much stuff on it, who needs all that? and our lives are not complicated enough to require the use of these devices. A major issue, which engendered much discussion, was the size of the device in relation to the large number of functions it tries to provide. As one postgraduate student put it: Its too small and too big carrying it around is a major issue. This seemed to be the case particularly for male students who were used to being able to carry a small mobile phone around in a pocket. When used as a phone the device was generally considered clumsy and too large, on the other hand the screen was too small to be used comfortably for the integrated Office functions (Word and Excel) or for web browsing a small laptop would make more sense. The reliability/trustworthiness of the device was another issue often raised: some of the functions were particularly prone to freezing or crashing, others had problems with GPRS (web) access and a number of students lost all data when the battery was not recharged in time. Other students underused the device because of concerns about inadvertently exceeding the data download allowance on the tariff. We purchased third-party software to monitor data traffic, but this too proved unreliable. As we were unable to get figures for use from the airtime service provider this created a climate of nervousness among the students; they were concerned about incurring debt if they used data above the agreed tariff. Most students tried at least half of the functions offered by the device (Figure 1), although e-mail came out a clear winner as its most useful feature (Figure 2).
3. Data collection
Throughout the course a great deal of time and attention was paid to the issue of data collection and evaluation. This was extensively discussed with the students as part of the process of developing their understanding of data collection issues. We covered the benefits and problems associated with different kinds of data, as well as attendant issues such as privacy and consent. The following data was collected: university e-mail traffic between course participants logged from week 4 onwards: when an email was sent between two or more people involved in the course we knew whether it was sent via the XDA or not e-mail checks on the COGS server were logged as coming from either the XDA or another device access to the course website was similarly logged as being with the device or not backups of the devices allowed logging of use of other functions, such as the calendar complete record of the e-mail exchanges via the online group data on student attitudes and learning preferences from a poll taken during the online session video and still photography from the postgraduate student-led session SMS data showing the patterns of collaboration during the student-led session data from an end-of-course questionnaire about student study habits, external access to technology and their attitudes to the XDA (it also covered preferred input methods and feelings about the usefulness of the software and functionality provided) qualitative data from notes taken during an end-of-course evaluation session with the postgraduate students.
90
4.2. What kind of learning resources do students value?

4.2.1 The online learning experience in week 8 Students were enthusiastic about this session (described earlier) and joined in during the normal lecture time (9.15am on a Wednesday). They accessed the Powerpoint slides from the website and voted in the polls. The most popular selections for the three key features vital to the success of a normal lecture and seminar-based, face-to-face teaching experience were: approachable, knowledgeable and enthusiastic tutors: 25% of the votes fully resourced course website: 13% of the votes opportunity to take part in group work: 12% of the votes. No students felt that interactive media in lectures or state-of-the-art technology resources were key features here. Similarly, the most popular selections for the three key features vital to the success of a distance and online course experience were: tutor support online: 24% of the votes web resources: 17% of the votes conference environment and e-mail: 14% of the votes. In both cases the tutors role was seen as the key feature and tools to support or opportunities to take part in, collaborative group work were seen as important along with webbased resources. Most (82%) of the students thought it was harder to design resources for an online learning experience than for a face-toface one. In addition to this, internet access was viewed as the most important technology for both online and face-to-face teaching situations. Students want to be connected and this is one of the key features devices such as the XDA can offer. Students were engaged and willing to continue on to the discussion forum where 52 messages were posted. Several students also took advantage of the chat room, though there is no record of this discussion. A content analysis of the messages posted to the discussion forum reveals that the largest category of talk was about the technology and its operation. However, there were also large amounts of discussion about the key features for learning in distance and face-to-face
Figure 1 Features of the XDA tried by students
Figure 2 The feature considered most useful
Having used the XDA, and explored how it might be used within an educational context, the students were asked whether they saw a clear educational use for the device (Figure 3).
Figure 3 Student views on educational use of the XDA
Luckin et al.
91
contexts. Examples from the different types of talk included the following.
4.3.
E-mail and web-logging data
Context/process Mmmm.... waking up almost an hour later than usual, looking at the slides while still in bed and listening to the audio over breakfast. Am now planning on taking the bus and continuing the lecture with my XDA ... how nice
Operational/technology I'm battling with a UNIX terminal, not good for media, to (sic) slow. Key features of a VLE/F2F the Asker polling system Well, I have answered the first set of questions and here are (sic) what I said and why: Enthusiastic tutors, interesting and accessible books, and up-to date content. I think these are the most important characteristics for a face-to-face course.
Figure 4 illustrates the content breakdown within the discussion forum.
This data is still in the process of being analysed, but the preliminary findings show some interesting usage patterns. The e-mail logging only began in week 4 of the course. This was due to extensive discussions held with the students about issues of privacy regarding their use of e-mail and who they were mailing. The decision was made to only log e-mail traffic between course participants, not content, nor e-mails sent to individuals outside the course. Not surprisingly, the course admin. team came out ahead in the average number of e-mails sent to course members, both using the device and not using it. The postgraduates were more frequent users of the device overall for e-mail, but the undergraduates were at the severe disadvantage of sharing devices so not being able to use it to access their personal e-mail. E-mail traffic from the undergraduate device came from a group alias set up for the course, therefore we need to consider carefully how this data might be used for comparative purposes. Access to the course website, on the other hand, did not have the same kind of restrictions 3 (Figures 5 and 6). The overall ratio of postgraduate to undergraduate use of the XDA for accessing the course website is 1:0.7, whereas for access to the website using other devices it is 1:1.5. Overall the undergraduates used the course website more when not using the XDA.
Figure 5 Average website access per XDA device Figure 4 Discussion forum messages (repeated text and header information excluded)
Figure 6 Average website access per user (not using the XDA) There are interesting weekly fluctuations in all the data; these will be examined in the light
92
of different course requirements for the undergraduate and postgraduate groups during those periods. Hourly patterns of use are also interesting, showing that students are active and online even in the early hours of the morning (Figure 7).
of phone calls and a 20Mb download limit, students were then to pay any excess usage. Although this turned out to be a generous limit overall, the lack of adequate software for logging data use made some students overcautious, in that they did not use the device very much in case they incurred charges. This over caution represented the extreme end of a pleasingly responsible use of the devices by the students; none of the devices were damaged or lost, although one SIM card went missing when a student removed it in a shop to try it in another device. Initial results on the use of mobile technology, such as those reported in Mlearn 2001, 2002 and in the 2002 IEEE workshop (Milrad et al. 2002) have been encouraging. Researchers have suggested, for example, that mobile learning enhances autonomous and collaborative learning (Cereijo Roibs and Arnedillo Snchez 2002), and that it can be applied to a wide age range of students (Inkpen 2000; Perlin and Fox 1993; Sharples et al. 2002 and Soloway et al. 2001). The evaluation of this learning experience, in particular the contribution of the technology, is ongoing. However, the initial analysis discussed here would suggest that the provision of coherent learning opportunities and episodes mediated by technology and accessible through multiple devices is possible. Students engaged well with the week 8 session on distance learning. They used the XDA and/or a desktop machine to interact with the Powerpoint presentation; they voted using the website asker and could watch as their peers did likewise and the representation of this data adapted accordingly. They joined in the discussion and reflected upon their differing views. Indeed the discussion continued long after the allocated session had finished. Other emerging positive findings are illustrated by the use of the device for accessing and interacting with information: the course website, and for course-based e-mail exchanges. The students who had sole use of an XDA used it for both types of activity, throughout the day and most of the night. The device enabled them to experience the promise of anywhere, anytime connectivity with learning resources both human and electronic. The technology can certainly support the iterative dialogue we know must take place for learning to be effective. However, this is not universally the case, with students reticent about using the device and failing to engage with much of its functionality. To be successful, designers of
Figure 7 Hourly patterns of use with the XDA device Data gathered from backing up the device was, unfortunately, patchy. Students did not always bring in the device at the required time; others forgot to recharge the battery, resulting in loss of data for that period.
5. Preliminary conclusions
The overall feeling from both the course team and the students was that this was a worthwhile exercise, allowing an investigation of the use of such devices within an educational context. In particular, it allowed students who were interested in becoming developers of such technology the opportunity to explore not only design and usability issues, but also the pitfalls encountered in the real life use of them. From the perspective of the course team a number of valuable lessons were learnt, the most important being that once you add the feature of online connectivity to a device the administrative burden increases dramatically. Particularly onerous was the task of dealing with the company responsible for airtime billing, trying to negotiate sensible tariffs at the start of the project, finding out usage information during it (not possible!) and then renegotiating tariffs when we found we would have been better off on a different scheme. An enormous amount of time was spent maintaining the devices in full working order. As was mentioned above there were numerous problems with the devices in particular GPRS access, installing third-party software, resolving problems with that software when it turned out to be buggy and, finally, negotiating the thorny issue of tariffs and billing. As the devices had been given to the students to use as their own there was the issue of who paid the bills. The project paid the basic tariff for a limited number
Luckin et al.
93
interactive learning experiences that involve this type of mobile connectivity need to provide: a strong focus for the activity to engage learners with the concepts of the domain to be learnt about, with regular reminders throughout the interaction activities that require a clear and simple use of a very limited set of the functions available through the technology regular support from peers and teachers both face to face and online.
www.cogs.susx.ac.uk/projects/equator/equator. htm, accessed 11 November 2003. 3 This is based on normalised data from nine postgraduates and 19 undergraduates.
They also need to be able to access a vast amount of technical support both before and during the course being offered.
6. References
Inkpen KM (2000). Designing handheld technologies for kids. Personal Technologies, 3(1&2), 8189 Laurillard D. (2002). Rethinking university teaching. Routledge. Milrad M, Hopper HU and Kinshuk (eds) (2002) Proceedings IEEE International Workshop on Wireless and Mobile Technologies in Education, August 2930, 2002, Vxj, Sweden. IEEE Computer Society 2002. Perlin K and Fox D (1993). Pad: An alternative approach to the computer interface. Proceedings of SIGGRAPH '93, 5764. New York. Cereijo Roibs A and Arnedillo Snchez I. (2002). Pathways to m-learning. Proceedings of the First European Workshop on Mobile and Contextual Learning, Birmingham, UK, June 2002 pp 5356 Sharples M (2003). Disruptive devices: mobile technology for conversational learning. International Journal of Continuing Engineering Education and Lifelong Learning, 12, 5/6, 504520. Sharples M, Corlett D and WestmancottO (2002). The design and implementation of a mobile learning resource. Personal and Ubiquitous Computing, 6, 220234. Soloway E, Norris C, Blumenfield P, Fishman B, Krajcik J and Marx R (2001). Handheld devices are ready at hand. Communications of the ACM, 44(6), 1520.
Notes
Details of the device can be found at: www.mmo2.com/docs/services/xda_details.html 2 For example, the projects associated with the Equator project, information available at:
1
Mayorga-Toledano, Fernndez-Morales
95
Learning tools for Java-enabled phones: an application for actuarial studies

Mara Cruz Mayorga-Toledano Dep. de Derecho Privado Especial. University of Mlaga, Spain Facultad de Econmicas, Campus El Ejido s.n. 29071 Mlaga, Spain E-mail: mcmayorga@uma.es Antonio Fernndez-Morales Dep. de Estadstica y Econometra. University of Mlaga, Spain Facultad de Econmicas, Campus El Ejido s.n. 29071 Mlaga, Spain E-mail: afdez@uma.es
Abstract
The aim of this paper is to explore the possibilities of Java midlets (small applications that can be stored in a mobile phone) as learning tools in higher education, via an experience we are developing for the actuarial degree of Mlaga University. Keywords: midlets, mobile learning, higher education.
1. Introduction
Mobile phone devices are becoming very popular within the communities of university students in Spain, as well as in nearby countries. As they are mainly used for social purposes, far less attention has been given to their use as learning tools. A previous paper, Mayorga-Toledano (2002), studied the possibilities of integrating the use of interactive tests designed for WAP phones into higher education (HE) strategies. In this paper we explore the educative use of Java midlets for two courses in the actuarial degree of Mlaga University.
or business). These students do not take all the scheduled courses for the academic year and do not attend classes regularly. Based on our experience of running this course, we decided to investigate new channels to communicate with our students and to deliver learning content. Nowadays, the interactions between teacher and students take place mainly in the classroom and on the internet via the virtual campus of the University of Mlaga. However, the widespread use of mobile phones among our students has led us to consider how this technology might help us to improve the motivation of and communication with our students.
Has mobile phone Phone with WAP Uses WAP Phone with Java Uses Java Will buy a new phone (2 years) Has PDA Uses PDA 0% 20% 40% 60% 80% 100 % Yes No Don't know
2. Previous research
The degree in actuarial studies was launched at the University of Mlaga in 1999. Many of the students on the course are already working in banks or insurance companies and have another university degree (mainly in economics
Figure 1. Mobiles used by students
96
The first step we took was to carry out a brief survey among the 112 students of the first course to understand their attitudes to, and knowledge of, mobile technologies. Figure 1 shows the main results of the first part of the questionnaire. It was found that all the respondents had a mobile telephone: 47% of the phones had WAP capabilities, and 18% were Java enabled. However, none of the respondents declared that they used WAP or Java. Also, only 6% of the students had PDAs, and no one reported a regular use of this kind of mobile device. These results indicate that the dominant mobile device among our students is the mobile phone. Although their phones are not currently equipped with the latest technologies, 77% of the students declared they will buy a new mobile phone within the next two years, and it is likely that many of these new devices will come with WAP and Java.
development of learning tools for mobile phone devices in our actuarial degree courses. In a previous study, Mayorga-Toledano (2002) developed WAP interactive tests for students of tourism at the University of Mlaga. The results were unsatisfactory, because the students found WAP connections very expensive and did not take the tests regularly. Now we are developing educative tools based on the Java platform to avoid those main drawbacks.
3. An educative application of Java midlets in higher education

Java technologies for mobile phones are opening up new opportunities for the development of educative applications in the field of mobile learning. Java midlets are small applications that can be stored in the mobile phone and they have two important advantages compared with WAP applications, which we explore in an earlier paper. First, once the midlet is stored, it can be used offline, without connection costs. Second, the popularity of Java games makes this technology very familiar to the students. The former can alleviate the problems noticed by Cher Ping and Chwee Beng (2002) and Loh (2000) in using mobile devices for m-learning through the WAP protocol. We are developing educative midlets for two courses in the actuarial degree of Mlaga University: Banking, Insurance and Securities Market Law (BISML), and Actuarial Statistics (AcStat), both courses being held in the first semester of the first year. Our goal is to make the midlets available by the beginning of the 2003/04 academic year. The midlets are integrated into a blended learning strategy that includes virtual (webbased activities and midlets) and non-virtual elements. The rationale behind this strategy is trying to ensure that every student can access contents independently of the channel he or she chooses to use. The midlets we are developing are deliberately simple and lightweight. They share a similar structure and interface, but their contents are specific to the two courses. Rings 2001 study about e-learning that combines web and mobile devices states that mobile devices are best used to support particular aspects of learning, like alerts, reminders, multiple-choice tests, or glossaries. The contents of our midlets are very similar to the ones mentioned in that study.
Micromodules Tests Glossary Tutor hours Calendar 0% Not useful 20% 40% 60% 80% 100% Very useful
Neither Useful useful nor not useful
Figure 2. Opinions about educative tools for the mobile phone The second part of the questionnaire focused on the attitudes of the students towards different kinds of educative tools designed specifically for the mobile phone. The results of this part are summarised in Figure 2. In general terms, the majority showed a good response to the five tools noted in the questionnaire. The tool with the highest percentage of very useful responses was the test (at 47%), and the one with the lowest was the micromodule (at 12%). A micromodule is a small independent application, related to a theoretical or practical concept, that complements the non-virtual teaching, like a collection of mini-charts showing the legal hierarchy in the banking market or a program that calculates probabilities and critical values of a normal model. The findings obtained from the survey were the main incentive for the
Mayorga-Toledano, Fernndez-Morales
97
have been designed specifically for the mobile phones, so their main features are the following: reduced number of questions in every test questions and answers expressed in deliberately simple, although strictly correct, language answers easily selected with the phone keys immediate feedback the user knows in real time if their choice is correct. Figure 3. Main screen of the midlet The midlet for the Law course includes a calendar, a glossary, and interactive tests (see Figure 3). The first module, the calendar, is essentially informative, while the other two, the glossary and the tests, are actually learning tools. The AcStat midlet has the same modules as the law midlet, but also includes a list of the main mathematical functions used in the course, classified by lessons (see Figure 6).
Figure 6. Example of the functions module
Figure 4. Calendar The calendar module includes useful information such as the dates of exams, the internet address of the course for additional online resources, calendar of classes, and so on (Figure 4).
4. Conclusions
There has never been a technology that has penetrated the world with the depth and speed of mobile telephony. The challenge for universities now is to develop didactic environments and tools for mobile devices and to integrate them into their learning strategies. For this reason it is important to develop and experiment with adequate tools to investigate their benefits and effectiveness. As educators, we need to be open to new teaching instruments and new ways of learning. In doing so, we are setting a good example to our students. Despite the limited possibilities of Javaenabled mobile phones, it is possible to design learning tools for this environment, like the ones proposed in this paper (mainly the interactive tests), which complement the integral process of learning. Students showed interest in them in a previous survey and we expect that this will continue when the tools become available, although this aspect deserves further research. In addition, if the tools proposed in this research reach a minimum level of use, we plan to
Figure 5. Example of a test The most interesting module of the midlet is the one that contains interactive tests. They
98
develop more tools like a mini-lab for experimenting with the main mathematical models of Actuarial Statistics in the AcStat midlet. Finally, it is worth mentioning that we believe in a flexible learning strategy. That is, all our didactic resources must be available in several types of media, to allow students access according to their own preferences, attitudes and requirements. This implies that the tests and other contents presented here will be also available via the internet, and in the campus laboratories.
5. References
Cher Ping L and Chwee Beng L (2002). Mediating E-discussions with WAP technology: the experience of the National Institute of Education (Singapore). Techknowlogia. At www.techknowlogia.org/TKL_active_pages2/Cur rentArticles 4(2) Mayorga-Toledano M (2002). Interactive tests for WAP devices: an application for law courses in tourism studies. In A Mndez, JA Mesa and I Solo (eds) Educational technology. Junta de Extremadura. Loh CS (2000). Learning tools for knowledge nomads. Educational Technology, 4(6), 514. Ring G (2001). Case study: combining web and WAP to deliver e-learning. Learning Circuits. American Society for Training and Development (ASTD), June 2001. Sariola J (2001). What are the limits of academic teaching? In search of the opportunities of mobile learning, compelling educational uses of wireless beyond the wireless LAN?, Vancouver: Telelearning.
Mifsud
99
Learning 2go: making reality of the scenarios?

Louise Mifsud Agder University College School of Management Department of Information Systems 4604 Kristiansand Norway E-mail: Louise.Mifsud@hia.no
Abstract
This paper draws on preliminary findings from a pilot study of a sixth-grade class at a 1 University of MichiganHi-CE project school. The study gives indications of what can be achieved with handheld technology in education a transition from disruptive technology to useful and enhancing educational technology. It focuses on the school arena because this has been the focus of much debate with regard to educational technology. This paper discusses the role of mobile and handheld technologies in education, exploring in which arenas handhelds are being used, and how and why are they used. This leads to the question: are handheld technologies changing classroom culture? Keywords: handhelds, learning culture, integral to daily learning, disruptive technology
own will not fulfil the promise many believe lies in the use of information and communications technology (ICT) in schools (Bransford et al. 2000). However, this paper is based on the premise that flexible access to handheld technology will provide the tools to help children construct knowledge throughout their daily activities, making such technology an integral part of daily learning (Soloway et al. 2001).
2. Mobile learning technologies in education

Many writers have developed scenarios describing the use of handheld technology both inside and outside the classroom, and the difference between learning inside and outside school has often been addressed (Lave and Wenger 1991; Resnick 1987). Learning has been described as being locked in the schools formal setting (Somekh 2002). The National Council of Research report How people learn (Bransford et al. 2000) emphasises that bringing students and teachers in contact with the broader community can enhance their learning, while the OECD report Learning to change: ICT in schools (2001) illustrates that ICT has established a new complementarity between formal learning in school and informal learning outside. Miettinen (1999) has pointed out that school learning is characterised by the memorisation and reproduction of school texts in which
1. The possibilities learning
of
mobile
Mobile and handheld computers offer new possibilities in education. Computer technology has been criticised for being segregated from the ongoing aspects of childrens lives, being relegated to the computer rooms in schools, and making the use of personal computers (PCs) anything but personal (Soloway et al. 2001). It has been suggested that access on its
100
teacher talk dominates, and students activity is largely limited to answering questions formulated by the teacher. This raises the question as to what kind of learning culture is predominant in the school or specific classroom. In this case culture refers to the way things are done in a particular setting, ie the social process and context. In such a learning culture as Miettinen describes, drawing on examples of mobile telephony in classrooms, handheld devices can be regarded as intruders in the learning culture, a disturbance (Mifsud 2002), and therefore a disruptive technology. The term disruptive technology has mainly been used in organisational theories. Christensen (1997) noted: Disruptive technologies are usually simpler and cheaper offer less capability they are usually shunned by well-managed companies which are often later destroyed by [the disruptive technologies]. While handhelds can be described as cheaper, and offering less capability, this definition does not entirely fit the picture of school. However, the term disruption as interrupting the flow or continuity (see T h e Concise Oxford Dictionary 1990) does seem to apply. Sharples (undated) points out that the assumption that computer-mediated learning will occur in the classroom, managed by a teacher is now being challenged by the growth in access to personal technologies. Many children already have access to a wide range of computing and communications devices. As well as the examples of mobile telephony in classrooms (Mifsud 2002), Mifsud (2003) suggests that the new types of mobile telephones and personal digital assistants (PDAs) offering extra functions, such as voice and image recording, can also act as a disruptive technology. Disruption or disruptive technologies can also be associated with control. Sharples (undated) suggests that handhelds may become a zone of conflict between teachers and learners, with both trying to get control of the opportunities it affords for managing and monitoring learning. He further points out that this potential for confrontation needs to be recognised and addressed. Mifsud (2002) addresses the same topic, referring to personal experience of the classroom culture, where for example communication is mainly controlled by the teacher and punctuated by raised hands from students wishing to contribute to a school or classroom-related discussion. In this setting, other forms of communication, between friends or classmate, for example, are not legitimate. Some teachers have also expressed worries at the implementation of PDAs in their
classrooms and learning culture because they might be used for things other than schoolrelated work, such as playing games, pranks, emailing friends in and out of school or cheating on tests (Trotter 2001). These worries are also reflected in the final Palm Education Pioneers Report (Vahey et al. 2002), where inappropriate use of handhelds is one of the drawbacks: games are played during class time, downloading inappropriate materials and inappropriate use of beaming (for passing notes, cheating on tests and copying by handing in assignments beamed from other students). These worries can also be related to Sharples zone of conflict over control (Sharples undated).
3. From scenario to reality?

3.1. The learning in the palm of your hand project
Hi-CE, the Center for Highly Interactive Computing in Education at the University of Michigan, is currently working with schools in Michigan to integrate Palm handheld computers into classrooms. In this project they are investigating two models of student use of Palm handhelds: the personal computer model, where each student is assigned a Palm computer to take home; and the class set model, where there is a class set of Palms that teachers use for specific curricular activities. HiCE is currently working with the third, sixth, eighth and ninth grades (ie students aged between 7 and 14). This paper focuses on a pilot project involving sixth-grade classes using the personal computer model. Hi-CE is also developing and researching a collection of applications for the classroom the Cool Dozen based on the Palm operating system (OS), along with instructions for each. One of these is PiCoMap, a concept-mapping program. Students working on a topic can first work on their own, making their own concept map. The concept map can then be beamed (sent through an infra-red port) to another PDA. The programs also include an offline browser 3 (FlingIt) ; a scrapbook maker (Go n tell) that can be used with a camera to create a story illustrated with pictures; FreeWrite, a wordprocessing program; Sketchy, an animation tool featuring geometric objects; many pen options, and an easy-to-use interface.
2
3.2. The pilot study

The aim of the pilot study was to find out in which arenas handhelds are used, how and why they are used and what role they can play. As Inkpen (1999) points out, referring to the
Mifsud
101
entertainment world of Nintendo and Game Boy, handheld computer technology for children is not a new idea. The 24 pupils in this pilot study are 12-yearold sixth graders at an Intermediate School in Michigan, USA. The school is a consolidated school, meaning that some of the pupils have a long journey to school sometimes as much as an hour. The class is a Hi-CE project class. The 4 teacher has been a Jason project teacher for nine years. She had received information about the Hi-CE project and initiated contact with them. She has been using handhelds in teaching for three years. The students used the Palm III handheld computer and had started using their handhelds at the beginning of the school year in September 2002. They could take their handhelds home if they want to. The teacher reported that she has had excellent technical help from the Hi-CE team.
interviewees was chosen because he appeared to be tapping away at his handheld most of the time; the rest of the group was randomly 6 selected. This was done to try to determine what would be most appropriate in the final study. The interviews in the pilot study were semi7 structured. The first interview was recorded on a mini-disc, but during the interview the students supported what they were saying by referring to their handheld, so it seemed that it would be more fruitful to film the interviews. As a result, the information saved on the students handhelds was also recorded.
3.4. Findings and discussion

3.4.1 Arenas of use All the pupils in the study indicated that they used the handheld in arenas other than school, such as home and in the car, with most using games. These games were used in situations that the pupils described as potentially boring, or when they did not have anything to do. Students also reported using their handhelds on their way to school, both for finishing assignments and for games, especially where the students lived far away from the school. The Palm Education Pioneers Report also concludes that handheld technology can be used in different contexts and in more places than in the classroom. (Vahey et al. 2002). As Inkpen (1999) points out, one of the main advantages of handheld devices is their ease of integration into a childs world the products themselves become a part of the childrens culture. The concept maps also suggest that the students had a clear understanding of which functions were used, for what purpose and in which context. Examples of this include concept-mapping, offline browsing and wordprocessing in the school arena, the address book at home, and games at home, at school and in the car.
3.3. Methodology
Classroom observations and informal interviews with pupils and teachers were undertaken. The first design of the pilot study opened up for observation not only inside the classroom but also outside in the canteen, library and school-yard. Jensen (2002) defines observation as referring to a set of research activities that involve the continuous and longterm presence, normally of one researcher, and generally in one delimited locale. The canteen and the school-yard proved however difficult to put into practice as following the students very closely in their free time means disturbing their natural behaviour and is also quite intrusive. Catching what the students are talking about also proved to be difficult in these arenas. To compensate for this, the students were asked to draw a concept map depicting where, for what and why they used their handheld computer (see an example in Figure 1). Concept mapping was used as a means of expressing ideas quickly, and providing evidence from each of the 5 pupils. According to the ImpaCT2 study, concept maps consist of putting words that represent concepts in boxes and linking these by means of words or phrases, so that the connections can be read. The study refers to Novak and Gowins 1984 work, where they found that this approach gave researchers more accurate insights into pupils thinking than traditional methods of testing, or a mind map. After two weeks of observation, some students were interviewed. Two different forms of interview were used group or focus interviews and the more traditional one-to-one interviews. The lessons observed were taken as the starting point of the interview. One of the
102
Figure 1. Concept map from a sixth grader
3.4.2. In the classroom arena In the classroom it was observed that the students appeared to work mainly in groups of four. The teacher and the students had agreed on a set of rules for use of the handheld, which specifically included the games. The students were allowed to play games on their handhelds as long as they had finished the task set, and most of the students were observed playing games towards the end of the lesson. The students appeared to move freely from one group to the other exchanging information (sometimes it appeared that they also 8 exchanged games or the results of games ). The program PicChat also allows immediate communication between the students (through the infra-red port), which the teacher has no control over. However there did not appear to be any conflicts over the zone of control. In fact, the teacher herself commented that she had to eventually let go of some of the control. The Palm Education Pioneers Report (Vahey et al. 2002) suggests that it is important that teachers find time to research available software and peripherals and take time to learn how to use them as well as how to integrate the handheld, software and peripherals into their learning activities [emphasis added]. Yet time is a commodity which is hard to come by for any teacher. The teacher in the project pointed out that she did not fully integrate the use of the handheld until she became more familiar with the many ways to use the Palm it just sort of came natural as I became more knowledgeable'. Observations at the Hi-CE project school suggest that the handheld computers were an integral part of the daily flow of school and classroom activities. That is, they were used as and when needed and in context; and also refers to seamlessness without disruption in the flow of activities. The teacher did not always
ask the students to use the handhelds, although she did sometimes make suggestions about what programs could be used. It was up to the students to find what they deemed to be the best way of achieving the task at hand, whether this meant using word-processing, animation tools, concept maps or role-play. This appears to be supported by findings from the Palm Education Pioneers Report (Vahey et al. 2002) where the teachers also indicated that handheld computers were more easily integrated with the flow of learning activities than desktop computers. The OECD report (2001) points out that in a world with easy access to huge stores of information, the skills of accessing ... are more important than the ability to recall in detail ever greater amounts across many fields of knowledge. In one of the lessons observed, the students were working on a project on medieval castles. They were also building castles from old milk cartons, aluminium foil and paint. The desktop PC was used for online browsing, with the information that the students found relevant downloaded to their handheld using an offline browser called FlingIt. Their handheld does not have unlimited memory, in this case the memory was 8MB, so the students had to be selective in what they chose to download and establish a set of criteria for making their choices. While it is too early to draw conclusions, there did seem to be an indication that the students learn to pick and choose the websites they need, and assess their relevance to the task at hand. The teacher did suggest in 9 the beginning that websites flung from the Internet to the handhelds are limited to two links and a few pictures, but it is ultimately the student who decides what is relevant for the task at hand. During one particular lesson, some of the students decided that they needed a depth of three links as well as pictures of castles but the observations of the information flung show that the students do generally limit the number of links and download pictures sparingly.
4. Reflections
The use of handheld technology in education is at a beginning phase, with more than a decade of research of desktop ICT behind it. Soloway et al. (2001) stated that we can now try to learn from the past mistakes of ICT implementation in schools. Several new questions are also raised as a result, including: Does the transition from disruptive to non-disruptive technology require familiarisation with the technology? Is letting go of some control indicative of
Mifsud
103
networks of learning. In Y Engestrm, R Miettinen, R Punamki (eds) Perspectives on activity theory. USA: Cambridge University Press. Mifsud L (2002) Alternative learning arenas Is the teacher who chooses to use pedagogical challenges to mobile learning handhelds in the classroom contributing in education. WMTE Proceedings 2002. to the flow of activities in the classroom IEEE. and thus seeking to change the learning Mifsud L (2003). Learning 2go: pedagogical culture? (The use of handhelds in this challenges to mobile learning technology in project was not imposed by education. In K Nyri (ed) Mobile learning: management.) essays on philosophy, psychology and education. Vienna: Passagen Verlag. OECD (2001). Learning to change: ICT in The research discussed in this paper s c h o o l s . Organisation for Economic indicates that handhelds can be a useful, Cooperation and Development. enhancing rather than disruptive technology and Resnick L (1987). Learning in school and out. an integral part of school, and life. Educational Researcher, 16(9), 1320. One of the first challenges is to view the Sharples M (undated). Disruptive devices: technology as a bridge between different mobile technology for conversational learning arenas, and for teachers to take time to learning. At find out how to integrate it into their learning www.eee.bham.ac.uk/sharplem/Papers/ijce activities. When access is no longer a problem, ell.pdf, accessed November 2003. the challenge lies in using mobile technologies Soloway E, Norris CP, Blumenfeld P, Fishman well, both as an enhancer in the classroom and B, Krajcik J, Marx R (2001). Devices are to bridge arenas that are usually referred to as ready-at-hand, Communications of the separate such as school and free time. ACM (June). Somekh B (2002). Unlocking learning from 5. Acknowledgements formal settings. At I am indebted to Professor Soloway and the www.itu.no/Aktiviteter/Konferanser/2GOHi-CE team (University of Michigan, USA) for foredrag/1035466042_97/view, accessed making this research possible. I would also like November 2003. to thank InterMedia, University of Oslo, for their Trotter, A (2001). Handheld computing: new support. I would like to extend my thanks to best tech tool or just a fad? Education Professor Sigmund Lieberg (University of Oslo, Week. At PFI) and Associate Professor Anders Mrch www.edweek.org/ew/newstory.cfm?slug=04 (University of Oslo, InterMedia). palm.h21, accessed November 2003. Vahey P, Crawford V, Lewis A, Toyama Y, 6. References Huang J and Whaley A (2002). P a l m Education Pioneers Program: final Bransford J, Brown AL, Cocking R (2000). How evaluation report. At people learn. National Research Council, http://palmgrants.sri.com/PEP_Final_Repor Washington DC: National Academy Press. t.pdf, accessed November 2003. Christensen CM (1997). The innovator's dilemma: when new technologies cause great firms to fail. Boston, MA: Harvard Business School Press. Notes Inkpen KM (1999). Designing handheld 1 Center for Highly Interactive Computers in technologies for kids. At Education at the University of Michigan, USA, for www.cs.sfu.ca/~inkpen/Papers/hcscw_inkpe more information see www.handheld.hice-dev.org/ n.pdf, accessed November 2003. 2 See www.handheld.hice-dev.org/ Jensen KB (2002). The qualitative research 3 This information can be downloaded from the process. In Jensen (ed) A handbook of internet to the PDA and is available offline, anywhere media and communication research: at any time. 4 qualitative and quantitative methodologies. According to their website, the aim of the Jason London: Routledge. Project is to spark the imagination of students and Lave J and Wenger E (1991). Situated learning: enhance the classroom experience. From oceans to rain forests, from polar regions to volcanoes, the legitimate peripheral participation. USA: JASON Project explores Planet Earth and aims to Cambridge University Press. expose students to leading scientists who work with Miettinen R (1999). Transcending traditional them to examine its biological and geological school learning: teachers work and a change in the learning culture? Does the zone of conflict over control have to disappear?
104
development. The JASON Project offers an interdisciplinary, multimedia approach to enhancing teaching and learning for students and teachers. More information is available at www.jasonproject.org/jason_project/jason_project.htm , accessed November 2003. 5 ImpaCT2 study. For more information see: www.becta.org.uk/research/impact2/index.cfm, accessed November 2003. 6 Students were to draw lots for who would start at which base in a particular lesson, and the students who drew the first lot were the students in the group interview. 7 A sequence of themes and some possible questions were identified, but there was also an openness to changes in the sequence and questions that would make it possible to follow up the interviewees narrative. 8 Some of the conversations between the students are not clear because an external microphone was not used. 9 Students, and the teacher, appropriated this term from the program FlingIt and thus flung websites from the internet to their handheld.
Mitchell
105
Exploring the potential of a games-oriented implementation for m-portal

Alice Mitchell Ultralab, Anglia Polytechnic University Bishop Hall Road, Chelmsford CM1 1SG E-mail: Alice@ultralab.net
Abstract
The m-learning project is a 3-year panEuropean project, supported by the European Commission's Information Society Technologies (IST) programme within the Fifth Framework. It is investigating how the use of mobile technologies might address the literacy and numeracy skills needs of young adults aged 1624 who are outside formal learning settings, start to change their attitudes towards learning and contribute towards their life chances. Mportal is Ultralabs contribution to the m-learning project. It forms the interface layer to the mlearning system, a virtual tutor and skills-based learning materials. Our aim is a user-friendly portal layer that is engaging and empowering, and will attract young people to learning. As part of our research activity, we have undertaken a literature review of the use of computer and video games for learning, including learners experiences with educational computer games and associated design issues. We have also undertaken associated field research with target audiences. The research findings are reported here and appear to support our view of the motivating potential and possible learning gains of games played on mobile devices with target audiences. The findings lead to recommendations that may be of interest to prospective software developers in the design of computer games, including games used on mobile devices, which can engage young adults in learning. In the light of the investigation the paper concludes that a games-oriented m-portal interface developed with such engagement in mind could have potential for encouraging a learning culture among target audiences. Some key challenges involved in seeking to implement the idea are highlighted. Keywords: computer games, video games, mobile games, multi-user role-play games, mportal, learning potential
1.
Background and rationale
Ultralab, a learning technology research centre based at Anglia Polytechnic University (APU), is developing a learning interface for mobile devices (m-portal). This is our contribution to the m-learning project, a 3-year pan-European project supported by the European Commission under the IST Programme, Fifth Framework. Coordinated by the UKs Learning and Skills Development Agency (LSDA), the m-learning project seeks to address some pressing socio-educational problems relating to many young Europeans in the age range 1624. These are: poor literacy and numeracy and non-participation in conventional education, leading to possible unemployment and social exclusion (OECD 2000). Many of these target audiences lack access to a computer, but do regularly use a mobile phone. Thus, the m-learning project is using readily available mobile technologies that many young adults find useful and attractive to attempt to re-engage them with learning. Our aim is to produce a user-friendly mportal that is powerful and empowering, and encourages active participation by its users. To address these challenges, and to ensure that m-portal is needs-driven in the sense that it meets the needs of the young adults, we undertook an iterative programme of research and development. This includes gamesoriented field research engaging co-researchers from the target audience, complemented by a review of the literature relating to computer and video games. This paper reports a selection of findings from the literature review and the gamesoriented field research. It draws conclusions and makes recommendations that may be of interest to prospective software developers. The terms games and computer games are used inclusively to denote all kinds of interactive computer games, regardless of
106
hardware platform. Playing and gaming are used to denote the playing of computer games in general; gaming also denotes the playing of online games.
2. The literature review: summary findings and recommendations

2.1 Review limitations and focus
enhance self-esteem and confidence (Ritchie and Dodge 1992) support cognitive apprenticeship (Greenfield et al. 1996) especially where users have control over the tools (Small and Samijo 1997).
As a joint initiative, LSDA undertook an intensive search of the literature about the use of computer and video games for learning, and Ultralab reviewed the resulting material. The full report on the use of computer and video games for learning will be published by LSDA later in 2004. These summary extracts focus on the: use of computer games to encourage learning users experiences of educational games: two studies designing edugames (educational games).
Prensky (2001) emphasises the importance of a user-relevant context and recommends selective use of games styles geared to both content and learning activities. Other authors concur with this view including Brownfield and Vik 1983; Randel et al. 1992; Griffiths 1996). Studies conducted by the British Educational Communications and Technology Agency (Becta) offer the following ideas for incorporating computer games into learning environments (Kirriemur 2002): multi-user online games used in class and linked to formal homework or an informal fun activity could enable students to access and exchange data with classmates or students elsewhere and then e-mail their work back to the teacher multi-user online games used with networked library services could encourage collaborative research activity (although research was advocated to test whether this was feasible).
In reviewing the literature, a number of limitations were found, including perceived inadequacies in research design and methodology, short-term focus and the use of non-random sample populations (Randel et al. 1992; Berson 1996; Dempsey et al. 1994; Harris 2001). Thus, assertions resulting from this review are open to debate.
2.2 Using computer encourage learning
games
to
2.3 Users learning experiences of educational games: two studies

Leddos (1996) study specifically investigated learners experiences of educational games. It was found that learners preferred games to standard classroom instruction, but and this is a big but students would never voluntarily play such a game outside of class. Essentially such games were disliked where the fun element was missing. Issues of race and gender were spontaneously raised. Females complained that commercial games were male-oriented. A balance of both gender and ethnicity were requested. Proposed solutions included to have the game played from the perspective of the main game characters eyes or to enable user choice via a character editor, a tool that allows modification of the character: gender, ethnicity, and so on. Variety in the context, mission and complexity was also requested in Leddos study. Students wanted novelty, surprise and humour, with little break in action instruction should flow with the game. Performance
A growing number of researchers and theorists (Dempsey et al. 1994; Doolittle 1995; Griffiths 1996; de Lisi and Cammarano 1996; Emes 1997; Mumtaz 2001; de Lisi and Wolford 2002; Kirriemur 2002; Ko 2002; Pillay 2003) ascribe significant benefits to use of computer games in educational settings. They have been found to: act as ice-breakers and rapport-builders (Spence 1988; Gardner 1991; Lynch 1981 and 1983; Phillips 1991; all cited by Griffiths 1996) stimulate curiosity, discovery learning and perseverance (Ehman and Glenn 1991 cited by Berson 1996; Ko 2002; Kirriemur 2002) enable risk-free experimentation (Berson 1996) promote spatial learning and cognitive processing (McClurg and Chaille 1987) provide motivation via immediate feedback (Roubidoux et al. 2002)
Mitchell
107
feedback was very important (Betz 1995; Berson 1 9 9 6 ). They also wanted the game to be challenging and to become more difficult with improved mastery of the game. Graphics and special effects were liked and virtually all students, male and female, wanted some sort of combat or shooting in the game. This is contrary to other findings (Dempsey et al. 2002) that females are put off by aggressive distracters (ie violent characters and incidents), but is in line with Chappells (1997) view that this may be a faulty over-generalisation. Dempsey et al.s (2002) study with adult respondents advocated the use of simulation, adventure, arcade, board, puzzle and word games for promoting problem-solving and decision-making skills, linked to specific topics. This study found that respondents wanted games with clear, concise instructions that were challenging and game-oriented. Player control over speed, level of difficulty, timing, sound effects and feedback were also desired, together with high-quality functionality to sustain engagement and game structure that was not too complex for different players ability levels.
Referring to Nintendos Super Mario Bros, a leisure game where bricks scattered through every scenario are integral to game play and enhance the fun, Fabricatore (2000, page 15) suggests that these: could be hiding anything else, for instance letters or numbers, and they would still be perceived as part of the gameplay. For proof, he designed and produced six Game-Boy games. He reports these were successfully tested in classrooms with around 300 children aged between six and eight, supervised by their teachers. The supervisors acknowledged unintentional as well as intentional learning gains and also: general improvements in terms of discipline, concentration and eagerness to understand technological issues related to the games they were playing. Fabricatore calls this alternative design approach: edugaming, where there is: no unnatural barrier separating learning from gaming. In similar vein, Prensky (2001, page 179) advocates that learning games should (in this order): Be fun enough to engage those other than target audiences. Allow users to consider themselves players as opposed to students or trainees. Be an addictive experience, producing word of mouth among users. Enable users skills in the learning content of the game to improve rapidly and significantly the more they play. Encourage reflection on what has been learned.
2.4
Designing edugames
Fabricatore (2000) suggests that learning gains associated with gaming should be exploited when designing educational games. He suggests that producers of edutainment software (ie software that engages a persons interest as well as being educational) often seek to make the game subservient to the educational process: where the resulting products do not incorporate some form of opposition, they lack cohesion between game and cognitive task and are not true games. According to Fabricatore, active participation, challenge and the role of struggle are the key concepts (see Randel et al. 1992). Thus, what are needed, Fabricatore says (2000, page 15), are new paradigms in educational game design: A good approach to create better educational games is not thinking what gaming experience can be the most motivating frame for some controlled learning activities, but rather how to create a virtual environment and a gaming experience in which the contents that we want to teach can be naturally embedded with some contextual relevance in terms of the game playing learning tasks must be contextual to the game in the sense that they must be perceived by the player as a true element of the game play.
This last point is important because reflection is seen as a disappearing skill in terms of the users. Prensky further recommends providing a non-game option for those who are not engaged by the electronic learning game in order to cater for different learner preferences. Finally, it is useful to note that Dempsey et al. (1994, pages 56) draw together other writers recommendations for the designers of educational computer games. Intrinsically motivating games (the game structure itself promotes learning) are preferable to extrinsically motivating games, (real or imaginary rewards are given).
108
Pay attention to gender preferences. Incorporate debriefing into game activities. Vary the methods of scoring and levels of challenge. Avoid complex rules and scoring to maximise limited learning time. Deploy or adapt existing gaming strategies to suit particular learning activities, for example: adventure games and games with a strong challenge are suited to learning activities that need to be made more attractive to the learner population, while theory-based simulation games assist with attitude change issues of conflict or control, allowing learners to make serious mistakes in a risk free environment and highly visual simulation games can help with group decision-making. Ensure the game has a satisfactory way of ending.
plurality of meanings. As a theoretical paradigm we found it especially relevant to m-portal development: our users may exhibit different learning styles and preferences and are likely to be disenchanted with formal, extrinsically motivated learning Furthermore, social constructivism considers socio-affective factors and the role of mediation of action through artefacts to be significant in encouraging learning. Again, this is highly relevant to our mportal project, which puts communication tools in the hands of learners. Specific social-constructivist theories that inform the field research include:
3 Our games-oriented field research: summary findings and recommendations

3.1 Research focus
To inform our development of m-portal, we primarily sought understanding and insights from the target audience concerning: the effect of a games application on the perception of a device: the influence that games may have on young adults perceptions of a device in terms of value and attractiveness interface issues: factors relating to the way a particular device interfaces between game and user time spent learning the games: the signs that target audiences will spend time learning rules of computer games rather than learning other things learnability of mobile games: what kinds of learning games might engage target audiences and what would be the possible learning gains? Theoretical considerations
experiential learning theory (Kolb 1984), understood here as: 'education that occurs as a direct participation in the events of life' (Houle 1980, page 221; cited by Smith 2001) and achieved through reflection upon that experience situated learning theory (Brown et al. 1989; Lave 1990), which sees the active learner graduating from newcomer to oldtimer within a learning community. Laurillards (1998) concept of a conversational framework; this is relevant, as it enables a continually iterative dialogue between teacher and students to reach shared understanding.
These theories and concepts already underpin the conceptual design of m-portal in Phase 1: the design incorporates a discursive functionality to enable people to engage in debate on their own terms and to scaffold each other's learning. 3.3 Methodology
3.2
We consider that one of the most useful contributions that m-portal research can make to the project is in designing a new and innovative m-learning environment guided by theory. Social constructivism (Vygotsky 1982) emphasises intrinsic learning through social interactions such as modeling or imitation and accepts the
The m-portal research team comprised developers and field researchers, including colleagues normally outside the project but interested in the field, and potential learners who were engaged as co-researchers. Such a collaboration of researchers and target audiences is well aligned to the views of socialconstructivism (Vygotsky 1982) which underpins the development of m-portal to ensure it is needs-led, rather than technologyled. Our approach falls within the naturalistic, interpretivist paradigm (Denzin and Lincoln 1994). We are seeking to develop ongoing, indepth relationships to arrive at holistic accounts that can afford understandings and insights from different perspectives and within naturalistic settings (Patton 1990; Burgess 1993). The researchers role is that of
Mitchell
109
participant observer, using the principle of reflection-in-action (Schn 1987) to guide aspects of the fieldwork. As will be appreciated, limited use is made of quantitative data; instead our approach is to draw inferences from the data rather than making broad generalisations. In operationalising the research, which followed APUs internal guidelines for ethics in research, access was sought to young adults with our target audiences characteristics. Time was taken to open channels of communication with colleges and youth centres that worked with such audiences. The organisations were either already known to Ultralab or had approached us following dissemination events. They had all expressed an interest in the m-learning project. Among these, a Youth Centre in Reading and a Community College in Devon offered commitment over several months. Access was facilitated by the Youth Centre leader and the College director. The young adults we worked with were of mixed ethnic background and aged between 16 and 18. This was opportunity sampling (Bell 1991): we were not able to recruit a sample of cases covering the full target audience range. Participants willingness to take part may have owed much to the fact that they knew their mentors or teachers had agreed to the project, moreover motivation can change over time (Burgess 1993). However, our experience was that all the participants evinced a strong interest in mobile technology developments throughout. Co-researcher status had a beneficial effect on their self-esteem and willingness to contribute. Data was collected from diverse activities such as conversations and discussions conducted with all the participants, with individuals and in groups, both face to face and online. Face-to-face meetings were conducted with individuals and groups in their natural environments: Youth Centre meeting rooms and the College training room with ICT equipment. Our expectations were that the young adults would play (ie experiment) with the mobile devices we gave them to look at, undertaking the software tasks that were made available. During and after this process, in face-to-face sessions and/or online, they would consider and report back on their perceived learning gains, sharing their experiences, views and ideas with each other and with ourselves. We used a loose structural framework: each session had a clear agenda. There was some use of aide-memoires to support conversations with a purpose (Burgess 1984). The
conversations were relatively informal with the researchers playing an active, reflexive part in the process, not acting as a neutral agent (Mason 1996) and keeping the conversations short and easy. In addition there was partial adoption of a questioning route, both in e-mail correspondence and also in face-to-face group discussion. According to Krueger and Casey (2000) a questioning route aids analysis and also: forces the moderator or research team to think about the words and phrases to be used ahead of time. Where questions were asked along the lines of discussion prepared in advance, this was kept compatible with the natural flow of conversation. It had been our original intention to use audio-visual recordings as a means of collecting non-verbal clues, but a number of participants felt uncomfortable with this approach. Instead, where permission was given, audio-recordings were used and transcripts made. Fieldnotes incorporated notes taken on the spot. Subsequent analysis was informed by critical use of an analysis protocol based on guidelines by Dick (2000), following Glasers (1992) grounded theory approach. The research process was iterative: review time was built into each session, allowing further questions to be raised and theories explored. There was also sufficient time between sessions to ensure these were geared to emerging issues. This accords with Glasers (1992) grounded theory, which allows theory to emerge in an iterative process that is responsive to the research situation. Subjectivity was inherent, both in implementation of field work and in the interpretation of evidence. This can be attributed to the Hawthorne effect. The Hawthorne effect refers to the work of Mayo (1933), where workers production rates at the Western Electric Company increased however such increase was not due to changes in working practices, but rather because of the psychological stimulus of being singled out and made to feel important. In this case, the Hawthorne effect can be applied in at least three important aspects: the roles of the Ultralab researchers, of the college participants and of the technologies themselves. Hence our attempts at the triangulation of data as a strategy that adds rigor, breadth, complexity, richness and depth to any inquiry (Denzin and Lincoln 1994, page 5, citing Flick, page 231).
110
3.4
Field research findings
3.4.1 Effect of the games application on perception of a device

A device that had games applications loaded was received more positively than one that had not. When asked for their general impressions of the different devices they were given to look at, students immediate responses were that games applications make a big difference; there was a favourable perception of a device when it had games, even where it may previously have been found too large or unwieldy. This finding was confirmed by further probing and subsequent voluntary comments.
3.4.2
Interface issues
Size of the device was considered important: ideally phones should fit into a pocket. Ease-of-use was also a key attractor, hence the smart-phone (a phone which also contains some of the attributes normally associated with a handheld computer) was liked because of the easy navigation offered by its Windows environment. Accessibility was another concern: speech bubbles were suggested for use with devices that had no sound and to support users with hearing difficulties. Icons were liked, also in combination with text, where they should be small enough to accommodate a large font. Input was difficult with small devices: a pen facility was preferred to a mini keyboard ease and speed making it more fun.
A colour screen was liked. Fun, speed and ease of use would be the key to sustaining engagement. A time element should be incorporated in graphic form something like Hangman (a game which need not be computer based, where the person has to guess a word and for each wrong guess part of the character is drawn the game ends when the word is either correctly guessed or the character completely drawn denoting the game is lost). Good design was appreciated, for example where the game moved from numeral or icon to word, thereby aiding spelling. The most engaging activity was one enabling them to paint virtual walls by solving a simple equation; the correlation between visual aspects of the solution and achieving a correct answer was the attractor. Short, simple games were preferred, where it was possible to complete quickly then move on to the next. Self-image was involved: completing lots of easy games boosts the users confidence. A plentiful number and variety of games would be important: the co-researchers were easily bored when revisiting the resources, except where the game was tricky. Most rejected the idea of inventing their own games as being too complex and time-consuming, preferring to explore what was available on the internet. However one participant was learning to build a game; he felt it enhanced prediction skills, team-building skills and fitting things together. Examples of the content that was proposed for m-portal included a set of interactive storybooks along the lines of a role-play game. Games connected with aspects of physics, such as angles and force were also proposed, as well as games navigating roads using maps and directions. Sport was felt to be useful in supporting the development of cognitive and spatial skills, for example via casino card games, snooker games, darts and a pinball game (this was thought best in black and white, to work well on a small screen). Simulation games like SIMS were thought unsuitable for most mobile screens again size was an issue. Compaqs iPAQ (pocket-sized personal computer) screen size was considered to be the minimum requirement for this type of game. Examples of skills you could learn in simulations would be handling money and communication skills, especially if there was multiplayer functionality. High concentration levels were sustained throughout the time the group spent exploring the devices and playing the games. There was confirmation of previous findings, ie there were signs of the motivating power of the new
3.4.3
Time spent learning the games
None of our sample population spent much time learning mobile games; on the contrary it was felt games needed clear instructions on how to play. A preference was expressed for multiple-choice options.
3.4.4
Learnability of mobile games
Mobile games were downloaded and played frequently by most co-researchers and therefore can be reasonably expected to engage target audiences. There were signs that females in particular enjoyed quiz games. Both males and females enjoyed fun, challenge and struggle against some kind of opposition. There need not be violence: the struggle can be to raise the level of ones own performance. They felt games had potential for great learnability. This was particularly true of those using multimedia games. Crucially, it appeared that learning games needed to be perceived to be as good as commercial programmes.
Mitchell
111
technologies and of mobile games, and that information and applications should be relevant to users interests (for example sport, travel, hobbies and work). As well as being challenging and fun to use, the games should also provide instant feedback, plus a form of reward. The reward need not be a certificate, but could be points, money, name in a draw, and so on. During the games-oriented sessions there appeared signs of social constructivist learning and skills learning through the sharing of discoveries and the exchange of pointers at getting round the various challenges in manipulating the devices. Furthermore the coresearchers awareness of the beneficial effect of their status on their own learning and selfesteem was apparent: they acknowledged with pride and enjoyment being part of a research project, to which they felt responsibility and commitment, relishing the new contacts and challenges. There were signs of heightened awareness of their existing strengths and competencies, including those developed via participation in the project. This boost in confidence was seen to further their interest in their own learning and in helping others to learn. Inspired by a related project that linked mentors and youngsters via mobile phones, the coresearchers proposed the use of texting to communicate with teachers. There was evidence of creative thinking, eg reconsidering their earlier request for clear instructions for games, they now asked for wizards to help people around the devices, as opposed to written guides, which they said are simply not used.
downloadable single-user games worked particularly well on the kinds of mobile devices owned by our co-researchers. Games need to be fast, fun, easy to use and challenging, to encourage cognitive skills. Cater for user diversity via various combinations of video, audio and text, use of colour, and so on. Provide different kinds of feedback: systeminitiated feedback as well as opportunities to access debrief. Where appropriate, link the game to a realworld activity, being played with other learners. Give users control over the learning tools.
4 The potential of a gamesoriented m-portal for encouraging learning

4.1 The motivating potential of a games-oriented m-portal
Our findings from both the desk and field research point to the motivating, even addictive features of many kinds of computer and video games. This is the case even with mobile games, despite the constraints of the devices small screen size, limited memory and battery life, connectivity issues, problems with input and navigation, and so on. We find in the literature (eg Kirriemur, 2002) strong indications that multi-user roleplay games in particular can engage target audiences in social-constructivist learning within strategic contexts. The finding is compatible with the experiences reported by our coresearchers, who all stated they enjoyed using their mobile phone for communication and games; most played regularly. The findings have strengthened our view that a multi-user role-play game version of mportal could be designed as a liberating structure (de Bono 1992), allowing users to try out different learning modes, thereby encouraging attitudinal change, confidence, curiosity and creativity. These qualities may be lacking in target audiences owing to exposure to curriculum approach that has become far too mechanistic (Barlex 2003) ie where the focus is on movement through grades rather than on experiential learning.
3.5 Recommendations for mobile learning developers

This section brings recommendations drawn from the field research, specifically in respect of developing mobile learning games for mlearning target audiences. In seeking to engage users in basic skills learning, lead on from arcade-type skills. Start-up procedure should be simple: target audiences threshold of interest and their concentration may be low. To retain interest, make available a variety of short learning games, still keeping it simple to minimise levels of frustration and to maximise the likelihood of satisfactory outcomes. Ensure that game context is relevant to users vocational and leisure interests. The type of game should suit both the content and type of device. Short,
112
4.2 Some key challenges in seeking to implement the idea

The crucial issue in seeking to develop such m-portal software appears to be that the design should follow the principles of edugaming (Fabricatore 2000), allowing the learner to function first and foremost as a player. Among other real challenges for developers are the following: A games-oriented m-portal would in effect be a simulation game. Although our co-researchers considered this game type to be unsuitable for mobile devices with their very small screens, there is nevertheless a basis for experimenting with the idea: we could design for decision-complexity, seen to be more important than statespace complexity as a determining factor in solving a game (van den Herik et al. 2002). We would have to contend with the developmental nature of much of the technology. We do share with our coresearchers some concerns about the ability of the emerging technologies to deliver a reliable and fast service. Importantly, developing a quality mportal gaming environment that stands up against commercial recreational software would require heavy resourcing in terms of both finance and expertise. This is something that could only be achieved in collaboration with commercial developers. For the resulting solution to be needsled, developers should achieve collaboration with target audiences.
work and mobile activity. A multi-user gamesoriented m-portal facility could afford opportunities for cognitive apprenticeship and participative learning, enabling users to hone team skills, social and communication skills and resource-sharing skills. Finally, and importantly, if the game assigns co-researcher status to players, this could have a beneficial effect on their self-esteem, confidence and willingness to participate; it could be a means of encouraging a learning culture among target audiences. The goal is ambitious, but worth exploring further.
References
All URLs were re-checked 3 November, 2003. Anglia Polytechnic University (APU). At www.apu.ac.uk/ Barlex D (2003). Creativity in crisis. Report from a joint seminar 10 September 2002. Warwickshire Design and Technology Association (DATA), Nuffield Foundation, ISBN 1 898788 63 4. Bell J (1991). Doing your research project. Milton Keynes: Open University Press. Berson MJ (1996). Effectiveness of computer technology in Social Studies: a review of the literature. Journal of Research on Computing in Education, 28(4), 486499. Betz JA (1995). Computer games: increase learning in an interactive multidisciplinary environment. Educational Technology Systems, 24(2), 195205. Brownfield S, Vik G (1983). Teaching basic skills with computer games. Training and Development Journal, 37 (2), 5256. Brown SJ, Collins A, Duguid J (1989). Situated cognition and the culture of learning. Educational Researcher, 3242. Burgess RG (1993). Research methods in education. Edinburgh: Nelson. Burgess RG (1984). In the field: an introduction to field research. London: Allen and Unwin. Cambridge Training and Development (CTAD). At www.ctad.co.uk/ Centro di Ricerca in Matematica Pura ed Applicata (CRMPA). At www.crmpa.it/
The last point brings the focus firmly back to end-users. Their preferences, experiences and psycho-social and health issues would be vital areas for future research. In seeking to develop new approaches to pedagogy, for example in exploring how we might vary feedback (immediate or otherwise) on decisions, we would need to investigate what the effects on learners might be.
4.3
In conclusion
Our research suggests that a multi-user mportal that is a true game would bring significant added value as an alternative means of attracting and engaging target audiences, particularly if used in a hybrid learning scenario such as a combination of face-to-face group
Mitchell
113
Chappell KK (1997). Investigating the impact of elements in educational mathematics software on girls attitudes. Journal of Educational Computing Research, 17(2), 119133. Compaq: iPAQ Pocket PC, information at welcome.hp.com/country/us/en/prodserv/ha ndheld.html de Bono E (1992). Serious creativity: using the power of lateral thinking to create new ideas. London: Harper Collins Business. de Lisi R, Cammarano DM (1996). Computer experience and gender differences in undergraduate mental rotation performance. Computers in Human Behavior, 12(3), 351361 de Lisi R, Wolford JL (2002). Improving childrens mental rotation accuracy with computer game playing. The Journal of Genetic Psychology, 163(3), 272282. Dempsey JV, Haynes LL, Lucassen BA and Casey, MS (2002). Forty simple computer games and what they could mean to educators. Simulation and Gaming. 33 (2): 157-168 Dempsey JV, Rasmusen K, Lucassen BA (1994). Instructional gaming: implications for instructional technology. Paper presented to the Annual Meeting of the Association for Educational Communications and Technology, 1620 February 1994, Nashville, USA. Denzin N (1970). The research act. In RG Burgess (1993) Research methods. Edinburgh: Nelson, 200201. Denzin NK, Lincoln YS (eds) (1994). Handbook of qualitative research. Thousand Oaks, CA: Sage. Dick B (2000). Grounded theory: a thumbnail sketch. At www.scu.edu.au/schools/gcm/ar/ arp/grounded.html Doolittle JH (1995). Using riddles and interactive computer games to teach problem-solving skills. Teaching of Psychology, 22(1), 3336. Emes CE (1997). Is Mr Pac Man eating our children? A review of the impact of video games on children. Canadian Journal of Psychiatry, 42(4), 409414.
Fabricatore C (2000). L e a r n i n g and videogames: an unexploited synergy. Learning Development Institute. A t www.learndev.org/dl Funk JB (1993). Reevaluating the impact of video games. Clinical Paediatrics, 32(2), 8690. Glaser BG (1992). Basics of grounded theory analysis: emergence vs. forcing. Mill Valley, CA: Sociology Press. Greenfield PM, Brannon C, Lohr D (1996). Twodimensional representation of movement through three-dimensional space: the role of video game expertise. In PM Greenfield and R Cocking (eds) Advances in applied developmental psychology: interacting with video. Norwood NJ: Ablex Publishing 169185. Griffiths MD, Hunt N (1995). Computer game playing in adolescence: prevalence and demographic indicators. Journal of Community and Applied Psychology, 5, 189193. Griffiths MD (1996). Computer game playing in children and adolescents: a review of the literature. In T Gill (ed) Electronic children: how children are responding to the information revolution. London: National Childrens Bureau, 4158. Griffiths MD, Hunt N (1998). Dependence on computer games by adolescents. Psychological Reports, 82(2), 475480. Griffiths MD (1999). Violent video games and aggression: a review of the literature. Aggression and violent behavior, 4(2), 203212. Harris J (2001). The effects of computer games on young children: a review of the research. RDS occasional paper 72. London: Home Office, Research, Development and Statistics Directorate, Communications Development Unit, Home Office. Houle C (1980). Continuing learning in the professions. San Francisco: Jossey-Bass, 221. Jones C (1998). Evaluating a collaborative online environment. Active Learning, 9 December 1998, 3135.
114
Kirriemur J (2002). The relevance of video games and gaming consoles to the higher and further education learning experience. 6 March 2002. Technology and Standards Watch (TSW) report 02.01. At www.jisc.ac.uk/index.cfm?name=techwatch _report_0201 Ko S (2002). An empirical analysis of childrens thinking and learning in a computer game context. Educational Psychology, 22(2), 219233. Kolb DA (1984). Experiential learning. N e w Jersey: Prentice-Hall, Inc. Krueger and Casey (2000). Focus groups. 3rd edition. Thousand Oaks, CA: Sage Publications Inc. Laurillard D (1998). A conversational framework for individual learning applied to the learning organisation and the learning society. Systems Research and Behavioural Science 16 (2), 113122. Learning and Skills Development Agency (LSDA) at www.lsda.org.uk/ Leddo J (1996). An intelligent tutoring game to teach scientific reasoning. Journal of Instruction Delivery Systems, 10(4), 2225. Mason J (1996). London: Sage. Qualitative researching.
Patton M (1990). Qualitative evaluation m e t h o d s . Newbury Park, CA: Sage Publications. Pillay H (2003). An investigation of cognitive processes engaged in by recreational computer games players: implications for skills of the future. Journal of Research on Technology in Education, 34(3), 336350. Prensky M (2001). Digital game-based learning. McGraw Hill, New York Randel JM, Morris BA, Wetzel CD, Whitehill BV (1992). The effectiveness of games for educational purposes: a review of recent research. Simulation and Gaming, 23(3), 261276. Ritchie D, Dodge (1992) Integrating technology usage across the curriculum through educational adventure games. P a p e r presented to the Annual Conference on Technology and Teacher Education, 1215 March 1992, Houston, USA. Roe K, Muijs D (1998). Children and computer games. European Journal of Communication, 13(2), 181200. Roubidoux MA, Chapman CM, Piontek ME (2002). Development and evaluation of an interactive web-based breast imaging game for medical students. Academic Radiology, 9(10), 11691178. Schn DA (1987). Educating the reflective practitioner. San Francisco: Jossey-Bass. Small RV, Samijo (1997), How information affects intrinsic motivation: two exploratory pilot studies. Paper presented to the National Convention of the Association for Educational Communications and Technology, 1418 February 1997, Albuquerque, USA. Smith MK (2001). David A. Kolb on experiential learning. The encyclopedia of informal e d u c a t i o n . At www.infed.org/biblio/bexplrn.htm Ultralab at www.ultralab.net/ van den Herik JH, Uiterwijk JWHM, van Rijswijck J (2002). Games solved: now and in the future. Artificial Intelligence 134, 277311
McClurg PA, Chaille C (1987). Computer games: environments for developing spatial c o g n i t i o n ? Educational Computing Research, 3(11), 95111. m-learning project at www.m-learning.org Mayo G (1933). The human problems of an industrial civilization. New York: Macmillan. Mumtaz S (2001). Childrens enjoyment and perception of computer use in the home and the school. Computers and Education, 36(4), 347362. Organisation for Economic Co-operation and Development (OECD). Literacy in the information age. Final report of the International Adult Literacy Survey, Paris 2000. At www1.oecd.org/media/publish/ pb00-09a.htm
Mitchell
115
Vygotsky LS (1982). Problems in the theory and history of psychology Collected works, I, New York: Plenum. Walsh D (2001). Kids dont read because they CANT read. Condensed from Principal, 81, 46, National Association of Elementary School Principals, 1615 Duke St, Alexria, VA 22314, January 2002. Young J, Upitis R (1999). The microworld of Phoenix Quest: social and cognitive considerations. Education and Information Technologies, 4(4), 391408. Yuji H (1996). Computer games and information processing skills. Perceptual and Motor Skills. 83(2), 643647.
Noessel
117
Mobile learning as a service offering with near-term technologies

Chris Noessel Interaction Design Institute Ivrea
via Montenavale 1, Ivrea (TO), Italy 10015
E-mail: c.noessel@interaction-ivrea.it:
Abstract
This work, the result of a masters project in interaction design, presents a service model and some new component concepts for lifelong mobile learning, assuming specific near-term (within 23 years time) technologies. Several prototypes of service components were developed and informally tested. Keywords: mobile learning, service design, interaction design
1.2. Technology market penetration

Early explorations in mobile learning, such as those conducted in 2001 by Melissa Regan (Regan 2001) at the Stanford Learning Lab, indicate that modern cell-phone displays with current network speeds are not engaging enough for mobile learning to be effective. A service model would actively further market penetration. One of the reasons learners do not upgrade their mobile equipment is that the perceived benefits of upgrading do not outweigh the costs. A service model provides customers with a clear value proposition; a hypothetical example of this might be If you choose the deluxe package with the new Nokia tall-screen Java-enabled phone, you will be able to download and use over 50 new learning modules. Since hardware costs can be distributed over the life of the service, perceived price decreases (Rifkin 2001). Additionally, as mobile devices eventually fall into obsolescence, the service can take advantage of the existing relationship in a trade-up offer to keep current technology in the hands of its clients, and maintain a more sustainable business practice. Low-tech service components offered in a starter package also aid technology penetration. As the learner uses such components with their current mobile devices, they notice points at which better hardware would give them more options, providing experiential incentive.
1. The benefits of mobile learning as a service

Mobile learning efforts in industry and academia over the past years have progressed with a business model of pay-per-device or payper-lesson. Rethinking these in terms of a longterm service builds on these successes while adding additional opportunities to facilitate blended learning, technology market penetration, learner prompting and tight integration between components.
1.1. Blended learning

The long-term relationship implied by subscription to a service provides a broader structure for blended learning across multiple devices and appropriate channels, ie the learners mobile device, the web, a physical location like a classroom or office, or even the post. This enables the service designers to place components in the environment where they work best, and learners to use the tools they prefer.
118
1.3. Learner prompting

From research into learning and attention, we can assume that the efficacy of new devices suffers as they fade from learners attentions. (Davachi et al. 2001) We can overcome the effect of fading by incorporating learner prompting into our designs. With devicestructured business models, we have to reinitiate contact with the learner to provide this prompting. My guess is that since there is no relationship established, such contact would be perceived as intrusive and unwanted, ie advertising. Framed within a service, however, prompts may be welcomed as a helpful aid to learning and can encourage continued use of the service.
In the second, research into learning theories was used to develop a list of learner needs, which were used to develop service components. These components were described using personas and narrative descriptions called scenarios. In the last stage, an umbrella service was conceived and described, under which customers could select and customise components. Three of the components that use near-term future technologies were prototyped and informally tested with users in an academic setting.
2.1.
Personas
1.4. Tight integration

Conceiving the components as a single service reveals opportunities for data sharing between the various inputs and outputs. For example, in this project, one component involves the delivery of location-based information and another allows the client to ask questions of the service. By logging the locations where questions were asked, we can build the database on which the location-based component relies. Similarly, one component allows users to ask free-form questions of the service, which is forwarded for real-time answers by experts. Since the service has relationships with many clients of diverse interests, we can use this client base as the pool of experts. Though many of the components described in this paper are not new, their inclusion as part of a broader network of integrated components is.
Three personas were developed to represent potential users of a service for mobile lifelong learning, based on a survey of current learning service demographics at four companies Empowering Technologies, Global Knowledge, Sylvan, and SkillSoft.
2.1.1
Ellen: the student
Ellen represents users who might use a mobile learning service to augment their public education. She conceives of learning as an isolated task, dissociated from the things in her life that she really enjoys.
2.1.2
John: the worker
John represents career-minded learners, who, in the words of the science journalist James Burke (1996), will need to reskill [themselves] constantly every decade just to keep a job. He has little time for training but is motivated to use the service in his spare moments.
2.1.3
Keiko: the lifelong learner
1.5. Extensibility
An ongoing relationship between service and member allows the service to announce as opposed to advertise new technologies, hardware, and service components as they become available.
Keiko represents self-identified lifelong learners. She learns for the joy of it and sees it as a way to connect with her friends and family. She joins the service for its mobile functionality and as a personal commitment to her ongoing learning.
2.2. Developing service components 2.2.1 Learner needs

Overviews of learning theories and methods led to more detailed readings, notably in Lave and Wengers situated learning (Lave and Wenger 1991) and communities of practice (Wenger 1998) theories. From these overviews and readings, I developed the following list of learner needs. 1. Positive attitude 2. Self-awareness 3. Goals 4. Learning skills mastery (see below) 5. Learning peers
2. Design process
The vision of the service model was developed in three broad stages. In the first, representative characters called personas were developed to embody the target users of such a service. Example marketing materials were developed to illustrate likely motivations for their joining.
Noessel
119
6. Learning resources (content) 7. Prompting 8. Help
Figure 1. The opportunity map

Learner need
Positive attitude Self-awareness Goals Skills All ears Learn Gety Learning modules SMS reference LC Challenge Learn Gety Learner profile
2.2.2
Learning skills
Mobile
Desktop
Articles Learner profile Question suggestions Learning modules
Offline
Articles
Though many learning theory sources referenced learning skills, few concretely named any of these skills, and none attempted an exhaustive list. Fortunately, an excellent course hosted at the Learning Disabilities Resource Community website by Greg Gay at the University of Toronto, and titled Learning to learn, does just this. The following list was developed by distilling the suggestions in Gays course and combining them with suggestions from other authors. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Positive attitude Active reading Asking good questions Concentration and relaxation Language Logic/reasoning Managing time and tasks Memory Metacognition Overcoming information anxiety Searching
Ready rooms
Learning circle
Constructionist chat Matching services
Group facilitation Lecture series Meeting places Wunderkasten
Learning resources
Body learning Genius loci SMS reference Media agent Question suggestions Topic drift The Cavalry
Resource database Topic node network
Prompting
Media agent Question suggestions Topic drift
Realtime links Wunderkasten
Help
The Cavalry
2.2.4
Service ecology
A map of each stakeholders relationship to the service, also known as a service ecology, was developed to illustrate the value exchange of the proposed system. Developing the ecology broadened the list of stakeholders to include indirect and peripheral users of the system, such as schools, employers, and content providers.
Though components were not developed for each of the skills, they were considered.
2.3. Service vision

The description of the service that follows includes how potential customers become aware of the service, how they sign up, and how they use the service.
2.2.3
Opportunity map
Identified needs were graphed against three different learning situations: mobile, desktop, and offline. This grid provided an opportunity map for service concept (see Figure 1). Where services did not already exist to fill an intersection, new components were developed. From this map, 13 new service components were developed. As most of these components are included in the service description, below, they are not detailed here. Please note that there is no room to describe the italicised service concepts All ears, Ready rooms and Wunderakasten in detail here but they are included in the online documentation of the project at www.freerangelearning.com
2.3.1
Awareness
2.3.1.1 Advertisements
Targeted advertisements could be placed learning websites, magazines and on college campuses to increase awareness of the service. Three example advertisements were created for the service to demonstrate the different appeals to the different personas. (See Figures 2, 3 and 4.) Figure 2. Poster targeting the Ellen persona
120
Figure 3. Poster targeting the John persona
2.3.1.5 Website
The service would need to maintain a web presence to inform potential users of the service and to allow for more complex customisation of the service for existing users. Additionally, the service could provide online tools that augment and complement mobile tools, including interface to a learning resource database, learning chat rooms, and matching services to find other members with similar interests. An example home page of this website is currently in development.
Figure 4. Poster targeting the Keiko persona
2.3.2
Ready rooms
At free-choice learning environments, such as museums and zoos, the service can sponsor small rooms on the grounds, which provide an overview of the environment, encourage goal setting, and provide access to further learning resources on discovered topics of interest. The rooms would be free for use, but provide certain services only to members. The room would contain non-intrusive advertising materials for the service.
2.3.1.2 Partnerships
A mobile learning company would need to coordinate many different constituent parts, including hardware vendors, mobile operating system companies, and cellular service providers. Each of these has an existing customer base that might be receptive to upgrading or migrating to the service. Advertisements could be included in these companies existing touch points.
2.3.3
Joining
For the simplest features of the service such as SMS reference, the Cavalry, and Media agent, users could sign up on the website. For other service features that require a proprietary interface or particular hardware, potential customers can visit one of the services storefronts. Customers would be prompted but not required to answer a few questions on joining so that the service could build a preliminary learner profile. The learner profile helps the algorithms that try to match learners to other learners and to items of interest.
2.3.1.3 Real-time links

Sparacino (2002) demonstrated how user preferences could be derived from user behaviour in a learning environment. Using the same model, known as a Bayesian network, the service could integrate with the computer systems of partner libraries, museums and even video rental stores to derive the learning interests of customers. Then, on checkout, the system could instantly include free information on the receipt about further local learning resources for the topic, with a URL for further information about the service.
2.3.4
Touch points
There are four main touch points between the service and the customers: the mobile device, the website (mentioned above in 2.3.1.5), the storefronts and monthly statements from the service.
2.3.4.1 Mobile device

The mobile device is assumed to be a cellular telephone, but the use of any mobile, wirelessly networked device such as a PDA is conceivable, as long as it could download and run the custom applications. The mobile device would have three main functions for the user, available from the main screen of the interface: allowing him or her to ask questions, watching for things of interest, and permitting further study.
2.3.1.4 Point of presence

Younger users, such as Ellen, who associate learning with their schools, may not take pride in joining a learning service at first. Other users like John and Keiko, however, may be proud of their participation. For these users, customer-exclusive ring tones and eye-catching idle screens on their mobile devices may signal their involvement and invite discussion of the service from others nearby.
Noessel
121
Asking questions On the main screen, users can indicate that they want to ask a question. The ask screen allows them to enter a question in text. If their device allows them to take photographs, it also provides an option to attach an image to the question. Once they complete their question, they have the option to send it to a computer for automated reference, to a group of pre-defined peers known as a learning circle, or, for a small charge, to the service for answering by an expert. The format of a question was deliberately chosen as the means of input because, according to inquiry-based learning theory (for example, Postman and Weingartner 1969), forming questions is a core learning skill that prepares the learner for an answer, encouraging understanding and recall. In each case, when a question is provided to the system from a mobile learner, the question is stored in the database with a note of the location from which it was asked. Doing this helps to build the location database on which another component, Genius Loci (see 2.4.3), relies. Of a computer This aspect of the service is called S M S reference. In it, users structure their queries so that the server can parse it easily, look up the answer, and send it via SMS back to them immediately. Reference options include dictionary, thesaurus, reverse dictionary, language translations and abbreviated encyclopaedia reference. Other references can be added to meet the interests of the customers. Of your learning circle Via the website, learners can identify the contact numbers of a group of peers who share their learning interests. Within the service, these groups are called learning circles. Customers provide an alias for the circle and invite them to accept or decline participation. By sending a question to the alias, the server automatically forwards the message to every member of the circle, facilitating easy group dialogue between groups of mobile learners. Of experts This component of the service is called The Cavalry, reinforcing the idea that it might be called on as a backup if neither SMS reference nor your learning circle provides an answer.
Upon sending the question to The Cavalry, the question is parsed for its likely topics and matched against the learning profiles of other members of the service who have identified that they wish to answer questions as experts. When these members are not themselves mobile and at their computers, they run a small application that alerts the server that they are available to answer questions in exchange for small credits to their accounts. When the server identifies a set of matching experts running the Cavalry application, the question is forwarded to them. On their screens, the question and any attached image are displayed. If they feel they can answer the question, they can claim the question by clicking a button, at which time the question is removed from the other experts screens. The expert answers the question as best as they can in real time by typing it into special fields in the application. The application sends the answer back to the server, which forwards it to the mobile learner. Upon completing the transaction, the mobile learner can rate the expert for the clarity and speed of their response. Experts with many low ratings are flagged for possible removal from the system. Experts with many high ratings can be given more credit for their participation. Requesting watchers In addition to asking questions, customers can request that their mobile device alert them to people, places and events of interest. The customers interests are either derived from their interactions with the server or through the customers directly entered learner profile. The components are referred to as watchers. Each relies on the learners permission for the system to track their location via cellular triangulation. Of interesting people This watcher is called Learn Gety, after the Japanese product Love Gety, which matched Japanese teenagers with others in their vicinity according to the settings of a small key chain device. When on, Learn Gety compares the learning interests of the mobile learner with the mobile interests of others in the vicinity and notifies both parties if there is a topic match. The interaction design supports control and safety for the participants, including postconversation ratings for collaborative filtering.
122
Of interesting places When mobile learners activate the Genius Loci watcher, their learner profile and location are compared against the database. Learners are alerted to the nearby item of interest via text and, if available, an image. Learners can set maximum limits for these alerts to avoid being barraged. It was noted above that learner questions are part of the database on which this service component relies. This assumes that something in the environment triggered the question in the first place. Since this is not always the case, ie people often think of things unrelated to their location, these items in the database are treated differently. They are represented tentatively to users of the Genius Loci in the form of a question, eg Do you see a church nearby? One user asked the following about a church in this location. If a number of learners answer no to such a question, the question is removed from the Genius Loci database. If a number of learners answer yes then the question is solidified in the database and treated as concrete. Of interesting events As learning events are entered into the service database, they are tagged for their topics. These are automatically compared against the learning profiles of customers who have activated the Media Agent watcher. These customers are notified of the event. Using the device interface, they can indicate if they would like to attend. If so, they can use the service to make reservations and any ticket purchases required. Once accepted, the system can compare the event topic against the learners other topic interests and check to see if there is any link in the database between them. If there is, the system can send the message to the learner around the time they are attending, helping to connect their current interests and encourage lateral thinking. If such a link does not exist in the database, the system can automatically submit it as a question to experts in the Cavalry pool. Further study While the screens of mobile devices are not large enough to present engaging content, some customers may wish to study more about their interests while mobile. For these customers, the service provides learning modules tailored for mobile use.
To overcome the limitations of the screen interface, the service includes at an extra charge the use of Body Learning modules. These are marked as such when lists are browsed online or via the mobile device. To use a Body Learning module, customers must pick up or receive via post a small backpack for their mobile device, which provides sensor information about the environment in which it is being used, such as bearing, tilt and even temperature. Body Learning modules can reference this data to try and engage the learner more effectively than could a screen and audio alone.
2.3.4.2 Offices
While the main interactions are conceived as being with the user through digital means, certain aspects of the service require a physical presence, including service and equipment maintenance, meeting places for learning circles and lectures, computer access and analogue references. The service would not necessarily need its own storefronts, as this is costly. Instead, the service could partner with businesses that have existing infrastructures to provide a presence at these locations, eg cellular service providers. In addition, the service could partner other entities such as libraries, schools, museums and community centres for the use of their spaces as appropriate.
2.3.4.3 Monthly statements

Being a service, customers are charged each month for use of the service in the previous month. These statements can arrive via e-mail or post, depending on the customers preference, and provide a small, recurring opportunity to update customers on service changes or upgrades as well as new opportunities. It is also an opportunity to congratulate them on their level of participation and success in the service, helping the learner with the first learner need identified above.
2.4. Experience prototyping

Testing services of this implied complexity is difficult without developing deep infrastructures. Given the scope of the project, experience prototypes of four of the components for the proposed service were built instead: SMS Reference, The Cavalry, Body Learning and Genius Loci. These were selected as they represented one component from each of the
Noessel
123
mobile device functions, and because their underlying technologies were near-term and fully realisable in 23 years development time. They were developed to a point of unsupervised usability and given to users to use in controlled environments. Afterwards, users were interviewed about usability and viability issues.
Figure 6. A request and response from the SMS Reference service
2.4.1
SMS Reference and The Cavalry
In this prototype, volunteers with cell phones were given access to a special phone number for 9 days. During this time, they could send SMS requests to the number for automated dictionary, thesaurus, ItalianEnglish or EnglishItalian translation lookups. The receiving cell phone sent the request via a cradle cable to a computer (see Figure 5) running a custom-written Java server. The program stored the request in a log file, parsed it, scraped responses from websites, formatted the responses, and sent the response back to the phone for immediate delivery (see Figure 6). Response time was typically within 10 seconds. Users could also use the same system to send open-ended questions to the service. In response to such questions, the system emailed the question to the author, who would research the answer and manually send an SMS response. Response times varied but were usually provided within an hour. Users were surveyed at the end of the week, in which they could review their logged queries, provide the location and circumstances of each query and answer questions about the experience.
2.4.1.1 Experience prototype results

Some small usability problems were discovered in the test. For example, the first iteration of the service used a dash as the command delimiter, eg d-mobile was the correct command to request a dictionary lookup of the word mobile. It was discovered that for most of the test subjects cell phone interfaces, dashes are difficult characters to find and enter, accessible only through submenu systems that made the requests unnecessarily complex. In subsequent tests, the use of a delimiter was omitted, so that the command d smile would suffice. Although this required slightly more complex programming so that it could handle multiple-word requests, the benefits in usability warranted the change. The responses indicated interest in such a service. Eight of nine volunteers said in post interviews that they would agree to have a small monthly fee added to their phone bill for the continued use of the service. The ninth volunteer became frustrated with the command syntax and when she finally mastered it, discovered that the word she requested, diegetic, was not in the referenced dictionary. In her response she clarified that given a better dictionary, she would be interested in such a service. All would opt for a charge per request rather than a monthly service fee, at an average acceptable price point of 14 cents (euro) per request. Volunteers reported a wide variety of circumstances that prompted their requests. Duplicate circumstances included native English speakers trying to explain a word with an intricate definition to non-native speakers, settling debates in conversation, and cooking using foreign-language and metric instructions. Significantly, three reported that they felt more empowered to ask questions about things around them.
Figure 5. The receiving cell phone connected to a PC running the custom Java application
124
The surveys revealed an unexpected aspect of the experience. Users reported that the time delays involved with the Cavalry service were not always important. While a few users needed their information immediately, most felt it was enough to be able to capture the question at the moment they thought of it. For example, one user lost his cell phone for two days. When he found it and read the response to an earlier query, he was immediately reminded of the moment he asked the question and felt the time delay reinforced his interest in the topic.
Figure 7. The sensor backpack affixed to the back of a Tablet PC
2.4.1.2 Near-term technologies

Java-enabled mobile devices: Testing revealed that the SMS platform limitation of 160 characters is a significant barrier to usability on long system responses as few users phones employed long SMS capabilities. On Javaenabled mobile devices with greater display capabilities, this service could be built with a custom interface so that the user does not have to deal with these limitations, facilitating use of long responses and integration of photo attachments. Easy text entry: In the United States, SMS is not yet a standard service and relatively few people know how to use it. In Europe, not all phones possess the ease of T9 text entry. As the services rely on text entry, these technologies incrementally add to the usability and thereby viability of the components. Middleterm future technologies may even enable onboard speech-to-text for even more ease of use.
Figure 8. A detail of the constellation browser, showing some constellations and the horizon line
2.4.2
Body Learning
In this experience prototype, a Tablet PC was augmented with sensors: an accelerometer and two compass chips, letting the device detect its tilt and cardinal direction (see Figure 7). This information was used to control a constellation browser module, which matched the tilt of the Tablet PC against a star map, enabling direct comparison with the night sky. By pressing one of the number keys on the interface, users could view the constellation lines and names (see Figure 8), which would fade over the course of a few seconds. A different button would display the same information without fading.
Four volunteer students were given this device and asked to use it one evening to find their star sign constellation in the sky. They were asked to try it with key control of the angle and once using the sensors (see Figure 9). Afterwards they were given questionnaires about their experience. Figure 9. The constellation browser in use
Noessel
125

The students spent 30 minutes in total with the device, switching between users, who acted as guides, and others, who spent time watching the sky. Each found their constellation using the sensors first. Three were able to find it in the sky afterwards. The fourth students constellation had not risen yet, but the student was pleased to know where it would rise. All enjoyed looking at the foreign constellations visible under the ground using the device. When the sensors were disabled and the students asked to use key control to move the display, they reported that the experience was as enjoyable, but entirely different. Manually, the task was to apply knowledge learned with the sensors. One of the users asked to switch back and forth between the manual and automatic mapping modes to test his guesses. The students spent time connecting their constellation with other nearby or well-known constellations, to aid their memory. For this they left the constellations visible rather than use the fading display. None reported preferring the fading mode. Some usability problems were uncovered in the questionnaire. The students eyes had difficulty adjusting between the backlit LCD screen (even with a black background and dimmed graphics) and the night sky. The size and the weight of the Tablet PC became uncomfortable during the demo, making them want to cradle the device and not hold it up to the sky. These issues would not be identical in the ideal implementation. In the questionnaire, the students were able to imagine some surprising applications for such a device for their own learning interests including 3D time-scrolling displays of historical sites while on vacations, distant-object labelling for panoramic views, and exploring famous artworks. All noted that they would probably not use the device in their daily lives on a regular basis, and so would be most interested in renting them while on vacation or visiting art galleries, rather than including the cost of a device in their service.
Java-enabled mobile devices: Mobility and usability dictate that users should be able to browse and download modules on the fly. This is easiest to accomplish on Java-enabled devices.
2.4.3
Genius Loci
As location-based information accumulates and overlaps, personalised filtering for topics of interest becomes paramount (Sheth 1994). In this experience prototype, a database of location-based, Italian-language information was built. Five adult students who were currently enrolled in an intermediate Italian class were given a Tablet PC containing custom-written Java and Macromedia Director applications (see Figure 10). These applications interfaced with an off-the-shelf wireless network positioning engine, called the Ekahau Positioning Engine. The students were given the Tablet PC and asked to tour the building, looking for space tags (see Figure 11). Their instructor accompanied them. Afterwards, they were given vocabulary tests of the items they encountered, and asked to complete a survey of the experience. Figure 10. Detail from screen of experience prototype
Figure 11. The Genius Loci prototype in use
2.4.2.2 Near-term technologies

The backpack: Though the compass chips and accelerometer components of the backpack are common technology and readily available, as tested they are costly, fragile, and bulky. An electrical engineer and product designer would need to develop smaller and more usable devices for particular mobile learning devices.
126

The students enjoyed the novelty of the device and the treasure hunt style of learning. One expressed a desire to be able to hear the words displayed. Other students agreed with this. Another student wished to read example sentences with the words. The accuracy of the engine is not perfect or instantaneous, and this, combined with the limitations of the prototype, led to some frustrations among the users, who expected immediate response from the system and more control. In the post-prototype questionnaires, the students remembered most of the words they encountered in English. They could only recall the Italian translations of the words less than 60% of the time, but no great emphasis was placed on the results. The post-prototype questionnaires explained the differences between this experience and the experience of the actual service. When asked to imagine the service throughout their town, all indicated that they would pay a small price on their monthly phone bill, as they felt it would be a positive aid to their Italian studies. Given control over the number of responses per day, they would request an average of four messages per day as when they were mobile. The average acceptable maximum price for such a service was 3 euros per month.
4. References
Burke J (1996). The pinball effect. Boston: Little, Brown. Davachi L, Maril A, Wagner AD (2001). When keeping in mind supports later bringing to mind: neural markers of phonological rehearsal predict subsequent remembering. Journal of Cognitive Neuroscience, 13, 105970. Lave J and Wenger E (1991). Situated learning: legitimate peripheral participation. Cambridge, Massachusetts: Cambridge University Press. Postman N and Weingartner C (1969). Teaching as a subversive activity. New York: Dell Publishing Company, Inc. Regan M (2001 and 2003). Tomorrow's teaching and learning. Online posting 8 January 2001. Tomorrow's professor. 26 February 2003 <http://cis.stanford.edu/ structure/tomprof/listserver.html> Rifkin J (2001). Age of access: everything is a service. London: Penguin Books.
2.4.3.2 Near term technologies

Ubiquitous wireless networks and continuous location-aware devices: To be truly mobile, Genius Loci needs its location awareness to transfer seamlessly between GPS networks and wireless networks. According to the FourthGeneration Mobile Forum, 4G wireless network standards promise the interoperability to do just this. The Forum estimates that 4G could become a reality as early as 2006.
Sheth B (1994). A learning approach to personalized information filtering. Master's thesis, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science. Available at http:// agents.www.media.mit.edu/groups/agents/ publications/newt-thesis/main.html Sparacino F (2002). The museum wearable: real-time sensor-driven understanding of visitors' interests for personalized visuallyaugmented museum experiences. Proceedings of Museums and the Web, (MW2002), April 1720, Boston, 2002. Wenger E (1998). Communities of practice: learning, meaning, and identity. Cambridge, Massachusetts: Cambridge University Press.
3. Conclusion
Informal tests of components indicate that students would be willing to subscribe to a mobile learning service. This supports the main hypothesis of this paper, that for several reasons, a service structure is more likely to be a viable business and lifelong learning model than pay-per-object or pay-per-lesson models. The tests conducted, however, are neither statistically significant nor a proper test of this larger hypothesis. Though these results are encouraging, further development and testing are needed.
Proctor, Burton
127
Tate Modern Multimedia Tour pilots 20022003

Nancy Proctor, New Product Development Manager Antenna Audio J307/9 Tower Bridge Business Complex 100 Clements Road London SE16 4DG E-mail: nproctor@antennaaudio.com Jane Burton, Curator of Interpretation Tate Modern Bankside London SE1 9TG E-mail: jane.burton@tate.org.uk
Abstract
This paper reports and discusses the deployment of Multimedia Tour Systems by Tate Modern in 2002 and 2003, using handheld computers or personal digital assistants (PDAs) on a wireless network in the galleries. Developed in collaboration with Antenna Audio, these projects have piloted interactive, contextual educational content and applications for visitors. The information gathered to date is the first step to providing a blueprint that will help to inform the development of handheld technologies for museums and other institutions in the coming years. Keywords: wireless, PDA, learning, access for deaf visitors interactive
wireless network and applications developed in 2002 to focus on interactive educational programmes and contextual learning for younger visitors (aged 1625) to Tate Moderns permanent collection galleries. The 2003 project will also test a text-based tour of the permanent collection, and a British Sign Language Guide offering signed interpretation of selected objects on display.
1.2. Multimedia Tour pilot 2002

In July 2002, Tate Modern (London) launched a unique, interactive, audio-visual tour of its galleries. Using the latest developments in wireless technologies and handheld computing, this 3-month pilot project was the first of its kind in any museum in the world. The tour was sponsored by Bloomberg and developed in association with Antenna Audio. The iPaq 3850s and network equipment used in the pilot were loaned by Hewlett Packard. Unlike the existing audio tours in UK museums, the Multimedia Tour (MMT) allowed background information about the works on display to be provided to visitors in a variety of different media on a portable screen-based device. Visitors could see video and still images that gave additional context for the works on display, and could listen to an expert talk about details of a work, while the details were simultaneously highlighted on their screen. Interactive screens encouraged visitors to respond to the art on view, for instance by answering questions or by layering a collection of sound clips to create their own soundtrack for a work.
1.
Tate Modern Multimedia Tours
Audio guides have been part of the interpretation and education strategy since Tate Modern opened in May 2000. Having established a reputation for delivering excellent audio tours (with award winning tours for children and for the visually impaired), Tate Modern would like to remain at the cutting edge of educational technology by helping to shape a new generation of multimedia tours. Preliminary research into multimedia by Tate saw the development of a multimedia tour pilot in collaboration with Antenna Audio, which was tested in the galleries between July and September 2002. Tate and Antenna Audio are now collaborating on a second-phase pilot in 2003 which will expand the
128
The location-sensitive wireless network meant that visitors no longer needed to spend time searching the multimedia tour to find the relevant information for a room, because the network pinpointed their exact location in the gallery and fed the correct information to them at the right time. Because this information came from a central server, rather than being stored in the memory of the hand-held device, practically limitless content could be provided, and could easily be kept up to date. A further benefit of connecting the tour to a network is that visitors could request the central server to send additional information about the art they saw to their home e-mail address. It also meant that Tate could broadcast messages to users during the tour, and send automated alerts when the film or other programmed events were about to start.
Figure 1 A visitor to Tate Modern takes the 2002 multimedia tour pilot
Although this technically innovative pilot often pushed the technology to its limits and beyond, visitors were enthusiastic about both the service and the tour. Visitors generally see this technology as an exciting and inevitable part of the future landscape in museums. The British Academy of Film and Television Arts (BAFTA) agreed that the multimedia tour enhanced the visitor experience at Tate Modern, remarking in its award to Tate Modern and Antenna Audio for technical innovation that:
Genuinely groundbreaking, this was an exciting demonstration of how new technology can be used to enhance museum and gallery visits. Using a handheld wireless device that knows just where you are on the tour, this offers a stimulating array of material to add to, but not confuse, the experience of a gallery visit. Commendably, Tate Modern is working with day-to-day feedback from visitors to develop a system that complements an already stunning physical learning space. BAFTA 2002
1.3.1
Visitor demographics
The largest group of visitors fell in the 2640 age bracket, with 26% of visitors aged 1825, 24% aged 4160, 9% aged 1017 and 4% over 61. 42% of visitors were female and 58% male 56% of visitors were British 18% were North American 17% were from continental Europe.
The remaining 8% were from the rest of the world (1% did not respond to this question)
1.3.2
Visitor satisfaction
1.3
Visitor feedback
The average amount of time visitors spent taking the tour was 55 minutes. Over 70% of visitors said they had spent longer in gallery than they would have without the MMT, and a similar percentage said that the MMT had improved their visit to Tate Modern. In general, older visitors found the technology more difficult to use than younger visitors. Overall 55% of visitors found the MMT easy to use, while 45% found it difficult.
The pilot tour was taken by 852 visitors who completed evaluation forms recording their experiences. In addition, qualitative focus group studies were conducted by the Susie Fisher Group. The software system used in the trial also logged all uses of the MMT and provided a statistical picture of how the tour was used, which rooms were visited, and how the visitor email system was used.
Proctor, Burton
129
1.4
Content findings and recommendations
In addition to testing the technology and visitors responses to it, the primary aim of the MMT pilot was to test a variety of approaches to content design. The content proved to be the primary draw of the MMT, and indeed it will be the quality of the content that ultimately determines the success or failure of the tour experience. The findings and recommendations made below regarding content design are based on Antenna staff experiences, the questionnaires, and feedback from focus groups conducted by the Susie Fisher Group.
video: eg using the screen to explain the process of making a work (this was considered by several visitors to be a good use of the screen, but also a potential distraction) intuitive, interactive interfaces: to help visitors find information quickly and easily.
1.4.2
What didnt work
Features that did not work: long messages: attention spans seem to be even shorter for interactive messages than for traditional audio tour messages blank screens: the screen should be usefully occupied at all times, but without distracting from the exhibits text: received a mixed response: some particularly more art experienced visitors liked having wall labels in the palm of their hand, while others wanted more exciting content help menu: a key to the navigation icons is essential to remind visitors of the functions and options available to them.
1.4.1
What worked
Interestingly, users did not seem to find multitasking and multi-tracking of different media (eg looking between screen and artwork) a problem as long as the message was well designed and the PDA was functioning properly. The multimedia tour clearly had the effect of making the visitor look longer at an object than s/he would have otherwise, even though the screen was also commanding attention. As Susie Fisher reported, Visitors can multi-track with great ease, even when the input tracks (audio, screen, painting) are not synchronized with one another (Fisher 2002, Chart 34). In this regard, audio acts like a friend, and indeed more use could be made of the audio to direct the users eye movements between the object, the screen and navigation through the gallery space. In both the questionnaires and focus groups, visitors favourite stops on the tour featured the following design approaches: audio-visual coherence: a strong logical link between the audio and the visual interactives: interactive messages, in which visitors had a chance to respond to artworks or register their opinions audio: interviews with artists, sitters, and related experts, as well as good audio navigational instructions
Moreover: Visitors wanted MORE of everything: more objects on the tour, and more information about each. Just as in audio tours, the multimedia tour can take attention away from other objects in the gallery that are not on the tour. Therefore careful tour design is essential.
1.5
Multimedia Tour pilot 2003
Tate and Antenna Audio are now building on the results of the 2002 pilot to create a secondstage pilot with a view to producing a product that can be rolled out for full public use in the galleries over an extended period of time. This year Tates multimedia project is again sponsored by Bloomberg, with hardware loans by Toshiba, including the e750 PDAs that visitors will use in the galleries. The information gathered to date is the first step to providing a blueprint that will help to inform handheld technologies for museums and other institutions in the coming years. The
130
development of this second-stage project includes a particular focus on the interactive potential of the devices. The areas indicated below will be key areas of research: enabling visitors to communicate directly with the gallery, eg posing and answering questions enabling visitors to page each other in peerto-peer communication enabling visitors to access online databases while in the gallery, and to e-mail themselves further information on objects and artists on the tour in order to follow up on artists and artworks of interest through the Tate website. improving processing speeds, tour interface, operating system stability, and location-sensitive content delivery systems.
In terms of content, the 2002 pilot tour explored a variety of contrasting approaches to delivering information about the art on display. We now want to pinpoint the most successful methods for a range of audiences, and refine them to create specifically tailored multimedia learning models for visitors. As in 2002, visitors to the 2003 tours are asked to fill in a questionnaire about their experience, and focus groups will be conducted with external evaluators. Early responses have indicated that the effort put into researching visitors preferences in 2002 is paying off; here are some selected comments from the first users of the Multimedia Tour, which opened to the public on October 1: I like very much the way it draws your attention to and from the screen and artwork. I am not usually a fan of audio guides but I found using the PDA very, very informative. Brings art alive through words, pictures and music a fabulous way of scoping the gallery. I found the experience of using the PDA very informative, and especially so, on peripheral issues to the works. I particularly liked hearing the artists' views and their tastes in music. I think as a result I probably spent three or four times longer in the gallery rooms. The 2003 tours run until 20 December, 2003. Articles summing up the results of these trials will be produced from early 2004.
In addition to this multimedia content, visitors to Tate Moderns second-phase trial will be able to try a text-based tour of the permanent collection, drawing from Tates databases of information held for every object on display. The second-phase trial will also include a test of a British Sign Language guide for deaf visitors. Deaf visitors will be able to see video footage of sign language interpretation about selected works in the permanent collection. The aim of the Sign Language guide is to increase access for deaf visitors not only to the objects on display, but also to the fields of study addressed in the galleries by enabling familiarity with the signs and art terms relevant to these discourses.
Figure 2 The 2003 Tate Modern pilots will include a test of the use of the wireless PDA for Sign Language guides to provide deaf visitors with interpretation on demand
2.
References
BAFTA (2002). British Academy of Film and Television Arts Interactive Entertainment Winners 2002. Technical Innovation. Available via www.bafta.org/ CIMI (2002). HANDSCAPE: Handheld access to the museum landscape. Available at www.cimi.org/wg/handscape/handscape_ long_desc_1201.html, accessed on 6 November 2003. Fisher Susie (2002). Tate Modern multimedia tour evaluation, unpublished findings. Tate (2002). Unpublished survey results. Research period 25 July to 2 September 2002.
Rainger
131
Usability and accessibility of personal digital assistants as assistive technologies in education

Peter Rainger TechDis, The Sussex Institute, School of Education, University of Sussex, Brighton BN1 9QQ E-mail: p.f.rainger@sussex.ac.uk
Abstract
Personal digital assistants (PDAs) are a reasonably new and emerging technology that is rapidly evolving, although their use as an educational tool or assistive technology has only just started. This paper looks at the accessibility and usability of the devices (from their physical characteristics to the graphical user interface and device controls or input methods). It also suggests possible uses for these handheld devices as an assistive technology (technologies for those with disabilities that make studying easier or more accessible). Keywords: accessibility, usability, PDA, education
This paper describes the accessibility problems related to common features of PDAs and how students with specific disabilities benefit from some features and are hindered by others. One aspect of the use of PDAs in education that has not been fully explored is their use as an assistive technology. This paper summarises the main areas in which PDAs might be a useful tool. The PDA market is always changing, more so perhaps than the desktop market. For the purpose of this research we only looked at PDAs: small enough to be held in the average adult hand easily operated at the forefront of the current technology. These PDAs are differentiated by their operating systems, such as Palm OS, Pocket PC and so on.
1. Introduction
A report from TechLearn states that: PDAs are important devices that can be used to enhance the learning and teaching environment. Students and staff enjoy using them and they seem to increase student motivation. When fitted with a keyboard, they are very useful for taking notes in lectures. (Smith 2003) In these situations consideration must be given to students who have a disability or specific learning difficulty.
2. Accessibility and common features of a PDA

2.1.1 Body (size, shape and weight), touch screen/display and styli)
PDAs should ideally fit easily into the averagesized hand and be easy to hold. Tactile grips
132
could provide greater support for those with manual dexterity problems. Ideally, PDAs and their peripheral devices should be easily portable and not too heavy, and this does seem to be the aim of most designers. However, at present most add-ons are fiddly to connect and cannot be considered very robust. 2.1.2 Hardware design
2.1.5
Operating systems/graphical user interfaces
People who have lost some sensitivity in their fingers or have manual dexterity problems, and those who are visually impaired, would benefit from better designed buttons and switches with clear markings and tactile additions, such as those offered on some mobile phones. Buttons should be raised or clearly identifiable (both visually and by feel) and be in ergonomic positions. Some PDAs require far too much fiddly scrolling or direction twiddling to produce actions on the screen. 2.1.3 Touch screens/displays
People with visual impairments will have problems reading the text and graphics on the display as well as identifying the functions of the hardware buttons. They may benefit from the ability to resize text or magnify graphics and change the colour or contrast of a display. External keyboards with shortcuts for navigation may be necessary as well as an external magnifying glass. But it has to be accepted that these devices may not be accessible or usable unless they have been specifically designed for the purpose, like the PAC Mate by Freedom Scientific. Pulse Data also provide a Palm application that connects BrailleNotes to Palm PDAs (via a serial cable) so that a user can show a sighted person what they have been writing in Braille. People with specific learning difficulties or dyslexia may find some of the complex graphical user interfaces (GUIs) provided on some PDAs confusing those trying Pocket PCs will come across cramped toolbars and menus with long lists. All small screens tend to result in a lack of 'white space' and there is usually little chance to choose a favourite font. Once again it may help to resize text or graphics and change the colour or contrast of a display. People with hearing impairments may not have any difficulties with the GUI interface and operating system. However, for those who tend to communicate in sign language and find English difficult, the PDA language may be confusing. There is a type of technological jargon that runs throughout all the websites, manuals and forums and it is with this issue in mind that we have produced a glossary, which can be found at: www.techdis.ac.uk/PDA/glossary.htm People with mobility and dexterity difficulties may not have any problems looking at the GUI interface but when they try to access it with a stylus, small button or keyboard, manipulation issues may arise. The options for mouse or switch access are limited but remote control, infra-red (IR) and short-range radio frequencies (eg Bluetooth), may be the way forward. In fact those with major mobility and communication difficulties have been using GUI interface communication aids for a long time. These are often larger than the usual PDA but now examples of the Pocket PC PDAs with speech output and a simple interchangeable grid
People with visual impairments and those with reading difficulties may find coping with text on a small PDA a problem. Screens should ideally have good-quality resolution, a reasonable colour depth and clear screen lighting. However, as with all computer monitors and text that scrolls down or across a screen, it is a question of testing the tools in different environments, if possible. A dark area needs a bright screen and a monochrome PDA chosen in a shop may not suit the user when the daylight fades. 2.1.4 Switches/buttons
Providing good support for hardware buttons to allow those operating PDAs with limited movement (perhaps manual dexterity problems or single handed) would increase a PDAs usability. All functions or operations should be available via keyboard commands (if a peripheral keyboard has been added or connected). The hardware (or fixed software screen) buttons should be intuitive and customisable to allow the user to perform all navigational functions and most operational commands without using a stylus.
Rainger
133
systems can be found at Sensory Software International (www.sensorysoftware.com/). Ideally the GUI should be intuitive and have user-friendly navigation and functionality with a clear, readable, uncluttered visual design that can be resized or enlarged. The operating system should support large enough graphics to allow easy viewing and stylus control. 2.1.6 Batteries
Ideally the auditory alarm should have a variable pitch and volume to allow for people with different hearing ranges. Visual alerts such as a flashing light emitting diode (LED) or flashing display screen would help users with visual and/or hearing difficulties and those who do not wish to disturb others in a library, for instance.
3. Disability and generic features

There follows a summary of: aspects that affect those with disabilities generic features that could be helpful to all users.
The longer the battery life, the better! Most students will be at university or college for up to eight hours a day and will have different usage needs. Some older PDAs use small AA batteries or have back-up batteries but most now depend on an AC charger and it can help to choose a model that does not require a cradle for charging. 2.1.7 Expansion slots, ports and connectors
It is grouped by disability: blind and visually impaired specific learning difficulties/dyslexia and other cognitive difficulties deaf/hearing impairment manual dexterity mobility impairment speech and language difficulties.
For those with disabilities who require extras to make their PDAs accessible and usable it is often essential to consider devices that offer: the means to add additional memory (useful if you wish to store a large volume of files, or run memory-hungry programs like text-to-speech or voice memos) additional means of backing up data to a desktop PC or other storage device, perhaps by an infra-red port or a cable the chance for the PDA to be connected to a network or mobile phone for local file access or/and the internet additional connectors for peripherals, such as a printer, modem, mass storage devices (hard drives), camera and more. Docking stations and synchronisation cradles
3.1.1
Blind and visually impaired

As has been said, a person with a visual impairment may find using a PDA problematic mainly because of the size and clarity of the display. They may also find the layout of hardware buttons on a PDA difficult to distinguish and use. Features that may hinder accessibility: small screen size low screen resolution small standard font size short sentence wrapping distance small touch screen sensitivity areas poor screen contrast control poor (font, back or side) lighting for the screen buttons with a low tactile quality buttons with small labelling or symbolism. Features that make accessibility better: live text-to-speech (screen reading and document reading) speech recognition (both text transcription and for 'actioning' commands) an external screen magnifier
2.1.8
As has been mentioned, the difficulties that tend to arise with this aspect of using a PDA are related to dexterity and being able to slot the PDA into the fitting. Plugging in cables and setting up the synchronisation through the hardware button or software synch program on the computer or PDA can be fraught with frustration if it does not go smoothly. 2.1.9 User alerts
People with hearing impairment would benefit from a vibrating alert to accompany an alarm.
134
keyboard commands with navigational prompts. Specific learning difficulties/dyslexia and other cognitive difficulties
3.1.2
People with specific learning difficulties may find that some of the accessibility features mentioned in the blind/visually impaired section will also apply because they may have a visual processing deficit or be a 'visual learner'. Features that may hinder accessibility: counter-intuitive layout of hardware buttons that action functional commands (eg badly aligned hardware) buttons for cursor navigation control counter-intuitive location or actions of fixed onscreen buttons poor use of symbolism/icons and visual representations of actions or commands lack of multimedia options poor quality calendar or diary functions that could be invaluable for those with short-term memory difficulties.
A person with manual dexterity problems may find manipulating or using a PDA in their hands cumbersome or difficult. They may lack the dexterity needed to coordinate simultaneously holding and using a PDA. Most of the PDAs we have looked at have touch screens and the GUI can be activated by touch or using the physical button. Features that may hinder accessibility: holding a PDA for 'in hand use' type of force or fine touch required to action buttons or other physical controls small size and/or non-ergonomic shapes of buttons where stylus or touch screen controls are the only option small, thin, hard-to-grip styli poor operating system support for hardware accessories (such as additional keyboards).
Features that make accessibility better: PDA cases designed with materials that increase friction and grip an overall shape that allows the device to be held comfortably in the average adult hand larger, more ergonomic styli that are more easily gripped the availability of keyboards or other hardware data input devices speech recognition (both text transcription and for 'actioning' commands).
Features that make accessibility better: 3.1.3 live text-to-speech (screen reading and document reading) speech recognition (both text transcription and for 'actioning' commands) simple graphical navigational aids clear menu structures.
Deaf/hearing impairment
3.1.5
Mobility impairment
Many of the difficulties that deaf users may encounter have already been mentioned and are often the same issues that arise when using mobile phones. Features that may hinder accessibility: alerts that are purely auditory (eg a sharp tone when user errors occur) complex use of PDA-specific language.
A person with mobility impairment may have difficulty in moving from place to place, due to a physical or medical constraint. They may find the portability of a PDA useful. On the other hand, gross motor impairments might cause operational difficulties. Features that may hinder accessibility: the 'handheld' nature of PDAs, often not toughened heavy weight short battery life requiring regular charging. Features that make accessibility better: the availability and/or feasibility of mounting brackets for use with a desk, wheelchair or in a fixed location
Features that make accessibility better: vibrating alert flashing LED flashing display and/or light.
3.1.4
Manual dexterity
Rainger
135

a means of portable battery recharging speech recognition (both text transcription and for actioning commands).
reference databases electronic material readers (e-doc readers, text editors and word processors) electronic reminders and user alarms to-do/task lists.
3.1.6 Speech and language difficulties A person with a speech or language difficulties may find it hard to cope with complex technical language and may prefer to use symbol or graphical-based communication systems. Features that may hinder accessibility: poor use of symbolism/icons and visual representations of actions or commands poor speech output from written text or picture grids, ie audible text-to-speech poor-quality built-in speakers lack of multimedia options.
The following application types can be useful for those with time management and organisational difficulties: to-do/task lists diary planners/calendars electronic reminders and user alarms.
The following application types can be useful for those with difficulties with writing skills and structuring thought processes: outline tools mind/concept mapping text editors (or word processors, for note-taking).
Features that make accessibility better: live text-to-speech (screen reading and document reading) better in-built memory to cope with speech output.
The following application types can be useful for those with problems with spelling and grammar: spelling checking software spelling correction software reference dictionaries (including language conversion) and thesauruses.
4.
The use of a PDA as an assistive technology
4.1.1 Assistive technology for all PDAs can be of use to many people with and without disabilities. The following lists uses that could benefit people with a range of disabilities (or none): note taking using a text editor or word processor for taking notes (in most cases this would also require a peripheral keyboard) viewing/storing reference materials using a customised PDA database, text editor files, or e-book/e-doc files to store and present information (such as sections of a textbook, old essays, lecture notes, etc) diary planner using an electronic diary and planner for an academic timetable.
The following can be useful for those with auditory and visual impairment or processing deficits: text-to-speech (auditory feedback) Rapid Serial Visual Presentation (RSVP) (where each letter of a word is briefly shown in sequence).
The following can be useful for those with concentration (including tiredness and fatigue) or attention difficulties: multimedia electronic documents text-to-speech (auditory feedback) RSVP (as above).
The following looks at the specific ways in which a PDA may be used as an assistive technology device in an educational setting and is grouped by functional difficulties. The following application types can be useful for those with short-term and working memory problems:
The following can be useful for those with difficulty in multitasking and physical coordination or dexterity: remote computer control remote environmental control (through the use of an infra-red controller).
136
5. Reflections on the results of the project

The accessibility of a PDA can be viewed from two perspectives: The accessibility and usability of the PDA device A person with a severe visual impairment may find it difficult to read the comparatively small text on a PDA display and may find the information it presents inaccessible thus making the device unusable in its original form. The use of a PDA as an assistive technology A person with a time management and organisational difficulty might benefit from the diary and calendar functions of most PDAs which would thus provide functional technology assistance to someone with a disability. These two possible strands of a PDAs functionality are not mutually exclusive since it is necessary that a PDA is functionally accessible to be of use as an assistive technology. Neither the manufacturers nor the developers of PDA handhelds, operating systems and their software have generally considered the accessibility of their products. The development of PDAs has been so fast since their evolution beyond personal information management (PIM) systems that little analysis has been made of their functionality for people with disabilities. Though much progress has been made on the accessibility of desktop computers with increasing amounts of assistive technology, the swift development of new PDA models has meant that little has been learnt from retrospective analysis. However, one notable advancement in the usability of a PDA is the implementation of a Jog thumb dial by Sony and Franklin eBookman on the side of their handhelds. 5.1. Evaluation of the way PDAs are currently used PDAs have the potential to provide a portable, flexible platform for personal information management, computing support and access to electronic materials. However, when assessing the usability of a PDA the user should be quick to realise that it is not a laptop. Attempting to use one as a primary computing tool would soon result in frustration and inefficiency in both time and effort. For
example, trying to use a PDA for the postproduction stages (spell-checking, large-scale word processing and publication design) of a publication would be most efficiently carried out on a desktop computer. For a PDA to be used to its full potential the user must have regular access to a host computer (whether personal or networked) on which to conduct synchronisation for backing up data and so on. For people who already own PDAs or are considering purchasing one, the huge variety of PDA models and software can be confusing. There are thousands of applications available in hundreds of permutations of combinations and additions. This means that general users are not aware of the range of software (and hardware) available that might make a PDA more accessible or aid their functional difficulties. The use of a PDA as an assistive technology has barely been explored in the mainstream educational sector. Expertise related to the suitability of operating systems and relevant software with accessibility features within the educational or supplier infrastructure does not at present exist, although TechDis is working with many parties to improve this situation. There are very limited examples of PDA use in the further and higher education sectors [within the UK]. Projects have begun at five FE colleges and three universities (Smith 2003). However the long-term effectiveness of PDAs in education has not been assessed. Under the Special Educational Needs and Disability Act (SENDA) consideration has to be given to students with disabilities. However, the project team felt that the lack of knowledge on this subject within the community meant that they would not have been able to deal successfully with this issue in anticipation of a students arrival. 5.2. What have we learnt? (Transferable knowledge) In the process of the project we have learned a great deal about issues surrounding the use of PDAs that is not included in this paper, eg the usability and accessibility of electronic material and specifically design for presentation on a PDA (ie PDA-friendly materials) The rationale behind the assessment process (for providing students with technology) needs to match the person to the PDA, by considering:
Rainger
137
hardware and software related to their functional needs the large range of PDA software and the limitations of each platform how a PDA could be integrated into a course curriculum or educational environment the specific features of software for the PDA platforms that lead to increased or decreases accessibility and usability. A brief glance into the future
5.3.
PDAs could be used as an assistive technology in the following ways. Peripheral cameras could be adapted to work as portable close-circuit televisions (CCTVs) using the PDA displays. Wireless PDAs could be used to provide simultaneous text transcription and presentation for users with a hearing impairment (or otherwise). Speech recognition could be used for personal note-taking and other writing tasks and not just for memos. Handheld optical character recognition (OCR) scanning could be done without the use of a third-party device.
6. References
Great Britain (2001). Special Educational Needs and Disability Act. Available at www.hmso.gov.uk/acts/acts2001/ 20010010.htm, accessed 11 November 2003. Smith T (2003). Personal digital assistants (PDAs) in further and higher education. Available at www.techlearn.ac.uk/NewDocs/ PDABriefing.pdf, accessed 11 November 2003.
7. Further information
The project website Usability and accessibility of PDAs in education can be found at: www.techdis.ac.uk/PDA, accessed 11 November 2003.
Sharples et al.
139
Evaluation of a mobile learning organiser and concept-mapping tools Mike Sharples, Tony Chan, Paul Rudman, Susan Bull University of Birmingham Educational Technology Research Group Department of Electronic, Electrical and Computer Engineering University of Birmingham, Edgbaston Birmingham B15 2TT E-mail: m.sharples@bham.ac.uk, chantks@eee-fs7.bham.ac.uk, rudmanp@dcs.gla.ac.uk, s.bull@bham.ac.uk
Abstract
We describe a learning organiser for handheld computers that has been trialled with university students. The aim was to investigate whether an integrated set of learning tools would be useful, which tools would be adopted and the contexts in which the tools would be used. The results show no single favoured application. The most frequent activities were reading e-mail, note-taking, managing deadlines and appointments, and listening to music. The main reported limitation, apart from battery life, weight and processor speed, was the loss of wireless LAN connectivity, and thus usefulness, outside the university department. A comparative evaluation was also carried out between one of the learning organiser tools, a star-structured concept map, and a more traditional, free-format concept map. The results suggest that different concept-mapping tools may be suited to different tasks and types of user. Keywords: learning organiser, wireless LAN, learning tools, concept maps
tools has been demonstrated through their widespread adoption; some companies now provide their entire workforce with mobile organisers. Learners at university, college or school have as broad a range of demands for selforganisation as professionals, but their requirements are somewhat different: to attend classes, meet course deadlines, read and understand teaching material, revise for exams, and manage individual and group projects. This paper describes a project to evaluate a learning organiser for university students (for details of the design and implementation, see Holme and Sharples 2002). The software runs on Pocket PC handheld computers and provides a set of tools for students to access course material, view their timetables, communicate via e-mail and instant messaging, and organise ideas and notes. The aim is to investigate whether students benefit from an integrated learning organiser and also to find out what other tools or services for handheld computers they choose to adopt. Thus, the main questions are as follows. Is an integrated learning organiser of value to university students? If so, what tools and services should it include? The paper also reports a comparison of one of the learning organiser tools, the concept map, with a more free-format concept-mapping tool, to see which would be more appropriate for note-taking and topic browsing on a handheld device.
1. Introduction
Mobile office organisers are becoming indispensable tools for many professionals. They provide a suite of work support tools, including a calendar, address book, notebook and to-do list on a handheld computer. Some devices, such as the Blackberry, also allow people to read and send e-mail and to read and update a shared calendar. The value of these
140
2. Design
The design aim of the learning organiser was to develop tools for handheld computers that might assist students to learn and manage their studies and to integrate these into a single software application. The four tools that were chosen for development were the Time Manager, Course Reader, Communication Centre, and Concept Mapper, since these span the range of student study and management activities. There was no principled reason for implementing these rather than other potentially useful tools, such as project management and groupwork aids, but rather the constraints of producing tools relevant to student study in the time available for the project. Students are expected to take responsibility for their own study and time management. Lecture patterns can vary from week to week, and student assignments and projects may last for a month or more, with a series of personal milestones and external deadlines. The Time Manager (Figure 1) was designed to show at a glance the structure of the teaching day, with nine hour-long periods. A time strip with boxes that are either green (free time), yellow (recurring sessions) or red (single event) is shown at the top of the Time Manager, and also on the main startup screen. A Deadlines tab displays a set of pending deadlines and tasks. A separate application enables a lecturer to publish a course timetable and deadlines on the university intranet, which students can download and then extend with their own events and tasks. The Time Manager is implemented to exchange data automatically with the PocketPC calendar and task applications, so that students can synchronise it with standard desktop calendar and task tools.
The Course Manager (Figure 2) provides a portal to teaching materials for the entire course and for individual modules. The students can download course materials onto their handheld computer, and can also access supplementary materials through weblinks, with the materials loaded via the wireless LAN connection. The teaching materials can be displayed in a PocketPC web browser, or in Microsoft Reader, Adobe Acrobat or Powerpoint format, depending on the type of document. The aim was not to substitute for printed or web-based teaching materials, but to provide a single store of materials for the course available to view on the handheld computer. For example, a student might browse the slides before a lecture, or quickly review a troublesome topic.
Figure 2: Screenshot of the Course Manager The Communication Centre gives an interface to run the standard Pocket PC e-mail, instant messaging and contact tools. The wireless LAN connectivity allows students to browse e-mail, or to check which students are online with Instant Messenger then hold a text chat session. Concept mapping, also known as topic mapping or mind mapping, has been developed over the past 30 years as a tool for learning. Students are able to learn effectively from course material presented as visual concept maps showing a network of nodes (representing topics or concepts) and labelled or unlabelled links (representing conceptual associations between the topics (see Fisher 2002, for an overview). The process of creating concept maps can also encourage students to reflect and understand. A series of studies have shown that students who were taught conceptmapping techniques for studying in lectures or from printed text performed better than those who used traditional note-taking, in identifying main ideas, recalling relations between ideas,
Figure 1 Screenshot of the Time Manager
Sharples et al.
141
and describing underlying mechanisms (Fisher 2002). There is evidence that computer conceptmapping tools for desktop computers, such as SemNet , can aid studying and notetaking (Gorodetsky and Fisher 1996), but this has not been tested for handheld devices. Map-It! was developed by Chan as a concept-mapping tool for small-screen devices, with a simple pointand-click interface to browse through related nodes. This was incorporated into the learning organiser. During the trials another concept mapping tool for Pocket PC computers, but with a very different interface, became available, developed as part of PhD research by Rudman. A small-scale comparison of the two tools was carried out.
were using the iPAQ at least once a day. This fell to 42% (6 out of 14) after 16 weeks (one student had left the course and two did not complete the questionnaire).
4.2.
Results
There was no single favoured application. The activities most frequently reported in the questionnaire were e-mailing, note-taking, managing appointments and deadlines, and listening to music. The diary reports also showed web browsing and reading (course notes and e-books) as frequent activities. Table 1 shows the reported usefulness of the main learning organiser tools, with the first figures being the percentage and actual numbers of students reporting useful or very useful after four weeks (n=17), and the second, the percentage and number after 16 weeks (n=14).
3. Equipment and software

Seventeen students taking an MSc in humancentred systems at the University of Birmingham were loaned an iPAQ handheld computer with a wireless LAN sleeve and docking cradle. The sleeve provides high-speed access to web pages, course material and e-mail within the university department. The docking cradle enables those students with computers at home to synchronise their calendar, notes and documents, and to transfer software. Students were provided with three types of software: the integrated learning organiser developed at the University of Birmingham: students could download material for teaching modules, including Powerpoint slides and supplementary texts, through the wireless LAN connection the standard set of PocketPC applications, including e-mail, Internet Explorer, Windows Media Player, and pocket versions of Word and Excel software that the students chose to download: it was made clear that they could also use the device for their own personal use and entertainment.
4 weeks E-mail Timetable Web browser Instant messaging Course materials Supplementary materials Concept mapper 76% (13) 59% (10) 65% (11) 59% (10) 59% (10) 53% (9) 35% (5)
16 weeks 79% (11) 64% (9) 64% (9) 50% (7) 43% (6) 43% (6) 14% (2)
Table 1 Perceived usefulness of tools (useful or very useful) after 4 weeks (n=17) and 16 weeks (n=14).
4.3. Discussion
It was not possible to measure improvements in time management and study habits directly (though a more extensive investigation may be able to do so, for example by a comparison of study patterns and missed deadlines), so the findings were based on a combination of surveys, interviews, and diary studies. The results show that e-mail, timetable and web browser retained or increased their perceived usefulness, while the instant messaging, teaching materials and concept mapper were judged to be less useful in the later survey. There are a number of possible
4. Evaluation: learning organiser

4.1. Method
Students completed detailed questionnaires about their iPAQ use after four and 16 weeks. They were also asked to keep logbooks recording each use of the iPAQ, their activity, the time spent on the task and the tools they employed. After four weeks, 64% (11 out of 17)
142
explanations for this: the novelty of some tools may have worn off while others continued to be useful, or the students pattern of study may have changed (the MSc is run with week-long intensive modules, so the students would have been studying different modules at each of the interviews). The main reported limitation, apart from battery life, weight, and processor speed, was the loss of connectivity, and thus usefulness, outside the department.
dragging into the eight arrows). Nodes may also be grouped for dragging. To add a new node at an unoccupied place on the map the user taps at that place, opening an input area for the node's text. Nodes may subsequently be linked by dragging one node on top of the other.
5. Comparison: concept-mapping tools

Given the low use and perceived usefulness of the concept-mapping tool, it was decided to carry out a comparison between Map-It! (which was provided as part of the learning organiser) and another concept-mapping tool also developed at the University of Birmingham: Concise Concept Mapper (CCM). While both mapping tools described here are optimised for pen-based interactions on a small screen, they operate significantly differently. HandLeR Map-It! (Chan and Sharples 2002) (Figure 3) uses a star structure in which one node is central and has linked surrounding nodes. To navigate, the user clicks on one of the outer nodes, which brings it to the centre, displaying the topics related to it. Clicking on the centre node displays any document associated with that node. The user can add a new node by selecting a document from the file list, which attaches it to the central node.
Figure 4 Screenshot of Concise Concept Mapper (CCM)
Figure 5 Screenshot of CCM showing the compressed view A zoom facility displays a compressed version of the entire map, giving an overview of its structure (Figure 5). This is designed to reduce usability problems inherent in working on small pieces of the map by separating the map structure from its details. Search text may also be input to highlight nodes containing specific words.
5.1. Method
A qualitative comparison was undertaken. Three MSc student volunteers were asked first to familiarise themselves with both software tools and to read some general information on using concept maps. The comparison then proceeded as follows: (1a) Participants were given 15 minutes to create a concept map based on a familiar
Figure 3 Screenshot of HandLeR Map-It! Concise Concept Mapper (CCM) (Figure 4) provides a free-form concept map based on user-positioned nodes and links (Rudman et al. 2002). Interaction is by pen gestures: a node is moved around the map by simply dragging it with the pen, scrolling the map as necessary (by
Sharples et al.
143
document from their MSc course (once for each software tool using different documents). (1b) Participants were given a ready-made concept map (about the geography and history of Guatemala and Puerto Rico countries they were not familiar with) and were given 15 minutes to answer five questions, using only the concept map for reference (again once per tool using different maps). (2) A few days later the participants were shown a video recording of their participation in 1b and asked to recall and describe their thoughts from that time.
important to provide search on Map-It! particularly for a larger concept map (more than three levels deep).
5.3. Discussion
Results suggest that each tool performed best in different situations. CCM performs better for note-taking and organising information with no obvious structure; it is also more suited to experienced users. On the other hand, Map-It! is better at presenting highly structured information and organising documents by semantic relations; it is also more suitable for novice or occasional users. Analysis of the concept maps the participants created during the early part of the experiment and their later interviews indicated that the lack of experience in creating concept maps is a major factor behind the low usage. Reflecting
5.2.
(1)
Results
CCM generally performed better than Map It! during note-taking because of the flexibility of placing text directly on the map without first creating a document or defining its relationship to other concepts. More correct answers were obtained using Map-It! (scores of 4, 4 and 5, out of 5) than CCM (scores of 3, 2 and 4). Participants suggested that Map-It! would be more suitable for presentation purposes, such as tutors providing notes, due to its imposed structure. Most participants (2 out of 3) were able to answer all of the questions in a shorter time using Map-It! than CCM at stage 1b. All participants suggested that at stage 1b. Map-It! appeared to have significantly less text than CCM. (In fact the amount of text on both software tools was similar.)
(2)
CCM
Map-It!
Learning Figure 6 Learning facilitation We conclude that combining the strengths of the two concept-mapping tools could further facilitate learning (Figure 6). The process of using notes from CCM to construct a map in Map-It! could be considered as a reflecting process as the user will be organising concepts and structures within their notes. On the other hand when using Map-It! as a delivery method, the process of taking notes from the concept map into CCM could be an aid to learning.
(3)
(4)
Participants also mentioned the importance of meaningful keywords on the maps, allowing them to predict what information lay beneath the node. Participants cited this as allowing them to answer questions more easily and quickly with Map-It! than with CCM. The requirement to condense the concept (or document) into a few words on the map imposed by Map-It! was the main reason for quicker navigation with this tool and why Map-It! appeared to have significantly less text. All the participants welcomed the search facility on CCM. Although during the experiment the lack of search in Map-It! did not greatly hinder the participants in completing their tasks, the participants suggested that it is also
6. General discussion
Students have always needed help to develop effective study skills, but the new trends in university teaching make this even more of an imperative. Worsening staffstudent ratios may mean that students have less guidance from tutors. The change in teaching patterns away from lectures, seminars and lab classes towards resource-based learning and flexible study mean that students have to manage a more complex set of learning
144
resources and patterns of teaching and assessment. The development of part-time, online and distance learning puts yet more burden on students to organise their limited time without the structure of a traditional full-time course. The premise of the study reported here is that if students have a personal organiser and communicator ready at hand, with appropriate study tools and resources, then they will become more effective in managing their time and study habits. The results are mixed. The continued popularity of e-mail and timetable, despite these also being available on desktop machines, suggests the importance of mobile organisation and communication tools to enable students to manage their learning. The consistent use of the web browser also shows the potential utility of integrated tools. Although fewer students continued to use the other tools, some students were still finding them useful. There was no single benefit or best tool. Instead the students appeared to employ the technology to suit their needs, with some using it primarily as a communications device while others made more use of the ability to browse learning materials or manage their calendar. This suggests that provision of an adaptable or even an adaptive version of the learning organiser is worth investigating. Indeed, in addition to differences in tool use, the logbooks showed that locations of use differed quite widely across users, and the relationship between tool and location also differed (Bull 2003). This further supports the proposal of implementing an adaptive version of the learning organiser in order to meet the individual needs of different students. Based on these results, work on such an adaptive system is now under way. It is not clear that students want or need an integrated learning organiser, separate from the standard calendar, address book or task list applications. Instead, it may be better to provide views on the standard tools, such as being able to view the course timetable within the calendar, and have assessment deadlines added to the task list. Further tools may be useful, such as a project manager for group projects and an electronic student logbook. We are currently investigating the design of such tools and services, for handheld devices and for pen tablet computers. The most telling finding was the disruption that some students experienced when they left
the Wireless LAN range and lost communication. Students learn and organise their studies across many locations and times, so for a learning organiser to be truly effective all its tools and services must be available anytime, anywhere.
7. References
Bull S (2003 forthcoming). User modelling and mobile learning. In UM2003: Proceedings of the Ninth International Conference. Berlin & Heidelberg: Springer-Verlag. Chan TKS and Sharples M (2002). A concept mapping tool for Pocket PC Computers. In Proceedings of the IEEE International Workshop on Mobile and Wireless Technologies in Education, (August 29-30), Vaxjo, Sweden, 163164. Fisher KM (2002). Prediction: a profound paradigm shift. In J Park (ed) and S Hunting (technical ed) Topic maps: creating and using topic maps for the web. Boston: Addison-Wesley. Gorodetsky M, Fisher KM (1996). Generating connections and learning in biology. In KM Fisher and M Kibby (ed) K n o w l e d g e acquisition, organization and use in biology. Berlin and Heidelberg: Springer Verlag, 135154. Holme O and Sharples M (2002). Implementing a student learning organiser on the Pocket PC Platform. In Proceedings of the European Workshop on Mobile and Contextual Learning, Birmingham, UK: 4446. Rudman PD, Sharples M, Baber C (2002). Supporting learning in conversations using personal technologies. In Proceedings of the European Workshop on Mobile and Contextual Learning, Birmingham, UK: 4446.
Stone
145
Designing scalable, effective mobile learning for multiple technologies

Andy Stone Learning Technology Research Group School of Computing and Information Systems Kingston University Penrhyn Road Kingston Upon Thames KT1 2EE E-mail: a.stone@kingston.ac.uk
Abstract
Mobile learning encompasses a range of technologies, to the extent that mobile learning and e-learning are now referred to as networked learning in some quarters. It is increasingly likely that this involves the deployment of reusable learning objects. There is a risk that as such objects are converted or modified for use in different mobile environments, some of the pedagogy may get lost in the process. Thus we need to consider how to design and deliver effective mobile learning in a scalable manner, to a multiplicity of technologies. This paper considers how Blooms (1976) cognitive taxonomy of learning and Ma et al.s (2000) framework for adaptive content delivery in heterogeneous network environments can be used in this context, and points to standards and the need for effective metadata to consolidate these factors. Keywords: learning objects, mobile learning, e-learning, networked learning
key issue of concern for interested parties is how we can scale up relatively small-scale projects into ones which can be used in a wider context. Those already involved in mobile learning projects are aware of a number of issues which act as constraints for developing mobile learning applications, and it is reasonable to assume that many of those interested in mobile learning have some experience with e-learning. In the e-learning community there is a great deal of interest in the notion of reusable learning objects; as more of these are developed and become available for reuse, it is reasonable to assume that some intended areas of reuse will include mobile learning. Although there is potential for mobile learning platforms to use e-learning resources by optimising them for the network and delivery platforms, we are concerned that this should not be done at the risk of losing the pedagogy designed into the original learning object. This paper attempts to draw together these issues by raising some considerations which may allow those involved in e-learning and mobile learning to maximise the potential for effective reuse of e-learning resources in mobile learning environments; that is ensuring that pedagogic benefits are retained or at least that users are aware of how they may change from intended and expected values.
1. Introduction
There has recently been significant growth in the number of initiatives that are using mobile devices to support teaching and learning, and an increasing amount of interest in this from many sectors. As with other emerging research and development areas of educational technology, a
146
This paper begins by considering what is meant by mobile learning at present, and illustrating the current diversity of technologies that exist in this area. I then consider how elearning, constructivism, and critiques of the notion of learning objects relate to mobile learning. Bloom's taxonomy of learning (1976) is considered in a mobile learning context; then I examine a framework that can enable the transformation of learning objects in terms of how it could affect their pedagogic efficacy when transformed for mobile learning usage. I conclude by identifying relevant emerging international standards and initiatives that may facilitate the reuse of learning objects to mobile learning environments.
There have been initiatives using SMS (text messaging): for example, revision support in secondary schools in Merseyside (Ananova 2001). At Kingston University, we are about to begin trials using SMS alerts to support firstyear degree students and provide scaffolding, that is learning supports (eg Soloway et al. 1996), in terms of time management and ensuring that essential core learning is not missed at an early stage, building upon previous research (Stone et al. 2002). Wireless Application Protocol (WAP) has been used in some cases to support postgraduate learners (see Noone 2001; Turner 2001). WML (Wireless Mark-up Language), which renders content on mobile telephones usable, has been useful in developing content using the stack of cards model, such as Apples Hypercard (Gigure 2000). WML is not restricted to the WAP network protocol: for example, one could use GPRS (General Packet Radio Service) to make the connection, and receive WML pages. As more handsets are capable of both WAP and GPRS, the distinctions become increasingly blurred from a user's point of view. Location-based services (LBS) are also being implemented using various network technologies, with varying degrees of accuracy, and thus degrees of sophistication, depending on the technologies in use. It should be noted that the use of mobile phones in education has mainly taken place in Europe and Asia. In North America, mobile learning more commonly denotes the use of handheld computers, personal digital assistants (PDAs), and so on, linked across wireless networks such as 802.11, rather than mobile phones (eg Wired 2002; Palm 2001). However, there are also cases where PDAs have been used in the UK and Europe to support mobile learning (eg Taylor et al. 2002; Collett and Stead 2002; Pinkwart et al. 2002). Third-generation mobile telephony (3G) has now arrived in the UK, with the new mobile operator 3 the first to offer handsets and services to the mass market (eg 3 2003). The current focus of applications is on mass-market audiences, with sport and entertainment being the main sectors marketed to prospective customers. However, the network and handsets offer the potential of high-speed data rates and the potential to offer streaming media in addition to large file transfers where needed.
2. What do we mean by mobile learning?

Mobile learning is a relatively new concept, and is closely related to e-learning. Milrad (2003) defines e-learning as learning supported by digital electronic tools and media, and mobile learning as e-learning using mobile devices and wireless transmission. Both these terms encompass a wide variety of technologies and initiatives, which is indicative of the state of the art in these fields. Polsani (2003a) expresses concern with the use of these terms, believing they are too restrictive to adequately characterise the new forms of learning because they refer either to the delivery format of content or the access devices (both terms) came to be thought as instances of traditional distance learning. To remedy this, Polsani argues that the term network learning (or nlearning) would be more appropriate and proposes that it can be defined as a form of education whose site of production, circulation, and consumption is the network. In this paper, my interest lies in w h a t is produced, circulated and consumed in the network, to facilitate the education process. There are a number of activities that are considered to be within the realm of mobile learning at present and they involve the use of a range of mobile devices, using a number of different mobile network technologies. Consideration: mobile learning can be thought of as a special type of e-learning, bound by a number of special properties (eg form factor, which is the size and physical arrangement and configuration) and the capability of devices, bandwidth and other characteristics of the network technologies being used, etc.
Stone
147
From a modest selection of examples, it is apparent that mobile learning is a very broad spectrum at present, and so presents another consideration for educational technologists. Consideration: mobile learning is a heterogeneous environment; it encompasses a range of devices and network technologies With these two considerations in mind, this paper considers two key aspects of e-learning that may be pre-requisites for its success, before looking at what in particular may be needed to make mobile learning successful.
3. Two key facets of e-learning: constructivism and learning objects

One of the most important planks underpinning much of e-learning theory and practice is pedagogic theory based on constructivism and social constructivism. Milrad (2003) states constructivism is at the core of the movement to shift the centre of instruction away from delivery in order to allow the learner to actively direct and choose a personal learning path (page 153). Social constructivism reflects the social nature of knowledge formation through personal experiences and collaborations which involve not just the exchange of information, but the design and construction of meaningful artefacts. Both social constructivism and constructivism require active learning on the part of the individual, with the teacher focusing on missing connections. However, the provision of technology and pedagogic theory also requires the production and provision of good-quality educational content to enable successful networked learning. Although the networked aspect can be a key element of facilitating the learning process among students, this is insufficient without the content to back up and stimulate constructivist behaviour. Content used in e-learning is increasingly being reused in ways it was not initially created for. There is a growing movement towards developing digital educational content as learning objects intended for reuse from the outset. Rather than considering the major standards and initiatives relating to reusable learning objects, recent critical work in this area can assist forming our considerations for developing effective reusable mobile learning content. The reader unfamiliar with current standards and initiatives can find these cited in the reference section of this paper.
Current definitions of learning objects are critiqued by Polsani (2003b), who believes they are confusing and arbitrary. He proposes a definition of a learning object as an independent and self-standing unit of learning content that is predisposed to reuse in multiple instructional contexts. Friesen (2003) raises a number of criticisms in relation to learning objects: contending specifications and applications that are pedagogically neutral cannot also be pedagogically relevant. He also questions the variety of definitions of learning objects, believing in order for the positive potential of learning objects to be realised, they need to be labelled, described, investigated and understood in ways that make the simplicity, compatibility and advantages claimed for them readily apparent. Links between constructivism and learning objects tend to be implicit in the literature, yet while Hodgins (2000) criticises the notion of a fully automated process which views the combination and use of learning objects in Lego-like fashion, he concedes there is potential in the student-directed constructivist use of small learning objects. Currin (2001) noted that the Nortel Networks training provider developed mobile e-learning which uses what they referred to as learning nuggets bite-sized chunks of information pertinent to specific tasks such as system specifications and installation procedures. Although corporate training material should not be considered as identical to constructivist learning models in further and higher education, this is one of the first concrete examples of the integration of a learning object-based approach combined with mobile delivery. Darby (2002) notes that the approach taken by Oxford University differs from the mainstream. While the latter considers learning objects to be potentially stand-alone objects and mini-courses in their own right, Oxford, under the Technology Assisted Lifelong Learning (TALL) programme, has viewed learning objects as components that define a learning activity, each designed to achieve a specific learning outcome. Darby refers to these as very small-scale learning objects, which are combined into a structured learning environment, with core spinal documents giving context to the content and activity elements. He asserts in this case that the learning objects may not have any intrinsic learning value, except when combined with other learning objects.
148
Consideration: e-learning is likely to be more effective when high-quality content is being used. Consideration: criticisms of the learning object world-view need to be considered; context must be borne in mind at all times. In the light of the critical literature, we must accept that key to a successful learning object economy is the notion of effective reuse of such objects, and that we have to accept that at least some of this reuse is likely to take place in a different context from its original use. There may be differences in the course in which it is used; how it is used as an educational resource; what platform it is deployed on; and so on. We therefore need to consider what factors can help make mobile learning more successful.
Table 1. Networked learning activities in relation to Blooms taxonomy of learning Provide repetitionservice on learned subjects Provide tests on learned material Give just-in-time learning that the learner can use in a practical situation Provide background information so that the user can evaluate the relevance of data to specific situations Give tools that help the user to develop new documents or projects Give the user different information on the same subject to let them evaluate and determine what information is relevant Level 1 knowledge
Level 2 comprehension Level 3 application
Level 4 analysis
Level 5 synthesis
4. What makes mobile learning successful?

The notion of successful mobile learning is intentionally provocative as the technologies and applications are still very new, exemplars of best practice are still emerging, and we suggest that it may be instructive to consider how various types of learning can be structured in terms of levels of sophistication, and how current thinking is being modified to consider how we can move towards ensuring more successful mobile learning. Bloom's taxonomy of learning (Bloom 1976) identified three domains of learning activities: cognitive, affective and psychomotor. The cognitive domain can be used to illustrate how networked learning activities (both elearning and mobile learning) may be categorised in terms of degrees of sophistication (see Table 1).
Level 6 evaluation
Irrespective of which levels of learning are addressed, mobile learning applications will, by definition, be bound by a set of constraints. These will have implications on how learning objects can be (re)used most effectively.
5. What are the constraints facing effective mobile learning particularly the reuse of learning objects?
The first set of constraints is that imposed by the network: different network technologies have different maximum potential amounts of bandwidth available to their users; however, there is also no guarantee that such maxima will be available to all users, at all times. Also, there may be differences between the width of the data pipe available in each direction; ie it may be possible to download some information much more quickly than for someone to upload the same amount. Second, the form factor of mobile devices presents constraints: mobile devices, by their nature, tend to be engineered with the key requirement that the actual device is small, and
Stone
149
the displays are increasingly large. However, display sizes and resolutions still differ widely, and unless one is working in an environment where all students have been provided with identical devices, it may be difficult to make assumptions regarding what students can and cannot do, and how easy it may be for them to perform certain actions. There is an analogy here with the early days of the worldwide web, as original HTML (HyperText Markup Language) specification did not take into account the use of graphics, sound, rich interactions or proprietary extensions for the different browsers. A similar phenomenon has been observed with the adoption of Java device-specific extensions by application developers (eg BBC 2002). Finally, and perhaps of most importance to mobile learning, a third constraint arises as an emergent property through consideration of the previous two, relating to the pedagogic efficacy of content reused in a mobile learning environment: ie does the learning object work" on this device does learning take place?. The two constraints above have quite profound design implications for those interested in working towards scalable, effective mobile learning for as wide a community as possible, if we are considering a world in which multiple technologies coexist. As in the more general discussion regarding learning objects in Section 3 (and, we would argue, of even greater importance here), we need a framework to ensure effective reuse, ie that not only can content be delivered in a meaningful way, but the pedagogy underpinning its creation and deployment is still there! It could be argued that the loss of pedagogic efficacy is inevitable as a result of repurposing, but notions of complementary media and complexity support a case for trialling, documenting lessons learned, and usage of this being archived as metadata to support reuse (Stone et al. 2002). However, as stated earlier, the potential for variable delivery quality may detract from this, ie the user experience, affected by issues relating to technical constraints, is another factor which must be taken into account. Consideration: technical constraints that may affect the mobile learning experience need to be taken into account. A framework exists which addresses this for heterogeneous network environments in general this can be used as a starting point to address the consideration above. We shall now consider the adaptive content delivery framework of Ma et
al. and consider how these points can be applied to mobile learning as a special case.
6. Adaptive content delivery framework some extensions for mobile learning reuse of learning objects
Ma et al. (2000) present a framework for adaptive content delivery in heterogeneous network environments a roadmap to manage technical issues. This framework presents five categories of what they call content adaptation techniques (CATs). I shall now consider each of these categories in the light of how they could be used to extend this framework for the delivery of learning objects in a mobile learning context.
6.1. CAT 1: information abstraction

This allows a preview of content to be generated and optimised for the device. At certain levels of teaching, learners may not be aware of all the factors relating to why a learning object is useful for them: ie the wider and deeper learning outcomes relating to a continuous learning strategy; and how the learning object maps to these, particularly in relation to a constructivist approach. Some parts of the activity may be key to the individual experience which the teacher will then facilitate reflection on, as part of the learning process. Information abstraction therefore needs to be undertaken by the learning providers and can then reside in the decision engine (defined in Section 6.6).
6.2. CAT 2: modality transform

Examples of this include transforming a video stream to sets of images, subtitles as key text, use of audio track, etc. Authors and teachers implementing the learning object may wish to prioritise this in terms of pedagogic utility. It could be argued that some value is better than none, although we also need to take into consideration learners' use of complementary media, which may differ from expected use. This may increase the aggregate utility of the learning objects delivered, even in a transformed manner (Stone et al. 2002).
6.3. CAT 3: purpose classification

In an education environment, classification of purpose should include learning outcomes this is overlooked in the model of Ma et al. model. These may vary according to context of
150
(re)use, and as such must be stored and made available for the decision engine. This does not prevent multiple entries being made according to set contexts of use (ie different level of use than originally intended, but context and usefulness recorded, eg as a pointer to information relating to this) and could be achieved with multiple entries in an appropriate metadata record (eg an IMS Learning Resource tag see IMS 2003), with each entry having an identification tag about context of use, reflections on that use. This could be expressed in an Resource Definition Framework, for example.
Stage 1 implies that the user may have a say in the type of modality transform and data transcoding techniques. Minimum levels (including lower bandwidth combinations which preserve the pedagogy) could be indicated in the learning objects metadata, which could then be applied by the decision engine if transformations need to take place. The user (or the network) could then indicate a Quality of Service (QoS) level at which content may be delivered, with a pre-qualified degree of pedagogic efficacy assured by the learning provider. Stage 2 involves a trade-off between information abstraction and download time, taking into account-processing time for performing information abstraction. Ma et al. suggest a user input where the quality vs. response time trade-off may be specified. By adding appropriate learning object metadata, this user input could remain, but could be mediated by recommendations from the learning providers as to what is meaningful content delivery that delivers the important parts for the learner to undertake a learning experience, which can be supported by the teacher in the reflective part of the learning process. Stage 3 is known as data prioritisation, with progressive interactive web delivery (Gilbert and Broderson 1999) suggested as an example. Our main consideration here is that the playback of the data on the device should take place without any glitches that may detract from the learner experience. If there should be a change in the network quality of service during the session, the data that is key to the delivery of an effective learning object could be delivered, even if it is at a lower quality than negotiated at the start of the session. However, while the pedagogic integrity of the learning content delivered may be intact, if the user has not been informed this may take place, expectations may be lowered which may detract from the user experience as a whole. Ma et al. consider both server and proxybased architectures for the above stages; ie whether the server is responsible for discovering client capabilities and available bandwidth or whether this is done by a proxy. Ma et al. also raise some issues that need to be considered in an educational technology context, namely: author previewing and control of content adaptation (better on a server, since the proxy architecture automates the adaptation process without being mediated, resulting in a
6.4. CAT 4: data prioritisation

This refers to the prioritisation of converting data through utilising the available network bandwidth. In Section 5, we noted that this is (and will continue to be) an issue, from variable reception on handsets, to the potential of available (and, very likely, affordable) bandwidth which may vary in different hot spots in 3G networks. This may also occur with handsets capable of switching between different network technologies and protocol, which may affect quality of service parameters over the usage session. For example, some of the current handsets that are marketed as 3G are actually capable of transferring between GPRS and 3G technologies, depending on network coverage.
6.5. CAT 5: data transcoding

In contrast to data prioritisation, data transcoding relates to the process of converting data according to the capability of the client device. However, there is the suggestion of loss of detail in video, for example, which could mean that detail which is crucial to the learning objectives designed into the learning object get lost. This could be protected by metadata prioritising a key level of quality, eg in video showing a technique for undertaking a procedure, a certain minimum acceptable level of resolution, frame rate, and so on. Where certain factors are regarded as essential for the learning object to have any pedagogic worth, we suggest that alternatives, which can be rendered as part of a modality transform (6.2), are provided by those responsible for providing the learning object in that context.
6.6. Herding CATs: decision engine

The above content adaptation techniques would be mediated by what Ma et al. call a decision engine, via a three-stage process I now consider how this may address the considerations I have outlined in this paper.
Stone
151
loss of control by the author over what is delivered to the reader) and copyright implications of content delivery. He suggests that liability may be avoided on a server, as the content provider has control over content transformation, although notes that on a proxybased architecture partnering with content providers may circumvent the copyright issue. In the UK education sector, agents such as Learning and Teaching Support Network (LTSN) subject centres (LTSN 2003), Joint Information Systems Committee (JISC) hubs (JISC 2003), and the like may be considered as such partnerships, although these agencies are exploring ways to address this issue.
for the potential for some degree of pedagogic integrity to be maintained, the profile resolution mechanism should integrate the considerations we have outlined in this paper and allow interaction with other mechanisms that support the effective resource discovery of learning object reuse.
8. Conclusion
This paper raises considerations about how we can design scalable, effective learning that can be used in networked learning, whether in the traditional e-learning domain, or in the rapidly increasing and diverse area of mobile learning. Regardless of the specific technologies in use, networked learning is underpinned by a constructivist approach combined with the use of digital content, more of which is being designed and/or acquired with reuse in mind. This paper has proposed that concerns relating to the maintenance of pedagogic integrity of materials need to be addressed. Networked learning activities can be mapped onto the cognitive domain of Blooms taxonomy of learning. Consideration of the mobile network and device constraints also act as an influence, particularly when the content in use may be reused. Frameworks for managing such issues exist, and can be used to facilitate the retention of pedagogic worth as digital content is reused across different network technologies. Such frameworks need to be integrated with resolution mechanisms such as that offered by DELI. Such integration requires a form of glue to facilitate such a process, ie an appropriate metadata set that can support what I envisage to be effective reuse. However, there are a number of issues surrounding the current state of play regarding the creation and quality of metadata of digital objects Currier and Barton (2003) provide an excellent overview. Even if these issues are resolved, we also require a culture and infrastructure that support the generation and sharing of metadata (as explored in Callahan et al. 1996), and effective meta-analysis of learning objects in particular contexts of use. If this can be implemented, a knowledge base could be generated which supports and strengthens a growing set of instances of good practice, which would maximise the potential for mobile learning across the board.
7. Other relevant standards and initiatives

Work is under way in the development of proposed standards which can address network quality of service and device dependency composite capabilities/preferences profiles (CC/PP) and user agent profiles (UAProf) respectively (eg Cowen 2002). These will be essential to underpin the framework Ma et al have outlined and to facilitate the move from repurposing learning objects to a variety of delivery platforms, towards delivering to a client device that may be able to support a variety of network technologies on one device (ie fourth generation mobile telephony/beyond 3G networks, or 4G/B3G). While a definitive implementation of the Ma et al. framework may still be some way off, Mark Butler of HP Labs has developed DELI, described as an open-source library that provides an Application Programming Interface (API) to allow Java servlets to resolve HyperText Transfer Protocol (HTTP) requests containing delivery context information from CC/PP or UAProf capable devices and query the resolved profile (Butler 2002). A servlet is a program that runs part of a network service and responds to requests from clients. This is achieved though profile resolution; with devices passing references to their particular profile (stored on the server) and any differences to that profile unique to that client. Cowen (2002) quotes Roger Gimson's observation of a stand-off between the client and server sides, in terms of making the first move towards supporting exchange of device capabilities, then asserts that in a device independent world, files will not necessarily look identical... on numerous devices and ... interfaces, but the content should remain usable regardless of the interface. To this end, in order
152
9. References
3 (2003). www.three.co.uk Aligonby J (2001). Phone texts end student agony. The Guardian. 22 October 2001. At www.guardian.co.uk/Archive/Article/0,4273, 4282067,00.html, accessed December 2003. Ananova (2001). Grades rise after text message teaching tip plan. Ananova.com, Thursday 23 August 2001. At www.ananova.com/ news/story/sm_381440.html, accessed December 2003. BBC (2002). Battle for the heart of the handset. BBC News Online. At http://news.bbc.co.uk/ 1/hi/technology/2336453.stm, accessed December 2003. Bloom B (1976). Human characteristics and school learning. New York: McGraw-Hill. Butler M (2002). DELI: a delivery context library for CC/PP and UAProf. HP Labs External Technical Report HPL-2001-260. At wwwuk.hpl.hp.com/people/marbut/DeliUserGuide WEB.htm, accessed December 2003. Callahan S, Johnson D, Shelley P (1996). Dataset publishing: a means to motivate metadata entry. Proceedings of First Institute of Electrical and Electronics Engineers Metadata Conference, Silver Spring, Maryland. At www.computer.org/ conferences/meta96/callahan/callahan.html, accessed December 2003. Collett M, Stead G (2002). Meeting the challenge: producing m-learning materials for young adults with numeracy and literacy needs. MLEARN 2002: Proceedings of the European Workshop on Mobile and Contextual Learning. Birmingham: University of Birmingham. ISSN 1463-9408. Cowen A (2002). Deli-style profile resolution: approaching device independence. mpulse magazine February 2002. At www.cooltown. hp.com/mpulse/0202-deli.asp, accessed December 2003. Currier S, Barton J (2003 ). Quality assurance for digital learning object repositories: how should metadata be created? In J Cook J and D McConnell (eds) Communities of Practice. Proceedings of the 10th Association for Learning Technologies Conference (ALT-C 2003), Sheffield, University of Sheffield: Association for Learning Technology.
Currin L (2001). Going mobile: m-learning's future may be bright, but other wireless applications must lead the way. eLearn Magazine, ACM, 11 September, 2001. At www.elearnmag.org/subpage/sub_page.cf m?article_pk=1880&page_number_nb=1 &title=FEATURE%20STORY, accessed December 2003. Darby J (2002). Networked learning in higher education: the mule in the barn. In C Steeples and C Jones (eds) Networked learning: perspectives and issues. London: Springer-Verlag, 1827. Friesen N (2003 forthcoming). Three objections to learning objects. In R McGreal (ed) Learning objects and metadata. London: Kogan Page. At http://phenom.educ.ualberta.ca/~nfriesen/, accessed December 2003. Gigure E. (2000). HDML/WML: HyperCard For wireless devices. Wireless Developer Network. At www.wirelessdevnet.com/ columns/mar2000/mobdev05.html, accessed December 2003. Gilbert JM, Brodersen RW (1999). Globally progressive interactive web delivery. Proceedings of IEEE Infocom 99. At www.ieee-infocom.org/1999/papers/ 09d_02.pdf, accessed December 2003. Gimson R (ed) (2003) Context overview for device independence; W3C working draft, 13 December 2002. At www.w3.org/ TR/2002/WD-di-dco-20021213, accessed December 2003. Hodgins HW (2000). The future of learning objects. In DA Wiley (ed) The instructional use of learning objects: online version. At http://reusability.org/read/chapters/hodgins. doc, accessed December 2003. IMS (2003). What does IMS stand for? IMS Frequently Asked Questions, IMS website. At www.imsglobal.org/faqs/imsnewpage. cfm?number=2 LAllier JJ (1997). A frame of reference: NETg's map to its products, their structures and core beliefs. NETg White Paper. At www.netg.com/research/whitepapers/frame ref.asp, accessed January 2004. Ma W-Y, Bedner I, Chang G, Kuchinsky A, Zhang HJ (2000). A framework for adaptive content delivery in heterogeneous network environments. Hewlett-Packard Laboratories White Paper. At
Stone
153
http://cooltown.hp.com/dev/wpapers/adcon/MMC N2000.asp, accessed December 2003. Milrad M (2003). Mobile learning: challenges, perspectives, and reality. In K Nyiri (ed) Mobile learning essays on philosophy, psychology and education. Vienna: Passagen Verlag, 151164. ISBN 3 85165 603 2. Noone (2001). ICUS uses phone tactics to elearn you. Australian Reseller News. At http://web.archive.org/web/20020109035813 /http://www.icus.net/pages/researchnews/ph one_tactics.htm, accessed December 2003. Palm (2001). Colleges give Palm handhelds to students. Palm Infocenter. 8 September 2001. At www.palminfocenter.com/view_ Story.asp?ID=2321, accessed December 2003. Pinkwart N, Schafer C, Hoppe HU (2002). Lightweight extensions of collaborative modeling systems for synchronous use on PDAs. Proceedings of IEEE International Workshop on Wireless and Mobile Technologies in Education. Vaxjo, Sweden, pp125129. ISBN 0 7695 1706 4. Polsani P (2003a). Network learning. In K Nyiri K (ed) Mobile learning essays on philosophy, psychology and education. Vienna: Passagen Verlag, 2003, 139150. ISBN 3 85165 603 2. Polsani P (2003b). Use and abuse of reusable learning objects. Journal of Digital Information, 3(4). At http://jodi.ecs.soton. ac.uk/Articles/v03/i04/Polsani/, accessed December 2003. Ramjee P and Munoz L (2003). WLANs and WPANs towards 4G Wireless. Universal Personal Communications Series. Norwood, MA: Artech House. ISBN 1 5805 3090 7. Soloway E, Jackson SL, Klein J, Quintana C, Reed J, Spitulnik J, Stratford S J, Studer S, Jul S, Eng J, Scala N (1996). Learning theory in practice: case studies of learnercentered design. CHI 96 Proceedings, ACM. At www.acm.org/sigchi/chi96/proceedings/ papers/Soloway/es_txt.htm, accessed December 2003. Stone A (2002). Mobile telephony and learning: nuisance or potential enhancement? Proceedings of 2002 IEEE International Symposium on Technology and Society:
Social Implications of Information and Communication Technology. Raleigh, North Carolina, pp262266. ISBN 0 7803 7284 0. Stone A, Alsop G, Briggs J and Tompsett C (2002). M-learning as a means of supporting learners: tomorrow's technologies are already here, how can we most effectively use them in the e-learning age? Proceedings of Networked Learning 2002 Conference, Sheffield. ISBN 0 9028 3141 0. At www.shef.ac.uk/nlc2002/proceedings/ papers/38.htm, accessed December 2003. Stone A and Briggs J (2002). ITZ GD 2 TXT: how to use SMS effectively in m-learning. MLEARN 2002: Proceedings of the European W o r k s h o p on Mobile and Contextual L e a r n i n g . Birmingham: University of Birmingham, pp1114. ISSN 1463 9408. Stone A, Stewart D, Livingstone D and Lynch K (2003). 3G Telephony on the Isle of Man: implications for individual notions of time, space, and place. Geography 88(3). ISSN 0016 7487. Taylor J, Peake L, Philip D, Robertshaw S (2002). Location Activated Nomadic Discovery (LAND): a mobile locationsensitive visitor information and navigation service for Cumbria. MLEARN 2002: Proceedings of the European Workshop on Mobile and Contextual Learning. Birmingham: University of Birmingham. ISSN 1463 9408. Turner A. (2001). Time to kill? Dial up WAP for e-learning on the run. IT News. 10 July 2001. At http://web.archive.org/web/ 20020309165624/http://it.mycareer.com.au/ news/2001/07/10/FFX90HO4XOC.html, accessed December 2003. Wang J (2001). B r o a d b a n d wireless communications: 3G, 4G and wireless LAN. The Kluwer International Series in Engineering and Computer Science. Boston, MA: Kluwer Academic Publishers. ISBN 0 7923 7391 X. Wired (2002). As American as a wireless U. Wired News. At www.wired.com/news/ wireless/0,1382,52234,00.html, accessed December 2003.
Syvanen et al.
155
Fragmentation in mobile learning

Antti Syvnen Hypermedia Laboratory FIN-33014 University of Tampere E-mail: antti.syvnen@uta.fi Marika Pehkonen Hypermedia Laboratory FIN-33014 University of Tampere E-mail: marika.pehkonen@uta.fi Hanne Turunen Hypermedia Laboratory FIN-33014 University of Tampere E-mail: hanne.turunen@uta.fi
Abstract
Learning with mobile devices is a highly fragmented process and this should be taken into account in designing as well as in developing evaluation methods for mobile learning materials and environments. Fragmentation in learning is understood as when the learning experience does not form a meaningful continuum because the amount of incoming information and the multiplied communication sources make the learning situations more distracting. This paper addresses the issue by describing a tentative list of mobile learning components (m-components), which offer the means to evaluate fragmentation. The mobile learning components have been operationalised into a self-rating questionnaire. The first version of the questionnaire was targeted at Finnish primary school pupils. The questionnaire was validated with a sample of 68 sixth graders from three primary schools. The preliminary results are presented in this paper along with supplementary qualitative data gathered from teacher interviews and observations on the sources of fragmentation. Keywords: mobile learning, fragmentation, evaluation
makes the learning situations more distracting. The issue of fragmentation has been brought up in an earlier pilot (Regan 2001), research paper (Leino et al. 2002) and also in the interviews with mobile learning experts carried out in our project. The aims of this paper are to: understand fragmentation as a typical phenomenon of mobile learning demonstrate how fragmentation is studied in our project and how it manifests itself in classes using different technologies (PCs, Communicators and laptops) discuss the implications of fragmentation for the design of mobile learning. Fragmentation in learning can be seen as an element affecting transfer. Transfer refers to the learners ability to use previously acquired skills and knowledge when performing a new task. In mobile learning the attention distribution in learning situations where the knowledge is processed is an important factor in constructing knowledge into observable learning outcomes. It can be said that fragmentation has occurred if the different learning situations do not form a meaningful continuum and the acquired knowledge does not accumulate. The hypothesis in the primary school pilot emphasises making learning more meaningful when introducing mobile devices in the classroom (Jonassen 1995; Leino et al. 2002; Turunen et al. 2003). The hypothesis has been presented with the model of mobile learning components (m-components). The mcomponents model has been operationalised in a mobile learning questionnaire (MLQ). The questionnaire includes measures of the possible effects of fragmentation on learning experiences and cognitions. The findings of the
1. Introduction
1.1. Aims of the study
Fragmentation in learning is defined in this article as when the learning experience does not form a meaningful continuum because the amount of incoming information and the number of communication sources are increasing which
156
preliminary data analysis of these measures will be presented. Supplementary qualitative data concerning the different sources of fragmentation in mobile learning has been acquired from interviews with teachers and by observing the field studies piloting mobile learning in two sixth-grade classes and at a small primary school in Raattama (18 pupils) that has acted as a remote partner in the Communicator pilot. The main research subjects have been 49 1112-year-old, sixth-grade primary school pupils who are using mobile devices to communicate and working collaboratively with pupils from different localities. In Pirkkala, a sixth-grade class (23 pupils) has used two Nokia 9210 Communicators, a laptop and a digital camera on their field trips, and processed the materials for online publication with eight computers in the computer class. In Hmeenlinna (Normaalikoulu, wireless future school), a sixth-grade class (26 pupils) has had personal laptops with a General Pack Radio Service (GPRS) card and Wireless Local Area Network (WLAN). In addition, a class using PCs at Turenki (26 pupils) took part in answering the MLQ. The findings have provoked discussion on the production of mobile learning materials as well as on the questions of usability and accessibility. Looking at our quantitative and qualitative findings we conclude with some preliminary guidelines for the design of mobile learning materials and present future research questions for studying fragmentation in mobile learning.
Sociological research on mobile use indicates the growth of flexibility but also of the micro-level inefficiency, for example when meetings are cancelled at the last minute (Cooper 2001; Kopomaa 2000; Laurier 2002). An imperative to be within reach all the time has resulted in different solutions so that people acquire, for example, separate mobiles for work and leisure (Kopomaa 2000). In management Davenport (2001) has paid attention to workers opportunities to focus on the right tasks rather than reacting to instant calls from the workplace or digital environment. It may be complicated to construct an environment in which people are able to combine their learning objectives, get the work done and live a fulfilling family and personal life. Therefore the promise to work or learn regardless of time and place should not automatically be included in the definition of mobility or mobile learning. Mobile learning is often defined as learning that takes place with the help of mobile devices (eg Quinn 2000). This does not necessarily capture the nature of the learning. A device that supports learning may be freely moved, but in the learning events the learner is mostly stationary, even though they are using a mobile device. Although the device is mobile and portable, the learning as an event cannot be described as mobile (Ahonen et al. 2 0 0 4 ) . Moreover, when people access information sources and learning objects via different devices from different locations, there are still many usability, compatibility and accessibilityrelated questions that hinder seamless mobility and mobile learning. Still, our hypothesis has been that as mobile devices are a pervasive medium, they can help to combine work, studying and leisure time in a meaningful way (Turunen et al. 2003). Therefore, mobile learning should be examined especially from the viewpoint of informal learning. According to Livingstone (2000) informal learning can be described as the activities aiming at creation of knowledge, understanding, and skills acquisition outside curricula or courses. Incidentally, initiated learning, irregularly timed learning, and the distinction between learning processes and learning outcomes have all been recognised as challenges for research concerning informal learning. In this sense the tools supporting learning, which possibly are mobile, can help both learner and teacher to perceive the observable processes of learning, which can be difficult to carry out (Ahonen et al. 2004).
1.2. Mobile learning and the challenges of context

When conceptualising mobility the computersupported cooperative work (CSCW) approach concentrates on depicting mobile technology and varying information needs in different situations (Churchill and Wakeford 2002; Luff and Heath 1998). But does the concept of mobile learning provide a narrow and technically defined utopian image of mobile learning based on a single characteristic of a mobile device? Laurier (2002) points out that remarks about technologies such as cars, mobile phones and Wireless Application Protocol (WAP) that they somehow cause work to be faster, more mobile and more connected-up commonly misinterpret the technologies and their users. It is forgotten how the management of time and place concerning the use of the technology is also joined with people slowing down, staying in one place and sometimes keeping the network connection closed.
Syvanen et al.
157
According to the initial hypothesis of making learning more meaningful, bringing this pervasiveness to a primary school would enable continuity between institutional learning and learning from real-world phenomena outside school hours. When the device is a flexible learning tool with which people repeatedly enhance their knowledge and skills according to their personal strengths, the goals of lifelong learning (Sharples 2000, 2002) and life-wide learning (Drake 1999) can also be achieved. Here mobile learning requires from the learner intention, self-discipline, reflection and learning to learn. Together with web-based services mobile learning can enable collaborative learning and access to different information sources in actual problem-solving situations (Jonassen 1995; Sharples 2000; Leino et al. 2002.) Our model (Figure 1) has been developed from the pedagogical point of view and also takes note of flexible and informal learning practices (Collis and Moonen 2001). Fragmentation is an overarching element in the model, especially in the continuity and contextuality components. It arises from distracting learning contexts and has potential effects on the continuum of long-term learning process.
On the other hand, mobile applications may enhance continuity when ideas need to be documented. One of the interviewed mobile learning experts stated: ideas need to be documented as they emerge for example when the learner is studying in the middle of the night or a studying group is having a cup of coffee. Nevertheless, even in the more consistent mobile learning situations the possibility of fragmentation is still present and the situational and environmental elements should be considered. The phenomenon of fragmentation is not new; it has only been redefined in the era of information technology. For example it was covered earlier in ecological psychology and especially in the time-budget studies. The ecological psychology perspective emphasised the wholeness of human functions, and stressed the content of the functions and their temporal and spatial organisation (Barker 1968). Actual environments have been seen as having a crucial role in psychological development. To understand the environments where human beings live requires a theoretical schema that will permit the systematic description and analysis of these contexts, their interconnections, and the processes through which these structures and linkages can affect the course of development, both directly and indirectly (Bronfenbrenner 1979). Our approach to mobile learning is same as in the earlier studies in ecological psychology where the actual environment was given emphasis. According to Cowan (1995) the presence of attention is a key element both at the time the preliminary information is processed and at the time of recall. As disturbing environmental elements can easily distract attention the metacognitive skills also become important.
2. What is fragmentation?
Regan (2000) raised the issue of fragmentation in her mobile learning pilots. Mobile learning can be seen as a highly fragmented experience: on-the-go learning situations are often disrupted and take place unexpectedly, and the focus of attention can easily be distracted. It can be questionable whether these on-the-go situations form a strong basis for meaningful mobile learning (Leino et. al. 2002). Figure 1. Mobile learning components
158
Data-driven processing and the formation of procedural knowledge can often go on to some extent with little or no attention, although it may be susceptible to attentional effects but conceptually driven processing and the formation of declarative knowledge typically require considerable attention to occur at all (Cowan 1995). This means also focusing on the right learning tasks or the information at hand. From the educational perspective the key issue here is: Is it really a fragment or was it just one element in a sequence within a continuum, as another of the interviewed experts in our study noted. This depends on how the information has initially been processed. In this way the fragmentation encompasses the issue of transfer. Salomon and Perkins (1989) have made distinctions between low-road and high-road transfer. Low-road transfer refers to a type of transfer that automatically emerges when two tasks are closely related to each other. Highroad transfer refers to the intentional application of previously acquired knowledge in new situations. According to Soini (1999) high-road transfer has usually been the goal of formal academic education and the skills taught in school are usually assumed to transfer in a wide, decontextualised manner. However, in mobile learning the low-road transfer should be seen as meaningful, as it emphasises the informal forms of learning discussed earlier. The earlier studies of transfer have taken either the metacognitive or the situational approach. The metacognitive approach regards knowledge as an abstract entity residing in individual cognition acquired in one task setting and conveyed to other task settings. Therefore the metacognitive approach considers the application of this knowledge (eg learning styles) in situations that are different from the learning context as the problem of transfer. The situated approach considers knowledge as being somewhat bound to situations (Brown et al. 1989). Then again the problem of transfer has been seen in earlier studies as to whether transfer can occur at all (Soini 1999). However, the preconditions for transfer in mobile learning are both metacognitive and situational. Therefore, we have taken an eclectic approach to studying transfer in relation to fragmentation. Recent discussions (eg Anderson et al. 2000) have shown convergence of the approaches. In a simplified manner fragmentation in mobile learning situations can be caused by the environmental disturbances, poor concentration of the learner and technical problems, eg bad
network connections, problems with the device or in the application. Considering fragmentation in relation to the question of transfer in mobile learning means investigating in which conditions fragmentation occurs and how does it affect transfer?
3. Methods for evaluating fragmentation in mobile learning

The study presented here is part of the Digital Learning 2 research project. The main objective of the project is to develop an evaluation tool (eValuator) for digital learning materials and environments. Our mobile group in the project explores mobile and web-based learning methods and develops the means to evaluate them as informal and collaborative learning. The aim of our studies has been to support the development of eValuator with the evolution of the empirically tested criteria that are grounded on m-components. (Digital Learning 2003). The mobile learning questionnaire was used in this study to reflect how fragmentation could be observed in relation to learning styles and experiences. Supplementary qualitative data has been acquired in teacher interviews and in observing the field studies and experiments. The qualitative data was used to track the sources of fragmentation according to mcomponents and to explain the results of MLQ, so producing a more complete picture of the investigated phenomena (Kelle 2001).
3.1. Mobile learning questionnaire

The mobile learning questionnaire is constructed in two parts. The first part, in accordance with mobile learning components, measures the assumed key competencies in using mobile devices in learning activities. The second part measures the learning experiences of the learners who have used the devices. The model works as a compilation of key elements presented in earlier studies of using mobile devices in learning; both possible strengths and weaknesses. In the questionnaire these issues have been operationalised. MLQ was developed to model learners using mobile devices and to evaluate their use. It was based on tests developed earlier and found to be valid. The phenomenon of fragmentation in learning was pursued using a scaling instrument based on students deep and surface approaches to learning developed by Entwistle and others (Entwistle and Ramsden 1983; Entwistle and Tait 1994) in their
Syvanen et al.
159
Table 1. Descriptives of the variables categorised by the device used

DEVICE Communicator DUALISM 20 ,109 1,184 -3,012 2,745 22 -,120 ,692 -2,406 ,624 26 ,018 1,091 -2,406 3,351 68 ,000 1,000 -3,012 3,351 SURFACE 20 -,226 1,047 -1,845 1,865 22 ,531 ,796 -,833 1,865 26 -,275 ,973 -2,857 1,865 68 ,000 1,000 -2,857 1,865 DEEP 20 -,113 ,865 -2,050 1,209 22 -,026 ,721 -,964 1,933 26 ,109 1,285 -2,774 2,295 68 ,000 1,000 -2,774 2,295 CONSTRUCT 21 -,145 1,353 -3,146 2,161 21 ,281 ,538 -,824 1,166 26 -,110 ,941 -1,819 2,161 68 ,000 1,000 -3,146 2,161 SEARCH 21 -,025 1,181 -2,477 2,113 21 ,307 ,698 -1,080 1,315 26 -,228 1,020 -1,479 2,712 68 ,000 1,000 -2,477 2,712 SHARING 21 -,536 1,029 -2,082 1,548 21 ,755 ,683 -,587 1,762 26 -,177 ,841 -1,655 2,403 68 ,000 1,000 -2,082 2,403
N Mean Std. Deviation Minimum Maximum
Laptop
PC
Total
approaches to studying inventory (ASI) and the seven item dualism scale (Ryan 1984; based on Perry 1970). The approaches to learning scales and the conception of knowledge scale (dualism) were selected in relation to Cowans (1995) arguments on attention and information processing. The dualism scale measures the relativistic and contextual knowledge orientations of the learners. The dualism refers to the m-component of contextuality as it measures how context-dependent the learners knowledge concepts are. Using the deep approach the learner tries actively to understand the material, whereas with the surface approach the learner tries to learn in order to repeat what they have learned. The learning approach scales refer to the m-component of continuity and adaptability as the initial processing of the material, which may have long-term effects on how the acquired knowledge will be used. The selected scales thus reflect the preconditions to transfer in which fragmentation may be manifested. Scales to measure knowledge construction, sharing and seeking activities in relation to using the devices were also created to capture the effects of possible fragmentation in learning experiences and to reflect the role of the metacognitive orientations in the learning process. During the spring of 2003 classes at Pirkkala, Hmeenlinna and Turenki completed the questionnaire as it went through its preliminary testing. At Pirkkala the questions referred to the
use of Communicators, at Hmeenlinna to the use of laptops and at Turenki to the use of PCs. Two respondents from the Pirkkala school were subtracted from the data because of uninterpretable answers. One pupil (at Pirkkala) did not complete the first part and another pupil (at Hmeenlinna) did not complete the second part of the questionnaire. Also four of the pupils at Hmeenlinna were not present at the time of completing the questionnaire. The scales showed fair reliability: deep approach Cr. alpha = .668, surface approach = .601 (1 item deleted), dualism = .575 (1 item deleted), knowledge construction = .680 (1 item deleted), sharing = .782 and seeking = 682. Ztransformation was conducted to the variables. The descriptives for the variables are shown in the Table 1 above. One sample t-test was used to find out whether classes responses in the MLQ were different from the expected theoretical populations means. Analyses of variance (ANOVA) were used to investigate the differences between the classes. One sample t-test revealed that there was a difference in knowledge sharing in class between pupils who used Communicators and theoretical population t(20) = -2.39, p < 0.05 This result indicated that the knowledge sharing in this class was rather unsuccessful. The difference in knowledge sharing also occurred with pupils using laptops t(20) = 5.07, p < 0.01. However this finding indicated being successful
160
knowledge sharing. The results in knowledge construction activities t(20) = 2.39 p < 0.05 in the class using laptops also indicated that the activities were successful. The pupils using PCs did not show any difference to the theoretical population, nor did the pupils using laptops and Communicators in any of the other variables measured. When the surface approach towards learning was examined, a one-way ANOVA test indicated that there were significant differences between classes F(2, 65) = 5,175 p < 0.01. According to the post-hoc tests (Tukeys multiple range test) the pupils using the laptops formed their own group, and there was no significant difference between the pupils using Communicators and PCs. This showed the pupils using laptops had more of a surface approach to learning than the other pupils in the other classes. Also, when the knowledge-sharing activities was examined, a one-way ANOVA indicated that there were significant differences among the classes (see Table 2 below). According to the post-hoc tests, the pupils using the laptops formed their own group and there was no significant difference between the pupils using Communicators and PCs. A two-way ANOVA showed that there was also a significant twoway interaction of the device used and deep processing on knowledge sharing activities. The result showed the pupils who used deeper approaches to learning were also found to use knowledge sharing activities.
The differences found in using laptops, Communicators and PCs should also be reflected through how continual their use was and how well it was implemented in the curricula. The laptops were used many times a day for studying in the school while the Communicators were used less than once a week and PCs once a week. The continual use of the laptops compared to the two other classes may have given a clearer idea of the possibilities of using mobile devices. Thus it is possible that the results regarding the class using laptops were also the clearest. The relationship between the deep approach and knowledge-sharing activities in this study might reflect the relation between how information was initially processed and then later shared with others. Although the surface approach to learning was strongest in the class using laptops, the knowledge-sharing activities seemed to work well. This could just reflect the fact that the pupils were flexible enough to use different learning strategies and simply used deep processing. The knowledge sharing, in general, also seemed to be an issue in this study, as again in the class using Communicators the knowledge sharing seemed unsuccessful. As the use of the devices was not coherent, many unpredictable situation-specific elements may have caused the differences between the classes, rather than the different media. These elements may have caused fragmentation and had an effect on the transfer.
Table 2. ANOVA tests of between-subject effects

Dependent Variable: SHARING Type III Sum of Squares a 45,714 1,381 13,081 14,183 15,127 18,505 64,260 64,219
Source Corrected Model Intercept DEVICE DEEP APPROACH DEVICE * DEEP APPROACH Error Total Corrected Total
df 27 1 2 12 13 39 67 66
Mean Square 1,693 1,381 6,541 1,182 1,164 ,474
F 3,568 2,910 13,785 2,491 2,452
Sig. ,000 ,096 ,000 ,016 ,015
a. R Squared = ,712 (Adjusted R Squared = ,512)
Syvanen et al.
161
Two such elements could have been how continual was the use of the media, and in what kind of contexts were the learning activities done? One explanation could be the continual use of the laptops compared to the intermittent use of the Communicators outdoors. The interviews and observations will clarify further the differences between the different media use. In this study it seemed that the pupils knowledge conceptions had no implications for the learning activities. The contextuality mcomponent did not get any empirical confirmation from the data gathered with the MLQ. The use of the devices for knowledge seeking and construction activities did not have a statistical significance either. However, as the samples were small, the preliminary findings cannot be generalised thoroughly.
jealously watched for the same chance to use the devices, or demanded the teachers attention at the same time. Moreover, the childrens attitudes towards the new gadgets may cause fragmentation in learning. As the children used their personal mobile phones basically for fun, they also wanted to use the Communicators mainly for entertainment purposes. Their enthusiasm towards these devices diminished slightly when the learning activities and tasks demanded more information processing (Turunen et al. 2003). Meanwhile, the laptops were perceived as tools and communication media over a full school year. At this point we can recognise another level of fragmentation, that is the level of user cultures and how the teacher has to react to these different users needs. From the user cultures point of view, an edutainment element might be useful when designing materials for mobile learning. However, teachers have noted that the earlier the children learn to use the devices, the better they get at the advanced methods of learning: crossdisciplinary project work and the process-writing method. Connections to the outside world and different sources of information may also cause chaos in the classroom (Sharples 2002; Mifsud 2002). The interviewed teachers agree that copying from the internet is a growing problem due to internet access. The primary school pilots stressed that the use of mobile devices, and accessibility to several different sources of information, require a flexible conception of knowledge. In MLQ the m-component of contextuality measures the users abilities to search for information and its organisation as well as their conception of knowledge. In the pilot the inquiry learning method that engages the pupils to reflect on their own thoughts and actions has been used. With this method the pupil learns to criticise certain facts and their understanding develops within collaborative group work. The teacher still had a central role in helping the pupil understand that using mobile devices can really assist the learning processes. The teachers of the Communicator pilot group pointed out that it is crucial to the success of the whole mobile learning process that the pupils do not lift their informative goals too high. During the pilot the children often thought that the facts they observed had to be either entertaining or really complex. This may also reflect the effect of information overflow in todays society and how the different media
3.2. Interviews and observations on mobile learning pilots

The findings of fragmentation in mobile learning pilots will be considered next in the context of the m-components. With the mcomponent of Continuity and adaptability we study a flexible transformation between different learning situations with the aid of a mobile device. In the MLQ it is measured as surface and deep approach in learning. In practice spontaneous learning requires the abilities to apply and reflect ones knowledge. Basing on the teachers interviews, the m-component of Continuity and adaptability proves an essential aspect in mobile learning, because of the crossover between the different contexts, which had been difficult to reach in the Communicator pilot. Children have had difficulties in understanding that they can learn everywhere and they can use the knowledge acquired in contexts other than school as well. Here fragmentation seems to intertwine with cultural issues: children should be taught to appreciate the knowledge they gain from their hobbies and everyday activities. According to the teacher this also means that learning has to be seen as a personal process, which happens in different situations and through the learner bringing his/her own personal experiences to the classroom situations. In mobile learning this can be supported, for example, by choosing inquirybased learning methods and presenting or publishing the learning outcomes online. The difficulties in continuity vary also depending on the maturity of the children both as individuals and as a group. Group dynamics may hinder the differentiation. In the Communicator pilot, for example, the children
162
sources present information in a more and more entertaining way. Therefore it is important to learn to recognise different sources and structures of knowledge. Rubin and Kaivo-oja (2000) point out that as the flow of information becomes increasingly fragmented and uncontrolled, also real-time and simultaneous, there is a danger that established ways for managing information do not suffice. The selection of relevant information becomes a random affair, and choices are made based on what appears to be fun, entertaining or useful at the time. This increases social disempowerment. From this point of view we can presumably question whether there is another kind of level of fragmentation. Is the learners experience so fragmented that they do not understand enough to process it, or do not have the time to process it properly? If the learner does not have the time to process the information enough it will not become a part of their knowledge. More attention should be paid to cultural and media literacy: pupils ability to distinguish different types of knowledge and to evaluate different kinds of information sources (Rubin and Kaivooja 2000). This requires new types of knowledge and skills from the teachers too, and for many teachers there is not enough time or easy tools at hand to take over the issues of media literacy, information searching and internet publishing. Supplementary education on the topic is required, as well as easy tools and learning materials! Based on our pilot findings accessibility, sufficient skill levels and desired possession of a device, crucially affect the possibilities of mobile learning. The teachers interviewed estimated that fifth- and sixth-grade children possess the required skill level for learning typical mobile learning activities, and at the age of 1112, pupils have an interest in and need for it too. However, it will not be easy to start mediated learning with mobile devices unless they have had previous experience with computers. Pupils who already had good skills got further with the Communicator, while those with poorer computer skills got tired of trying to figure out the complicated logic and user interface of the device. The Communicator pilot did not reach the content goals of the mobile learning; but for those using the laptops their active content production increased and pupils having difficulties with it clearly improved their skills. The differences also diminished with pupils who did not have computers at home. Personal laptops were perceived as inspiring throughout the year, whereas eagerness about the irregular and scattered use of Communicators diminished
after an enthusiastic beginning. Based on the interviews and observations, therefore, it seems that sufficient personal possession of the device is required to eliminate fragmentation, ie to gain routine in using a device and be able to focus on learning contents. For the future it would be useful to provide equal possibilities for mediated learning to all the pupils to capture the learning part of mobile learning properly. When offering tools for active and intentional self-guided learning (time and learning management) we see that mobile technology can be well utilised in supporting differentiation because it is a rather flexible and motivating medium. Since pupils' technical and learning skills vary, the teacher has a lot of work to tailor lessons to meet everybodys needs. The teachers tend not to believe in using readymade learning content for mobile learning, but they have seen the materials, which are produced in mobile learning. For the more selfguided pupil, directions on independent learning tasks might work. From the viewpoint of flexible interaction the lack or inoperability of GPRS connections were the main sources of fragmentation. In the Communicator pilot, the data transfer connections were not available. The pupils from Pirkkala communicated with their peers in Kittil (Raattamas school) by calling and sending text messages (SMSs). With the laptops WLAN worked fine in the school area, but because of weak GPRS connections pupils were only able to browse single web pages or read their emails outside the school. The intended mobile use of one web-based learning environment was not possible because of its heavyweight multimedia content. Still, the mobile devices did enable meaningful learning tasks for small groups and those working in pairs. Later on the children willingly expressed themselves and proudly showed at school what they had found out from the internet or created with the devices. The possibility of publishing the work on the internet and showing it to parents boosted their motivation and the level of outcomes.
4. Discussion and future work

According to the preliminary findings from the MLQ, the knowledge-sharing activities became more emphasised when the laptops were used in the classroom in a continual manner, even though the pupils did not have a specific application designed to support this. When properly instructed and implemented in the curriculum, the laptops seemed to have
Syvanen et al.
163
advantages in knowledge-sharing practices. However their use outside the classroom could affect this in an unpredictable way. As the use of the devices was not coherent, the findings of the preliminary analysis are hard to interpret: many unpredictable situation-specific elements may have caused the differences between the classes, rather than the different devices. Drawing on our findings we can draft some preliminary guidelines for the design of mobile learning practices and materials in the primary school setting that could prevent fragmentation. When seeking the sources of fragmentation in pilot studies, accessibility becomes a central theme. In the Communicator pilot the children with slight difficulties in concentration got confused and interrupted the learning process when they got lost in the user interface. This might have also been reflected in the learning activities measured in MLQ. The learning content goals were also difficult to reach due to the short period of use. The skills acquired with the PC helped them become familiar with mobile learning. However, sufficient possession and use of the device is required to enable the transformation to routine use when a learner is able to concentrate in learning contents. Here, the usability of a device is of great help in adopting mobile learning practices. A sufficient amount of time to process the learning content is also required to prevent fragmentation in knowledge construction. As the knowledge sharing and supporting the initial processing of the knowledge aspects seemed to be issues in our study, one should keep in mind the future distribution of knowledge to others. Moreover, the problem in applying flexible interaction implies the importance of good connections and networks, but also the need for accessible webbased learning materials and environments. When operating with the device the environment and context offer other elements that can distract the users attention. Although there was no statistical significance found in the pupils conceptions of knowledge, the interviews and observations stressed the importance of a flexible conception of knowledge. The classroom culture and different user cultures set clashing expectations towards mobility and mobile learning. From the wider cultural perspective the access to different sources of information requires new approaches towards knowledge building and learning among both teachers and pupils. Based on our pilot findings, the teachers did not believe in using ready-made learning contents, rather using the materials produced in mobile
learning. The earlier the children learn to use the devices as tools, the more flexibly they get into the advanced methods of learning: crossdisciplinary project work and the process writing method. More attention should be paid to cultural and media literacy, and to the childrens ability to distinguish different types of knowledge and information sources. The learning tools for information searching, content processing and internet publishing can, therefore, be presumed to be very useful. As mobile learning is well suited to small groups and working in pairs, its potential lies in supporting social contacts and collaborative learning. To restrain the possible effects of fragmentation in mobile learning the user should have tools designed specifically for mobile situations. The different knowledge construction and learning management tools being developed are important in highlighting this need. The MLQ showed consistency and sufficient validity to be used in the next mobile learning research pilot. However, it should be used in evaluation with other methods and the findings interpreted in relation to other results. The questionnaire will be used in autumn 2003 in a pre-post test research frame where different devices (laptops, iPAQs and PCs) are piloted in comprehensive school classes. The devices and applications will be used and instructed more coherently than in the preliminary study. Fragmentation will be evaluated by interviewing the teachers and by making observations of the performed learning tasks. Future research questions and actions concerning contextuality in informal and mobile learning will be targeted on adult learners and their learning processes. Acknowledgements We would like to thank Tiina Jrvinen, Mikko Ahonen and Vesa Korhonen for their comments and invaluable help. Digital Learning 2 research project is a joint project of Hme Polytechnic and University of Tampere. Researchers in the Hypermedia Laboratory at the University of Tampere have specialised in the area of mobile learning. The research work in the Digital Learning 2 project has been funded by TEKES, the National Technology Agency in Finland, European Regional Development Fund and partners in cooperation: DNA Finland Ltd, HPOYhtym, Nokia, eWSOY, Discendum Oy, Summer University of Hme, Hmeenlinna Technology Centre Ltd, IT Hmeenlinna Oy, Mediamaisteri Ltd, and Hme Polytechnic.
164
References
Ahonen M, Joyce B, Leino M and Turunen H (2002). Mobile learning: a different viewpoint. In HKynslahti (ed) Mobile learning. Helsinki: ITPress. Ahonen M, Pehkonen M, Syvnen A, Turunen H (2004 forthcoming). Mobile learning and evaluation. Interim report. Digital Learning 2 project. Working papers. University of Tampere: Hypermedia Laboratory. Anderson JR, Greeno JG, Reder ML, Simon HA (2000). Perspectives on learning, thinking, and activity. Educational Researcher 29 (4), 1113. Barker R (1968). Ecological psychology: concepts and methods for studying the environment and human behavior. Stanford: Stanford University Press. Bronfenbrenner U (1979). The ecology of human development. Cambridge: Harvard University Press. Brown JS, Collins A and Duguid D (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 3242. Churchill E and Wakeford N (2002). Framing mobile collaborations and mobile technologies. In B Brown, N Green and R Harper (eds) Wireless world: social and interactional aspects of the mobile age. London: Springer-Verlag. Collis B and Moonen J (2001). Flexible learning in a digital world: experiences and expectations. London: Kogan. Cooper G (2002). The mutable mobile: social theory in the wireless world. In B Brown, N Green and R Harper (eds) Wireless world: social and interactional aspects of the mobile age. London: Springer-Verlag. Cowan N (1995). Attention and memory: an integrated framework. New York: Oxford University Press. Davenport TH (2001). eLearning and the attention economy: here, there and everywhere? LiNE Zine Summer 2001. At www.linezine.com/5.2/articles/tdeatae.htm, accessed November 2003.
Digital Learning 2 (2003). Project webpages. At http://dll.hamk.fi/dl2/en/, accessed November 2003. Drake C (1999). Values education and lifewide learning. Paper for the 16th Annual Conference of the Hong Kong Educational Research Association. At www.livingvalues.net/pdf/lvlearning.pdf, accessed November 2003. Entwistle N and Tait H (1994). The revised approaches to studying inventory. Edinburgh: Centre for Research into Learning and Instruction, University of Edinburgh. Entwistle N and Ramsden P (1983). Understanding student learning. London: Croom Helm. Jonassen DH (1995). Supporting communities of learners with technology: a vision for integrating technology with learning in schools. Educational Technology 35 (4), 6063. Kelle U (2001). Sociological explanations between micro and macro and the integration of qualitative and quantitative methods. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research [online journal], 2(1). At www.qualitative-research.net/fqs-texte/1-01/101kelle-e.htm, accessed October 2003. Kopomaa T (2000). City in your pocket: birth of the mobile information society. Helsinki: Gaudeamus. Laurier E (2002). The region as a sociotechnical accomplishment of mobile workers. In B Brown, N Green and R Harper (eds) Wireless world: social and interactional aspects of the mobile age. London: Springer-Verlag. Leino M, Turunen H, Ahonen M, Levonen J (2002). Mobiililaitteet oppimisen ja opetuksen tukena. In P Seppl (ed) Mobiili opiskelu: joustavasti liikkeess. Helsingin yliopisto, Opetusteknologiakeskus. Helsinki: Yliopistopaino, 4758. Livingstone D (2000). Exploring the icebergs of adult learning: findings of the first Canadian survey of informal learning practices. N A L L working paper 10-2000. Toronto: Ontario Institute for Studies in Education, University of Toronto. Luff P and Heath C (1998). Mobility in collaboration. In CSCW 1998. 305314.
Syvanen et al.
165
Mifsud L (2002). Learning '2go': pedagogical challenges to mobile learning technology in education. In K Nyri (ed) Mobile learning: essays on philosophy, psychology and education. Communications in the 21st Century. Vienna: Passagen Verlag. 165173. Perry W C (1970). Forms of intellectual and ethical development in college years. New York: Holt, Rinehart and Winston. Regan M (2001). Mobile learning explorations at the Stanford Learning Lab. Speaking of C o m p u t e r s , Issue 55, Jan 8, 2001.. A t http://sll.stanford.edu/projects/tomprof/newtompr of/postings/289.html, accessed January 2004. Rubin A and Kaivo-oja J (2000). Young people caught between the new and the old: how to empower citizens of the future. In A Rubin Growing up in social transition: in search of late-modern identity. Doctoral dissertation, University of Turku, Finland. Annales Universitatis Turkuensis, Ser B, 234, Humaniora, 97127. Ryan MP (1984). Monitoring text comprehension: individual differences in epistemological standards. Journal of Educational Psychology 76, 248258. Salomon G and Perkins DN (1989). Rocky roads to transfer: rethinking mechanisms of a neglected phenomenon. E d u c a t i o n a l Psychologist, 24, 113142. Sharples M (2000). The design of personal mobile technologies for lifelong learning. Computers and Education, 34, 177193. At: www.eee.bham.ac.uk/sharplem/papers/handler %20comped.pdf, accessed November 2003. Sharples M (2002). Disruptive devices: mobile technology for conversational learning. Paper accepted for publication in International Journal of Continuing Engineering Education and Life-long Learning. Special issue on collaborative learning in networked environments. (draft). At www.eee.bham.ac.uk/ handler/ePapers/ijceell.pdf, accessed February 2003. Soini T (1999). Preconditions for active transfer in learning processes. Helsinki: The Finnish Society of Sciences and Letters.
Turunen H, Syvnen A, Ahonen M (2003). Supporting observation tasks in a primary school with the help of mobile devices. In K Nyri (ed) Mobile learning: essays on philosophy, psychology and education. Communications in the 21st Century. Vienna: Passagen Verlag, 209221.
Taylor
167
A task-centred approach to evaluating a mobile learning environment for pedagogical soundness

Josie Taylor IET UserLab The Open University, Milton Keynes E-mail: j.taylor @open.ac.uk Abstract
This focus of this paper is how to evaluate the pedagogical soundness of a mobile learning environment in which many users (both teachers and learners) may not have previously encountered mobile technology, so may be uncertain how best to deploy it to achieve their goals. Drawing on concepts from Activity Theory and the socio-cognitive engineering method described by Sharples (2000), it describes an approach which enables an enriched view of users current and future activities, which in turn will allow us to understand the range of actions and opportunities open to mobile learners, and seek ways of extending this range to support what learners want to do even if they themselves do not yet know what that is. Keywords: learning, socio-cognitive engineering, Activity Theory, pedagogy, evaluation, mobile environments 1. Introduction A major goal of the worldwide, European-led research and development project MOBIlearn is: the creation of a virtual network for the diffusion of knowledge and learning via a mobile environment where, through common themes, it is possible to demonstrate the convergence and merging of learning supported by new technology, knowledge management, and new forms of mobile communication. MOBIlearn Technical Annex 1, page 7 The project aims to evaluate the pedagogic effectiveness of the learning environment thus developed to ensure that it is sound. Although there are tried and tested methods for pedagogic evaluation of specific applications of technology for learning (eg Draper et al. 1996; Scanlon et al. 2000), there are no existing comprehensive frameworks for broader formative evaluation in the mobile environment, largely because of its novelty relatively few teachers and learners have experience of working in this way, so we are simultaneously introducing new ways of engaging in learning with new artefacts and evaluating technical and pedagogic effectiveness. This requires careful consideration so as not to skew the evaluation data gathered from users, who may find themselves fascinated by the new devices in a way which they may find interesting, and even fun, but which produces no lasting valuable impact on their work practices. They may simply then avoid using the technology in anger once the evaluation study is complete. Therefore, to make progress in achieving our goals, we must develop a thorough understanding of: the learning opportunities presented by the new mobile technology its (potential) impact on the way people perform learning tasks its (potential) impact on human social processes and interactions how these in turn are changed or modified by the technology.
In the rest of this paper we briefly indicate how it is possible to develop this understanding driven by task-centred user requirements rather than technological advances, so describing the approach that underpins our evaluation strategy for MOBIlearn. 2. Pedagogy in the mobile environment Developments in pedagogy have moved away from the transmissive mode of teaching and learning and toward the constructivist or sociocognitive models, placing the active learner at the heart of activities. In this view learning is: a personal idiosyncratic experience, characterised by individuals developing knowledge and understanding it through the forming and re-forming of concepts. The focus of constructivism is on learner control, with learners making decisions that match their own cognitive states and needs. Farmer and Taylor 2002
168
The socio-cognitive view would also add that learning takes place in a social context (see Rogers 2002), and the forming and re-forming of concepts need not necessarily take place only at the level of the individual. Collaborative group work and sharing with peers (and others) can be a powerful way of confronting ones own conceptions (pre-conceptions), contributing to the perceived need to restructure ones cognitive schemas. So learning can be perceived as being as much about communication as it is about content. In fact, some more radical pedagogical approaches, facilitated by mobile computing, would go a step further, and suggest that no content is a useful starting point for learning. A group of learners may decide themselves what they are going to learn, and how they are going to learn it, bringing their own material to bear in whatever way they feel appropriate. The MOBIlearn project embraces this view of learning, with its emphasis on rapid communication and access to resources. In this context, however, although usability is an important issue for evaluators, it is not enough to say that because the usability requirements have been satisfied, the MOBIlearn project has been successful from the pedagogic perspective. Pedagogical evaluation demands to understand not only whether or not a learner has succeeded in learning, but why. Understanding the reasons for success or failure depends on deep knowledge of the appropriate relationship of tasks to technology an area of knowledge that spans both the pedagogic/educational, and the technical fields. From the point of view of usability, educators and learners have raised the concern that the handheld elements of the mobile environment have very small screens which do not facilitate easy access to text, and small keyboards which impede input of, or annotation of, content and do not support skim reading (see Kukulska-Hulme 2002). These are real ergonomic concerns but they are not fatal for the learning enterprise because it depends what role the handheld is playing in the activity. For example, few would argue that using a current personal digital assistant (PDA) as an ersatz laptop, to access and read large documents, is an optimal use of the device. However, using the PDA to find or share documents to download onto a desk-top or laptop computer for later perusal is perfectly feasible. We must beware trying to make devices perform beyond their capacity to deliver what is required, but, rather, we should examine potential activities that could be
supported, and evaluate the pedagogic benefits of these activities, which may be distributed across several devices. The whole experience needs to be evaluated, not just the component parts. This will mean ensuring that mobile technologies are used appropriately to exploit their potential, for example supporting activities that might simply be impossible without them. This is quite a challenge for evaluation because we have to recognise that the integration of new tools into existing activities creates a dialectic the tool introduces new possibilities for action, and new constraints (see Waycott et al. 2002) which change how the activity is performed. We must also take into account that, in adopting the human-centred view, it would be philosophically unacceptable for us to disregard learners existing tasks and their structures, and impose tasks upon them that we as designers or teachers think are beneficial ie possibly favouring the capabilities of the technology rather than the users. As stated earlier, the active learner is at the heart of the enterprise, so we need to observe and analyse the effect of technology on learner actions, activities, intentions and goals as they engage in learning. Sometimes they will change, for good reason; sometimes they will not. Understanding activities Addressing this issue, we have adopted the socio-cognitive engineering method for system design (Sharples 2000; Sharples et al. 2002) which describes a two-stage process: first, activity analysis sets constraints on the system design and analyses how people work and interact with their current tools and technologies; and, second, design of new technology is integrated into the users or learners environment and activity structures. One technique for activity analysis is the Future Technology Workshop (Vavoula et al. 2002). In these workshops, participants are encouraged to consider the range of, and benefits of, their existing activities before being supported in thinking about how those activities could be more effective when supported by new technologies and services. This allows participants to approach the concept of a new activity structure in a way that has their goals at the forefront of the discussion, rather than subsumed beneath the glamour and glitz of new technology. In addition to this method, an Activity Theory view (see Mwanza 2001) informs our analysis of the environment in which the activities are taking place, other potential collaborators in the activity, and the ways in which 3.
Taylor
169
organisational requirements can impinge on those activities. Through this enriched view of users and their current and future activities, in which learning is viewed as a distributed activity, we can better understand the range of actions and opportunities on offer to mobile learners, and seek ways of extending this range to support what learners want to do even if they themselves do not yet know what that is. This broadening of the scope of the learning system enables a much deeper understanding of users needs, and the constraints that govern their behaviour. From the evaluators point of view, then, the task is to evaluate the effectiveness with which learners are able to achieve their goals, and complete learning activities, irrespective of the specific devices that might have been used in doing so. Indeed, the same or similar activities could be instantiated in a variety of different ways depending on availability of technical support (eg access to wireless Local Area Network, LAN) and user preferences. In so doing, we will necessarily be evaluating the validity of the tasks themselves as vehicles for learning. 4. Conclusion The evaluation framework for the MOBIlearn project is driven both top-down and bottom-up. The theoretical perspectives of
Activity Theory and constructivism, here represented by the socio-cognitive method, allow us to analyse learners in their appropriate contexts and to understand the nature of their learning tasks, and how they go about them. The Future Technology Workshops provide us with much useful data on the views of potential mobile learners and what they see as crucial elements in their learning activities. At the same time, usability studies are, of course, essential. As the MOBIlearn system is being developed, standard usability testing is being performed on component software and devices, in parallel with higher-level evaluations of pedagogic benefit. Figure 1 illustrates the complexity of the task before us. At the bottom levels are subsystems being purpose-built for the MOBIlearn system. These need to be technically verified and tested. There are also existing sub-systems being deployed within the overall architecture which we can assume have already been technically validated. When we have brought all the subsystems up to a common level, we will test the communication protocols between them, both in pairs and all together. At this point, we will have a basic instantiation of the MOBIlearn system.
Figure 1: The evaluation framework for an instantiation of the abstract framework Evaluation
Pedagogic validity
System plus environment
Socio/pedagogic usability Fully operational system
technical verification
technical verification
Usability
Usefulness
Stable sub-system
Technical evaluation and verification

Stable sub-system developmental subsystem Stable sub-system
developmental subsystem
170
But, of course, that is only half the story. We then need to embed that system in an environment that can be used for our learning purposes. At that point, we will begin to engage in the higher-level evaluation involving socio-pedagogic perspectives and pedagogic validity. The important point to remember is captured in Figure 2, which illustrates the flow of evaluation data around the system. Here we can see that the more technical testing, which
might very well involve users, flows information up to the higher levels of the evaluation design. In turn, the more abstract analyses meaning those further distant from the actual implementation issues are flowing data down to inform the design. A key issue for the project in the future will be to ensure that the two levels can meet intelligently in the middle with a mutually informing discourse. We believe that the taskcentred approach will facilitate this marriage.
Figure 2: Flow of evaluation data around the MOBIlearn system
Evaluation
Pedagogic validity
System plus environment
Socio/pedagogic usability
Usefulness Usability
Technical
Data
Taylor
171
5.
References Sharples M (2000). The design of personal mobile technologies for lifelong learning. Computers and Education, 34, 177193. Sharples M, Jeffery N, du Boulay JBH, Teather D, Teather B, & du Boulay, GH (2002) Socio-cognitive engineering: a methodology for the design of human-centred technology. European Journal of Operational Research, 136, 310323. Vavoula GN, Sharples M, and Rudman PD (2002). Developing the 'Future Technology Workshop' method. In MM Bekker, P Markopoulos and M Kersten-Tsikalkina (eds) Proceedings of the International Workshop on Interaction Design and Children (IDC2002). Aug 2829, Eindhoven, Netherlands: Shaker Publishing, 6572. Waycott J, Scanlon E, and Jones A (2002). Using PDAs as learning and workplace tools: an Activity Theory perspective. Proceedings of MLearn 2002, European Workshop on Mobile and Contextual Learning, Educational Research Papers of the University of Birmingham, 14. Birmingham: University of Birmingham, ISSN 1463-9408.
MOBIlearn Technical Annex (2002). Nextgeneration paradigms and interfaces for technology supported learning in a mobile environment exploring the potential of ambient intelligence. IST-2001-37187. Draper S, Brown MI, Henderson FP and McAteer E (1996). Integrative evaluation: an emerging role for classroom studies of CAL. Computers and Education, 26, 13, 1732. Farmer M and Taylor B (2002). A creative learning environment (CLE) for anywhere anytime learning. Proceedings of MLearn 2002, European Workshop on Mobile and Contextual Learning, Educational Research Papers of the University of Birmingham, 14. Birmingham: University of Birmingham, ISSN 1463-9408. Kukulska-Hulme A (2002). Cognitive, ergonomic and affective aspects of PDA use for Llearning. Proceedings of MLearn 2002, European Workshop on Mobile and Contextual Learning, Educational Research Papers of the University of Birmingham, 14. Birmingham: University of Birmingham, ISSN 1463-9408. Mwanza D (2001). Changing tools, changing attitudes: effects of introducing a computer system to promote learning at work. In Pl Dillenbour, A Eurelings and K Hakkiarainen European Perspectives on ComputerSupported Collaborative Learning. Maastricht, Netherlands: Euro-CSCL. Rogers T (2002). Mobile technologies for informal learning: reflections on past research. Proceedings of MLearn 2002, European Workshop on Mobile and Contextual Learning, Educational Research Papers of the University of Birmingham, 14. Birmingham: University of Birmingham, ISSN 1463-9408. Scanlon E, Jones A, Barnard J, Thompson J and Calder J (2000). Evaluating information and communication technologies for learning. Educational Technology & Society, 3(4), ISSN 1436-4522.
Thomas et al.
173
Designing for learning or designing for fun? Setting usability guidelines for mobile educational games
Siobhan Thomas and Gareth Schott Institute of Education University of London 20 Bedford Way London WC1H OAL, UK E-mail: four@nucleus.com; s.thomas@ioe.ac.uk g.schott@ioe.ac.uk Maria Kambouri National Research and Development Centre in Adult Literacy and Numeracy Institute of Education University of London 20 Bedford Way London WC1H OAL, UK E-mail: m.kambouri@ioe.ac.uk
Abstract
While this paper looks at the definitions of heuristics and usability as they apply to digital games, its primary focus is expanding the usability dialogue into the arena of mobile educational games. Recent discussions of game heuristics have made some useful connections between work completed in heuristics and game design theory. This paper expands the usability dialogue, beginning where these discussions have left off, and draws on interviews with educational game developers, game design theory, and game analyses to put forward design principles intended to be useful for the development and evaluation of mobile educational games. Keywords: mobile educational games, mobile learning, games, heuristics, learning principles, usability, learning, education, motivation
Introduction
This paper considers the potential application of industry practice to the development of mobile educational games, and its appropriateness, and identifies guiding principles that can be used in the development of educational games. The development of a set of educational game heuristics, or usability guidelines, is a useful undertaking not only because it creates a guide for the investigation of usability issues, but also because, once identified, game heuristics can help developers avoid usability problems in the first place. Heuristics is grounded in a process of inductive reasoning. Heuristic principles are developed through problem-solving situations are examined,
experiences are drawn on and usable solutions are uncovered through trial and error. Usability is therefore, in a sense, the extent to which heuristics can successfully operate. Heuristic evaluation traditionally, evaluation in which a small team of independent evaluators compare user interfaces with a set of usability guidelines, the heuristics has been recognised as an effective method for the formative evaluation of educational software (Quinn 1996; Albion 1999; Squires and Preece 1999). Heuristic evaluation using six evaluators uncovers 75% of usability problems (Nielsen 1994) and is considered a costeffective method of evaluation that yields reliable results for minimum investment (Quinn 1996). But while heuristics has gained some attention as a useful tool in the educational software arena for examining user interfaces, usability, in general, is still a relatively foreign concept in the game development community (Federoff 2002). Furthermore, a comprehensive list of heuristic principles dealing with the usability of digital educational games, not to mention those available on mobile platforms, is virtually nonexistent. Recent discussions of game heuristics have made some useful connections between Nielsens heuristics and Malones heuristics, and Csikszentmihalyis flow theory, and conventional game development theory and practice (Jarvinen et al. 2002; Federoff 2002). We are interested in contributing to the dialogue surrounding usability and heuristics, primarily from an educational gaming perspective. This paper focuses on providing conceptual solutions to potential design problems and puts forward for discussion principles that will benefit educationalists involved in commissioning, developing or evaluating mobile learning games.
174
Towards an educational gaming strategy There is little consensus among practitioners and researchers on the use of games for learning when the delivery mechanisms are consoles or PCs; consider how much more complicated educational game play becomes when you add in the variable of mobility. For over 20 years educationalists have been discussing the potential for the application of digital games to learning. Yet in this same time a time, which incidentally has seen remarkable advances on the technological side of gaming there has been little progress toward the realisation of digital games learning potential. Although there are an increasing number of positive reports and initiatives within the area of educational gaming technology, instructional designers, academics, teachers and governmental agencies are still embroiled in debates about whether games can, both practically and ethically, be used in education. The overriding sentiment against the use of games to teach seems to be that learning is a serious endeavour that requires serious tools. Even in situations where games are commissioned, design specifications are broad and games are viewed as a homogenous entity, an easy, one size fits all, motivational band-aid (or sticking plaster) perfectly suited to the thumb-twitch generation. The result is that all too often the games developed neither instruct nor engage the learner (Garris et al. 2002) because consideration has not been given to the needs of the learner. If learning is a situated activity, then it follows that it is not possible to prescribe a game typology that suits all learning situations (Squires and McDougall 1996). The difficulty is that while we can conceptualise a good learning game there are few models we can follow in the production of these games. Educational game designers often turn to the commercial game industry for guidance. But while the mass-market model of gaming practised by the gaming industry is immensely valuable, it is not automatically transferable to the learning context in all senses. There are several reasons for this. First, because of the commercial nature of gaming, game developers are concerned with producing material that will be palatable to the mass market. This becomes even more intensified with games for mobile phones, where development cycles are a month or two, instead of the two to three year average development cycle for console games. In a sense the commercial games industry struggles with many of the teething pains that exist for educational gaming. Developers can articulate what makes a good game although not all agree. For instance, Freeman (1997) suggests:
a good game empowers your imagination a good game makes you feel in charge a good game is transparent. You only feel your own mind, the other player and the ideas a good game lets you into its creators imagination a good game lets its players feel each others personality a good game fits the human being like a glove.
But as Federoff (2002) points out, game development companies could benefit from a greater understanding of what makes their games usable. Further, if game development companies are not able to articulate what exactly makes their games fun, how much harder does this task become when they are commissioned to make mobile games that help people learn? What conditions are needed to promote not just user-centred design but learnercentred design? Learner-centred design The vocabulary of software usability centres on effectiveness, efficiency, and satisfaction (ISO 924111) and looks at how easy in general it is to use software, how effectively users can achieve goals and how easy it is for them to learn to do so. Researchers, particularly in the field of humancomputer interaction (HCI), have developed comprehensive sets of usability guidelines to help designers produce better, more usable systems (McGrenere 1996). (For more information on HCI see the usability, user interface design and HCI bibliography at www.humanfactors.com/downloads/ bibliography.asp) But efficiency, effectiveness and user satisfaction dont necessarily add up to a good mobile learning game. To accomplish this we must move away from considerations of user-centred design and look toward learner-centred design (Soloway et al. 1994). This means constructing learning environments that are adaptive, scalable, robust, reflexive and feature modularity, automation and variability (Manovich 2001). Such environments are built by valuing an individuals creative energy; for instance, learners are part of a cotext rather than a context (Akman and Bazzanella 2003; Walz 2002) and the games design takes into account players emissions rather than only considering their immersion (Walz 2002; Csikszentmihalyi 1991). A key problem in the development of educational games is balancing how much of the game is a game and how much of the game is learning (Squire et al. 2003). Developers who become involved in educational games projects must wrestle with how to incorporate learning into games and be able to differentiate between the different types of learning they can incorporate (ie knowledge, skills, curriculum
Thomas et al.
175
content) while retaining the qualities that make games fun. As a result, many learning games, particularly those targeted to a teen or adult audience, lack the qualities that make commercial games so enticing and end up dumbing down educational content: most existing edutainment products combine the entertainment value of a bad lecture with the educational value of a bad game relying on drill and memorization and have graphics and gameplay that fall well below industry standards (Jenkins 2002). It is easy to surmise that these challenges will not simply disappear but, instead, will be compounded when mobility is added to the design mix. Mobility For all intents and purposes, the commercial mobile games industry is overrun by PC and arcade games from days gone by. These games have simply been ported to mobile devices: they are not truly mobile; they are merely portable. The problem is that these games were originally designed to be played on PCs or consoles capable of rendering intensive graphics rather than on mobile devices with limited performance and limited graphic complexity. Further, during development little attention is paid to the devices small screen size, restricted performance, and limited means of input (Liljedal 2002). Mobile games have evolved considerably during their 20-year existence. The first contender in the mobile gaming arena was Nintendos Game & Watch handheld and today commercial devices used for mobile gaming include mobile phones, personal digital assistants (PDAs), handheld computers, smart phones, game-specific devices such as Nintendos Game Boy Advance SP, and hybrid devices such as Nokias N-gage, TTPComs bngo, MyOrigos mydevice, and Tapwaves Palm OS Zodiac gaming device. Future offerings include Sonys PlayStation Portable (PSP), which, if publicity is to be believed, will revolutionise the handheld industry with specifications that Sony Computer Entertain-ment Europe president Chris Deering claims are like the PlayStation 2 minus a bit. These new device specifications should encourage designers and developers to look beyond the current practice of designing for stand-alone gaming experiences. The first step in this process is to consider connectivity, to simply think about connecting learners either to each other or to a central server. Most handheld devices permit connectivity between devices through a USB port or wirelessly via Bluetooth, WiFi (wireless fidelity), or infra-red. Further, network connectivity means that learners are no longer isolated; they can still play games as individuals but they can also benefit from all the functionality that being connected to the internet or a back-end database can offer, such as customised learning and increased capabilities for adaptive learning.
Mobile learning, not portable learning Portable learning experiences offer advantages in price and accessibility (Klopfer and Squire 2003) it is more cost effective, for instance, to buy handhelds than to kit out every student in a classroom with a laptop or PC. However, they dont make use of what mobile devices can really offer, namely, connectivity, location sensitivity and context awareness; nor do they allow for ubiquity, which would make the real-world environment an intrinsic and meaningful game element (Bjork et al. 2002). Although connectivity opens the door to limitless possibilities for interaction, it is valuable to remember it is only one piece of the mobile learning equation. Mobile games are played while learners are on the move and such things as the players direction, speed, location, or proximity to objects in the physical world (Liljedal 2002) can be incorporated into gameplay. Incorporating this type of functionality also raises a variety of questions that require answers. Who and what will the player interact with? At what proximity? For how long? Commercial games, such as Nokia game, Picofun, Botfighters and Blue factory, make use of location, position (both ordinary and relative), and movement between locations, but they dont take into account the environment that the player is playing in. Context awareness can be an integral part of the mobile gaming experience and includes factors such as speed, direction, timing, changing surroundings, acceleration, manipulation of objects, and issues such as dealing with multiple entries and exits, and no on or off switch (Liljedal 2002). In a context-aware gaming situation consideration is not only given to the players relationship with the objects that he or she interacts with but to the players progress while playing and to cooperation between players (Liljedal 2002). The reasons for not incorporating context into games are obvious, particularly on the commercial front: context-aware games are time-consuming and expensive to make, there are few models to follow, and commercial operators and publishers are unwilling to take risks. But the introduction of true mobility (versus portability) offers unparalleled learning opportunities, marrying the benefits of gaming with some of the value that comes out of the classroom reflection, mediation, collaboration and opportunities for enhancing learning transfer. General approach: exogenous play One of the practical challenges that developers face when making learning games is how to fuse gameplay and learning seamlessly, as in the case of Riebers exogenous play, play which is not removed from a learning experience, but inherent to it (Squire et al. 2003). Edutainment games typically have two obvious components: gaming and learning. The player can see where the learning begins and can easily separate this from the gameplay. A seamless
176
learning experience, in contrast, is much more difficult to construct. The gameplay cannot be separated from the learning content. The games structure, interface and so on, is the instructional content. The player is immersed in a learning experience because the entirety of the game is learning. Such experiences are beneficial because learning is tied to intrinsic rather than extrinsic motivation the player wants to complete the game for feelings of personal satisfaction rather for an external reward (Deci 1972; Malone 1981). There are few learning games that have achieved Riebers holy grail of exogenous play. However, there are groups, in particular MITs Games-To-Teach project, that are working toward this (see http://cms.mit.edu/games/ education/proto.html for examples of prototypes). One area of gaming that has had reasonable success marrying games and learning is simulation gaming. Crookall et al. (1987) argue that a simulation is a representation of some real-world system that can also take on some aspects of reality for participants or users. Simulations are characterised by reality of function: if a learner adopts the role of a chairman she really is a chairman with all the power, authority, and duties to complete the task (Jones 1984). One of the key benefits of using simulations for learning is that in simulations learners see the consequences of decisions and actions they make without experiencing the real-world consequences of their mistakes. Games, in contrast, do not intend to represent any real-world systems; they are systems in their own right. Squire et al. point out that structurally games differ from simulations in that games (usually) have an additional narrative back story and context, one or more challenges, and various failure and win states and players immerse themselves within games and their more immediate participation expands the opportunities for mastering the content (Squire et al. 2003). However, simulations can take on game features. For instance, Garris et al. developed a game-based submarine periscope trainer for the US Navy and found that it provided more effective training than a training simulation without any game characteristics (Garris et al. 2002). Commercial game simulations, from Life and death to Rollercoaster tycoon, have educational value because, as Prensky points out in Digital game-based learning (2001), to keep most learners engagement you have to keep making it fun fun from the players, not the creators, perspective. Perhaps counter-intuitively, having an extremely highfidelity simulation that exactly imitates life can sometimes take the fun out of it. So can not giving the player enough choices, or enough humorous or even outrageous possibilities.
Principles of mobile game learning

To understand better what elements contribute to effective mobile game learning we reviewed literature related to games and learning, interviewed key figures involved in the development of learning games, conducted critical game-analyses, and looked at good practice within the commercial game industry, validating its relevance and increasing its scope to address mobile game learning. It is not our intention in this paper to put forward a comprehensive list of game learning principles (for such a list see the work completed at www.pervasivelearning.org), but rather to present a few categories of the principles that our research has identified as particularly relevant to the development of mobile learning games. 1. Adaptation One important characteristic of games is that they can adapt. If games are adaptive they support learner preferences for different access pathways and allow the learner to find relevant information while at the same time remaining immersed in the game (Quinn 1996). In adaptive games the level of difficulty increases or decreases depending on a players performance. The game intervenes when a player is in trouble. For instance, in Byzantine: the betrayal a helper character steps in to assist the player in tackling difficult situations if help is needed. In Tech deck skateboarding the game adapts to the player by adjusting the amount of time permitted for completing events as the player advances to higher levels. Poccer, a Pocket PC football game, adapts by speeding up or slowing down the computer opponent depending on the players score. However, adaptability is more than simply increasing or decreasing a games difficulty level. The effectiveness of mobile learning games can be improved by the introduction of an even greater level of adaptability to players actions, particularly if the game explicitly monitors students interactions and learning patterns, and intervenes when constructive reasoning and reflection need to be triggered (Conati and Klawe 2000). Adaptiveness, therefore, is not only a structural response to a learners actions but plays a crucial role in mediating the learning experience. Learning outcomes can be associated with key behavioural indicators. These indicators will, in turn, cause the game to adapt if the player is having difficulty or needs greater challenge (Pagulayan et al. 2002). If the indicators do not suggest the player is experiencing difficulty, the game does not adapt and continues on its course. An extension of this type of adaptive mediation is to generate the game completely on the fly, tailoring learning to the user. In this type of adaptive learning situation, the cognition activities that users have to perform, the difficulty of the problems behind the game, the sceneries presented and the organization of these elements, among others, can be
Thomas et al.
177
dynamically selected or generated for each particular user depending on his/her personal features and behaviors (Carro et al. 2002). Challenge and mastery Learners with varying skill levels play games. There are a number of different strategies for designing games to accommodate this variability while still keeping players challenged. Ryan (1999) advocates designing a game for players with a median skill level, determined through an iterative design process which features play testing both extremes (skilled and unskilled players) and using the results to identify where the game needs to be made easier or harder. Laramee identifies three types of difficulty curves for games: the flat curve, the linear progression and the s-curve (Laramee 2002). In the flat curve the games difficulty does not change and traditional game levels are replaced with a series of activities or tasks with an equal difficulty level; this is the typical approach in edutainment games that are targeted to a very narrow age group with specific knowledge and ability expectations (Laramee 2002). In the linear progression type of difficulty, challenge increases steadily throughout the game. While easy to implement, this model is dangerous because it may not give the player the necessary amount of time to learn the game before progressing to a harder level and, as a result, the player may never be able to complete the game. The s-curve starts slowly and lets the player learn how to play the game, sometimes through a tutorial, at other times through training levels. The difficulty level gets steeper during the bulk of the game and in the last two to five hours flattens out, allowing players who make it through most of the game to survive through to the finish (Laramee 2002). Games motivate when they challenge players and, at the same time, maintain the illusion of winnability (Nawrocki and Winner 1983; Crawford 1982). Games should not offer one single way of winning (Crawford 1982; Malone 1982; Shelley 2001), because if a game is winnable it will lose its appeal (Crawford 1982). Players are challenged and strive to improve when complexity increases (Squire et al. 2003). In the simplest of terms, games have to get harder to keep a players interest. The first time a player sets foot in a Diablo dungeon, a skeleton is a powerful foe. However, it does not remain so for very long; if the player had to keep hacking away at basic skeletons throughout the game, boredom would soon set in (Laramee 2002). Challenges (and challengers) should be introduced slowly, often in isolation, so that players get the opportunity to study their behaviour (Laramee 2002). It takes trial and error for players to find the best ways to defeat a games challenges, but players face each progressive challenge with the knowledge learned from challenges already accepted. 2.
Bushnell points out that a good game should be easy to learn and hard to master (in Federoff 2002). This ties in nicely with the idea of replayability: games are intended to be played over and over again. Players engage in cycles of gameplay repeated judgement-behaviour-feedback loops that see users making decisions based on scenarios put forward in the game, acting on those decisions, and getting feedback based on what theyve done (Garris et al. 2002). Users enter the gameplay cycle to beat the challenges presented to them. The point of the game, what keeps the boys playing, is the promise the intimation that with enough energy, enough focus, and enough lives, he might master this machine (Weinbren 1995). Goals A basic rule of instructional design put forward by Gagne in 1965 seems like commonsense but is still extremely relevant today: inform learners of objectives. Gagne suggested that when learners were made aware of objectives they had an expectation for learning. A quite similar fundamental rule exists for game design: Games should provide enticing long-term goals (Barwood and Falstein 2003). Further, these goals need to be presented early (Clanton 1998; Malone 1982), need to be clearly stated, and should be personally meaningful, obvious, and easily generated (Malone 1981). Designers need to tune the message to the content (Crawford 2003); in other words, game goals and learning goals need to be one and the same. But while games should have one clear overriding goal for instance, rescue the Princess Zelda they also need to have clear short-term goals: subgames that build towards the overall goal (Barwood and Falstein 2003).. Short-term objectives guide players through the game (instead of being faced with the insurmountable task of saving the world they find out that first they need to meet the oracle). They help players avoid the frustration of uncertainty and reassure players that they are making progress (Barwood and Falstein 2003). Short-term goals can be explicit or implicit: players can be told directly (for instance, in the Legend of Zelda: link to the past you meet characters who tell you what your game tasks are, eg you need to climb to the tower to find the Moon Pearl) or given cues by the environments they are exploring (for instance, in Halo the landscape itself and suggestions from ingame companions push players toward the next short-term goal (Barwood and Falstein 2003). Along the way the game should also provide performance feedback on how close the user is to achieving the goal (Malone 1981). 3.
178
4.
Community and collaboration One of the advantages of using mobile devices for gaming is the opportunities they offer for community and collaboration. Cooperative learning environments situations where learners work in collaboration to achieve learning goals and receive rewards or recognition based on their groups performance (Slavin 1980) have been found to foster positive interdependence among learners, which translates into positive interpersonal relations and attitudes (McGrenere 1996). Mobile multiplayer learning games require positive goal interdependence (Cohen 1994), positive resource interdependence (Cohen, 1994), positive reward interdependence (Cohen 1994), group evaluation opportunities, and individual accountability (Hymel et al. 1993). Collaboration, therefore, does not disregard the value of individualist learning structures Yeuh and Alessi (1988) maintained that a combination of group and individual rewards produced higher achievement and increased peertutoring. Joint discovery and exploration is a valuable activity that allows one players understanding of a game, and the inherent learning within it, to shape another players understanding (Liljedal 2002). In her review of educational electronic multiplayer games McGrenere (1996) expanded on Grudins paradox of collaboration, pointing out we interact with other people continually and usually rather effortlessly, but designing computer support for collaboration is very difficult because we have to actually understand how groups and organizations function. Collaborative activities fail because designers dont understand the fundamentals of group behaviour. Interestingly, inroads have been made on the collaborative front in commercial roleplaying games which recognise that the story comes not from the game, but from the struggle of multiplayer opponents and collaborators (Levine 2001). Massively multiplayer online roleplaying games (MMORPGs) such as Everquest, Asherons call, and Planetside offer the quintessential collaborative experience, featuring thousands of players who go head to head in worlds that are available 24 hours a day. As Oliver points out: Within the roleplay of the MMORPG, character development is better termed capacity development; the player is wilfully locked into a system of performance centred around growth which sees collaboration driven by proximity: players meet in safe public spaces such as lobbies or town squares and organise teams for mission (Oliver 2002). These scenarios also tackle central issues of collaborative learning activity coordination problems; within-group communication problems; the difficulty of properly organising individual work with joint group activities; negotiation problems; lack of group synchronization; lack of interaction with other group members (Zurita et al. 2003). Interesting too is how coordination is achieved in these vast
multiplayer expanses after all players could roam randomly without needing to make contact with other players for days. Designers know well how to force players to collaborate. First, they make the spaces for collaboration safe, as opposed to the other parts of the game which are inordinately scary (Oliver 2002). Second, they use a design technique which Squire et al. (2003) point out goes back to Dungeons and dragons (or farther): they give players a motivation to play together. Players must collaborate precisely because they need what other players offer. One of the core game design mechanisms for encouraging collaboration is the notion of differentiating between different players roles, so that players must collaborate to succeed in a world (Squire et al. 2003). Another important consideration in the development of collaborative game structures is that learning is based on frequency of task-related interaction. Given a problem with no right answer and a learning task that will require all students to exchange resources, achievement gains will depend on how often players engage in interactions related to the task (Cohen 1994). Also important is ensuring that players understand the game goals and that the learning objectives are inherent in the goals. Players must know the point of the mission. If they know they have to take over an enemy prison they can adopt appropriate roles and develop strategies: if learning is for understanding and involves higher order thinking, then tasks and instructions which foster maximum interaction, mutual exchange, and elaborated discussions will be more beneficial than tasks and instructions which constrain and routinize interaction (McGrenere 1996). 5. Context Squires (1997) points out that both components of a learning environment people and artefacts interact and contribute to the learning process. This environment, including all the implicit situational information that learners use to communicate, is considered context (Dey and Abowd 2000). Most games make little use of context; they do not incorporate the learners environment into the gameplay experience. However, when learners have increased freedom of mobility, in situations where the users contexts, such as location and the people and objects around them, are more dynamic (Dey and Abowd 2000), games need to consider context (see table below) and adapt appropriately.
Thomas et al.
179
Considerations for context-aware mobile games Presence Who is playing the game? Location Activity level Actions Intentions Changes Objects Extents Abilities Sphere of influence Expectations Where are learners playing the game? How active are they? What are they doing? What will they do next? Where will they be? What changes are they making, and where? What objects are they using? What objects could they use? What can they see? How far can they reach? What can they do? Where can players make changes and how? What do players want the game to be able to do? Adapted from Cutwin and Greenberg 1996
Context awareness also involves more than accounting for a players immediate context. Contextaware multiplayer games need to take into account the context of all players playing the game. They need to go beyond thinking about being there and consider beyond being there so that players at a distance are not at a disadvantage and the game offers more than just playing face-to-face (Hollan and Stornetta 1992).
1994; McDougall and Squires 1986). While Malone and Lepper (1987) suggest that it is the perception of control rather than actual control that is most important, perception of control is affected by how responsive the game is to player choices and, in fact, how many choices are on offer to the player. There is a sense that this ideal of perception was driven in part by the technological deficiencies that existed in the late 1980s and that it has continued to be the ideal expressed over the last 20 years because the possibility of players really controlling the games they play could not be practically realised. Mobile learning games have the potential to offer players agency rather than the trickery and perception of control. Situated learning experiences that give players the opportunity not only to write the content of their own stories, but to create structurally the games they play are only a mobile device away. Before this is achieved and embraced by the learning community, however, frameworks for the design and delivery of meaningful pedagogical materials need to be carefully researched, tested and evaluated. The principles outlined in this paper represent an important first step towards this, particularly in the area of mobile learning.
References
Akman V and Bazzanella C (2003). The complexity of context. Guest editors' introduction. Journal of Pragmatics. 35, 321329. Albion PR (1999). Heuristic evaluation of educational multimedia: from theory to practice. Australasian Society for Computers in Learning in Tertiary Education, Brisbane. Barwood H and Falstein N (2003). The 400 project. At www.theinspiracy.com/400_so_far.htm, accessed 30 June 2003. Bjrk S, Holopainen J, Ljungstrand P and kesson K-P (2002). Designing ubiquitous computing games: a report from a workshop exploring ubiquitous computing entertainment. Personal and Ubiquitous Computing, 6, 443458. Blease D (1988). Evaluating educational software. London: Croom Helm. Bushnell, N (1996). Relationships between fun and the computer business. Communications of the ACM, 39(8). ACM Press. Carro RM, Breda AM, Castillo G and Bajuelos AL (2002). A methodology for developing adaptive educational-game environments. In P De Bra, P Brusilovsky and R Conejo (eds) Adaptive hypermedia and adaptive web-based systems. Lecture notes in computer science 2347. Berlin: Springer-Verlag, 9099.
Discussion
While the principles presented here provide a conceptual overview of what could become good practice in relation to the development and evaluation of mobile learning games, only a handful of principles have been illustrated. Feedback, consistency and competition are other key game characteristics that we have found integral to the game development process. The aim of this paper has been to highlight the need for comprehensive research and evaluation of usability principles and initiate a dialogue in which usability frameworks can be modified, enhanced and validated by the mobile learning community. There is value too in identifying how learning concepts such as reflection, scaffolding, mediation and debriefing can be addressed. One consideration that should not be overlooked when designing mobile learning games is the educational potential offered by including the player in the game creation process. Instead of the game programming the player, the player should program the game (Papert 1993). The extent to which learners can develop a sense of ownership in the game environment is closely linked to the level of control they have in their interactions (Blease 1988; Chandler 1984; Goforth
180
Chandler D (1984). Young learners and the microcomputer. Milton Keynes: Open University Press. Clanton C (1998). An interpreted demonstration of computer game design. Proceedings of the conference on CHI 98 summary: human factors in computing systems. Chi 98, 12. Cohen EG (1994). Restructuring the classroom: conditions for productive small groups. Review of Educational Research, 64, 135. Conati C. and Klawe M (2000). Socially intelligent agents to improve the effectiveness of educational games. Proceedings of the 2000 AAAI Fall Symposium Socially Intelligent Agents: the Human in the Loop, North Falmouth, USA: AAAI Press. Crawford, C (1982). The art of computer game design. At www.vancouver.wsu.edu/fac/peabody/gamebook/Coverpage.html>, accessed 12 December 2003. Crawford C (2003). Foresight and Governance Project (2003). Subjectivity and simulation. At www.seriousgames.org/reading.html, accessed 29 November 2003. Crookall D, Oxford RL and Saunders D (1987). Towards a reconceptualization of simulation: from representation to reality. Simulation/Games for Learning, 17, 147171. Csikszentmihalyi M (1991). Flow: the psychology of optimal experience. New York: Harper Perennial. Cutwin C and Greenberg S (1996). Workspace awareness for groupware. CHI96 Conference Companion. Vancouver, Canada, April 1318, 208209. Deci EL and Ryan RM (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum. Dey AK and Abowd GD (2000). Towards a better understanding of context and context-awareness. CHI 2000 Workshop on the What, Who, Where, When, and How of Context Awareness. The Hague, Netherlands. Federoff MA (2002). Heuristics and usability guidelines for the creation and evaluation of fun in video games. Indiana: Department of Telecommunications Indiana University. Freeman T (1997). Monitoring devices in games. Game Developer. AprilMay, 2231. Gagne R (1965). The conditions of learning. New York: Holt, Rinehart and Winston. Garris R, Ahlers R and Driskell JE (2002). Games, motivation, and learning: a research and practice model. Simulation & Gaming. 33, 441467.
Goforth D (1994). Learner control = decision making+information: a model and meta-analysis. Journal of Educational Computing Research. 11, 126. Hollan J and Stornetta S (1992). Beyond being there. CHI92 Conference Proceedings, Monterey, California. Hymel S, Zinck B and Ditner E (1993). Cooperation versus competition in the classroom. Exceptionality Education Canada, 3, 103128. Jarvinen A, Helio S and Mayra F (2002). Communication and community in digital entertainment services, Tampere: University of Tampere Hypermedia Laboratory. Jenkins H (2002). The technology review. Game Theory. At www.technologyreview.com/articles/wo_jenkins0 32902.asp, accessed 22 June 2003. Jones K (1984). Simulations vs. professional educators. In D Jacques and E Tipper (eds) Learning for the future with games and simulations. Loughborough: SAGSET/Loughborough University of Technology, 4550. Klopfer E and Squire K (2003). Augmented reality simulations on handheld computers. Paper delivered at the annual meeting of the American Educational Research Association, Chicago, IL, April 2003. Laramee FD (2002). Elements of level design. In Game design perspectives. Hingham, Massachusetts: Charles River Media Inc. Levine K (2001). New opportunities for Storytelling. Paper presented at the Electronic Entertainment Exposition, Los Angeles, CA. Liljedal A (2002). Design implications for context aware mobile games. Uppsala, Sweden: Department of Information Science, Computer Science Division, Uppsala University, 57. Malone TW (1981). Toward a theory in intrinsically motivating instruction. Cognitive Science, 4, 333369. Malone TW (1982). Heuristics for designing enjoyable user interfaces: lessons from computer games. In JC Thomas and ML Schneider (eds) Human factors in computing systems, Norwood, NJ: Ablex Publishing Corporation. Malone T, Lepper, M (1987). Making learning fun: a taxonomy of intrinsic motivation for learning. In RE Snow and MF Farr (eds), Aptitude, learning, and instruction. Vol 1: Cognitive and affective process analysis. Hillsdale, NJ: McGraw-Hill. Manovich L (2001). The language of new media. Cambridge, MA and London: The MIT Press.
Thomas et al.
181
McDougall A and Squires D (1986). Student control in computer-based learning environments. In AD Salvas and C Dowling (eds) Computers in education: on the crest of a wave. Melbourne: Computer Education Group of Victoria, 26972. McGrenere JL (1996). Design: educational electronic multi-player games: a literature review. Vancouver BC: Department of Computer Science, University of British Columbia. Nawrocki LH, Winner JL (1983). Video games: Instructional potential and classification. Journal of Computer-Based Instruction, 10, 8082. Nielsen J (1994). Heuristic evaluation. In J Nielsen and RL Mack (eds) Usability inspection methods. New York: John Wiley & Sons Inc. Oliver J.H (2002). The similar eye: proxy life and public space in the MMORPG. F Mayra (ed), CGDC Conference Proceedings, Tampere, Finland: Tampere University Press. Pagulayan RJ, Keeker K, Wixon D, Romero R. and Fuller T (2002). User-centered design in games. In J Jacko and A Sears (eds) Handbook for humancomputer interaction in interactive systems. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Papert S (1993). Mindstorms: children, computers and powerful ideas. New York: Basic Books. Prensky M (2001). Digital game-based learning. New York: McGraw-Hill. Quinn CN (1996). Pragmatic evaluation: lessons from usability. 13th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, Adelaide: Australasian Society for Computers in Learning in Tertiary Education. Ryan T (1999). Beginning level design part 2: rules to design by and parting advice. Gamasutra. At www.gamasutra.com/features/19990416/level_de sign_01.htm, accessed 29 November 2003. Shelley B. (2001). Guidelines for developing successful games. Gamasutra. At www.gamasutra.com/features/20010815/shelley_ 01.htm, accessed 20 December 2003. Soloway E, Guzdial M and Hay KE (1994). Learnercentered design: the challenge for HCI in the 21st century. Interactions. April, 3648. Squire K, Jenkins H, and the Games-To-Teach Team (2003). Designing educational games: design principles from the Games-to-Teach project. Educational Technology. SeptemberOctober. Squires D, McDougall A. (1996). Software evaluation: a situated approach. Journal of Computer Assisted Learning, 12, 141161. Squires D and Preece J (1999). Predicting quality in educational software: evaluating for learning, usability and the synergy between them. Interacting with Computers. 11.
Walz SP (2002). On the move between proximate distance and distant proximity. Master's thesis published by the Dept of Cultural Anthropology, University of Tuebingen, Germany and commissioned by the Dept of Mobile Application Design (HGKZ-MAD), University for Art Media and Design, Zurich, Switzerland,. At www.madcountdown.de, accessed 25 September 2003. Weinbren G (1995). Mastery: computer games, intuitive interfaces, and interactive multimedia. Leonardo, 28, 403408. Yueh J and Alessi SM (1988). The effect of reward structure and group ability composition on cooperative computer-assisted instruction. Journal of Computer-Based Instruction, 15(1),1822. Zurita G, Nussbaum M and Sharples M (2003). Encouraging face-to-face collaborative learning through the use of handheld computers in the classroom. Udine, Italy: Mobile HCI 2003.
Traxler
183
Mobile learning evaluating the effectiveness and the cost

John Traxler National ICT Research Centre Priorslee Campus Telford Shropshire TF2 9NT E-mail: John.Traxler@wlv.ac.uk
Abstract
The EU m-learning project coordinated by LSDA is developing a mobile system for adult basic skills in which learners access content and discussion transparently across a range of mobile and computing devices. For any commercial exploitation, it is necessary to explore and assess the factors that determine the various developmental efforts and their respective educational benefits. This paper suggests a way to integrate and apply work on multimedia educational software cost-estimation, cost models of networked learning, the Laurillard conversational framework and blended learning development tools. These factors must underpin any objective economic evaluation of mobile learning and establish the foundations for understanding the basis of commercial exploitation. Keywords: costbenefit, software costestimation, conversational framework, mediamix
1. Introduction
Mobile learning is currently at a stage of small-scale projects working to establish aspects of technical feasibility in specific educational subjects and settings. The EU m-learning 1 project, coordinated by the Learning and Skills 2 Development Agency, and the EU MOBIlearn project are two exceptions to this European
generalisation. In North America for example 3 the University of South Dakota and perhaps the Far East for example at the Kinjo Gakuin 4 University in Japan mobile learning, or at least learning with mobile devices, is breaking through to some visibility at an institutional level. If these projects are educationally and technically successful and act as the focus for a consensus of what in practice constitutes mobile learning, the next phase must be largescale social or institutional use and commercial exploitation. This may not necessarily take the form of direct free-market commercialisation but instead may involve publicly funded support within government initiatives and ongoing provision. Whichever is the case, exploitation will only take place on a sustainable basis if there is an understanding of the relations between the costs of mobile learning, in all their different forms, and the educational, and perhaps social, benefits. Hence, commercial exploitation is a question of educational costbenefit analysis and in particular that part of educational costbenefit analysis that deals with large-scale or industrialised educational systems explored, for example, by Rumble (1997) and Peters (1998). Subsequent work on costs has looked at specific settings, for example information and communications technology (ICT) (Nicol and Coen 2003) and networked learning (Bacsich et al. 1999) within the confines of conventional universities. Other work has looked at cost-effectiveness with computer-aided learning (CAL) materials (Hunt
184
and Clarke 1997). All this work is characterised by differing definitions of the system boundary and by differing attention to the various portions of the lifecycle from conception via delivery and operation to revision and maintenance. There is also the recurrent difficulty of identifying and setting tangible costs against intangible educational benefits. This paper looks at mobile learning as a system composed of systemspecific content and human teaching and support. Moving the system boundary, for example to situate mobile learning within an institutional setting, would create a different analysis. However, mobile learning is a technical system with significant computing components and much can learned by looking at the literature of software engineering, especially software cost-estimation and project management, summarised in Pressman (2000) and Sommerville (1992). Implicit in the following analysis is the proposition that the cost of developing and deploying mobile learning systems can be broken down into: content development costs teaching costs software development costs hardware costs usage costs, eg phone charges. This assumes that a mobile learning system is a reusable generic shell and that new content can be developed and delivered as and when needed. The declining real costs of massproduced hardware such as PDAs and mobile phones, the artificiality of phone tariffs and the economies of scale associated with commercial exploitation will mean that it is the first two elements that will be the decisive determinants of mobile learning costs.
2. Software cost estimation

For commercial and industrial software developers, there has always been a considerable economic advantage in being able to predict and control the effort and thus the cost of software development. This opening section looks at methods for predicting the effort and hence cost of developing programs. Over the last three decades, a variety of predictive methods have been devised and tested, based on a variety of assumptions and principles. One of the most effective has been Constructive Cost Modelling (COCOMO), devised by Barry Boehm (1981). This looks at a program specification a description of what the proposed piece of software should actually do and analyses it to generate some indicator or metric of program size. This metric might be the
anticipated eventual size of the program (KLOC kilo lines-of-code) or the extent of its functionality (FP function points). From these it is possible to calculate effort, as personmonths, and duration, as months, and hence the size of the development team, as persons. Unfortunately, COCOMO in its basic form requires some subjective judgements about the nature of the development team and about the nature of the proposed software and so the method is less analytic and mechanical than it at first seems. In this basic form, it is also less exact than would be desirable. Subsequent versions of COCOMO developed by Boehm (1996, 2000) attempted to add sensitivity by introducing the idea of cost-drivers. These were environmental and technical factors, attributes of the project such as its tools and methods, attributes of the personnel such as their capability and experience, attributes of the target hardware such as its performance and memory and attributes of the product such as reliability and complexity. These were all scored for a proposed development and put into the appropriate equations to give the effort, duration and staffing of a project and hence its cost. The argument being made in this section is implicitly that the appropriate analogy for the development of educational software, for example mobile learning material, is that of the mainstream software engineering project. This may not be the case. There are other possibilities, for example the development of mass-market retail software described by Cusamano and Selby (1996). If educational software development is tied to a specific development methodology, for example the Dynamic Systems Development Method (DSDM) (Boyle 1997) or structured methods (Stoner 1996), then perhaps more accurate but less transferable predictions are possible.
3. Educational effort estimation

This section looks at an application of software cost-estimation techniques to educational software and identifies the underlying characteristics of multimedia educational packages that drive educational software development costs upwards or downwards. This is the work of Dr Ian Marshall and his collaborators at the University of Abertay (1995a, 1995b). These characteristics are fundamental to different conceptions of teaching and learning, and are shared by many components of mobile learning systems. Marshalls work used COCOMO on 14 projects in the 1990s, mainly with a nominal
Traxler
185
learner time of an hour. It tested four possible cost-drivers: course difficulty development expertise subject expertise interactivity. These were further sub-divided into 24 subheadings. His results showed the significant cost-drivers to be: development environment: o instructional design method o size of developer team o powerful development tools o methodology course difficulty: o number of objectives o level of objectives o existing course material. Interactivity was not strongly indicated in the results. The implications, for example, for the mlearning project and its development costs are clear. Costs are driven down for packages with fewer objectives, for objectives that are at a lower level or based on existing material developed using powerful development tools with superior design methods.
Laurillard (1993b) gives this explanation of the Conversational Framework background: the characterisation of the teachinglearning process as a conversation is hardly new. Gordon Pask formalised it as 'conversation theory' some time ago (Pask 1976), including the separation of 'descriptions' and 'model-building behaviours', and the definition of understanding as determined by a two level of agreement. Vygotsky described learning in terms of social interaction (Vygotsky 1962). The framework was used in examining educational television (Laurillard 1993b) and computer-aided learning (Laurillard 1987). It is most often used to explore the adequacy and coverage of strategies (Britain and Liber 1999) in proposed HE courses or classes and is sometimes extended with a representation of learner-to-learner exchanges supporting social learning. Print, lectures and web pages appear as an unidirectional broadcast from the conceptual world of the teacher to the conceptual world of the learner; seminars as a bi-directional exchange between the same worlds. In fieldwork and simulations, there is bi-directional exchange between the two worlds of practice or action, and perhaps reflection will then inform the learners world of conception. Virtual learning environments (VLEs) and now mobile learning offer greater diversity and richness of exchanges since they embrace a range of constituent technologies. It is instructive to evaluate the coverage of the proposed mobile learning system against the full Laurillard framework. Laurillard herself (Laurillard 2002) says that the framework has not yet been used outside British higher education and so an extension into adult basic skills would be an exciting innovation. Nevertheless, the framework provides a basis for widening the work of Marshall (who in Laurillards terms only tackles didactic or broadcast exchanges or conversations) since in theory one could look for the costs and then the cost-drivers associated with each form of exchange. An analysis along these lines would undoubtedly be helpful but not complete since it would ignore some major issues including: longevity of content and material individual learning styles economies of scale relationships between creation, maintenance and delivery subject matter o suitability o volatility, changeability.
4. Analysing mobile learning the Conversational Framework

The previous section identified potential costdrivers for the didactic components of mobile learning systems, or rather, their discrete didactic software components. It was, however, an incomplete account of mobile learning economics because it failed to address other components of such systems and their probable synergy. It also failed to address the educational effectiveness or efficiency of the various components. This section will look at a more general framework developed by Laurillard (1993a) that allows us to categorise and understand the activities within mobile learning. Laurillards analysis of learning draws on constructivism and on the view that teaching and learning is based around conversations or exchanges between two different types of world, the discursive and interior mental world of descriptions and conceptions and the interactive and exterior physical world of action. For any given individual these are linked by reflection and adaptation. These worlds can also be divided into those of teachers and those of learners. Their interior worlds communicate by articulation and re-articulation as concepts are defined and refined, while their exterior worlds communicate by action and feedback as learners learn experientially from environments created by teachers.
186
A detailed examination of any commercial exploitation of mobile learning would have to address these factors. Much of the content might seek to use topicality to capture learners interest. In purely economic terms, this may be a costly strategy but a more thorough analysis would need to balance the short lifetime of topical content with its enhanced effectiveness with learners. Similarly, dealing with the multiplicity of anticipated learning styles would be very expensive since it would entail the techniques of artificial intelligence (AI) but more pragmatic and procedural solutions might prove cost-effective, especially if the delicacy of myriad learning styles were in any case masked by the constraints and peculiarities of mobile learning technologies and interfaces.
5. Using the Conversational Framework Media Adviser and CRAM

The previous section provided a theoretical model for discussing the costs and choices involved in developing a mobile learning system. This section looks at two current attempts to combine Laurillards conversational framework with data relating to a range of developmental activities. They provide examples of how an economic analysis of mobile learning might proceed. This is a significant step towards a transparent procedure for optimising the elements used within a teaching system and a considerable improvement on the pragmatic media-mix solutions of Reisner and Gagne (1983) which lacked flexibility or any obvious theoretical basis. One of these attempts is institutional, the other individual, neither is ever likely to reach maturity or widespread use but nevertheless both show how the basic principles can be carried forward and possibly applied to mobile learning. The Course Resource Appraisal Model (CRAM) was developed by the Open University (OU) for internal resource management. It is implemented as an Excel spreadsheet and assists course development teams in exploring the resource implications, early on, of their various learning technology options. Their system starts from the required overall study time, usually the OU norms for 15-credit and 30credit courses. It uses historical institutional OU figures for academic and production activity (graphic design, editing, etc) dating back to the mid-1990s which were never updated. The multimedia development times, for example, were based on data from developing a first-year science course. The intended outputs are the person-hours across the various skills and tasks
for a range of technology options. The parameters are manually adjustable. The CRAM tool is intended as guidance for individuals leading multidisciplinary production teams including designers, developers, authors and others. As such, it deals only with average figures for cost and productivity. Interestingly, it does attempt to allow for the increasing cost associated with the complexity of using larger numbers of technology options. The drawbacks of the CRAM tool are that it does not deal with any issues of economies of scale, nor include longevity of material or the educational efficiency of specific media as factors in the cost-benefit equation. There are probably problems unpacking the CRAM data for computer-mediated conferencing (CMC) because it deals with several different teaching formats, especially as the OU is tied closely to one specific commercial system, and with VLEs for similar reasons. The tool has not been used widely within the OU, partly because of user resistance (it is perceived as a management tool), and it is unlikely it will receive much further investment, development or promotion. The Media Adviser tool (Conole and Oliver 1999) developed by Grainne Conole, Martin Oliver and others at the University of North London as part of the Learning and Teaching Innovation and Development (LATiD) project simplifies Laurillards conversations down to: delivery discussion activity feedback. It covers a range of contemporary media that users can customise, adding their own. For each medium, the Media Selector section of the tool provides default data about: search and evaluation time (in hours) familiarisation and preparation time cost (pounds sterling) delivery resources (hardware, software, etc) and so on for existing materials or media. For new materials, it provides for: development time (in hours) development resources (hardware, software, etc). This provides a simple database of learning technologies. The Media Rater section allows users to rate media in terms of their support, that is their suitability and effectiveness, for each of the four types of conversation in the interests of creating a balanced course. Then the Course Modeller section estimates development costs, both effort and expenditure,
Traxler
187
based on the users preferred balance and ratings. This simple tool is intended as the focus for staff development and reflective practice in higher education. Perhaps it should not be taken too seriously as an objective analytic tool. Nevertheless, it may express some useful principles and procedures that could underpin the development of tools to predict mobile learning economics. In particular, it allows for the cumulative capture and refinement of experience. One common feature of all forms of effort and cost estimation is the need for stability and measurement. Although theory provides preliminary estimates, these become progressively more useful and accurate only if they are refined in the light of continued monitoring in a stable development environment.
Another aspect of learner behaviour that might constrain discursive, though not didactic, mobile learning is the reluctance of learners to engage actively within the chat, forum and discussion facilities of computer-mediated conferencing. Informal surveys on mailbases such as those of the Joint Information Services Committee (JISC) suggest that from 10% to 40% of learners lurk without actively participating and that for online learning considerable skill and effort (and hence, cost) are required on the part of tutors to convert these lurkers into active users (Salmon 2000). These skills must be adapted and costed before they can be used in mobile learning.
7. Conclusion
This paper has identified elements of a theoretical basis for estimating and predicting the effectiveness, efficiency and economics of mobile learning. It also shows that even at this early stage of implementation, it is possible to identify those procedures likely to enhance the efficiency of any commercial exploitation of mobile learning. Increased exploitation will improve on any early estimates and refine any procedures.
6. Human factors
This section looks briefly at the factors that complicate this apparently rational and systematic analysis of the economics of mobile learning, namely the preferences, attitudes and behaviour of teachers and learners, and how they are currently manifest in e-learning. The current m-learning project is clearly tied to the adult basic skills curriculum. There are many subject areas where mobile learning is impossible, however attractive it might be on cost grounds. These include subjects and courses with elements that are: hands-on, eg music-making in vivo, eg wet biology, practical dentistry and so on interpersonal, eg interview skills social, eg team work, marketing expressive, eg ballet, dance. There are others, as well as many exam and assessment situations, where computer-based learning of any sort is inappropriate. In addition, the time and cost elements borne by the learners themselves might be unacceptable without public subsidy (which might have to include setting up, maintaining and supporting systems in use). Another constraint on deploying the most cost-effective technologies is learner resistance and preference. The subjects of one survey though they are not identified as coming from the m-learning project target groups show clear preferences for books, lectures and videos over computers as their preferred learning tools (Daniel 1996). Unpublished survey results from LSDA make the opposite case for potential mlearning learners and indicate a remarkably high preference for mobile phones, though not yet for PDAs as the medium for communication and learning.
8.
References
Bacsich P, Ash C, Kaplan L (1999). The costs of networked learning. Sheffield: Sheffield Hallam University. Boehm B (1996). Anchoring the software process. IEEE Software, 13(4), 7382. Boehm B (2000). Software cost estimation in COCOMO II. New Jersey: Prentice Hall International. Boehm B (1981). Software engineering economics. New Jersey: Prentice Hall International. Boyle T (1997). Design for Multimedia Learning, New Jersey: Prentice Hall International. Britain S, Liber O (1999). A framework for pedagogical evaluation of virtual learning environments. Bristol: JISC Technology Applications Programme. Conole G, Oliver, M (1999). Media advisor (2.1). London: University of North London; Connected Learning Ltd. Cusamano MA, Selby RW (1996). Microsoft secrets. London: HarperCollins. Daniel JS (1996). Mega-universities and knowledge media. London: Kogan Page. Hunt M, Clarke A (1997). A guide to the cost effectiveness of technology-based training. Coventry: National Council for Educational Technology. Laurillard D (1987). Computers and the emancipation of students: giving control to the learner. Instructional Science, 16, 318.
188
Laurillard D (1993a). Rethinking university teaching. London: Routledge. Laurillard D (1993b). Balancing the media. Journal of Educational Television, 19(2), 8193. Laurillard D (2002) private communication to the author by e-mail 3 August 2002. Marshall IM, Sampson WB, Dugard PI (1995a). Multimedia courseware. Association for Learning Technology Journal, 3(1), 110117. Marshall IM, Sampson WB, Dugard PI (1995b). Predicting the developmental effort of multimedia courseware. Information and Software Technology, 36(5), 251258. Nicol D, Coen M. (2003). A model for evaluating the institutional costs and benefits of ICT initiatives in teaching and learning in higher education. Association for Learning Technology Journal, 11(2), 4660. Pask G (1976). Conversation techniques in the study and practice of education. British Journal of Educational Psychology, 46, 1225. Peters O (1998). In F Lockwood (ed) Learning and teaching in distance education: analyses and interpretations from an international perspective. London: Kogan Page. Pressman R (2000). Software engineering: a th practitioner's approach (5 ed). Maidenhead, England: McGraw-Hill Publishing Company. Reisner RA, Gagne RM (1983). Selecting media for instruction. New Jersey: Prentice Hall International. Rumble G (1997). In F Lockwood (ed) The costs and economics of open and distance Learning. London: Kogan Page. Salmon G (2000). In F Lockwood (ed) emoderating: the key to teaching and learning online. London: Kogan Page. Sommerville I (1992). Software engineering. London: Addison-Wesley. Stoner G (1996). A systems analysis and design approach to the integration of learning technology., Glasgow: Association for Learning Technology. Vygotsky L (1962). Thought and language Cambridge, MA: MIT Press.
Supported by the EC Directorate-General Information Society, number IST 2000-25270 2 Supported by the EC Directorate-General Information Society, number IST 2001-37187 3 www.usd.edu/pda/, accessed November 2003 4 wwwst.kinjo-u.ac.jp/~thornton/, accessed November 2003
1
Vainio, Ahonen
189
A critical approach to an adaptive user interface design
Teija Vainio Hypermedia Laboratory University of Tampere 33014 University of Tampere, Finland E-mail: teija.vainio@uta.fi
Mikko Ahonen Hypermedia Laboratory University of Tampere 33014 University of Tampere, Finland E-mail: mikko.ahonen@uta.fi
Abstract
In our study we present a critical approach to usability issues and usability evaluation particularly involving adaptive user interface design for mobile learning environments. We describe some design challenges for adaptive user interfaces and key usability issues in the MOBIlearn project. In addition, we compare these challenges in three different learning contexts. Our hypothesis is that the concept of learning is not clearly defined in most usability studies. Additionally, the influences of different learning contexts are not considered enough in usability evaluation. Our aim is to develop mobile systems that are adaptable to more than one learning context. Keywords: adaptive user interface, usability, learning context
This paper introduces one approach to a user interface design in the MOBIlearn project, eg adaptive human interface design. Our work aims to design user interfaces that take account of different kind of users, contexts, content and devices. In the MOBIlearn project results are verified with real users in field trials involving: blended learning (enhancing formal courses) adventitious, location-dependent learning (during visits to museums) learning to interpret information sources and advice (acquiring medical information for everyday needs) (MOBIlearn 2003). In our opinion, usability design should aim to support all these three types of learning and user activity within the same adaptive mobile learning environments. In a sense the learning itself is different in each of these contexts: the first is focused on blended, formal learning, the second more on informal learning and the last on lifelong learning. For that reason, learner and context modelling are the primary concerns in our usability and user interface design process. We try to avoid building three totally different user interfaces, instead providing the user with different navigation alternatives and levels of discovery. De Carolis et al. (2001) state that adapting information presentation to the users is not enough; besides their experience, information needs, interests and so on, other features have to be considered. These features include users location, the activity in which they are involved, their
1. Introduction: user interface design in the MOBIlearn project

MOBIlearn is a EU IST research project focused on next-generation paradigms and interfaces for technology-supported learning in a mobile environment exploring the potential of ambient intelligence. By ambient intelligence we mean digital environments in which applications are sensitive to users needs and to their requirements and behaviour.
190
emotional state and, finally, the technical characteristics of the device they are using. This information about the user in context can be employed to contextualise the way the information is accessed and presented. In the MOBIlearn project the context issues are well addressed. The University of Birmingham is working closely with us (the University of Tampere) and they have focused on building a context-awareness subsystem (CAS). In broad terms, the aim of the CAS is to provide a means by which users of mobile devices can maintain their attention on the world around or the task at hand, while the mobile devices provide timely and effective computer support (Lonsdale et al. 2003). The user interface design challenges in our research are strongly connected to those mentioned by de Carolis et al. (2001): which type of navigation through the hypermedia (free or guided); what kind of structure (linear, hierarchical or circular); and what level of orientation support should be provided for users. Context-awareness information is needed to make this kind of decision and to support users activities.
and are used only to a limited degree by experienced users. One reason for that is the cognitive load these features cause for the users. This could keep them away from their main task, which in the MOBIlearn project, is learning. Since the learning systems with mobile devices are expected to be used for the short-term, we argued that it might be better let the system adapt itself (adaptivity) to the user rather than forcing the users to change the system behaviour (adaptability). Design of adaptive human interfaces is based on assumptions of a users behaviour which are then used to form a theoretical model. A user model describes what is known about the user and users interest (Patamaa et al. 2001). In mobile learning applications, such as those in the MOBIlearn project, definitions of the user models are the most challenging ones. For example, the users interest can be classified as a short-term interest such as a current task, a long-term interest which is stable such as work, a hobby or a learning path, or a hybrid interest, which is both of these. What the user already knows is also important concerning, for example, new information (Macskassy et al. 2002.). User models can be based on the distinction of novice and expert users, device, screen layout, and interaction technique preferences (Sukaviriya and Foley 1993).
2. Adaptation, adaptability a user model in focus

Studies of web-based educational systems and mobile learning environments have recently started to focus on systems described as adaptive. Adaptive systems can be seen as an alternative to the one-size-fits-all approach (Brusilowsky et al. 2000). An adaptive user interface can be defined as a software artefact that improves its ability to interact with a user by constructing a user model based on partial experience with that user (Langley 1999). The term adaptation refers to a systems capacity to change its behaviour dynamically to keep the quality of service above a certain level. In many cases adaptation is seen as a part of contextawareness in mobile applications and systems. Adaptivity refers to a system that adapts itself according to the user. Furthermore, adaptability refers to a system where the users have to change the system behaviour. Adaptive user interfaces can be focused on, for example, information or content-based filtering, recommendation, social or collaborative filtering, or optimising (Langley 1999). Recent studies have stated that adaptable features, such as tools for adapting the user interface, are not often used by novice users
3. Adaptation and adaptability as a part of usability

We see adaptive user interface design as a vital part of the usability design of an application or a service. Usability can be seen as a relative issue in many different ways. Acceptance by end users of an application can be seen as a primary goal of interactive systems design. A simple model of system acceptability consists of social acceptability and practical acceptability. Social acceptability refers to system features acceptability in cultural and social contexts. Practical acceptability can be analysed within various categories such as cost, support, reliability, compatibility and usefulness. (Shackel 1991). Usefulness is defined as the ability to achieve a goal by using the system. Usefulness can be divided into utility and usability. Utility refers to the functionality of a system, and usability to the ability to make use of that functionality (Nielsen 1995).
Vainio, Ahonen
191
The usability of a system is determined by the usability of both the underlying system engine and the contents and structure of the information base. Usability has been associated with five attributes: ease of learning, efficiency, easy to remember, consistency (few errors) and pleasantness of use (Nielsen 1995). In user-adaptive systems there are several key issues to be resolved in the design of such systems. Jameson (2001) argues that the following issues are essential and common. 1. What functions are to be served by the adaptation? 2. What properties of the user should be modelled? 3. What input data about the user should be obtained? 4. What techniques should be employed to make inferences about the user? 5. How should decisions about appropriate adaptive system behaviour be made? 6. What empirical studies should be conducted? Furthermore, in mobile learning environments adaptive user interfaces should support lifelong, informal learning. We mentioned earlier the focus of the MOBIlearn project on informal learning. When learning takes place over a long time, the learners abilities and skills will change gradually. To adapt to a learners changing skills and knowledge, the system must be able to maintain a profile or model of the learner that can determine the way in which the accumulated knowledge and learning material are stored and then presented back to the learner in new contexts. (Sharples et al. 2002).
Some recent studies argue that current adaptive (hypermedia) systems are based on a stereotypical user model with limited levels of user differentiation and some additional research is suggested in the evaluation of the educational effectiveness of system adaptation (Triantafillou et al. 2002). The psychological effects of an interface adaptation on user performance have also been studied. Two competing possible effects of using adaptive user interfaces can be found: social facilitation and chocking. The former refers to consequences that occur because the user performance is monitored by the interface. The latter, chocking, refers to consequences caused by the interface, which adapts to a users performance (Jettmar and Nass 2002).
5. Conclusions and future work

In our study we examined examples of existing adaptive user interface design practices and some key usability issues. Our aim is to develop usability of adaptive user interfaces in mobile learning systems that adapt to the users changing needs over a long time. Usability focuses on making applications easy for people to use. Accessibility on the other hand focuses on making applications equally easy for everyone to use, including people with a disability. This kind of accessibility view often focuses on areas like multi-modality and material conversions. In the context of mobile collaboration, access and accessibility are often more essential issues than ease of use (Ahonen 2003). As the importance of informal learning in a mobile context has been recognised, there has been research on systems that support a persons everyday learning over a lifetime. Vavoula and Sharples (2002) have inspected learning episodes and personal learning projects, and have developed the following criteria for lifelong learning organisers (LLOs): LLOs should be available and functional any time, during any day of the week. To us these kinds of accessibility requirements mean that adaptation logic is integrated into the device and its software, not only based on the hyperlinking structure or users profile on a server. Adaptation in future studies may need to be inspected in the context of learning agents and artificial intelligence.
4.
Critical voices
Some criticism of user adaptive systems and user profiles has already appeared. According to Piomo et al. (2000), in existing web-based education environments a student profile has been taken into consideration and some common features are encountered such as initial level of knowledge and learning objectives. Few systems have considered cognitive aspects when student profiles are modelled. We argue that the learning contexts, which are based on usability design including adaptive user interface design and adaptive system design, should be defined more clearly. Furthermore, we argue that in traditional usability testing the long-term usage is underestimated.
192
However, agents pose some new user interface design considerations like understanding, trust, control, distraction and personification (Wexelblat and Maes 2003). We will continue our work in the MOBIlearn project by providing some guidelines for adaptive user interface design. Furthermore, we will study some design challenges of mobile learning architecture and related adaptivity and accessibility themes.
6. Acknowledgements
We would like to thank reviewers, our colleagues in the MOBIlearn project and Ms Tiina Jrvinen, University of Tampere, for their comments.
7. References
Ahonen M (2003). Accessibility challenges with mobile lifelong learning tools and related collaboration. Communities and Learning Conference. Ubiquitous and Mobile workshop. Amsterdam. Paper submitted for review. Brusilowsky P, Stock O, Strapparava C (eds) (2000). Adaptive hypermedia and adaptive webbased systems. Proceedings of the International Conference, AH 2000. Trento, Italy: Springer. De Carolis B, de Rosis F, Pizzutilo S (2001). Context-sensitive information presentation. UM Conference proceedings. A t http://orgwis.fit.fraunhofer.de/~gross/um2001ws/ papers/carolis.pdf, accessed January 2004. Jameson A (2001). Systems that adapt to their users. Description of an IJCAI 01 tutorial on user-adaptive systems. Last update: 7 March 2001. At www.dfki.de/~jameson/ijcai01-tutorialjameson.pdf, accessed 12 December 2002. Jettmar E and Nass C (2002). Adaptive testing: effects on user performance. Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves. Minneapolis, Minnesota: ACM Press, 129134 Langley P (1999). User modeling in adaptive i n t e r f a c e s . Proceedings of the Seventh International Conference on User Modeling, Banff, Alberta: Springer, 357370. Lonsdale P, Baber C and Sharples M (2003). Context-awareness architecture for facilitating mobile learning. MLEARN 2003 Conference. Book of abstracts, 4546.
Macskassy SA, Danyluk AA, Hirsh H (2002). Information valets for intelligent information access. Papers from the 2000 AAAI Spring Symposium on Adaptive User Interfaces. Menlo Park, CA: AAAI Press. At wwwcsli.stanford.edu/cll/schedule.html, accessed 26 November 2002. MOBIlearn (2003). MOBIlearn EU IST project overview. At www.mobilearn.org, accessed January 2004. Nielsen J (1995). Multimedia and hypertext. the internet and beyond. Boston: AP Professional, 280281. Patamaa L, Lankoski P, Ermi L (2001). Personointi henkilkohtaisessa navigoinnissa. In P Lankoski (ed) Ihminen, aika ja paikka. Kohti henkilkohtaisen navigoinnin kyttliittymn suunnitteluperiaatteita. Tampere: University of Tampere: Computer Centre Publication Series 9. Piomo C, Batatia H, Ayache A (2000). Adapting instructional hypermedia content to cognitive profiles. ABIS-Proceedings: OnlineVersion. ABIS-Workshop 2002: Personalization for the mobile world. At www.kbs.uni-hannover. de/~henze/lla02/proceedings/abis.pdf, 3942, accessed 26 July 2003. Shackel B (1991). Usability context, framework, design and evaluation. In B Shackel and S Richardson (eds) Human factors for informatics usability. Cambridge: Cambridge University Press, 2138. Sukaviriya P, Foley JD (1993). Supporting adaptive interfaces in a knowledge-based user interface environment. Proceedings of the 1st international conference on intelligent user interfaces. Orlando, Florida: 107113. Syvnen A, Ahonen M, Jppinen A, Pehkonen M, Turunen H, and Vainio T (2003). Accessibility and Mobile Learning. Proceedings of IFIP eTrain03 Conference. Pori, Finland. Triantafillou E, Pomportsis A, Georgiadou E, AES-CS: adaptive educational system based on cognitive styles. In P Brusilovsky, N Henze and E Milln (eds): Proceedings of the Workshop on Adaptive Systems for Web-based Education, held in conjunction with AH'2002, Mlaga, Spain, 112. Vavoula G , Sharples M (2002). Requirements for the design of lifelong learning org a n i s e r s . S Anastopoulou, M Sharples, Vavoula (eds) MLearn 2002: Proceedings of the European Workshop on Mobile and Contextual Learning. Birmingham: 2326. Wexelblat A, Maes P (2003). Issues for software agent UI. Draft. At http://citeseer.nj.nec.com/128549.html, accessed 28 July 2003.
Zancanero et al.
193
Mobile cinematic presentations in a museum guide

Massimo Zancanaro, Oliviero Stock and Ivana Alfaro ITC-irst Via Sommarive, 18 Pant di Povo, Trento I-38050 Italy {zancana|stock|alfaro}@itc.it
Abstract
In this paper we introduce the idea of enhancing the audio presentation of a multimedia museum guide by using the personal digital assistant (PDA) screen to travel throughout a fresco and identify the various details in it. During the presentation, a sequence of pictures is synchronised with the audio commentary and the transitions between the pictures are planned according to cinematic techniques. Preliminary studies and pilot tests show encouraging results and interesting effects. The use of cinematic techniques in the audio guide seems to help users better understand and localise the frescos details, while the localisation mechanism employed in the museum allowed them quickly to identify the panel of the fresco referred to in the audio. Keywords: multimedia museum guides, cinematography
1. Introduction
Many research projects are exploring the new possibilities offered by personal digital assistants (PDAs) in a museum setting (for example, Grinter et al. 2002; Cheverst et al. 2000). These projects usually have multimedia guides that use static images, while others employ pre-recorded, short video clips about museum exhibits. In previous work (Not et al. 1998), we explored different techniques for automatically building location-aware multimedia presentations in a museum setting. The advent of more powerful devices since has allowed researchers to experiment with new forms of multimedia, in particular time-based media such as animations. In this paper we introduce the idea of enhancing an audio presentation (dynamically
assembled pre-recorded or synthesised speech) of a complex fresco by using the PDA screen to travel throughout the fresco itself and identify various details in it. Our hypothesis is that this type of animation used to present the description of a painting allows the visitor to identify the details introduced by the audio counterpart to the presentation more accurately. In this manner, the system becomes both more effective and more satisfying (Nielsen 1994), while also providing an enhanced learning experience for the visitor. At present, we have completed a first prototype for the famous 15th-century fresco The cycle of the months at Torre Aquila in Trento, Italy. This fresco illustrates the lives and activities of aristocrats and peasants throughout the year, and covers all four walls of a tower. The numerous characters introduced throughout the fresco are seen harvesting wine, hunting with falcons and generally occupied in medieval activities. A web-based demo of the prototype is available at: http://peach.itc.it/preview.html
2. Rhetorical Structure Theory

Rhetorical Structure Theory (RST) (Mann and Thompson 1987) analyses discourse structure in terms of dependency trees, with each node of the tree being a segment of text. Each branch of the tree represents the relationship between the two nodes, where one node is called the nucleus and the other is called the satellite. The information in the satellite relates to that found in the nucleus in that it expresses an idea related to what was said in the nucleus. This rhetorical relation between them specifies the relation of coherence which exists between the two portions of text contained in the nodes. For
194
example, a Cause rhetorical relation holds true when the satellite describes the cause of the event contained in the nucleus. In the original formulation by Mann and Thompson (1987), the theory posited 20 different rhetorical relations between a satellite and a nucleus, while other scholars have since added to this theory. RST was originally developed as part of the work carried out in the computer-based text generation field. A previous article (Not and Zancanaro 2001) described a set of techniques that dynamically compose adaptive presentations of artworks from a repository of multimedia data annotated with rhetorical relations. These techniques have been employed in an audio-based, location-aware adaptive audio guide described in Not et al. (1998). The audio commentaries produced by that audio guide were automatically annotated with the rhetorical structure. We are now investigating a system that automatically composes video clips out of these audio commentaries (Rocchi and Zancanaro 2003). For our current work, we used RST to inform the design of our cinematic presentations and organise the various scenes that composed each video clip in accordance with the logical model that exists in dialogue. RST provided us with the blueprint for a better analysis of the audio presentations and thus construction of the video counterparts. In the next section we will discuss how this information can be used to create more effective video clips to accompany the commentary.
Figure 1. Screen of mobile device To have a more engaging presentation, the visual part should not only focus on the right detail at the right time, but it should also support the presentation of new audio information by illustrating its relation to information that has been already given. In this manner, continuity between the pieces of information is built, which in turn facilitates the viewing of the video clip while stimulating the absorption of new information. For strategies in the second class, it is often necessary to make reference to the discourse structure of the audio part of the presentation, such as enumeration of properties, background knowledge and elaboration of related information. For example, consider an audio commentary describing a detail of the fresco, such as: At the bottom on the right is a blacksmiths workshop, a plebeian antithesis to the tournament going on in the upper part of the painting. The choice of the tournament for the month of February is related to the jousts that took place during carnival. The two sentences are linked by a rhetorical relation of type Elaboration since the second sentence further elaborates the topic introduced by the first. This commentary can be visually represented with two shots of the same image (that is, the tournament) linked by a long cross fade. Technically, having two shots is not necessary, since the image is the same, but the cross fade helps the user understand that background information is going to be provided. The first image is thus presented while the first paragraph is heard over the audio, then when the audio switches to, in this case, the background information, the image is enlarged to cover the entire panel and finally refocused on the detail once the audio has stopped.
3. Cinematic presentations
The language of cinematography, including shot segmentation, camera movements and transition effects, is employed to plan the animation and to synchronise the visual and the verbal parts of the presentation (Metz 1974). In building the animations, a set of strategies similar to those used in documentaries were thus employed. Two broad classes of strategies have been identified. The first class of strategy encompasses constraints imposed by the grammar of cinematography, while the second deals with conventions usually used in guiding camera movements in the production of documentaries. For instance, a strategy in the first class would discourage a zoom-in facility immediately followed by a zoom-out facility, while a strategy in the second class would recommend the use of sequential scene cuts, rather than a fade-out effect, to enumerate different characters in a scene visually.
Zancanero et al.
195
4. Mobile presentations
The guide was implemented on a PDA that by means of infra-red sensors is capable of identifying its position within the frescoed tower of the castle. Interaction with the system is both proposed by the system itself, and accepted by the user, thus sharing the responsibility of information access. When the system detects that the visitor is in front of one of the four walls, a picture of that wall is displayed on the PDA and, after a moment, if the user has not changed position, the panel is highlighted (see Figure 2). At this point, the visitor can click on the panel and receive a multimedia presentation of the chosen panel . As one infra-red was placed in front of each panel, the visitor needed only to be within an approximately two-metre range of the infra-red to connect with the sensor. Given the characteristics of the tower, each panel was clearly separated and while the sensors took a moment or two to activate the system, we did not experience any accuracy problems. This modality for localisation was chosen to allow the visitor to retain control of an inherently proactive guide.
small details on the fresco with the help of the PDA. This demonstrates that use of cinematic techniques in a multimedia guide can be effective, particularly when explaining a complex painting. A formal study started in May 2003 in Torre Aquila and will involve approximately 60 subjects. The purpose of the study is to investigate the correlation between rhetorical devices and the visual attention of the guide user. We will study how the application of the rhetorical transitions described above affect the users attention by observing the patterns of eye movements to and from the fresco and the guide.
5.1. Subjective evaluation: preliminary results

As the study is still in progress we have to date only analysed the subjective evaluation of the first 30 participants. The age of the participants ranges from 21 to 64 years with the average age being 37.1 years (standard deviation 43.7 years). Of these 14 are male and 16 female; 20% have had a primary school education, 47% a college education and 33% a university degree. Self-declared previous knowledge of the tower in the fresco is evenly distributed between no knowledge at all and a very good knowledge. After a short training phase, participants interact freely with the mobile multimedia guide in Torre Aquila and, when finished, are asked to fill out a questionnaire about the experience. The questionnaire is composed of 21 statements with which the participants have to express their agreement using 10- and 5-point Likert scales. The most interesting aspect to emerge so far from the analysis regards the role of the cinematic techniques. In the statement The videos helped me to better understand the frescos details, it was found that 76.7% of participants totally agreed with this statement (see Figure 3).
90 80 70 60 50 40 30 20 10 0 76,7
Figure 2. Mixed-responsibility in fresco selection
5. Evaluation
Preliminary studies and pilot tests indicate encouraging results and interesting effects. It was found that all users became acquainted with the system very quickly. The main limitation with using infra-red beamers as a localisation technology is their directionality, that is, the user must face the beamer to be localised. Most of the visitors, however, overcome this limitation by naturally using the PDA as a type of remote control, pointing it directly at the infra-red emitters to speed up the localisation. Before actually using the system most of the users interviewed so far complained that a video sequence on a PDA would distract their attention from the real artwork. However, after a short interaction with the system, they appreciated the possibility of quickly localising
Percentage
13,3 3,3 Strongly disagree 6,7 0 Disagree Undecided Agree Strongly agree
Figure 3. Agreement on the sentence The videos helped me to better understand the frescos details (5-point Likert scale)
196
In the statement the Indication provided by the system allowed me to identify less visible details, it was found that 60% of participants totally agreed with this statement (see Figure 4). In general there is a negative correlation between the age of the subjects and the usefulness of the cinematography (ie the correlation is -0.592, with a significance level of 0.05, in the statement noted in Figure 3).
70 60
Percentage
techniques on the overall system efficacy. This will be further analysed with the objective evaluation in the next phase.
6. Acknowledgments
This work has been supported by the PEACH and TICCA projects, funded by the Autonomous Province of Trento, Italy.
7. References
60
50 40 30 20 10 0 Strongly disagree Disagree Undecided Agree Strongly agree 6,7 0 0 33,3
Figure 4. Agreement on the sentence Indication provided by the system allowed me to identify less visible details (5-point Likert scale) Also the localisation mechanism appears to play a significant role in improving the usability of the mobile guide.
50 45 40 35 30 25 20 15 10 5 0 46,7 33,3 20
0 Strongly disagree
0 Disagree Undecided Agree Strongly agree
Figure 5. Agreement on the sentence The localisation mechanism allows me to quickly identify the panel of the fresco referred to in the audio (5-point Likert scale) In the statement The localisation mechanism allows me to quickly identify the panel of the fresco referred to in the audio, it was found that 46.7% of participants totally agreed with this statement, and 33.3% partially agreed (see Figure 5). The complete analysis will be published at the end of the study. Although these are preliminary findings, they do provide a reasonable indication that the use of cinematography techniques with an audio guide can enhance the overall museum experience of a user. The subjective evaluation does not allow us to estimate the impact of each individual component that makes up the cinematographic
Cheverst K, Davies N, Mitchell K, Friday A and Efstratiou C (2000). Developing a contextaware electronic tourist guide: some issues and experiences. Proceedings of CHI 2000, Amsterdam 1724. Grinter RE, Aoki PM, Hurst A, Szymanski MH, Thornton JD and Woodruff A (2002). Revisiting the visit: understanding how technology can shape the museum visit. Proceedings of the ACM Conference on Computer Supported Cooperative Work. New Orleans, LA. Mann WC and Thompson S (1987). Rhetorical Structure Theory: a theory of text organization. In L Polanyi (ed) The structure of discourse. Norwood, NJ: Ablex. Metz C (1974). Film language: a semiotics of the cinema. New York: Oxford University Press. Nielsen J (1994). Usability engineering. San Francisco: Morgan Kaufmann. Not E, Petrelli D, Sarini M, Stock O, Strapparava C, Zancanaro M (1998). Hypernavigation in the physical space: adapting presentations to the user and the situational context. New Review of Hypermedia and Multimedia, 4. Not E, Zancanaro M (2001). Building adaptive information presentations from existing information repositories. In NO Bernsen, O Stock (eds) Proceedings of the International Workshop on Information Presentation and Multimodal Dialogue. Verona, Italy: 95100. Rocchi C, Zancanaro M (2003). Generation of video documentaries from discourse structures. Proceedings of Ninth European workshop on Natural Language Generation. Budapest.
Percentage
197
Author index
193 189 79 3 13 21, 79 27 61 61 27 87 35, 139 127 Alfaro, I Ahonen, M Arvanitis, TN Attewell, J Avellis, G Baber, C Bjrgvinsson, EB Bokma, A Bowerman, C Brandt, E Brewster, D Bull, S Burton, J 3 13 173 21, 79, 139 87 69 43 193 145 155 167 173 183 155 Savill-Smith, C Scaramuzzi, A Schott, G Sharples, M Siddons-Corby, R Specht, M Stead, G Stock, O Stone, A Syvnen, A Taylor, J Thomas, S Traxler, J Turunen, H
139 Chan, T 43 Colley, J 47 Danielsson, K 51 Dawabi, P 87 du Boulay, B 95 Fernndez-Morales, A 13 Finkelstein, A 21 Glew, P 61 Graham, P 47 Hedestig, U 27 Hillgren, P-A 69 Hine, N 47 Juslin, M 73 Kadyte, V 173 Kambouri, M 79 Lonsdale, P 87 Luckin, R 95 Mayorga-Toledano, MC 99 Mifsud, L 105 Mitchell, A 55 Neuhold, E 117 Noessel, C 47 Orre, CJ 87 Pearce, D 155 Pehkonen, M 127 Proctor, N 131 35 69 139 Rainger, P Reid, E Rentoul, R Rudman, P
189 Vainio, T 21 Vavoula, G 55 Wessner, M 193 Zancanaro, M
Mobile learning is a relatively new concept but one which is gathering momentum and attracting the interest of researchers, educators and companies developing learning systems and materials. The MLEARN 2003 conference brought together people who were interested in mobile learning from around the world. All the papers in this edited book are based on presentations given at the MLEARN 2003 conference; together they provide a fascinating mix of research reports, theory, work in progress and news of products under development.
ISBN 1 85338 833 5 M-learning and MOBIlearn are projects supported by the European Commission Directorate-General Information Society IST-2000-25270 (m-learning) IST-2001-37187 (MOBIlearn)

Learning With Mobile Device

Uploaded by

Copyright:

Available Formats

Learning With Mobile Device

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Learning With Mobile Device

Uploaded by

Copyright:

Available Formats

learning with mobile devices research anddevelopment

a book of papers edited by Jill Attewell and Carol Savill-Smith

learning with mobile devices research anddevelopment

MLEARN 2003 conference programme committee

Keith Duckitt Head of ICT Learning and Skills Council, UK

Kate Anderson Director, Research Learning and Skills Development Agency, UK

Mobile learning and social inclusion: focusing on learners and learning

1. An important issue for educators

2. Background to the m-learning project

Learning with mobile devices: a book of papers

3. Investigating the context of mobile learning

4. The findings of the research reviews

Learning with mobile devices: a book of papers

Rules Goals Interaction

How do young adults use their mobile phones?

Our findings the survey

Learning with mobile devices: a book of papers

Figure 1 is a diagrammatic representation of the phase 2 learner research.

Learning with mobile devices: book of papers

Learning with mobile devices: a book of papers

3. Evaluation of mobile learning contents and multimedia educational software

Learning with mobile devices: a book of papers

3.1 Mobile e-learning in practice

3.2 Evaluation issues

4. A scheme for critical NFR representation

Learning with mobile devices: a book of papers

Figure 1 Non-functional requirements representation scheme

5. Conclusions and further research

Learning with mobile devices: a book of papers

1. Defining the problem

Learning with mobile devices: a book of papers

Type of learning X Project Different technologies Z

Activity Context Presentation Management Communications Y

2. Quality function deployment

3. Defining user requirements

Learning with mobile devices: a book of papers

5. Building a House of Quality

97 173 109 74 74 7413713217419114824258169183273 Evaluation criteria for functions

Figure 2: House of Quality for learning technologies

Self-produced video to augment peer-to-peer learning

Learning with mobile devices: a book of papers

3. Mobile computers with video

Self-produced learning material

Learning with mobile devices: a book of papers

Learning with mobile devices: a book of papers

5. Using mobile video

Learning with mobile devices: a book of papers

Individualised revision material for use on a handheld computer

Learning with mobile devices: a book of papers

3. Mobile revision material

Figure 2. Mobile graphical performance overview

Figure 3. Mobile textual performance overview

Learning with mobile devices: a book of papers

4. Initial pen and paper evaluation

Figure 5. Tailored mobile revision material: less detailed

4.2. Materials and method

Learning with mobile devices: a book of papers

Table 2. Expected utility of mobile MoreMaths components by subject

5. Summary and further work

Learning with mobile devices: a book of papers