life, leveling the playing ield for people, companies, and countries worldwide. Education for a Digital World 2.0 Innovations in Education Volume 1 Edited by Sandy Hirtz & Kevin Kelly The irst edition, learning objects | dialogue | ePortfolios | engag ODL | gaming | social networking | learning com activism | praxis | relection | mobiles | wikis blogs | LMS | media | virtual worlds | co-creatio collaboration | learning | authentic | assessmen workshops | Web 2.0 ISBN 0-7726-6488-9 9 780772 664884 7540006117 Education for a Digital World 2.0 Innovations in Education Vol. 1 Published by Open School BC Acknowledgements We are unlocking formidable new capabilities that could lead to much more exciting lives and glorious civilizations. – James Martin Education for a Digital World 2.0: Innovations in Education, embodies an interactive and collaborative model of writing and publishing that demonstrates the powerful opportunities aforded by online technologies, innovative thinking and cooperation around the world. I would like to express my gratitude to the 98 people from 13 countries (listed in the contributor section), who collaborated on this project, provided encouragement, and assisted in the peer review and copyediting process. I am grateful for the friendship and support of Kevin Kelly who tirelessly helped me lead this endeavor. A special thank you to Paul Beaufait and Rick Lavin for maintaining the wiki and coordinating the ever growing list of contributors, and to June Kaminski for the cover art. Sincerest appreciation and thanks to the Ministry of Education and Open School BC for inancial and production support, and to Dennis Evans for maintaining enthusiasm as the book expanded, burst its seams, and metamorphosed into two volumes. Sandy Hirtz, Editor 2 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 If we teach today as we taught yesterday, we rob our children of tomorrow. – John Dewey he need to know the capital of Florida died when my phone learned the answer. Rather, the students of tomorrow need to be able to think creatively; they will need to learn on their own, adapt to new challenges and innovate onthe-ly. – Anthony Chivetta, high school student in Missouri hese two quotes, separated by almost 100 years, speak to the importance and timeliness of Education for a Digital World 2.0: Innovations in Education. he world around us is changing, and what better way to prepare educators than to ask other educators to share their knowledge and experiences? I was convinced of this as an author and peer reviewer of the irst edition, and am grateful to Sandy Hirtz for inviting me to join her as an editor for this second edition. As mentioned above, this entire process—this book—was truly collaborative, evidenced even by the wiki URL: http://collaborativebook.wikispaces.com/. I hope to see many others emulate our approach to co-creating knowledge. I add my own thanks to all the participants. We would not have a second edition without the many authors’ contributions. I also would like to thank the collaboration coordinators, Paul and Rick; the peer reviewers, experts from around the globe in diferent areas of education and technology; and the copy editors—all of whom volunteered time to make this efort more rigorous. Finally, thanks go to Dennis, and everyone at the Ministry of Education and Open School BC for bringing our work to the world. Kevin Kelly, Editor Copyright Permission he print version of these volumes is © All rights reserved, and the digital version is CC-BY-NCND. Any part may be reproduced without permission but with attribution to the authors. For derivative work, please write to the editors. Cover design by June Kaminski EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 3 Education for a Digital World 2.0: Innovations in Education Vol. 1 Foreword (Volume 1) Emerging Technologies, Learning Management, and Developing Content in a Digital World . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Tony Bates Chapter Abstracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Part One: Emerging Technologies and Practices Emerging Technologies in eLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Patricia Delich, Kevin Kelly, and Don McIntosh Digital Interaction: Learning and Social Communication in the Information Age . . . . 71 Christopher Boyle and Gerald Wurf Managing Your Digital Footprint: Ostriches v. Eagles . . . . . . . . . . . . . . . . . . . . 89 Julia Hengstler Language Learning 2.0 – Innovative Approaches for Tomorrow’s Global Citizens. . . . 141 Sandy Hirtz, Sarah Elaine Eaton, with a contribution from Brittany Espe Immersive Gameworlds for Worldwide Change. . . . . . . . . . . . . . . . . . . . . . . 167 Natasha Boskic Media Education in India: A Peer Perspective. . . . . . . . . . . . . . . . . . . . . . . . 191 Ankuran Dutta and Anamika Ray mLearning: Small Technologies – Massive Contributions . . . . . . . . . . . . . . . . . 213 Brown Onguko, Symon Ngatia and Susan Crichton Part Two: Implementing and Managing eLearning he Pearls and Perils of Digital Education . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Pheo Martin Learning Management Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Don McIntosh with contributions from Kevin Kelly, Patricia Delich, and Randy Labonte Stakeholder Issues for Adopting a LearningManagement System . . . . . . . . . . . . . 309 Joshua L. Mindel and Kevin Kelly Towards an implementation of eLearning at a multi-campus university . . . . . . . . . 327 Azra Naseem, Shariq Khoja, Melaine D’Cruze, and Karim Wallani Case Study of Implementation of a Learning Management System in India . . . . . . . 351 Nilay M. Yajnik 4 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part hree: Developing and Monitoring eLearning Content Co-Creation of Content to Promote Learning, Activism and Advocacy . . . . . . . . . 363 June Kaminski and Amandah Hoogbruin he OER Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Paul G. West and Lorraine Victor Open Licences for Educators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 Julien Hofman and Paul West Quality in eLearning: Engaging Learners . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Randy LaBonte Learning Objects and Personal Learning Environments . . . . . . . . . . . . . . . . . . 511 Valerie Taylor, Richard S. Lavin, and Nellie Deutsch ODL For Corporate Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 Anil Prasad P EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 5 Stakeholder Issues for Adopting a Learning Management System Joshua L. Mindel and Kevin Kelly he pace of academe is perhaps best measured by the 25 years it took to get the overhead projectors out of the bowling alley and into the classroom. - Gilbert & Green (1997, p. 25) Introduction Selecting a learning management system (LMS) for an organization is not an easy process, especially if that organization has already been using one for several years. LMS vendors beneit from many institutions’ fears of the pains of change and user backlash when faced with changing systems. Beyond the iscal cost, an organization must also weigh the labor costs required of the users and support personnel to make such a change. his decisionmaking process can take several years to complete, depending on the amount of planning performed and the results of early pilot tests. herefore, this article looks at the decisionmaking process of enterprise-level academic technology adoption rather than simply documenting the comparison of possible LMS solutions to the legacy system. As San Francisco State University (SF State) went through its investigation to ind an alternative LMS solution, it soon became apparent that diferent stakeholders asked diferent questions relating to the same concepts. To organize these questions during the investigation itself, the authors put them into a matrix that served as a helpful tool for conducting a rigorous analysis. As new people joined the process, the matrix provided evidence that the decision-making process considered their interests. More broadly, the EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 309 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System analysis described in this article is applicable to any organization in need of learning infrastructure tools or systems, whether that organization is the obvious educational institution or a training division within a business, government, non-proit, etc. Ater providing the context in which SF State made its decision, the authors describe the three stakeholder perception categories, hereater called “lenses,” used to make sure everyone’s viewpoint was considered. he paper then investigates a number of issues through each lens as part of the decision-making process. he paper concludes with implications for future technology adoption decisions. Institutional Context San Francisco State University (SF State), a large, urban institution of higher education, accommodates the learning needs of roughly 30,000 undergraduate and graduate students. Roughly 80% of these students work half- or full-time, and many have additional obligations, such as taking care of children or family members, that take time away from their studies. he student body is also diverse; there is no dominant ethnic group on campus and there are over 100 languages other than English spoken as the irst language at home. he campus footprint is expanding to include several additional teaching and learning locations throughout the San Francisco Bay Area. hese conditions necessitate an atmosphere of lexible learning at SF State. Since the mid-1990s, SF State faculty and students have used a learning management system (LMS) to supplement face-to-face courses with online resources, activities, and assessment strategies, and to facilitate hybrid courses (that are partly online) and fully online courses. Ater careful analysis of the best LMS solutions available at the time in 1997, SF State made a transition from an earlier, vendor-based product to the Blackboard LMS. While that transition constituted a signiicant change, it impacted a smaller portion of the faculty and students than it would today. he Center for the Enhancement of Teaching (CET) at SF State put a great deal of energy into the move to Blackboard (which today is considered the legacy LMS). hese initiatives included both technology adoption and efective learning workshops. By academic year 2000-2001, almost one-third of SF State faculty and students used Blackboard, ranging from providing course-related documents for face-to-face classes to conducting fully online courses. he campus teaching and learning community faced serious LMS service quality issues during the 2002 to 2003 period. hese involved a mix of operational and sotware quality issues that SF State attempted to resolve under the contractual constraints of the California State University (CSU) system-wide contract with the Blackboard LMS vendor. By December 2003, the SF State Academic Technology team organized a series of focus groups to assess whether these issues warranted campus exploration of alternative LMS solutions. Approximately 40 faculty members participated in these meetings, and the response was airmative. he AT team identiied three options for how to provide LMS services to the university. he irst option was to host the legacy LMS at the SF State campus, rather than paying the vendor to host it as an application service provider. he campus did not seriously pursue this solution, as it would only address issues with service and issues with the product would still remain. he second option involved investigating other proprietary, vendorbased LMS products. hese vendor-based solutions were generally seen as disadvantageous 310 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System because SF State would remain captive to vendor responses to enhancements, customizations, and bug ixes. he third option was to look at open source sotware (OSS) LMS solutions. If the products had suicient feature sets and were supported by a sizable community of users and developers, then an open source LMS solution could help SF State overcome the captivity associated with commercial vendor solutions. Ater initial investigations of equal numbers of commercial and open source products, the academic technology staf concentrated on the third LMS option, which ultimately led to the Moodle evaluation described in this article. At the time (spring 2004), there were approximately 1,000 known Moodle sites and 10,000 registered users on the Moodle.org community website. hese metrics indicated an active community of support, which relates to the product’s potential long-term viability and support structure. Exploratory projects, both vendor-driven and open source, posed a risk of waning interest, but Moodle appeared to have gained the critical mass necessary to overcome this risk. From an administrative vantage point, a global community of users (and developers) provides numerous tangible and intangible beneits. For example, at the time of the decision to investigate Moodle further, the New Zealand Open Source Virtual Learning Environment project successfully used Moodle with 30,000 learners. Since that number was equivalent to the size of the entire SF State student body in spring 2004, the AT group conidently anticipated that Moodle would scale for the SF State campus. Within the CSU system, Humboldt State was another early investigator of Moodle for their campus LMS. Both Humboldt State and SF State selected Moodle over Sakai, another OSS product that held tremendous promise, because it was a more mature product at the time of the investigation. Problems with the legacy LMS dictated that the campus could not risk waiting to see if and how the Sakai project developed successfully and on-schedule. he social constructionist philosophy behind Moodle was also a very attractive aspect from the instructional technology vantage point shared by many in the AT group. SF State successfully rolled out its exploratory Moodle LMS to approximately half the faculty and student body between 2004 and 2006; both Blackboard and Moodle were supported on the SF State campus during this time of parallel operation (ETAC, 2006). Efective fall 2007, Blackboard was completely phased out and Moodle became the sole LMS serving the university community. he evaluation that was undertaken over the next several years is described in this paper, and related to the theoretical literature on technology adoption. he SF State director of academic technology (Beers, 2010) recently emphasized the success of the Moodle LMS to the Academic Senate by citing the following student usage statistics for fall 2009: • 95% of 30,000 students and 90% of 1,600 instructors used Moodle for one or more courses, through which those students: ▸ Completed 334,504 quizzes. ▸ Submitted 164,026 assignments. ▸ Posted 114,099 forum comments. ▸ Watched 20,767 streamed lectures captured in CourseStream system. • 1,500 students also built electronic portfolios. EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 311 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System At times that success has presented challenges for the Academic Technology staf responsible for maintaining Moodle and supporting its users. For example, during peak periods such as inals week, there were as many as 16,000 quiz attempts in a twelve-hour period—in addition to load generated by students uploading written assignments, posting forum comments, and checking their grades. With each challenge, the SF State team has moved the Moodle system forward with respect to scalability, usability, or both. SF State has worked with other campuses within the CSU system to help them pilot, adopt, implement, or maintain their own Moodle instances. In academic year 2010-2011, SF State took a leadership role in the newly formed CSU Moodle Common Interest Group, a consortium of Moodle campuses. Moodle and Learning Management here is a rich literature on technology adoption. Rogers (1962) produced the seminal work, Difusion of Innovations, in which he identiies ive key perceptions that inluence technology adoption: relative advantage, compatibility, complexity, trialability, and observability. Numerous other studies (Ostlund, 1974; Tornatzky & Klein, 1982; Davis, 1989; Moore & Benbasat, 1991) have followed suit with reinements. Key studies from LMS adoption literature (Harasim, Hiltz, Teles, & Turof, 1995; Anderson, Varnhagen, & Campbell 1998; Wilson & Stacey, 2003) partially overlap with the technology adoption literature since both sets involve adoption of information systems. Based on previous experiences with LMS vendor tools and support, SF State determined that lexibility and extensibility were critical considerations. Flexibility was loosely deined as the ability to modify and ix the LMS, while extensibility was deined as the ability to customize the LMS for current use as well as plan for unforeseen future needs. Open source LMS solutions provide both options. Brandon Hall (2007, para. 3) described Moodle as “the best-known open-source LMS.” When the academic technology team began its investigation in 2004, Sakai was the best-known solution. However, early tests with Sakai showed that it had not yet become feature-parallel to the legacy LMS. Rosen (2006, para. 8) described Moodle as more lexible than vendor products, as well as Sakai: So the diference is lexibility versus heaviness. As an Open Source project Sakai will be able to take advantage of some of the power of the open source paradigm, but its heavier architecture may mean that it will always be closely compared to Blackboard and WebCT (while Moodle moves ahead into fresher waters). Stakeholder Lenses Complexity in the mission-critical technology adoption decision at SF State emanates from the LMS infrastructure attributes and from the broad set of stakeholders afected by the infrastructure. For example, the LMS infrastructure has capabilities, limitations, risks, and implications that need to be clearly understood. Diferent stakeholders may view the LMS infrastructure diferently based on their ailiated roles. How someone evaluates any given issue or criterion related to the technology adoption decision partly depends on his or her vantage point in the organization, or the lens through 312 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System which he or she views the decision. As noted above, SF State identiied three primary lenses through which technology decisions would have to be viewed: the teaching and learning lens (primarily faculty); the technology management lens (primarily technology support staf); and the university administration lens (administrative decision-making staf). hese lenses are used to identify a wide range of concerns rather than to pigeonhole any particular stakeholder into a single viewpoint. On the contrary, some stakeholders will use multiple lenses to fulill their multiple responsibilities. For example, university administrators and academic technology managers frequently come through the ranks of faculty prior to assuming their university leadership or management responsibilities. • he teaching and learning lens addresses the core purpose of the learning management system—to help instructors and students improve learning. he LMS inluences how learning activities are created, how the instructor facilitates interactions, how the instructor provides learning resources, and inally how the instructor assesses learning. • he technology management lens relects the operational support required to maintain the learning management system, such as providing training and technical support for students and faculty, making the LMS universally accessible, administering sotware components, evaluating and potentially modifying LMS functionality, and inally interacting with the LMS vendor or community. • he university administration lens relects the executive level view of the learning management system within the university. his view includes strategic concerns such as teaching and learning (overlapping with teaching and learning lens), policy development, impact of LMS on matriculation, risk exposures, costs (e.g., total cost of ownership, return on investment), and other high-level considerations. hough these three lenses were developed and used independently by academic technology practitioners at SF State, they are consistent with the literature on LMS adoption decisions. Coates et al. (2005), for example, state that LMS adoption decisions deal with interrelated educational, technological, and administrative issues that clearly map to the three lenses identiied in this paper. Landon, Henderson, and Poulin (2006) asked MIT’s peer institutions to answer questions about choosing an LMS. hey put the reported key drivers into three categories—systems administration drivers, organizational drivers, and pedagogical drivers—that mirror SF State’s lenses. Evaluating an LMS through the Lenses Once the Moodle evaluation gained momentum and included external activities, such as the CSU Summit on Open Source LMS solutions in November 2005 and early discussions of the research documented in this paper, the list of evaluation issues and three lenses were made explicit for discussion purposes. his ex ante or in-progress consideration of teaching and learning issues is apparently somewhat unusual in light of the predominantly ex poste consideration that Coates et al. (2005) argue is given to many LMS adoption decisions. Having said that, the characterization of issues in this paper beneits from ex poste relection during the research that led to writing this paper. EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 313 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System Issue by Issue Each evaluation issue is described here, some in groups for conciseness. Given that speciic issues will frequently look diferent based on the lens through which the stakeholder views them, we characterize the issues with representative questions that a stakeholder would ask. We do this to ensure that the analysis will be useful to other institutions seeking guidance. Issues are not listed in order of importance since that depends on the stakeholder view. Stakeholder prioritization of these issues is addressed explicitly in “Prioritization of the Issues” below. Appendix A collects questions that diferent stakeholders might ask with respect to each issue. Accessibility Accessibility has emerged as a key issue for CSU campuses, both in the classroom and for online teaching and learning. he CSU Chancellor’s Oice issued Executive Order 926 to express its responsibility related to accessibility. It then began the Accessible Technology Initiative to help the campuses reach speciic accessibility goals. However, legal concerns are not the primary motivation for addressing accessibility. Universal design for learning principles dictate that everyone should have equal access to course materials and activities. hese principles—“teach every student”—accommodate everyone, including people who have disabilities, speak English as a second or third language, have limited Internet connectivity, or have diferent learning preferences. hrough the teaching and learning lens, as well as the university administration lens, accessibility ensures that persons with disabilities can use all LMS resources. Speciic questions that a teaching and learning stakeholder might ask include: Can instructors and students with disabilities use the LMS to accomplish all teaching and learning tasks? Do instructors and students have access to the necessary assistive technology tools and information to make the LMS useful? Questions that a university administrator might ask include: How can the university prevent lawsuits related to unequal access to LMS? What types of accessibility information must be disseminated to the instructors and students with disabilities? To technology management stakeholders, the issue of accessibility can be interpreted as one in which the technology is the focus. Representative questions include: Does the LMS work with all browsers and platforms? Is it feasible to modify the system to improve accessibility? Do campus labs have stations with the necessary assistive technology tools and information to make the LMS useful? During the investigation of alternative LMS solutions, SF State conducted an accessibility study of Blackboard and Moodle. Staf from Academic Technology and the Disability Programs and Resource Center chose not to rely solely on the results from a validation tool such as Bobby or WebXACT. Using a scripted process, they asked students who regularly use assistive technology, such as Dragon Naturally Speaking for speech recognition or the JAWS screen reader, to complete common academic tasks within an LMS. he students performed tasks related to content (e.g., download a ile), interaction (e.g., participate in a forum), and assessment (e.g., take a quiz) in parallel courses established in both LMS solutions. While the number of accessibility study participants was not statistically signiicant, the preliminary results showed that Moodle was easier to use for assistive technology users. 314 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System Availability of Support For instructors using the LMS, the community of interest is the set of instructors using a similar tool or teaching similar material with whom ideas and materials can potentially be shared. For students, the community is likely within a given class, college, or the university campus. So the key question for those viewing the LMS through the teaching and learning lens is whether the LMS fosters a sense of community amongst peers. Is there anyone available, either in-house or in an external support community, who can help users learn the LMS and resolve issues if they arise? For technology management stakeholders, the in-house questions to address include: Do we have the staf and resources to provide 24/7 support for the LMS? What contingency plans must we establish to deal with a server crash if no one is on campus? Can we train existing staf with this LMS, or will we need to acquire new staf? What outside resources are available to help when in-house resources require assistance? When considering external community support: Can the community provide supplemental or backup technical support for operations as well as modiications or extensions to the LMS? Community members may be users or third-party service providers, and in the case of an open source LMS such as Moodle, the community also includes those volunteers that actively participate in the development and maintenance of the system. University administrators share much with the teaching and learning and technology management stakeholders here. What is the appropriate number of staf members to support the teaching and technology aspects of mission-critical LMS use, based on number of actual users on campus? Can the university competently support the LMS? Are there related or similar universities using a comparable LMS with whom we can establish virtual communities for faculty (if not students) that span campuses? Is there support of any type (vendor, community, other) to which staf can turn when they need additional resources? Is there the budget to leverage available support? How will an open source LMS be supported since there is no single vendor “standing behind” the LMS product? Worldwide, there were more than 24,000 active Moodle sites in 175 countries (Moodle, 2007b) when the legacy LMS contract was set to expire, so the potential for external community support was there. By 2011, that number had doubled to over 49,000 sites in 212 countries. Closer to home, San Francisco State engaged with other CSU campuses and several Northern California community colleges to create the Digital Teaching and Learning Consortium (DTLC). he DTLC acted as an umbrella organization that helped coordinate common academic technology projects, starting with collaboration around Moodle and electronic portfolios. Each stakeholder group derived beneit from these eforts. For example, when SF State shared its print-based training materials with a community college, the community college then used them to create podcasts, or audio versions of the materials, which also beneited SF State faculty. SF State and another CSU campus also collaborated on grant-funded work on Moodle projects. his local community provided another layer of support beyond the global Moodle community. In 2010-11, the CSU system formed a more oicial Moodle consortium, the CSU Moodle Common Interest Group, in which SF State has played a leadership role. EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 315 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System Customizability, Extensibility hrough the teaching and learning lens, the customizability and extensibility issues can be understood through the following line of questioning: Is it possible for instructors and students to suggest changes or new functionalities to help accomplish instructional goals? To what extent can the LMS be customized to suit the heterogeneous needs of the campus learning community? For example, can the online content in an engineering course be organized diferently than the writing portfolios in an English writing course? Can the LMS be linked to other teaching and learning tools, such as the online library system? For technology management stakeholders, these are representative questions: Can programmers or system administrators make appropriate changes to sotware to meet educational needs expressed by instructors and students as requirements evolve? Will customizations of open source create a branch that breaks too far away from the core, isolating the university from external support? University administrators may have a more strategic and risk-oriented view of LMS customizability and extensibility. For example, in anticipation that campus needs will evolve over time, how much lexibility and freedom does the campus have in enhancing the LMS to serve unanticipated needs? In terms of risk assessment, there are upsides and downsides to local modiications to the LMS. Beneits include being able to tailor the LMS to local requirements, plus there are likely motivational factors that will positively inluence staf retention. However, there are potential downsides to this capability too. Downsides include the potential inability to meet unreasonable expectations and the risk of customizing the local LMS so much that the university can no longer use future updates to the system. Quality, Reliability, Scalability, Security, and Stability For teaching and learning stakeholders, is the LMS reliable so that instructors can be assured that students will have access to course materials at all times? Can large classes be conducted in the LMS as easily as small classes? Are there features that ease block uploading or downloading of information? Are there adequate information security controls in place to ensure that suitable privacy is maintained for student data as well as for instructor data? From the perspective of teaching and learning, quality is addressed by ease of use (which is treated in the next subsection) and reliability. Stability is also addressed by reliability. From the perspective of technology management, will the LMS be up and available 99.9% (or whatever availability measure is targeted)? If the system is hosted on campus, how oten should the academic technology team require maintenance windows? What are the most stable underlying platform and operating systems? Can the LMS technology and support organization scale to serve the needs of a campus of our size, and can it expand if we grow? Does the LMS design require load balancing to handle high demand? Do certain LMS features or functions fail with high volume use? At what point should the application, data, and database layers be given their own server(s)? How can the academic technology team make the system scalable if adding more hardware is not an available option? What are the symptoms of inability to scale? (e.g., Frequent system crashes? Slow system performance? Inadequate response to instructor or student users? Resource issues?) What security questions do the team need to ask the vendor if it is hosting the LMS? How susceptible 316 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System are the LMS and supporting infrastructure systems to security breaches? Is there adequate support to rapidly report and address security vulnerabilities? For university administrators: Does the product relect the high standards of our organization? Will instructors and students feel empowered and pleased to use the LMS? Can the LMS technology and support organization scale to serve the needs of a campus of our size? Will the LMS sotware require additional costs for hardware to make it truly scalable? What eforts need to be coordinated among various centralized and decentralized technology staf (e.g., lab managers, college technology staf) to promote good security practices across the campus? Will the LMS help us comply with all suitable privacy and safeguard regulations? If the LMS does not meet stability expectations, how will administration address vocalized concerns? In 2005 and 2006, SF State undertook scalability tests for the Moodle LMS that demonstrated successful operation with almost 10,000 students, each of whom were enrolled in at least one course. A list of other institutions that are successfully running large Moodle installations is available at the Moodle website (Moodle, 2007a). SF State migrated completely to Moodle as of fall 2007. During the fall 2007 term, Nearly twothirds of the 1600 total faculty (~66.50%) and more than three-quarters of 30,000 total students (78.33%) used it for one or more classes. Overall, this equated to use in 1,525 out of 11,000 total class sections (13.85%). Long-Term Viability For teachers and learners: Will we need to learn yet another LMS again in the near term, or will this system stick around for a long enough period to warrant time investment? For the technology management team: Will we need to initiate yet another LMS investigation process again because this product does not have market sustainability? Long-term viability is very important for minimizing disruption to campus teaching and learning, and avoiding wasting valuable academic technology resources on evaluating and deploying a new system. For university administrators: Will the campus soon have to go through the investigation process again because the LMS product may disappear? Long-term viability helps in academic technology infrastructure planning and budgeting. It also aids in keeping academic technology staf motivated, since they are maintaining and building relevant market skills. Blackboard announced its intention to acquire and merge with its competitor, WebCT, on October 12, 2005—the same day the CSU system was facilitating an LMS summit in Southern California to assist campuses considering the need to change products. he merger/acquisition demonstrated that even the second largest LMS provider could not maintain long-term viability in the market. Since that time, Blackboard has also acquired Angel, another leading LMS vendor, and the two largest Web conferencing solutions (Elluminate and Wimba) as well. Mature open source products like Moodle and Sakai may actually have greater long-term viability than several commercial products, as their communities or segments therein could maintain the code independently regardless of potential acquisition-like activity. EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 317 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System Functionality For teachers and learners: Do the LMS features allow instructors and students to perform tasks as they would like, or must instructors and students modify tasks to it system limitations? To what extent does the LMS provide a set of tools that enables instructors to establish a vibrant and efective online community outside the classroom? In addition to the rudimentary ile upload and download capabilities, are there advanced community features such as forums, wikis, instant messaging, RSS feeds, and gradebooks? Is there consistency? For example, does restricting access to a selected resource restrict access in all situations? For technology managers: How easy is it to develop and implement faculty and student training to use the technical features of the LMS? To what extent do the features meet or exceed requirements expressed by instructors? If not, then is there suicient capability to add on features to provide such functionality? To what extent is there LMS support to add functionality as requirements evolve? For university administrators: Can any of the features be used to provide institution-level reporting for accreditation or for generating reports useful to administrators of the LMS and executive management in the university? At SF State, Academic Technology regularly conducts interviews with faculty users, as well as faculty who have stopped using the LMS, to determine which features meet or do not meet teaching and learning needs. It can Feasibility of Integration his issue has to do with how well the LMS will integrate with a university’s other online information services. For teachers and learners: Are students automatically registered in LMS course shells when they register for a course through the oicial university registration system, or must they separately register for an LMS course? Are course shells automatically created for professors, or must instructors request them? Are there linkages between the LMS content that professors post and the library’s electronic databases? For technology managers: What level of efort is required to integrate the LMS with student management databases, campus authentication processes, or other academic technology solutions? How easy is it to complete each of these integrations? What level of efort is required to implement university policies (e.g., if a student drops a course)? Does the LMS leverage current and standards-based technology to ease integration today and in the future? For university administrators: Can the LMS be integrated with student management databases, campus authentication processes, or other academic technology solutions? If so, can the campus perform these integrations in-house or does it need to hire outside consultants? Can information from the LMS be made available and other reporting tools used to administer the university? Can LMS integration be used to implement university policies? Early in the adoption process, SF State integrated the Moodle LMS with the Student Information Management System (SIMS) database to replicate the daily updates that were being done with the legacy LMS. his kept the Moodle student enrollment synchronized with university class enrollments throughout the semester as students added and dropped classes. Prior to this integration, students needed to explicitly enroll in each Moodle course shell associated with an SF State class. While this integration is now considered essential, 318 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System it was not as common in 2005 and 2006. Additional integration steps have been completed since then, such as linking Moodle with the College of Behavioral and Social Sciences’ Digital Information Virtual Archive (http://diva.sfsu.edu); a commercial lecture capture solution (Echo360); a plagiarism detection service (Turnitin); .and a library services feature that allows faculty to select journals and books for students to consume as part of a class. Ease of Use For teachers and learners: Is the LMS environment intuitive enough for instructors and students to complete tasks with little or no training? For technology managers: Is the LMS written in a common language for which it will be easy to ind qualiied programmers? Is the LMS easy to install and maintain? Is there a graphic user interface (GUI) for administrators? What resources need to be allocated to training users? For university administrators: How accessible is the aggregated data that is collected? Is it easy to extract and summarize logs (e.g., usage levels) and other data for return-oninvestment analysis? What is the cost of training users and technology staf? Open Source or Proprietary? Strictly speaking, this is more of an issue for the technology management and university administrators than it is for the teaching and learning stakeholders since the university administration lens will capture the users’ view. Users primarily focus on the information services availed by the LMS rather than on the underlying technology used to provide that information service. In practice, though, some faculty may pay attention to whether the underlying product is open source or proprietary. his may be because the open source LMS provides for the possibility of technical faculty suggesting modules or features to add. Another reason is that the open source vs. proprietary debate sometimes becomes an ideological one. A third reason is that faculty may recognize the relatively low procurement and licensing fees for the LMS sotware and seek reallocation of those funds to teaching enhancement. For technology managers: Can the open source LMS be used “right out of the box” (i.e., without much modiication)? Does the current technical team have the skill sets to support the system? If the university is currently paying the vendor to host the legacy LMS, is it possible to pay a diferent group to host the open source alternative? How can the campus best take advantage of the control gained by using open source? For university administrators: What are the beneits and hidden costs associated with any open source product? Recognizing that the cost structure of procuring and maintaining an open source product is diferent than that of a proprietary product, how should the campus reallocate the money? Prioritization of the Issues he previous section described the evaluation issues, but neglected any prioritization of them; this was intentional, since each set of stakeholders would likely prioritize them diferently. In this section, we clarify the bias of each lens and then prioritize the issues based on this bias. It is up to the university to seek the appropriate balance between the EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 319 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System three lenses when making the ultimate decision to adopt an LMS. To highlight the bias, we raise two questions for each stakeholder: 1. What are the beneits of viewing the LMS adoption decision predominantly from the perspective of your own lens? 2. What pitfalls may arise if concerns of either or both of the other two stakeholders dominate the decision? Teaching and Learning Teachers and learners will say that their lens should take priority because the university exists to serve teaching and learning, and technology management and university administration are overhead that exist to serve teaching and learning. herefore, basing the decision predominantly on teaching and learning will ensure that the LMS furthers the core mission of the university. If the other perspectives are allowed to dominate, the technology management stakeholders may focus too heavily on interesting technology (i.e., bells and whistles) or technology that is easy to support, without meeting teaching and learning needs. And university administration may focus too much on costs or vendor relationships. Prioritizing the evaluation issues into three levels: • Key issues for teachers and learners will be accessibility, ease of use, functionality, quality, reliability, scalability, security, stability, and support. • Less important but necessary issues to be considered are availability of support (community or otherwise), customizability, extensibility, integration with other online information services at the university, and long-term viability. • Whether the product is open source or proprietary will be the least important consideration for teachers and learners. Technology Management To the irst question above, technology management stakeholders will respond that the LMS is mission-critical infrastructure that needs to be managed properly to support educational needs. Paying attention to technology management needs will ensure that this service can be provided reliably. Technology management staf have the concern that a predominantly teaching and learning focus may not address what is technologically feasible or what its within university guidelines. Technology management staf may also have the concern that the university administration lens could focus too heavily on short-term cost and not enough on total cost of ownership or impact on the tech management team. Prioritizing the evaluation issues into three levels: • Technology management staf ’s key issues are accessibility, ease of use, availability of support, customizability, extensibility, integration with other online information services at the university, functionality, quality, reliability, scalability, security, stability, support. 320 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System • It will be less important but still necessary to consider the long-term viability of the product and whether it is open source or proprietary. • No issues are irrelevant to the technology managers. University Administration University administration will likely conclude that serving the teaching and learning community is key, but that cost and strategic direction of the university are also important. However, administration may also have the concern that a strictly teaching and learning focus may not relect technology management requirements or university cost constraints. And from an administration standpoint, the technology management lens is important but not all-inclusive, as it may not take into account university strategic plans or operational requirements. Technology managers may pull the campus in directions it is not prepared to travel. Prioritizing the evaluation issues into three levels: • University administration and technology management staf will share many key issues including accessibility, ease of use, customizability, extensibility, integration with other online information services at the university, functionality, long-term viability, quality, reliability, scalability, security, stability, support. • he university administration will also need to consider whether or not support is available and whether the product is open source or proprietary. • No issues are irrelevant for university administration. Evaluation Against the Rogers Dimensions We previously characterized each of the SF State evaluation issues through three separate lenses: teaching and learning, technology management, and university administration. With that in mind, we now demonstrate that these are consistent with – but more detailed and speciic to an LMS – the ive technology dimensions proposed in the seminal work (Rogers, 1962): relative advantage, compatibility, complexity, trialability, and observability. We do this to clarify that the SF State issues are useful in practice and consistent with prior theoretical work. Relative Advantage refers to the perceived beneits of a technology over earlier technologies or ideas. Many of the SF State evaluation issues address aspects of relative advantage: accessibility, community support, customizability, ease of use, extensibility, functionality, feasibility of integrating LMS with other online information services at the university, quality, reliability, scalability, security, and support. Compatibility refers to consistency with practices and norms. Accessibility is an aspect of compatibility in that SF State must serve people with disabilities. Community support is an aspect of compatibility because SF State has a mission that includes interactions with the SF Bay area. One could argue that many of the other SF State evaluation criteria (e.g., ease of use, quality) support compatibility, but the reasoning behind that is captured by the relative advantage construct. EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 321 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System Complexity refers to ease of use or learning a new technology. Ease of use and support required obviously address this, but so do customizability and extensibility since they can be aided or hindered by complexity. Another grouping of issues, centered around quality (quality, reliability, scalability, security, stability), also relates to complexity in that more complex systems will hinder technology support staf ’s ability to achieve high levels of these criteria. Trialability refers to the having the opportunity to experiment with a technology before committing to it. his is indirectly addressed by the open source vs. proprietary product discussion, since the former typically entails a lower upfront sotware cost. When a demo version of a vendor-centric LMS is available, that partially addresses trialability. But it is very unlikely that the extent of trialability of a vendor-centric LMS will ever reach the level of an open source system. SF State’s operational evaluation of an LMS over a three-year period was only possible because it was open source. Observability refers to the extent to which features and beneits are visible. Over a very short period of time, ease of use and functionality address this. Over an extended period of time, most of the SF State evaluation issues address observability. For example, the extent to which an LMS is accessible, customizable, or scalable can be observed by the stakeholders over a period of time. hough the speciic comparative analysis of Moodle vs. Blackboard is beyond the scope of this paper, one such analysis has been undertaken through the teaching and learning lens. he Educational Technology Advisory Committee (ETAC) at SF State serves as the liaison between faculty and other stakeholders on campus. his committee produced a report (ETAC, 2006) that included the application of Rogers’ ive attributes to the two systems. Cultural Changes in Teaching and Learning Not all cultural shits on a university campus are positive, of course. Coates et al. (2005) argued that one of the more persuasive reasons for LMS adoptions in general is to provide university administrators with “a means of regulating and packaging pedagogical activities by ofering templates that assure order and neatness, and facilitate the control of quality.” Further, Smart and Meyer (2005) described challenges associated with changing from one LMS to another. If an organization has acculturated itself around one system, the users will not want to lose all their work or any familiar functionality in moving to another. he academic technology team at SF State used the aforementioned focus groups to determine what functionality faculty would not want to lose when moving to another LMS. Once the campus started Moodle pilot tests, the same team worked with a computer science graduate student to create a conversion utility so instructors would not lose the work they had done in the legacy system. As it turned out, at least half of the faculty members chose to start over from scratch, rather than convert their legacy LMS course to Moodle. To capitalize on this period of change, the academic technology team asked faculty to think about changing the way they used the LMS to supplement their teaching. Namely, they asked faculty to take this mentality a step further by going beyond using the LMS solely for document delivery. Many faculty members began including interaction or assessment activities to supplement their face-to-face instruction. 322 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System During the transition from Moodle pilot tests to scalability tests and exclusive use of Moodle as the campus LMS, the number of transition faculty was closely matched by faculty who had never used an LMS at all. Some had heard from their peers that Moodle, a learning-centered environment, was easier to use than the tool-centered legacy system. Since there were fewer barriers to entry than the legacy LMS, these novices to online coursework wanted to add new elements to their teaching. O’Quinn and Corry (2002) identiied concern about workload as the second highest factor keeping traditional classroom faculty from engaging in online work. With reports that Moodle took one-third the time to set up, the threshold had become low enough for these novice users to enter the world of online teaching. Surprisingly, one of the most frequently used activities by both veteran and novice instructors did not even exist in the legacy LMS. Wikis had become a frequently utilized Moodle activity, ater discussion forums, assignments, and quizzes. Instructors were able to facilitate several variations of group work, such as the entire class generating a literature review, or small teams posting their project summary (Mindel & Verma, 2006). One novice faculty member reported wanting to use wikis again, even ater having diiculties related to (a) lost formatting when copying and pasting from Microsot Word and (b) having MLA citations become hyperlinks due to the use of square brackets, a wiki convention. Conclusion and Implications Organizations that are in the middle of, or just beginning, the decision-making process about adopting a new or diferent learning management system will ind using the stakeholder lenses to be helpful. hrough this approach, San Francisco State University was able to identify the issues that required further research, to address previously unmet needs, and to generate greater buy-in among the interested parties. With everyone on board from the beginning, the transition between LMS solutions was very successful. To maintain this level of success, the campus continued to engage people from the three lens categories as new decisions were made about upgrades, feature additions, and integrations with other applications. Further, the campus can use the same process if and when it must make other adoption decisions for other technologies, such as electronic portfolios, online meeting spaces, or online course evaluations. To do so may require adding or subtracting a couple categories from the original matrix, but by and large the process will remain intact. Since every institution is diferent, the authors recommend using the matrix of questions as a starting point. Organizations should ask stakeholder groups to modify the questions regarding each issue and to add issues, if necessary. Answering the questions themselves can be accomplished in diferent ways. he project leaders can facilitate a face-toface process, such as a series of large meetings or focus groups. hey can set up an asynchronous activity, such as a wiki, discussion forum, or online qualitative survey. For best results, organizations may want to use some combination of the two. Acknowledgement he authors would like to thank groups involved in the multi-year LMS evaluation at SF State—Academic Technology (formerly Center for the Enhancement of Teaching) for performing the work related to establishing the pilots and following studies, hosting and maintaining the Moodle LMS, supporting the faculty and student users, and conducting EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 323 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System the campus evaluation; the faculty-led Educational Technology Advisory Committee for reviewing the evaluation results at regular intervals and providing valuable feedback; the Disability Programs and Resource Center for assistance with accessibility study; the early adopter faculty members who helped move the project forward through word-of-mouth recommendations; the Instructional Technologies Department and a Usability Studies student from UC Berkeley for assistance with the heuristic evaluation; and the administrators who supported the LMS investigation when things worked and when they did not. Authors Joshua L. Mindel, Ph.D., Assistant Professor of Information Systems, College of Business, San Francisco State University, 1600 Holloway Avenue, San Francisco, CA 94132. Email: jmindel@alumni.cmu.edu Kevin Kelly, Ed.D., Manager, Online Teaching and Learning/Media Distribution and Support—Academic Technology, San Francisco State University, 1600 Holloway Avenue, San Francisco, CA 94132. Email: kkelly@sfsu.edu 324 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System References Anderson, T., Varnhagen, S., and Campbell, K. (1998). Faculty adoption of teaching and learning technologies: Contrasting earlier adopters and mainstream faculty. he Canadian Journal of Higher Education, 2-3, 71-98. Beers, M. (2010, February). he role of academic technology in assisting the online course evaluation policy and the online teaching policy. Presentation given to the San Francisco State University Academic Senate, San Francisco, CA. Black, E. W., Beck, D., Dawson, K., Jinks, S., & DiPietro, M. (2007, Mar/Apr). he other side of the LMS: Considering implementation and use in the adoption of an LMS in online and blended learning environments. TechTrends, 51(2), 35-39, 53. Coates, H., James, R., and Baldwin, G. (2005). A critical examination of the efects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19-36. Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), 319-339. ETAC. (2006). Viability of an Exclusive iLearn-based LMS Environment. Educational Technology Advisory Committee, San Francisco State University, retrieved on 13 April 2007 http://www.sfsu.edu/~etac/ETAC_Report-iLearn-Bb-inal.pdf. Fisher, S. (2006). Open technologies and resources for the humanities - and cooperative consequences. Arts and Humanities in Higher Education, 5, 127-145. Gilbert, S. W., Green, K. C. (1997). Moving information technology into the classroom. In Linda Enghagen (Ed.), Technology and higher education (pp. 25-56). Washington, DC: National Education Association. Hall, B. (2007, March). Five open-source learning management systems. Chief Learning Oicer. Retrieved from http://www.clomedia.com/content/templates/clo_article. asp?articleid=1714&zoneid=190 Harasim, L., Hiltz, S. R., Teles, L., & Turof, M. (1995). Learning networks: A ield guide to teaching and learning on-line. Cambridge, MA: he MIT Press. Landon, B., Henderson, T., & Poulin, R. (2006, July 19). Massachusetts Institute of Technology & Edutools: Peer Comparison of Course/Learning Management Systems, Course Materials Life Cycle, and Related Costs. Boulder, CO: Western Cooperative for Educational Telecommunications (WCET). Mindel, J. L., & Verma, S. (2006). Wikis for teaching and learning. Communications of the Association for Information Systems, 18, 1-23. Moodle. (2007a). Moodle Documentation: Installations 10000 plus. Retrieved April 13, 2007, from Moodle.org: http://docs.moodle.org/en/Installations_10000_plus EDUCATION FOR A DIGITAL WORLD 2.0 VOL.1 325 Part Two: Implementing and Managing eLearning Stakeholder Issues for Adopting a LearningManagement System Moodle. (2007b). Moodle Sites. Retrieved April 13, 2007, from Moodle.org, http://moodle. org/sites/ Moore, G. C. & Benbasat, I. (1991). Development of an instrument to measure the perceived characteristics of adopting an information technology innovation. Information Systems Research, 2(3), 192-222. O’Quinn, L. & Corry, M. (2002). Factors that deter faculty from participating in distance education. Online Journal of Distance Learning Administration, 5(4). Retrieved from http://www.westga.edu/%7Edistance/ojdla/winter54/Quinn54.htm Ostlund, L. E. (1974). Perceived innovation attributes as predictors of innovativeness. Journal of Consumer Research, 1(2), 23-29. Rogers, E. (1962). Difusion of Innovations. New York, NY: Free Press. Rosen, Z. (2006, Jan). Comparison of Sakai vs. Moodle. Educause CIO Archives. Retrieved 04 May 2007 from http://listserv.educause.edu/cgi-bin/wa.exe?A2=ind0601&L=cio&T= 0&F=&S=&P=20617 Smart, K. A. & Meyer, K. A. (2005). Changing learning management systems: Lessons learned. Educause Quarterly, 28(2), 68-70. Tornatzky, L. G. & Klein, K. J. (1982). Innovation characteristics and innovation adoptionimplementation: A meta-analysis of indings. IEEE Transactions on Engineering Management, 29(1), 28-46. Wilson, G. & Stacey, E. (2003). Online interaction impacts on learning: Teaching the teachers to teach online. In G.Crisp, D.hiele, I.Scholten, S.Barker & J.Baron (Eds), Interact, integrate, impact: Proceedings of the 20th annual conference of the australasian society for computers in learning in tertiary education (pp. 541-551). Adelaide, 7-10 December 2003. 326 EDUCATION FOR A DIGITAL WORLD 2.0 VOL. 1