Technology and Human Issues in Reusing Learning Objects

Betty Collis and Allard Strijker

University of Twente
7500 AE Enschede
The Netherlands

collis@edte.utwente.nl
strijker@edte.utwente.nl

Abstract: Reusing learning objects is as old as retelling a story or making use of libraries and textbooks, and in electronic form has received an enormous new impetus because of the World Wide Web and Web technologies. Are we at the brink of changing the "shape and form of learning, ... of being able to truly increase and improve human learning and performance" (Hodgins, 2000)? We are sceptical, for human and educational reasons. One of our arguments is that human aspects not technology will constrain what will be done with learning objects. Our other argument is that the learning philosophy that seems to underlie many of the discussions and the technology relating to learning objects will limit their depth of development and impact. In this paper, we examine the life cycle of a reusable electronic learning object, including steps involved with creating, capturing, indexing, archiving, finding, wanting to use, using, revising, and maintaining it. We also explore the human issues as well as the technology-related aids in each of the above phases. We illustrate the influence of context -- higher education, corporate learning, military training, in these life cycles, together with the effect of two educational philosophies, namely those of acquisition and participation/contribution.

Keywords: Learning Objects, lifecycle, pedagogy, organizational context

1. A New World or an Old Problem?

The reuse of digital learning material has been an issue for more than two decades (Collis, 1995). In the 1980s a number of initiatives occurred with the aim of promoting the reuse of educational software outside of its original market. These came to little success for a number of reasons. One certainly was the technology of the time, with incompatibilities in operating systems and storage media forming major barriers. In addition, there were problems in awareness and access. Potential users had little opportunity to be aware of what was available or to see or try it out. The most substantial problems however were related to incompatibilities with the local context and culture of the end users (Ely, 1989; Griffiths, Heppel, Millwood, & Mladenova, 1994). The reusability of an electronic learning resource depends on its fit with the language, culture, curriculum, computer-use practices, and pedagogical approaches of the potential learners and their instructors. Making this fit has proven to be very difficult. The major reasons for this difficulty relate to the way that electronic learning objects are used in practice, which in turn are directly related to the organizational settings of those who create, label, and offer learning objects on one hand, and of those who select and use them on the other. For example in terms of the granularity of learning objects:

The size and shape of an "object" is open to each organization to define. This decision is based upon the needs, tools, processes, and business goals of the organization (Barritt, 2001) .

Granularity is just one of many issues related to learning objects that will have different meanings and implications in different organizational contexts. In the following section three different organizational contexts for the use and reuse of learning objects are briefly reviewed.

2. Organizational Contexts

Three of the major different organizational contexts for the use of learning objects are the university, corporate learning, and military training establishments. These will be contrasted in a number of ways, each of which affects the potential impact of learning objects, particularly the nature of the learning material, the dominant adopted philosophy of learning, the delivery of courses, the technology involved, and typical practices which relate to the ownership and access of learning objects.

2.1 University context

The designers and instructors of university courses are generally part of the academic staff and their freedom within course development is large. The instructor's professional identity is predominately based on his research. "In the university model, the long-term work is research, assembly, investigation, exposition, criticism, publication, and integration of knowledge?.What is most persistent in the university model is the process of knowledge production" (Roschelle, Henderson, Spohrer, & Lilly, 1997, p. 27). Within this research orientation, instructors also design, develop, and deliver courses, frequently bringing in their research into the course materials.

The instructor can choose how to structure the course in terms of organisation, course material, and assessment and will strive to integrate aspects of his current research and projects into courses. This means that research-specific and sometimes instructor-specific knowledge is used for many courses. Because of the research aspects, many courses are revised every year and upgraded with new articles and relevant material. The course materials most used are PowerPoint presentations, word-processed documents created primarily by the instructor, textbooks, copies of scientific articles, and increasingly, professional resources available via the World Wide Web (De Boer, 2003). There is very little use of educational software of a tutorial nature (see for example, Draper, 1998, who discusses the lack of use of educational software in higher education beyond its developers). Throughout higher education, there is an on-going emphasis towards the development of meta-cognitive skills, critical thinking and reflection, together with an apprenticeship into a community of professionals (Sfard, 1998); content but is no longer just acquired is to be critically applied in knowledge production and problem solving.

2.2 Corporate context

Courses offered in the corporate setting are based on demand where the demand partly comes from changes in the field and partly comes from needs identified within the organization, such as through competence-gap analyses or profiling done by the human-resources division. While some competences and their associated tasks are generic to a type of industry, many involve company-specific proprietary information and sometimes information that is tacit rather than explicit. Courses are under constant revision because of new developments in the workplace. Courses and learning objects involving generic knowledge are frequently outsourced; courses and materials with domain-specific and corporate-specific knowledge are generally created in house. In-house course resources are corporate property and individual ownership is not usually claimed; sharing and reusing resources within a discipline group or a learning centre is common. An industry exists for producing courses and learning resources and the technologies needed for creating and managing electronic learning objects (Chapman & Hall, 2001).

Typically, attending a course means going to a classroom-based setting away from work for a period of days, although other combinations involving computer-based (e-) learning can occur with the employee still in his own workplace. A recent surge of interest in anytime, anyplace e-learning has been fuelled by the rapid development of the market for LMS (learning management systems) and LCMSs (learning content management systems). However, both classroom learning and learning based on learning objects associated with LMSs and LCMSs operate separately from the informal learning that takes place via peer contacts, in-house discussion forums, and those activities which involve the use of knowledge-management tools and systems. Knowledge-management systems, are in general terms either for codification of information or personalization (Kankanhalli, Tanudidjaja, Sutanto, & Tan, 2003). A codification approach emphasizes efficient access to information while a personalization approach emphasizes knowledge sharing and facilitating contacts with others. Traditional classroom-based courses in corporate settings may include both codified and personalized approaches but "e-learning" via learning objects typically reflects characteristics of a codification approach rather than an approach that emphasizes human contacts. Blending aspects of the various types of learning in one learning event is an emerging idea (Chapman & Hall, 2001, p. 10; Collis & Margaryan, 2003).

2.3 Military context

The military context is a setting that changes slowly. The organisation is strict and has a command structure that is hierarchical. This means that all procedures are well defined and that uncertainty is minimized to exclude errors in orders and commands. This command structure is based on the fact that misinterpretation of definitions can result in casualties. The training that occurs and course materials that are developed are based on this strict form of communication and use the same predefined definitions to make terminology as consistent as possible. Courses and learning resources are highly specialized and localized. Course materials include text materials such as handbooks and manuals and also multimedia materials, particularly visual materials (videos and images). Simulators and hands-on experiences are often used (Verwijs, 1998). Courses and course materials are made by in-house teams that include subject-matter specialists, instructional designers, and multimedia designers and developers (Visscher, 2002). The trainers who run the courses with the participants are generally not part of the development teams. Trainers are not generally involved in other work besides training. Training is highly task-focused. Participants have a prescribed series of courses based on their specialist subject and attainment level although their supervisors can also request additional courses to fit particular needs (Visscher, 2002). External companies produce tools and systems such as simulations and development tools and are beginning to supply LMSs and LCMSs to military training settings.

2.4 Comparing the organizational contexts

Although many points of comparison can be made among the three organizational settings, three particular aspects are of key importance.

The first relates to the nature of the course and of reuse of learning objects within it:

In the university setting, the course is very much influenced by the instructor who in turn is relatively autonomous in his choice of instructional approach and learning resources. The instructor is professionally defined by the originality and productivity of his research. He has little or no help from others in terms of developing or delivering his course. Reuse usually relates to reuse of his own materials in different courses. In contrast, in the corporate setting, courses and learning objects are based on business needs and competence analyses. Courses are client-oriented and course developers must respond quickly to new requirements and requests. This limits the freedom of developers and of those who eventually teach the course. Suppliers of courses, learning objects, and learning-object technologies from outside the corporation stimulate the reuse setting: "aiming at what they perceive to be a much bigger market: content aggregation 'on the fly' by individual learners or training providers" (Rehak & Mason, 2002, p. 22). In contrast while "universities in the forefront of learning object development are designing templates and other systems to support academics in the preparation of learning material geared to reuse?[training company providers are creating systems] where learners indicate their personal parameters, needs, background knowledge, etc., and courses are created automatically from a database of learning objects" (Rehak & Mason, 2002, p. 22). As Rehak and Mason point out, "a number of significant implications about the nature of learning objects arise from these scenarios" (p. 22). In the military setting, courses and content are highly structured, and slow to change or adapt. Instructors have little or no say in the selection of the content of the courses they facilitate or of the learning objects involved. Reusable objects, such as repositories of photographs related to critical equipment and deployment of the equipment, are managed in a systematic way using database technology as well as other systems.

The second point of comparison relates to the relationship of learning objects to other aspects of the course:

In universities, instructor-delivered courses with lectures remain the norm but Web-based course-management systems are routinely used to support the courses throughout their cycles (De Boer, 2002). In corporate settings, there are typically two parallel but different delivery forms: a classroom with an instructor or an e-learning environment without an instructor. A mixture of these two scenarios which can be termed as "blended learning" is now appearing, but unlike the university setting, a course-management system is not typically used. Instead an LMS may be employed to deliver and track the e-learning component. The classroom-portion of the blend generally does not make use of electronic learning objects or electronic delivery systems (Rossett, Douglis, & Frazee, 2003). In the military setting, there are classroom (or practical) settings or CBT (computer-based training) but typically there is no combination of the two in a single course.

The third point of comparison relates to ownership of learning objects:

In the university setting, the instructor as creator of a learning object sees the object as his intellectual property. Many times the object will include specific aspects of the creator's own research and writing. As the creator is generally also the instructor, his willingness to share his work with others is bounded by the wish and need to retain acknowledgement of his intellectual property. In the corporate setting, in-house learning objects are company property, there to be used when needed. Concerns are high that company-specific knowledge and material remains inside the corporation and is not used to the advantage of other corporations. In the military, the need to protect state secrets also limits access to in-house learning objects and objects are copyrighted. However, reuse within a particular military-training setting is seen as desirable.

These differences lead to substantial differentiations between the roles, nature, and use of learning objects which in turn have strong influences on the choice of standards and the use of metadata. They thus shape and constrain the meaning of the Semantic Web in the different contexts. They also are associated with different philosophies of learning.

3. Learning philosophies

Underlying the differences in the instructional and delivery approaches that were discussed in the last section, are major differences in philosophy of learning. Sfard (1998) describes these as the knowledge-acquisition approach and the participation approach. Collis and Moonen (2001) extend the participation approach to include an emphasis on learners contributing objects for reuse to the overall learning experience. The participation and contribution approaches both focus on active, constructive, intentional, authentic, and collaborative activities (Jonassen, Peck, & Wilson, 1999) where these activities take place in the real world as much as possible and not within a learning object. Table 1 contrasts the different approaches.

	Acquisition	Participation
Key definition of learning:	Learning as knowledge acquisition and concept development; having obtained knowledge and made it one's own; individualised	Learning as participation, the process of becoming a member of a community, "the ability to communicate in the language of this community and act according to its norms" (Sfard, p. 6); "the permanence of having gives way to the constant flux of doing" (p. 6)
Key words:	Knowledge, concept, misconception, meaning, fact, contents; acquisition, construction, internalization, transmission, attainment, accumulation;	Apprenticeship, situatedness, contextuality, cultural embeddedness, discourse, communication, social constructivism, cooperative learning
Stress on?	"The individual mind and what goes into it" (Sfard, p. 6); the "inward movement of knowledge" (p. 6)	"The evolving bonds between the individual and others" (p. 6); "the dialectic nature of the learning interaction: The whole and the parts affect and inform each other" (p. 6)
Ideal	Individualized learning	Mutuality; community building
Role of instructor	Delivering, conveying, facilitating, clarifying	Facilitator, mentor, "Expert participant, preserver of practice/discourse" (p. 7)
Nature of knowing	Having, possessing	Belonging, participating, communicating

Table 1Comparing the Acquisition and Participation Models (Collis & Moonen, 2001, p. 22; summarised from Sfard, 1998, pp. 5-7)

In the approaches described in the right-side column of Table 1, digital learning objects can serve important roles as resources, examples, discussion foci , or the products of learning when created by the learners themselves, but do not serve as the core of the learning event. "Collaborative construction, reconstruction and negotiation of information?are powerful constructivist and generative principles that provide an alternative view of the capabilities of learning object systems for learning" (Bannan-Ritland, Dabbagh, & Murphy, 2000, p. 37). "Learners themselves function as designers using technologies as tools for analyzing the world, accessing information, interpreting and organizing their personal knowledge and representing what they know to others" (Jonassen & Reeves, 1996, p. 694).

This educational philosophy is very different from that underlying most if not all LCMSs or LMSs. Most implicitly or explicitly see a combination of digital learning objects as adequate in itself for learning and reflect the "acquisition" column in Table 1. The acquisition approach can fit with what Euler (2003) describes as the least-complex of three levels of learning, followed by individually oriented constructivist approaches where the goal is self-regulated learning; and the highest and most-desirable level: collaborative learning, participation in a community, and knowledge creation and sharing. Euler's three levels can be mapped onto the university, corporate, and military contexts as shown in Figure 1.

	1. Knowledge acquisition (Content-oriented)	2. Individually oriented, problem based	3. Collaboration, contribution, community, knowledge sharing
University: Graduate or professional levels		Professionally focused; self-responsible for complex problem solving and research
Corporate, current	Classroom courses, e-learning		Informal learning, new directions in HRD and KM
Corporate, emerging	Blends of formal and informal learning
Military	Classroom courses; simulations and other forms of CBT

Figure 1 Pedagogies related to organizational context

The "corporate, emerging" row can be seen in new models of blended learning in corporations where the integration of formal and informal learning through work-based activities can involve all of Euler's levels (Collis & Margaryan 2003). This row and the row relating to university courses at the graduate or professional level, represent the difference between learning as acquiring content, and learning?

"as a human experience laden?with emotive colouring, and nested in an intricate, ever-changing web of relationships?All learning has context, and it has historicity. In both dimensions, [it] is imbued with meaning and emotion far beyond its informational content, and it is netted in a social understanding of the world. ..It has a past and a future. It means different things to different people?.The snapshots and freeze frames of knowledge objects?are not to be mistaken for the processes of learning" (Lambe, 2002, pp. 5-6)

The context of learning and the philosophy of learning both play a strong role at each phase of the learning-object lifecycle.

4. Learning-Object Lifecycle

A learning object can be seen as going through six distinct stages in its lifecycle: Obtaining or creating, labelling, offering, selecting, using, and retaining (Figure 2). Figure 2 is repeated in the following sections to show which stage in the lifecycle is being described.

Figure 2 Stages within the learning-object lifecycle (Strijker, 2003)

For each of these stages, Why?, What?, How?, Who?, and Where? questions can be indicated from two perspectives: The human perspective dealing with the Why? and Who?, and the technical perspective dealing with the How?, What?, and Where? questions. Table 2 shows some key questions that relate to the perspectives.

Table 2 Human and technical perspectives related to Why?, What?, How?, Who?, and Where? questions

Perspective	Question	Description
Human	Why?	What is the reason for reuse? Why should humans invest time and effort during the different stages of the learning-object lifecycle?
	Who?	Who are involved in the reuse process? What roles can be identified in the different stages of a learning-object lifecycle? What are the incentives for carrying out these roles?
Technical	What?	What material is reused? What are the granularity and types of the reused material?
	How?	How is the material actually reused in terms of tools? What kind of technical support is possible and in place in the different stages of the learning-object lifecycle?
	Where?	Where does reuse place in terms of systems? What systems are available to support reuse and what services are offered by the systems during the different stages of the learning-object lifecycle?

These questions will have different answers depending on the organizational context and philosophy of learning.

4.1 Obtaining

The first stage of the lifecycle is obtaining or creating a learning object. All the following stages depend on this. Material is obtained in a digital form for easy distribution and adaptability. Also the desire for quality in terms of professional behaviour and consistency in presentation play a role. In different organisations various kinds of templates are available for example, to create PowerPoint presentations, documents, and proposals. The use of templates provides structure and can help users to create consistent pieces of material. Templates are therefore an important tool for obtaining a new object.

How learning objects are created depends on the developer. However for the answers to "Why?" create or retrieve certain learning objects, depend upon different aspects of the various learning contexts. In the university scenario the instructor wishes to supplement the textbook in a number of ways , through supporting a variety of different class activities and processes and including new developments and research in the subject area. In the corporate setting, the organization is concerned with the quality and attractiveness of learning objects, and with realizing cost-effectiveness by reducing travel expenses by replacing the instructor and classroom with learning objects. It values the efficiency of reusable learning objects particularly if managed by an LMS, if the externally produced object is editable, costs are acceptable, it is compatible with existing technical systems, and is uniform in terms of house style and branding. In the military context, the organization chooses CBT and within it, learning objects, for their efficiencies and for the standardization of self-managed learning (costs are not as much of an issue as internal consistency and localization).

Where learning objects are obtained from again varies per context. In the university, learning objects come from one's own resources, also from colleagues, projects, conferences, the Web/Internet, consortium resources, and discipline-specific portals or other forms of collections such as research papers. In the corporate context sources of objects are consortia supported by the company (see for example, PetroSkills, http://www.petroskills.com/, a consortium of companies in the oil business); external vendors, or from knowledge-management (KM) systems within the company. Searching on the Internet does not happen very often.. In the military context, almost nothing is obtained from the Internet, everything isconstructed in-house including standard manuals..These manuals are often used as learning resources so the reuse of actual manuals is the typical source of learning objects in this context.

4.2 Labelling

After a learning object is obtained, the second stage of the lifecycle is labelling the object. The most arbitrary form of labelling is just to provide a filename or subject for the learning object. Tagging tools can facilitate and support the addition of metadata to the objects. Text processors like Microsoft Word also have metadata tagging tools for documents

The use of database-oriented developers' tools makes it possible to gather metadata from different locations and resources for a learning object. Also the use of profiles can make labelling of material easier. Profiles can be seen as predefined sets of data that are filled in automatically when a learning object is tagged. Inheriting metadata from a learning object to a similar learning object can reduce the time that is invested in labelling a new object. Inheritance can play an important role because the required set of metadata for learning objects can overlap for certain settings. Meta-tagging in some tools is based on a set of predefined ontologies (Kabel, Riemersma, & Wielinga, (2001). Another form of support for tagging can be found in the use of vocabularies (LTSC, 2002). A vocabulary is a recommended list of appropriate values for metadata. Vocabularies are being developed in various initiatives (ARIADNE, 2002; IMS, 2002), based on a number of experiences and good practice. For specific domains only a small group of subject-matter experts is able to assign the right metadata to a learning object. This is in contrast with the idea of a more centralized approach where a librarian labels the learning material.

Why labelling? In the university, Labelling is primarily undertaken in order to locate a an existing resource, to aid the instructor's memory about where to find a particular object, and to organize material (i.e, folders on the hard drive). In the corporate context, labelling occurs to relate objects to the competency framework, (objects are justified and retrievable relative to their fit within the competency framework), and to anticipate accreditation, maintenance, and quality control. In the military context, labelling is often associated with the archiving and reproduction of images (i.e., noting the time and shutter speed of a photograph), in order to make subsequent production of images easier, more effective, and more efficient. The military has collections of expensive media resources, primarily photographs or films, that it could not or would not want to make again. Therefore labelling is a necessary activity to find and document what is readily available.

4.3 Offering

When a learning object is obtained and labelled it can be offered for selection and eventual use. Different people or organizations can offer learning objects, for example course developers, professional societies, and trainers, but also 3^rd-party vendors specializing in creating course material. Once a collection of learning objects has been tagged it can be offered to an audience.

"Who offers these learning objects?" In the university context it is primarily the instructors who offer their articles and other forms of work as learning objects. There are also repository services, such as the Digitale Universiteit (http://www.digiuni.nl) in The Netherlands, or other service providers, European Union consortia, domain publishers, owners of discipline-specific portals, and conference organizers who offer learning objects. Often these are free, or included in the cost of a subscription to a conference or society. In the corporate context, vendors, publishers, whoever maintains the LCMS (publishers offer material through the LCMS); in-house electronic forums, announcements; brochures, advertisers using (regular) mail, and vendors at trade shows are common groups that offer learning objects. While in-house materials while be produced at low tariffs, in general learning objects for the corporate sector involve a number of costs. In the context of military training; this is done internally so course developers are in effect offering their learning objects only to themselves. This electronic process is a more-efficient way of sharing resources from a central repository than having to ask each other for certain resources. Some effort is now being made to share learning objects between the NATO- partners.

The first three steps described in the above lifecycle relate primarily to the providers of learning objects while the next steps, focus upon the users of these objects..

4.4 Selecting

The selection process is the fourth stage in the learning-object lifecycle. Tools can support course developers or course takers in selecting material from repositories that contain learning objects. For example, Petroskills, (2003) offers a "Competency Assessment Tool" that identifies an individual's gap between current and required competencies so that a course can be provided to fill the gap. The selection process can also be based upon the needs of a course developer. Other support tools are being developed to help course developers to select material based on sets of given criteria. For example the Candle Authoring Tool (CAT), developed within the CANDLE (2003) Project sponsored by the European Union (http://web.candle.eu.org/), offers a wide variety of selection criteria including combinations of ontologies, semantic relations, and keywords within the categories: General, classification, life cycle, pedagogical, and technical. In the university context, selection often occurs via use of the instructor's own bookmark or file collection, via professionally maintained portal sites on the World Wide Web, or by using a search engine for Web resources.

"Selecting" goes a step further in terms of deciding which objects on offer are potentially usable. The influence of colleagues (in universities) or advertising, vendor contacts, or trade exhibits (for the corporate setting) also have a role in the selection process.. Issues relating to mismatches in content, tone and style of communication, presentation, and granularity as well as costs and ownership all influence this selection process (Calverly & Shepherd, 2003).

4.5 Using

Material can be used in two different ways: Directly, the so-called "pure" use of the object, or in an adapted form. "Adapted" means here that the object is edited or adapted after selection for its new environment. The use of a so called pure learning object is easier than an adapted object because the modification of learning objects requires specific tools and skills that may not be available to the developer. Within the corporate and university contexts most multimedia assets like movies and pictures are used without any modifications. Within the military context, multimedia assets are modified when needed for reuse. Therefore the multimedia-development team uses authoring tools. The adaptation or editing of learning objects implies that the course developer does have full access to the objects with no restrictions regarding this access. The developer needs (a copy) of the original learning object to make the necessary adjustments. Therefore the packaging of learning objects (IMS, 2002) is an essential method for distributing learning objects between systems. Distribution of packages includes the copying of learning objects instead of linking. By editing the learning object, a new instance or version of the learning object is created. Linking material is seen as an appropriate way to reuse material more than once. Only one copy of material is available which is maintained and revised when needed. IMS (2002) has developed Content Packaging specifications to support the exchange of learning objects using copies instead of links.

Major aspects involved with the use of a learning object relate to whether it is employed in a self-contained manner, as provided by a LMS, or if it is chosen by an instructor or design team to be used in combination with other learning elements (including an instructor). The way a learning object is used reflects the underlying assumptions about how learning can be instantiated within a given context.

4.6 Retaining

After or during the actual use of a certain learning object that object can become outdated and should therefore be deleted or revised. Decisions about retaining an object are influenced by new insights, experiences, or research from both the developer or user of the object. New instances or versions may be created to revise the original object. Authoring tools can be used to revise the original learning object. The quality control of the learning objects can be measured by tracking the use of learning objects. Also rating tools can provide information about the usability and quality of learning objects. Organizations need to identify who is responsible for the maintenance, versioning and access control of collections of learning objects.

4.7 Summarizing the lifecycle

From the above discussions, the lifecycle of a learning object can have different aspects and emphases in the four different learning contexts /philosophy settings suggested by Figure 1. For the university setting, Figure 3 illustrates key answers to the questions shown generically in Table 2.

Why?	To increase professional awareness and identity			Instructor, for pedagogical or professional reasons
Who?	Instructor and learners	Instructor, also the larger professional community
	Larger professional community may provide support
What?	Packages, content aggregations, activities, SCAs, assets.
How?	Templates
	Scanner, office tools (Word, PowerPoint), hard disk	Using profiles, databases, back-office systems, content-analysing tools, dedicated labelling	Software packages, CDs, DVD	Dedicated selection tools, search engines, browsing tools	Browser, dedicated clients			Tracking and tracing learning objects, editing learning objects
Where?	Within environments associated with authoring tools				Within LMSs		Within authoring tool environments
		Within LCMSs			Within CMSs
Learning objects	Obtaining	Labelling	Offering	Selecting	Using Pure	Using Adapted		Retaining

Figure 3 University context, participation/contribution pedagogy

In contrast, the current corporate context can be summarized in Figure 4

Why?	To match competence framework				Because of learning approach: (Acquisition); also for efficiency reasons
Who?	Provided by LCMS, LMS, specialist staff				Learning manager
	Specialist staff working with LCMS, LMS,
What?	E-modules
	Packages, content aggregations, activities, SCAs, assets.
How?	Templates
	Office tools (Word, PowerPoint) Scanner, digital camera, software programs, 3rd-party commercial development	Using profiles, databases, back-office systems, dedicated labelling tools	Software packages, CDs, DVD	Competency assessment tool, search engines, competence gap analysis, browsing tools	Browser, dedicated clients		Tracking and tracing learning objects, editing learning objects
Where?	Within authoring tool environments					Within authoring tool environments
		Within LCMSs			Within LMSs
Learning objects	Obtaining	Labelling	Offering	Selecting	Using pure	Using Adapted	Retaining

Figure 4 Corporate context, knowledge-acquisition use of learning objects

In contrast, the emerging context in corporate settings involves not only the aspects shown in Figure 4, but also the roles and issues associated with the corporation's knowledge-management tools and systems. It therefore incorporates aspects of the university context as illustrated by Figure 3 in terms of an emphasis on contacts and resources from the larger professional community.

Why?	HR policy reasons Pedagogy of community of practice
Who?	KM professionals, competence specialists			Support via forum moderators, discussion groups for knowledge sharing
What?	E-modules
	Short courses, multimedia, animations, simulations			Training modules	Multimedia assets	Courses, learning objects
How?	Templates
	Office tools (Word, PowerPoint)	Using profiles, databases, back-office systems	Software packages, CDs, DVD	Competency assessment tool, search engines, competence gap analysis, browsing tools	Browser, dedicated clients		Tracking and tracing learning objects, editing learning objects
Where?	Within authoring tools					Within authoring tools
		Within LCMSs			Within CMSs
	Scanner, digital camera, software programs, 3rd-party commercial development	Within dedicated labelling tools	Web		Within LMS
Learning objects	Obtaining	Labelling	Offering	Selecting	Using Pure	Using Adapted	Retaining

Figure 5 Corporate context, knowledge sharing and community of practice orientation

For the military context learning objects are created by authoring tools and used within CBT systems. Reuse is not automated by technical systems. Figure 6 shows this.

Why?	Efficiency reasons Acquisition pedagogy
Who?					Policy maker
	In-house development team, media specialists
What?	Packages, content aggregations, activities, SCAs, assets.
	Templates
How?	Scanner, digital camera, software programs, 3rd-party commercial development tools	Ontology-based agents, dedicated labelling tools	Software packages, CDs, DVD	Search engines, browsing tools	E-modules		Tracking and tracing learning objects, editing learning objects
Where?	Within authoring tool environments		Catalogues, Web		Within authoring tool environments
Learning objects	Obtaining	Labelling	Offering	Selecting	Using Pure	Using Adapted	Revising

Figure 6 Military context

Figures 3, 4, 5 and 6 (from Strijker, 2004) show that there is a difference in the systems used in particular to the offering and selection stages which is directly related to their different contexts.. The types of material which is reused differs within each type of organization. These materials vary from assets to complete packages.

4.8 Issues

Many different issues can be mapped to the lifecycles of different learning objects and will vary in their severity and resolution in the different context/learning-philosophy settings. Some are technical in nature, such as:

Specifications and standardsSpecifications are evolving quickly and are rather technical. The implementation of the standards, development of content, and development of tools is therefore difficult. Content developers need the assistance of technical persons for the implementation of standards. Tools also need to be tailored to the new specifications in order to create content according to these new standards.

GranularityThe specifications can deal with different types, sorts, and sizes of objects. While handling different t levels of granularity is not so much a technical problem, but rather more one for the human operators who will require different tools and skills for dealing with objects which can be at the course, module, lesson, or object level. ReuseTechnical issues involving reuse can be related not only to the interoperability between systems, but also to the services that handle the exchange between systems in terms of copyright and usage restrictions The exchange of material in terms of the payment, hosting, and usage restrictions needs to be handled, for example within the brokerage service that could be part of the LCMS. The use of Digital Rights Management (DRM) can help to solve copyright issues.

Meta-taggingTwo sorts of metadata can be identified: Objective metadata and subjective metadata (Hodgins, 2000). Objective metadata can be obtained from different sources like backend repositories and database functionalities. The issue here is that most authoring tools do not use databases for creating content which means the objective metadata is not available. A more-difficult sort of metadata are the subjective metadata. This metadata depends on the metadata provider who may not have the skills to assign the data correctly or uses criteria that do not fit other settings.

Access and privilegesThe biggest issues from a technical perspective relate to the access and privileges required to (re)use material. Hardware and software issues can be identified when it comes to access and privileges. Confidential material, classified material, commercially competitive material, copyright-protected material, embargos, terrorism, hacker attacks, network security, lack of software, and private networks are issues that interfere with exchange of learning objects.

Other issues relate more to the human perspective, such as:

Usability Tools have to be easy to use because of the risk of user cognitive overload. Another issue is the computer skills of those involved with creating learning objects particularly the use of developer tools when there is not sufficient experience with these types of tasks.

Time and effort The effort invested in providing metadata for reuse should be minimised as the payoff is often not directly visible. Rewarding such effort should be discussed within the organization. One question that needs to be asked is whether the process of searching for the right content and adapting it is taking as much time as creating it from scratch?

Pedagogical aspects The opinions of those involved about the potential pedagogical value of learning objects can vary enormously, particularly in different organizational contexts.

Organizational payoff What is in it for the organization? Is there an organizationalstrategy? What is the reason for implementing a reuse strategy in the organization? Is it to do with status, professional approach, recognitionor efficiency?

Intrinsic motivation Besides commercial reasons, why should creators want to share their material? Spontaneous sharing on the public World Wide Web rarely occurs in the corporate or military sectors, although in corporate knowledge-management systems the use of forums or networks to support communities of practice provides a setting for those intrinsically motivated to share content.

Willingness Are the developers of learning objects willing to share their knowledge or is it a protected domain of knowledge? Are there organizational policies that limit sharing, even if there is willingness? For researchers, when will sharing interfere with intellectual property and with the uniqueness of an individual's research?

Support services What are the resources available for the human support of the different phases of the lifecycle? How are they offered?

Access and privileges Who can, or cannot, have access to learning objects? Should students be able to see the work of other students? Does this lead to plagiarism or to lack of control over assessment?

Ownership and copyright Who owns the material, the creator, the development group, the subject matter expert, the publisher, the internet provider, the host organization, or the organisation itself ? (Rowe, Webb, and Hartwell-Hunnicutt, 1998, identify this as a critical issue related to learning objects created in the university setting ). What is the essence of what is owned: The idea, the actual content, its representation conceptually, or its representation in terms of digital presentation? If adaptations are made to an object, does the owner or copyright holder have to give explicit permission? If so, how is this managed and how is version control maintained?

Thus, learning objects have a complicated life cycle that differs in different contexts and is profoundly influenced by the philosophy of learning of the dominant decision maker. Many issues confront any mainstream use of learning objects. Given all this complexity, how does the Semantic Web come in? Is it going to be the answer that will change the shape and form of learning?

5. Semantic Web and Ontologies: An answer? To what question?

In discussions of the Semantic Web, it seems that the focus is predominately on only two of the six lifecycle stages: "select", and before that, "label". The assumption seems to be (perhaps this is an unfair interpretation) that if these functions work well, then this is the key that will "forever change the shape and form of learning" (Hodgins, 2000). However, our argument is that all stages are important, particularly the "use" stage; and also that context and learning philosophy give very different views of these stages. For many of the issues identified in the previous section, the Semantic Web and ontologies have little or no relation to the sorts of questions that are raised.

There are lessons already being learned from the current work with standards and metadata. All of the standard bodies are developing taxonomies for their metadata. While these taxonomies may seem appropriate from a logical perspective, in practice they may not reflect the way human users think about learning objects if they go to find them, or have to label them. There are two major issues: Can a taxonomy be generalized across all potential users? How much detail is necessary and how much detail is it feasible to collect?

In terms of the first question, a number of groups have tried to define taxonomies for metadata based on pedagogical analyses of potential end users. In the CANDLE Project (2000-2003), sponsored by the European Union, considerable effort was put into the modelling of different user groups in order to provide input for the set of metadata to be used (Scott & Van Helvert, 2001). To help users in the CANDLE Project assign the metadata to a potential learning object, a software Wizard was created to guide assigners through each of the metadata categories (Liu, 2003). As far as possible, pull-down menus were available in the Wizard, and for each metadata category, an example and set of definitions were supplied. However, even with this level of detail, the use of the Wizard by an instructor intending to use an eventual object as a potential resource, particularly for a generative or contribution-type activity, turned out to be problematic in user trials (Brostoff & Kent, 2003). One reason is that with a generative or collaborative approach, the activity is not inherent to the learning object itself, but depends upon what the learner does with the learning object. It may be useful, for example, that a broad selection of learning objects be made available, so that the learner can decide for himself which are the most useful for his task.

Another problem is the selection of a taxonomy. Sets of tags that might appear generally appropriate in a university context would lack many elements that would be necessary in a corporate or a military setting. In a corporate setting, objects are likely to be labelled in terms of their relation to a competency framework (Mulder, 1999) where personal authorship is of little importance. More fundamentally, there is considerable debate about the possibility of developing taxonomies that involve the same ontologies for different groups of users. Kraan (2003) notes that objects are "best described by using multiple vocabularies. There is no way to determine which vocabulary will be relevant to either an author or user of a given object?What may be a learning object to you, is a news article, archive context or a use case for somebody else. An object's meaning, in other words, depends on its context of use".

Berners-Lee, Hendler, and Lassila, (2001) in their work with "The Semantic Web" see ontologies as one solution to this problem. "Ontologies are a shared and common understanding of a domain that can be communicated between people and application systems" (Davies, Fensel, & Harmelen, 2003, pp. 4-5). Much of the current research on ontology development follows a rational approach (see for example, Berners-Lee, Hendler, & Lassila, 2001). Engers and Lech (2003) however note that "within current approaches to the Semantic Web, it is debatable what should be central --the human using the Web or the possibility of performing machine processing on Web content. In the former case, logical representations are probably not the most intuitive for use with humans, and different, more 'cognitive' representations of such knowledge might be more convenient" (p. 114).

However, even with tools focused on ontology development and a relatively well-defined knowledge domain (ontologies about skills, job functions, and education in a knowledge-management setting), Reimer, Brockhauser, Lau, and Reich (2003) point out that many human problems occurred when trying to use a Semantic Web approach to ontologies. Problem areas were a lack of domain experts to build the ontology, difficulties with ontology evaluation beyond a certain range of core concepts, and user difficulties in selecting the right concepts. Doctorow (2002) anticipates these problems when he notes that "there is more than one way to express something". Another difficulty is the problem of "ontological drift" (Fensel, Stask, Studer, Harmelen, & Davies, 2003).

The latter see the combination of peer-to-peer collaboration and ontology development as the future: "Only by bringing together Semantic Web (specifically ontologies) and P2P (peer-to-peer) technology can we fully realize the potential?by giving participants freedom to use their own ontology structures" (p. 264). User-tailored descriptions for metadata are a form of peer-to-peer collaboration being studied in a number of locations. Recker, Walker, and Wiley (2000) describe an approach similar to that used on the Web in public sites such as Amazon Books in which patterns of choices and responses of users are used to identify which objects might be of interest to which persons. Called "collaborative filtering", the approach involves "developing and evaluating a collaborative filtering system, which enables users to share ratings, opinions, and recommendations about resources". However, if such a system would be taken up in widespread practice throughout an organization is not clear. An incentive for content specialists to take the time to add comments about a particular object is likely to be lacking.

With regard to incentives for the labelling of learning objects with metadata, a major issue is the amount of metadata that is feasible to expect, given the time constraints of those who enter metadata and given the interests of those who make use of the metadata for the selection of objects. Bois (2002) says that "all" that is needed is that learned societies develop domain ontologies, authors use the new tag editing application to complete their texts with tags, and retrievers use the new browsers that allow the selection of documents by specifying tag contents and relations. However, she acknowledges that while "this is simple it doesn't mean that there is no effort" (p. 343). The effort involved needs organizational embedding and incentives in order to occur.

All of these problems have been studied for many years within the domain of information retrieval. Swanson, in 1988, summarizing 30 years of fundamental research on information retrieval concluded that:

"Our relevance judgements and our thinking entail, among other things, artful leaps of the imagination unconstrained by logic, reasoning, or the clammy hand of consistency; more important, they entail knowing who we are, what kind of world we live in, and why we want what we seek. It is hardly imaginable that a mechanism other than a human could acquire such self-knowledge, be given it, or do the job without it." (p. 95)

This insight is not out of date; it is the basis of a new research line at the University of Twente in The Netherlands (Huibers, 2003). Due to the insight of this research as well as our on-going analyses of the impact of context and learning philosophy on the lifecycle of learning objects (Collis & Strijker, 1999, 2001-2002, 2002, 2003; Strijker, 2000, 2001, 2002, 2004) we remain sceptical about how a focus on the Semantic Web or ontology development will act as keys to change the way people learn.

6. Conclusion

It is not that we are sceptical about the power of improving agents to select objects from the Web based on semantic approaches. The site KartOO (http://www.kartoo.com/en/servlet/H) for example shows that currently available tools can help locate and select objects but also expose a network that you didn't know existed in terms of who is linking to objects you find particularly useful, something that goes beyond finding a particular object. There are new efficiencies, new power, new ways of thinking and "new forms of intelligence and meaning being added to display and navigation of context in the current World Wide Web" (Anderson & Whitelock, 2003). We encourage continued development toward these ends, but we are constrained by two sets of concerns: (a) the process should not be over formalized; and (b) intelligence and creativity are more important during the use process than during the find and select processes, and intelligence and creativity will come from humans, individually or collectively, outside of the Web (whatever sort, Semantic or World Wide). In a participation or contribution approach to learning, learning objects are only a tool; human processes involving communication, sharing, and collaboration are more important.

With regard to procedural/conceptual difficulties and the dangers of over-formalization, it appears to us that the Semantic Web as now described depends too much on a pre-formed structure; maybe finding this will succeed in certain cases, but for this to happen, too much must be organized, too many people (user groups, etc) must be in agreement about the structure, and a clear description in a shared language of the domain is needed. Shanks, Tansley, and Weber (2003) note that ontology theory requires the following rules when modelling a domain: "Composites and aggregates should be modelled as entities, not relationships, Relationship should not be modelled with attributes, Entities should not be modelled with optional attributes, Conceptual models should clearly distinguish between classes and instances, and Things and their properties should be clearly distinguished in the conceptual model" (p. 88). What does all this mean? Shanks, Tansley, and Weber continue by noting problems in practice in carrying out these rules, such as misclassifications and dual classifications. Putting groups together to form the ontology may be possible but requires too much discipline to be feasible in practice. Ontological drift and human drift will be unavoidable.

With regard to the underlying learning model, we recognize that in many cases knowledge transfer is the goal and thus an acquisition-based learning model is appropriate. However we agree with Euler (2003) that this is the lowest level of learning. In the knowledge-building and sharing model represented by the right-side column if Table 1 and Figure 1, the essence of learning is not so much concerned with finding or being presented with objects but in learning situations where collaboratively creating and constructing the objects may be a larger goal. We see this kind of learning occurring in a setting where a great deal of formalism isn't needed to make sense of objects, because humans are around to supply the sense and be aware of the tacit understandings involved. Human-to-human "ontology" that comes from personal shared understandings and communication is not likely to be simulated/paralleled by technology. Thirty years of attempts to model learners for intelligent tutoring systems shows us the limitations of trying (Park, 1996). Even if we can find objects more quickly and more accurately doesn't mean a higher quality learning experience. For many types of cognitive development, finding and deciding about the appropriateness of knowledge is a major learning goal in itself, and striving for a situation where an agent or system presents "what you need" without mental effort or responsibility on the learner's part will not even be desirable. We also agree that the use of technology in the form of agents and their capabilities never will and can replace human-to-human communication. "Human-to-human communication will always be a important component of the educational experience" (Anderson & Whitelock, 2003). The promises of the semantic web are high but the costs to achieve such a kind of automatism may be unobtainable in practice. Even more fundamentally, the focus on content may not be the solution for the needs of a pedagogy based on a participation or contribution-oriented educational philosophy (Anderson & Whitelock, 2003).

7. References

Anderson, T. and Whitelock, D. (2004). The Educational Semantic Web: Visioning and Practicing the Future of Education: Journal of Interactive Media in Education, 2004 (1), Special Issue on the Educational Semantic Web. ISSN:1365-893X [www-jime.open.ac.uk/2004/1]

ARIADNE. (2002). Alliance of remote instructional authoring and distribution networks for Europe. Retrieved September 29, 2003, from http://www.ariadne-eu.org/. [cited]

Bannan-Ritland, B., Dabbagh, N., & Murphy, K. (2000). Learning object systems as constructivist learning environments: Related assumptions, theories and applications. In D. Wiley (Ed.), The instructional use of learning objects - Online version. Retrieved 2 October 2003 from Retrieved September 29, 2003, from http://reusability.org/read/chapters/bannan.doc. [cited]

Barritt, C. (2001). Reusable learning object strategy: Designing information and learning objects through concept, fact, procedure, process, and principle templates. Retrieved September 24, 2003, from http://business.cisco.com/servletwl3/FileDownloader/iqprd/86575/86575_kbns.pdf. [cited]

Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. Scientific American Feature article. Retrieved February 27, 2002, from http://www.sciam.com/2001/0501issue/0501berners-lee.html [cited] [cited]

Boer, W.F. de (2002). ICT in teaching and learning: Part of a blend. In B. Collis, & M van der Wende (Eds.) (2002). Models of technology and change in higher education: An international comparative survey on the current and future use of ICT in higher education. Enschede: CHEPS. [cited]

Boer, W.F. de (2004). Flexibility support for a changing university.. PhD dissertation, Faculty of Behavioural Sciences, University of Twente, Enschede, NL.

Bois, C. (2002). Learning about Sancho Panza and foreign cousins: A case for introducing the future of e-learning within the semantic web. In: Wagner E. & Szücs A. (eds.), Open and Distance Learning and Beyond Rethinking International Co-operation. Proceedings of the 2002 EDEN Annual Conference, Granada, Spain, 16-19 June 2002. EDEN, 2002. (pp. 340-345). [cited]

Brostoff, S., & Kent, P. (2003). Deliverable 7.3: Evaluation report. IST-1999-11276-CANDLE. Universität Karlsruhe, Karlsruhe, Germany.

Calverly, G., & Shepherd, K. (2003). Assisting the uptake of on-line resources: Why good learning resources are not enough. Computers & Education, 41, 205-224. [cited]

CANDLE. (2003). The CANDLE project - collaborative and network distributed learning environment. Retrieved September 29, 2003, from http://www.candle.eu.org/. [cited]

Chapman, B., & Hall, B. (2001). Learning content management systems: comparative analysis of systems used to construct, organize and reuse "learning objects". Sunnyvale, CA: Brandon-Hall.com. [cited] [cited]

Collis, B. (1995). The evolution of educational software productivity. In D. Ely & B. Minor (Eds.), Educational media and technology yearbook Volume 21 (pp. 76-97). Englewood, CO: Libraries Unlimited. [cited]

Collis, B., & Margaryan, A. (2003, 9 September). Work-based activities and the technologies that support them: A bridge between formal and informal learning in the corporate context. Paper presented at LearnIT: Information and Communication Technologies and the Transformation of Learning Practices, Gothenburg, Sweden. [cited] [cited]

Collis, B., & Moonen, J. (2001). Flexible learning in a digital world: Experiences and expectations. London: Kogan Page. [cited]

Collis, B., & Strijker, A. (2001). Generic re-usability situations for Web-supported learning: Metadata requirements and user-support tools. Internal research proposal, Faculty of Educational Science and Technology, University of Twente, Enschede, NL.

Collis, B., & Strijker, A. (2001-2002). New pedagogies and reusable learning objects: Toward a new economy in education. Journal of Educational Technology Systems, 30 (2), 137-157.

Collis, B., & Strijker, A. (2002, 27 June). New pedagogies and re-usable learning objects: Toward a different role for an LMS. In P. Barker & S. Rebelsky (Eds.), Proceedings of ED-MEDIA 2002, (pp. 334-339). Norfolk, VA: AACE.

Collis, B., & Strijker, A. (2003). Re-usable learning objects in context. International Journal on E-Learning,(In Press)

Davies, J., Fensel, D., & Harmelen, F. van. (2003). Towards the semantic web: Ontology driven knowledge management. Chicester, UK: John Wiley & Sons, Ltd. [cited]

Doctorow, C. (2002). Metacrap: Putting the torch to seven straw men of the meta-utopia. E-Learning Guru articles. Retrieved February 14, 2003, from http://www.e-learningguru.com/articles/metacrap.htm [cited]

Draper, S. W. (1998). Niche-based success in CAL. Computers & Education, 30(1/2), 5-8. [cited]

Ely, D. P. (1989). Protocols and processes for promoting interactive cross-cultural media transfer. Educational Media International, 26(1), 6-12. [cited]

Engers, R. H. P., & Lech, T. Ch. (2003). Generating ontologies for the semantic web: OntoBuilder. In J. Davies, D. Fensel, & F. van Harmelen (Eds.), Towards the semantic web: Ontology driven knowledge management (pp. 91-115). Chicester, UK: John Wiley & Sons, Ltd.

Euler, D. (2003, 8 October). E-Learning: A catalyst fo developing quality in in universities? Presentation at the SCIL (Swiss Centre for Innovations in Learning) Congress, St. Gallin, Switzerland. [cited] [cited]

Fensel, D., Stark, S., Studer, R., Harmelen, F. van, & Davies, J. (2003). A future perspective: Exploiting peer-to-peer and the semantic web for knowledge management. In J. Davies, D. Fensel, & F. van Harmelen (Eds.), Towards the semantic web: Ontology driven knowledge management (pp. 245-264). Chicester, UK: John Wiley & Sons, Ltd.

Griffiths, D., Heppell, S., Millwood, R., & Mladenova, G. (1994). Translating software: What it means and what it costs for small cultures and large cultures. Computers & Education, 22 (1/2), 9-17.

Hodgins, H. W. (2000). Into the future: A vision paper. Commission on Technology and Adult Learning. Retrieved October 9, 2002, from http://www.learnativity.com/download/MP7.PDF. [cited] [cited] [cited]

Huibers, T. (2003, 2 October). Information Retrieval. Wat vindt u? [Information Retrieval. Wat do you find?]. Enschede: Universiteit Twente. [cited]

IMS. (2002). Instructional Management Systems Project. Retrieved September 29, 2003, from http://www.imsproject.org/. [cited] [cited] [cited]

Jonassen, D. H., & Reeves, T. C. (1996). Learning with technology: Using computers as cognitive tools. In D. H. Jonassen, (Ed.), Handbook of research on educational communications and technology (pp. 693-719). New York: Macmillan. [cited]

Jonassen, D. H., Peck, K. L., & Wilson, B. G. (1999). Learning with technology: A constructivist perspective. Upper Saddle, NY: Prentice Hall. [cited]

Kabel, S., Riemersma, J., & Wielinga, B. (2001). Final set of ontologies. IMAT deliverable O3. ISLA. Retrieved September 29, 2003, from http://www-old.swi.psy.uva.nl/usr/kabel/O3final.htm. [cited]

Kankanhalli, A., Tanudidjaja, F., Sutanto, J., & Tan, B. C. Y. (2003). The role of IT in successful knowledge management initiatives. Communications of the ACM, 46 (9), 69-74. [cited]

Kraan, W. (2003). The one standard, LOM and the semantic web. CETIS. Retrieved February 13, 2003, from http://www.cetis.ac.uk/content/20030127164729 [cited]

Lambe, P. (2002). The autism of knowledge management. Retrieved October 30, 2002, from http://www.straitsknowledge.com [cited]

LTSC. (2002). Learning Technology Standards Committee. Retrieved September 29, 2003, from http://ltsc.ieee.org/ [cited]

Lui, L. (2003). Collaborative and Network Distributed Learning Environment (CANDLE) pedagogical wizard. Masters thesis, Faculty of Behavioural Sciences, University of Twente, Enschede, NL.

Mulder, M. (1999). Competentie ontwikkeling in organisaties. [Competence development in organizations]. Assen: Van Goram. [cited]

Park, O-C. (1996). Adaptive instructional systems. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 634-664). New York: Macmillan. [cited]

Petroskills. (2003). PetroSkills, LL: A world leader in petroleum training. Retrieved September 29, 2003, from http://www.petroskills.com/ [cited]

Recker, M. M., Walker, A., & Wiley. D. (2000). Collaboratively filtering learning objects. In D. Wiley (Ed.), The instructional use of learning objects - Online version. Retrieved 2 October 2003 from Retrieved September 29, 2003, from http://reusability.org/read/chapters/recker.doc. [cited]

Rehak, D., R., & Mason, R. (2002). Keeping the learning in learning objects. In A. Littlejohn (Ed.), Reusing online resources: A sustainable approach to eLearning, (pp. 20-34). London: Kogan Page [cited] [cited]

Reimer, U., Brockhausen, P., Lau, T., & Reich, J. R. (2003). Ontology-based knowledge management at work: The Swiss Life case studies. In J. Davies, D. Fensel, & F. van Harmelen (Eds.), Towards the semantic web: Ontology driven knowledge management (pp. 197-218). Chicester, UK: John Wiley & Sons, Ltd.

Roschelle, J., Henderson, B., Spohrer, J., & Lilly, J. (1997). Banking on educational software: A wired economy unfolds. Technos, 6(4), 25-28. [cited]

Rossett, A., Douglis, F., & Frazee, R. V. (2003). Strategies for building blended learning. ASTD Learning Circuits.. Retrieved August 20, 2003, from http://www.learningcircuits.org/2003/jul2003/rossett.htm [cited]

Rowe, J., Webb, L. D., & Hartwell-Hunnicutt, K. (1998). Who owns multimedia courseware - Faculty of the institution? A critical issue in development and delivery of technology-based education. In T. Ottmann & I. Tomak (Eds.), Proceedings of EDMedia 1998 (pp. 1197-1202). Charlottesville, VA: AACE. [cited]

Scott, J., & Helvert, J. van. (2001). Needs assessment report. CANDLE Project Deliverable 1.2. IST-1999-11296-CANDLE. Universitat Karlsruhe, Karlsruhe, Germany.

Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27 (2), 4-13. [cited] [cited] [cited]

Shanks, G., Tansley, E., & Weber, R. (2003). Using ontology to validate conceptual models. Communications of the ACM, 46 (10), 85-89. [cited]

Strijker, A. (2000). Re-usability problems in large-scale content management of database driven Web based environments. In J. Bourdeau & R. Heller (Eds.), Proceedings, ED-MEDIA 2000: World Conference on Educational Multimedia, Hypermedia & Telecommunications Vol.2. (pp. 1718-1719). Charlottesville, VA: AACE [cited]

Strijker, A. (2001). TeleTOP & reuse: Standards, SCORM and exchanging material. Report for the TeleTOP Alpha Beta Project, Workpackage T6, No. 01-14, University of Twente, Enschede, NL.

Strijker, A. (2002, September 9-11). Learning content management systems for web-supported learning: experiences with reuse and user-support tools. Presentation at ALT-C 2002, University of Sunderland.

Strijker, A. (2004). Reuse in organizational context: Human and technical perspectives. (In Preparation).PHD Dissertation, Faculty of Behavioural Sciences, University of Twente, Enschede, NL. [cited] [cited]

Swanson, D. R. (1988). Historical note: Information retrieval and the future of an illusion. Journal of the American Society for Information Science, 39, 92-98. [cited]

Verwijs, C. (1998). A mix of core and complementary media: New perspectives on media decision making. PhD dissertation, Faculty of Educational Science and Technology, University of Twente, Enschede, NL. [cited]

Visscher, R. (2002). Pilot LCMS/SCO generator. Final project for the Master of Science degree. Ministry of Defense, Staff COKL, Utrecht, NL. [cited] [cited]