COALE: collaborative and adaptive learning environment

COALE: Collaborative and Adaptive Learning Environment Nobuko Furugori 1, Hirotaka Sato 1, Hiroaki Ogata 2, Yoji Ochi 2, Yoneo Yano 2 1 2 Systems Research Section, INES Corp. Department of Information Science and Intelligent Systems, Tokushima University 1 {furugori, sato_hiro}@ines.co.jp, 2{ogata, ochi, yano}@is.tokushima-u.ac.jp ABSTRACT This paper proposes a new adaptive WBT (Web Based Training) environment for collaborative learning named COALE: Collaborative and Adaptive Learning Environment. COALE is an integrated environment of collaborative learning into individual learning based WBT with Active Personalized Awareness Provider. We propose a personalized active recommendation system, which gives proper awareness at right timing for each learner in order to support dynamic course organization aimed at effective and efficient learning. The recommendations are generated based on learners' dynamic learning activities. The prototype system for our environment was developed using object oriented database system, JAVA serve let, and Web server system. Experimental learning session was performed at a University class for the evaluation. Results show the effectiveness of our proposed environments. Keywords Collaborative learning, personalization, recommendation, awareness INTRODUCTION Traditional learning assistant systems have not been pragmatic because they are based only pre-set materials and preset courses, hence far from the needs of the learners who seek flexible environments. Recently, WBT (Web Based Training) system has been emerged and attracting attentions as the tool for distance learning, lifelong learning and correspondence courses. WBT system can expand the learning opportunity beyond the limitation of time and place because it is easy to reach for everyone who uses the usual Web browser through Internet. But general WBT system is based on the traditional learning assistant paradigm; therefore the course organizations are still not flexible enough. We propose the COALE (Collaborative Adaptive Learning Environment) based on WBT with Active Personalized Awareness Provider, to support dynamic course organization aimed at effective and efficient learning. Here, we define a Course as a sequence of learning materials of a target domain, consequently organized by a learner through his or her learning activities. COALE is based on the learner-centered concept. So the learners take the initiative of their own process of learning with proper supports from the environment, instead of given the next step from the system automatically by the intention of an author of the course. COALE has two keywords in its name: adaptive and collaborative. Adaptive feature is realized by personalization. COALE supports learners' to select the next step learning material by personalized recommendation. The next step material selections settle the main road of the Course, step by step. COALE provides collaborative learning support to learners': to post shared knowledge and to discuss with colearners. Note that we consider the posted shared knowledge as a part of the learning materials. Discussions and advices works to spread and deepen the learners' knowledge, therefore collaboration is considered to spread the width of the main road or to form branch roads of the Course. Collaboration The Internet connects persons not only to information but also to other persons. From the viewpoint of learning, Internet provides a basis of collaborative environment where learners may find proper co-learners who have the similar target of interest. Thus, we can activate humans and their knowledge as a part of the learning resources through collaborative learning. In the individual learning WBT, a common problem of difficulty of continuation or completion of courses is experienced, and one of the practical solution is reported to assign human coach or co-learners to advise and discuss with the learner with his or her problem (Nagashima, 2001). Therefore, integrating collaborative learning to WBT proposes a solution to this common problem. Traditional WBT is based on individual learning paradigm, and the extent of integration of collaborative learning into WBT is analyzed and represented by a locus of collaboration named PoC (Point of Cooperation) by Wesser, M. (Wesser, 2000). In comparison with his three level of integration: generic, spontaneous, and intended cooperation, our proposed environment is between spontaneous and intended cooperation. Though discussions are spontaneously started due to the learner-centered characteristics, the current-step-specific recommendation of co-learners is provided at each step of learning. Awareness The implementing realization of the above two kinds of recommendations is Active Personalized Awareness Provider. We provide Contents Awareness and Learning-mate Awareness correspondingly. The way of presentation of awareness information called the intervention type (Jermann, 2001) is "graphical visualization" and the level is monitoring among the three levels: mirroring, monitoring, and advising. Approaches fall into the similar category are SharlokII (Ogata, 2000), Visualization of discussion threads (Simoff, 1999) and Visualization of Communication network (Wortham, 1999). Learning-mate Awareness Provider is developed after the Active Knowledge Awareness proposed for open-ended learning environment Sharlok II (Ogata, 2000) (Ogata, 1999), and Sharlok (Ogata, 1998). For the rest of this paper, we firstly propose the learning model of COALE and characteristics of ideal learner group. Then we present Personalized Active Awareness Provider: recommendation mechanism, profiles and indicators. After explaining system configuration of the prototype system, we report the experiment for evaluation and its results. Finally, we give concluding remarks. COALE Model of Learning We assume two phases model of learning (see Fig. 1). The first phase is contents selection phase and the second phase is learning practice phase. These two phases are iterated through the learning course. Action items in Fig. 1 follows the example case of evaluation experiment. The learning contents include exercise questions, its answer and explanation, and shared knowledge posted by collaborative activities. The basic learning materials are, in this case, exercise questions that are organized in tree type list according to the structure of the domain category. Awareness support Learning Style Phase of Learning Action At the first phase, learners finish with one Contents learning material (here Select the next step Contents Awareness Map an exercise question) and selection phase select the next question. Individual learning The second phase Read question includes two styles of Practice phase learning i.e. individual Solve and put an answer learning and Check the answer collaborative learning, Read explanation and includes much kind Collaborative learning of actions. These Read shared knowledge learning actions are took in random order, random Put own knowledge Learning-mate number of times. Select a partner Awareness Map Request discussion The awareness supports Discuss are provided at each phase. Contents Fig. 1. Model of Learning recommendation is provided at the first phase, and the Learning-mate recommendation is provided at the second phase to support starting collaboration. Characteristics of ideal learner group COALE is aimed at the following learning group. The participants are 3 types: learners, coach or learning environment manager, and operating manager. As a learner group, we consider members who belong to a group with a common target domain of learning e.g. students of the same laboratory of a University or members in the same department of a company. Members of the group are almost fixed for a regular period of time, and some portion of the members change periodically. So the group consists of members of different level from experts to novices concerning to their target domain. This condition enhances the members to be able to teach or coach each other; hence collaborative learning may work well. We further assume the existence of a coach or a learning environment manager who provides the basic learning contents, take part in collaborative learning to behave as a moderator, and adjust the systems behavior according to the learning strategy of e.g. the organization. An operating manager is a person who actually handles the system to continue and change rules by a request of the coach. PERSONALIZED ACTIVE AWARENESS PROVIDER We provide awareness information based on the learners’ behavior. Personalization is popular technique of Web customization or e-commerce where user interface or contents recommendation is personalized according to the users' former activities (Hirsh, 2000) (Smith, 2000). The major difference between such systems and COALE is the filtering criteria for the recommendation. General criteria of personalization of e-commerce are aimed at fitting to the users' taste based on the segmentation of users to distinguish the most profitable segment (Kramer, 2000). In COALE, as a learning environment, the filtering criteria have concerns to the progress and capacity of learners, the learning conditions and the didactic strategy. We take this into account of the recommendation process. We support learners’ selection by restricting the choices within narrow limits from large amount of information. The final choice is always left to the learners. The result of the selection compared to the recommendation is also recorded for future feed back. Learner Recommendation Process Monitoring The recommendation in COALE is based Action event mechanism on the relation between the learner and the learning contents. Learning contents Learners’ Monitoring Mechanism action records We developed an action-status transition model corresponding to the learners’ action through the system windows. When an action event occurs, the system Learner profile Contents profile catches the action of a learner. At the next event, the action-status is interpreted and the action history is Indicators recorded to the DB. At the same time, the learner profile and the profile of the target contents of the action are revised. Filtering mechanism for Filtering mechanism for We make learner profile by summing up Learning-mate recommendation the action records along with the learner, Contents recommendation and contents profile by summing up that along with the contents. We further edit Fig. 2. Recommendation filtering for awareness maps the indicators that are used in filtering based on the learner-contents relation. for recommendations, from the above profiles. Indicators used in filtering We define several indicators as follows. LOC (Level of Comprehension) of a learner to content is defined: LOC = (A + LOCe) / 20. (correct answer) (wrong answer) (not answered). z +1 Easy -1 0 where A (Answer indicator) = DBC Near Same category y LOC High +1 preference plane +1 10  3 0  and LOCe (Level of comprehension of an explanation of a contents) from (understand) a reply of a little questionnaire presented with (could not understand) an explanation contents, (neutral or not replied). LOCe = 10 Relative LOD x Difficult -1 Low Different Category  5 0  -1 Far Fig. 3 Contents recommendation space category) is defined as a rate of correct answer: LOCc = We further define LOC of a category of contents. LOCc (LOC of a learner to a Total number of correctly answere Total number of answered contents We use LOCc as an alternative of the LOC of contents which is not learned yet. LOI (Level of Interest) of a learner to a content is defined: Total number of active actions of the learner to a contents LOI = Maximum number of the above numbers among the learner where active actions contain solve, learn, look, create, request, and discuss. BL (Busy Level) of a learner is derived from the learner's current action status. We assign lower value to the higher busy level. For example, a learner in discussion is considered very busy hence assigned busy level 1. On the other hand, a learner who is browsing category tree, seems easy to transfer to a discussion, is assigned busy level 10. We further convert the value dividing by 10, to be within the range from 0 to 1. LOD (Level of Difficulty) of each content is a rate of correct answer: Number of correctly answered learner LOD = Number of answered learner DBC (Distance between Contents) for a pair of contents is defined: DBC = + 1  − 1 if two contents belong to the same category if two contents belong to the different category. Contents Recommendation At the first phase of the learning model, the system presents candidates for the next step contents for learning. The learner can request the Map by pushing the request button. We consider the 3D contents recommendation space where 3 coordinate axes correspond Relative LOD, LOC, and DBC (see Fig. 3). Here, the Relative LOD is derived as follows. We define the Proper LOD for a learner as an average value of LOD's of contents already learned and correctly answered by the learner. The Relative LOD is derived from the LOD so as to be normalized within the range from 1 to +1, with the Proper LOD corresponding to the origin of coordinates. All contents not yet learned or not yet correctly answered are plotted in this 3D space. We modeled two kinds of strategy concerning the learning sequence named after tracing strategy of tree branch: Depth-first and Width-first. Following the Depth-first strategy, the next step contents should be in the same category and at the same or higher LOD. By the Width-first strategy, we should recommend contents in the different category and at the same or lower LOD. To reflect the learning strategy, we consider the Preference Plane with coordinates Relative LOD and LOC. Note this plane corresponds to the x-y plane in the above 3D space. Naturally, an easier content is easier to understand. We point the learner's position on the line which connects (-1,-1) and (+1,+1) on the preference plane. Learners of Depth-first strategy take position in the right half of the plane, and learners of Width-first strategy take in the left half. The distance from the point (0,0) represents the strength of the strategy. Values of a parameter to set the position of learners are given in a rule by the learning environment manager. The recommendation indicator is the distance from the learner's position to the plotted point of contents. The shorter distance has higher priority. In the case of Depth-first strategy, the target contents are on the plane where z=+1, on the other hand in Width-first strategy, the target plane has z=-1. Learning-mate Recommendation In the second phase of our learning model, the system always presents candidates for the synchronous collaboration on the Learning-mate Awareness Map according to the learners’ current and past conditions. The requirements for candidates are logging in the environment, having high LOC and LOI to the current contents of the requesting learner, and being not so busy enough to BL z accept discussion. We consider the 3D recommendation space where LOC, LOI and BL are 3 coordinate axes (see Fig. 4). All logged in learners are + plotted in the space. The distance from the origin of the coordinates represents LOI y the recommendation indicator. The candidates are selected who has the larger value of the indicator. + 1 SYSTEM CONFIGURATION We developed a prototype system of COALE in client/server style, in which the client side is based on a Web browser. The COALE prototype system consists of Windows 2000 Server and OODB. We describe the main functions of both client and server sides. 0 + LOC x Fig. 4 Learning-mate recommendation space Special Feature of COALE As a software system, COALE prototype system has the following three features. • The learners can add or insert their own know-how, knowledge, and/or questions and replies into the preset Web-learning contents for common use. • To effectively use the learning contents, and to find proper co-learners among the learning members, the system has personalized active recommendation sub-system that give proper awareness at proper timing for each learner. The recommendations are based on learners' dynamic learning activities. • To ease burdens of the managers of the learning group, in adjusting behaviors of the system according to their policy or strategy, easier way using ECA rule setting is provided. In this paper, we concentrate to the awareness support. Client System Client system consists of 4 functions. Individual Learning Function We provide question-answering exercise as the basic function of individual learning support system. The basic learning contents are sets of question, answer and explanation of the answer. Exercise questions are arranged into a tree type list according to the domain structure. Learners select a question from the awareness map or from the list, read, think, and put their answer. Then the system evaluates the answer, informs learners whether the answer is correct or not, and at the same time keep the answer records as a part of learners history. Sequence of selected questions and learning actions of each learner are recorded as the source information of personalization. Collaborative Learning Function We provide knowledge sharing function and real-time chat function for collaborative learning support. Learners can add their own knowledge such as interpretation of the answer, explanation of technical terms, and can share among co-learners learning in the same environment. As for collaborative discussion, chat tool is provided for synchronous discussions and the knowledge sharing function mentioned above works also as a tool for asynchronous discussion. Learners’ action records database Individual learning function Contents awareness map function Learner Web server, Servelet Collaborative learning function OODB management Learning materials, System Shared knowledge database Learning-mate awareness map function Rule database Server Client Fig. 5 System Configuration Contents Awareness Map Function This map presents recommended candidate of exercise questions for the next step. The map shows the previous question and its category, next candidate questions with recommended order number and the level of difficulty using GUI. Recommendation is formulated adaptively to the learners' behavior history and learner profiles. Learners can choose one from the restricted candidates. Learning-mate Awareness Map Function We provide awareness of co-learners aiming to support effective collaboration. This map displays recommended candidates of real-time discussion with recommended order number and background knowledge of each candidate using GUI. Recommendation is based also on the monitoring of co-learners' current status. Server System Server system consists of 3 databases. The management system of these databases is as follows. Learning Material and Shared Knowledge Management Function This function manages the registration, adjustment, deletion and retrieving of the basic learning contents and shared knowledge. Only the manager of the learning environment can handle the basic contents. The shared knowledge is provided by co-learners and registered automatically by the system. Learners' Action Records Management Function The system is monitoring the learners’ behavior and keeps action records in the database. Learners' profiles are produced from these records and held in the same database. Rule Management Function The coach uses the rules to adjust the learning strategy. Rules are expressed as a combination set of event, condition and action, after active database systems. When an event occurs, the system evaluates whether the condition is fulfilled, the indicated action event, e.g. pop-up presentation of the awareness map to learners, will be fired. User Interface Fig. 6 shows a display shot of the COALE prototype system. Main Window Learning-mate Awareness Map Learning-mate Category Exercise Question Background Knowledge Show Shared Knowledge Button Knowledge Input Button ƒ A ƒC )C (B –Ê ‰æ ƒ“ ? Category Exercise Question Answer Fig. 6. Display Shot Main Window _A_ The right side window is the main place for individual learning action. A learner read and solves an exercise question, put an answer. The system checks the answer whether it is correct or not. By pushing the "explanation request" button, the system shows the explanation of the answer. Pushing the "show shared knowledge" button, the system presents a list of the shared knowledge for learners' selection. To put a shared knowledge, push the "knowledge input" button then a window for input will be opened. As for collaborative learning action, "request discussion" button works to open a chat request window for the first step of opening a discussion. Contents Awareness Map _B_ To select the next step exercise, learners select one of the recommended contents from the Contents Awareness Map. On the Map, a square mark represents a category, a circle represents an exercise question that has not been correctly answered, and a diamond represents a question that has not been learned. The level of difficulty of a question is reflected on their color. The orders of recommendation are displayed as a number at a side of the question title. The questions that are graded as first and second are presented in red text and given the order number. Learners can select a circle or a diamond to open the corresponding exercise question. Learning-mate Awareness Map _C_ To select a proper partner of discussion, learners can consult the Learning-mate Awareness Map. The nodes represent co-learners, exercise questions, and categories of questions. Corresponding marks are circle, diamond and square. Firstly and secondly recommended learners are in yellow color, presented with the order number. Moreover, up to two contents for these two recommended learners are displayed as their background knowledge. From third to seventh recommended learners are shown in gray circle. EXPERIMENT We settled an experimental learning class of basic information processing. The objective of the class is to acquire basic knowledge of information processing within a given period of time. The course is offered to a group with 32 members of students including a few teaching staffs who belong to the same research laboratory of a university. The class was held 5days and each student learned 1hour a day using the system. The experiment is aimed to evaluate the supportive efficiency of our proposed environment. The system support intends learners to expand the scope of knowledge through effective collaborative learning, and environments to raise efficiency of individual learning. The 3 points of the evaluation are effectiveness as a learning support system, effectiveness of the Contents Awareness Map, and Table 1. The 4 subject groups (A1- B2) and their conditions in the experiment effectiveness of the LearningSystem mate Awareness Map. t Usual Adaptive Learning style Practice A1 We installed basic learning A2 Individual Learning contents: 200 questions selected A list of Contents awareness map questions from the Examination for BÇ P B2 National Certification of A list of questions Contents awareness map Individual Learning Information Processing A list of co-learners Learning-mate awareness map Collaborative Learning Engineer 1st class and 2nd class held these 3 years. We separate 32 subjects (participants of the experiment) into the following 4 groups. Each group is given the following learning environment (see Table 1). A1) Usual individual learning environment Learners in this group select exercise questions from a list of the whole questions and learn by themselves. Each question is marked already learned or not. A2) Adaptive individual learning environment The Context Awareness Map supported A2 GROUP learners. The learners could select next step from small number of recommended questions presented in a form of colorful map using GUI. B1) Usual collaborative learning environment B1 group was allowed both individual and collaborative learning. The system provided a chat tool for collaboration. The learners chose their learning mate from a list of all learners. B2) Adaptive and collaborative learning environment B2 group learners do individual and collaborative learning with full support from the system. They select the next step questions from the Contents Awareness Map and choose learning mates for discussion from the Learningmate Awareness Map. Learning-mate awareness map shows appropriate colearners in a form of colorful map using GUI. The maps' recommendations Fig. 6. Number of correctly dynamically correspond to the learners' behavior. answered questions. EXPERIMENTAL RESULTS We performed both objective and subjective evaluation. Objective Evaluation Results of 3 written examinations and history of the subjects' behavior are the sources of the objective evaluation. Result from Written examination We carried out 3 written examinations, which we call pre-test, middle-test and post-test. Each test consists of 40 questionnaires covering 6 categories of contents. Fig. 6 presents the result of 3 written examinations. Average number of correct answers show that B2 group made progress through middle and post-test. B2 group was supported by two awareness maps. Therefore we would state the two awareness maps are effective to learning as a whole. Results from History of actions Fig. 7 is a graph of the numbers of exercise questions that were learned by each subject of each group. The whole average number was 122 questions per subject. B groups count smaller numbers comparing to a groups, because they took time for collaboration besides individual question solving. Comparing A2 with A1 and B2 with B1, we can see that A2 and B2 group solved larger numbers than A1 and B1 groups correspondingly. Both A2 and B2 groups had support of Contents Awareness Map. As learners were in navigable environment so they could finish more questions. Fig. 7. Total number of exercise Average of the rate of correctly answered questions are on Fig. 8. The questions whole average was 0.65. That means each subject correctly answered to 65 questions out of 100 questions. The rate increases in the order of A1, B1, A2 and B2. Individual learning environment with Contents Awareness Map (Group A2), individual and collaborative learning environment with two Awareness Maps (Group B2) marked high ratios. Therefore, we confirmed that the proposed environment could provide effective support to the learners. Contents Awareness Map for collaborative learning In A2 group, the average number of usage (acceptance) of the Map was 150 times and the average number of questions learned was 147. On the other hand in B2 group, the average usage was 118 times and the average number was 125. Considering with the above rate of correct answer, we can say the recommendation given on the Map was appropriate. Next, we consider collaborative learning of B1 and B2 groups. The number of discussions in the groups and the total number of participants was almost same, 17 times and 46 persons for B1 group, and 13 times and 49 persons for B2 group correspondingly. Fig. 8. Rate of correct answer. On the other hand, the number of remarks per subject differed approximately twice between B1 (59) and B2 (117). In B2 group, the Learning-mate Awareness Map enhanced appropriate generation of discussion. Moreover, the rates of acceptance out of requests of discussion were 36% for B1 and 68% for B2. The results shows that the requests were made to the proper co-learner in B2 group supported by the Awareness Map recommendation, and that the discussions were generated effectively. The rate of acceptance is calculated by the following expression. Rate of acceptance = Number of accepted discussions / Number of requests * 100. Subjective Evaluation Questionnaire Survey We prepared a questionnaire survey based of the following 4 viewpoints: the whole system, Contents awareness map, collaborative learning environment, and Learning-mate awareness map. Each of 24 questionnaires has graded 5 choices and some requests free style comments. The target of the first viewpoint questionnaire is all subjects participated in this experiment. As for the degree of difficulty of the exercise questions, every group selected high choice: difficult. Interest to the domain of exercise was increased in B2 group. We consider this was the influence of the collaborative learning enhanced by the Awareness Map support. The subjects' evaluations to the merit of collaborative learning were almost same between B1 and B2 groups. The differences between two groups were observed on the difficulty of choosing the partner of a discussion, for this point B2 group suffered less. The Contents Awareness Map were evaluated to be useful to look for the next step exercise and worked well as a navigator. From 75% to 80% of the subjects followed the recommendation. As for the Learning-mate Awareness Map, acceptances of the recommendation were as many as above. The support by the Map was estimated relatively high. CONCLUDING REMARKS We have developed a prototype system of Collaborative and Adaptive Learning Environment to support dynamic course organization aimed at effective and efficient learning. We proposed two kinds of personalized active awareness provider. One recommends learning contents for the next step and the other recommends learning-mate for discussion. Both of them are presented as a visualized map using GUI, according to the history and the current state of learners' behavior. Because COALE follows the learner-centered concept, the final decision of selection is left to the learners. An experimental class was performed at a laboratory in a university, participated in 32 students and staffs. The results of the objective and the subjective evaluations show the basic effectiveness of our proposed environment to support learning. We have got valuable suggestions from the participants of the experiment in the questionnaire survey. Our next task includes tackling these suggestions e.g. the fitness of the recommendation, the confusing similarity of the design of two maps. One of the main future works is adjustment and elaboration of the recommendation mechanism. We need further experiments for a longer period, and in other domains for evaluation and elaboration. ACKNOWLEDGMENTS This work was performed in cooperation with the Information Technology Consortium, as a part of a project of the Information-technology Promotion Agency Japan. REFERENCES Hirsh, H., Basu, C., and Davison, B. D. (2000) Learning to Personalize. Communications of the ACM, 43, 8, 102106. Jermann, P., Soller, A., and Muehlenbrock, M. (2001) From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning. Proceedings of the Computer Support for Collaborative Learning (CSCL) 2000, 324-331. Kramer, J., Noronha, S., and Vergo, J. (2000) A User-Centered Design Approach to Personalization. Communications of the ACM, 43, 8, 45-48. Nagashima, K. (2001) The latest trend of application and development of e-Learning in the corporate in-service training (Overseas investigation). Study Report on the Practical Use of The Component Based System Development, 221-297. (in Japanese) Ogata, H., Matsuura, K., and Yano, Y. (2000) Active Knowledge Awareness Map: Visualizing Learners Activities in a Web Based CSCL Environment. Proceedings of The International Workshop on New Technologies for Collaborative Learning (NTCL) 2000. Ogata, H., Imai, K., Matsuura, K., and Yano, Y. (1999) Knowledge Awareness Map for Open-ended and Collaborative Learning on World Wide Web. ICCE99,1, 319-326. Ogata, H. and Yano, Y. (1998) Supporting Awareness for Augmenting Participation in Collaborative Learning. Proceedings of ED-Media98. Simoff, S. (1999) Monitoring and Evaluation in Collaborative Learning Environment. Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999. Smith, B., and Cotter, P. (2000) A Personalized Television Listening Service. Communications of the ACM, 43, 8, 107-111. Wesser, M., Pfister, H. (2000) Points of Cooperation: Integrating Cooperative Learning to Web-Based Courses. Proc. of The International Workshop on New Technologies for Collaborative Learning (NTCL) 2000. Worthham, D. W. (1999) Nodal and matrix analysis of communication patterns in small groups. Proceedings of the Computer Support for Collaborative Learning (CSCL)1999.