Knowledge building and the quantity, content and quality of the interaction and participation of students in an online collaborative learning environment

Computers & Education 97 (2016) 31e48 Contents lists available at ScienceDirect Computers & Education journal homepage: www.elsevier.com/locate/compedu Knowledge building and the quantity, content and quality of the interaction and participation of students in an online collaborative learning environment Ümmühan Avcı Yücel a, *, Yasemin Koçak Usluel b a Department of Computer Education and Instructional Technology, Faculty of Education, Baskent University, Baglica, 06810, Ankara, Turkey b Department of Computer Education and Instructional Technology, Faculty of Education, Hacettepe University, Beytepe, 06800, Ankara, Turkey a r t i c l e i n f o a b s t r a c t Article history: Received 27 April 2015 Received in revised form 25 February 2016 Accepted 26 February 2016 Available online 3 March 2016 The processes of knowledge building, the interaction and participation of students in an online collaborative learning environment and the relations among them are investigated. The investigation involved a procedure carried out over 14 weeks in an online collaborative learning environment. During the procedure, the knowledge building process was executed over Knowledge Forum and the planning process over Moodle LMS. The scaffolds, quantity, content and quality of interaction and participation, and relations between scaffolds and the quantity of interaction and participation were reviewed. The participants were 145 prospective teachers. In this research, a convergent parallel design method was employed. Data were collected from multiple sources, including the log records and the content analysis of Knowledge Forum postings. Development was observed in terms of the use of opinion building and expressing forms and note creation and build-on of the students in the knowledge building processes within the online collaborative learning environment. Furthermore, most of the academic content and the quality of interaction and participation were indicators of progress of the individual student and groups of students. There was a significant increase in the quality of interaction and participation from the beginning of the term to the end of term. It is suggested that the 14-week knowledge building process contributed to learning. It was found that there was a significant relation between the use of opinion building and expressing forms and the creation and build-on of notes by the students. Results show that the course offered in an online collaborative knowledge building environment contributed to opinion building and expression, the quantity, content and quality of interaction and participation, and thus the learning of students. © 2016 Elsevier Ltd. All rights reserved. Keywords: Computer-mediated communication Cooperative/collaborative learning Learning communities Post-secondary education * Corresponding author. E-mail addresses: uavci@baskent.edu.tr (Ü.A. Yücel), kocak@hacettepe.edu.tr (Y.K. Usluel). http://dx.doi.org/10.1016/j.compedu.2016.02.015 0360-1315/© 2016 Elsevier Ltd. All rights reserved. 32 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 1. Introduction Previous studies on the effects of online collaborative learning environments on the cognition, comprehension and learning of students (Cacciamani, Donatella, Francesca, Tiziana, & Nobuko, 2012; So, Seah, & Heng, 2010; Stahl, Koschmann, & Suthers, 2006) have focused on the knowledge building skills of students. Knowledge building is a group activity in which knowledge is intentionally developed and students collaboratively work to solve, discuss, and compare common problems and detail their ideas (Scardamalia & Bereiter, 2003; 2006). The aim of knowledge building is to create valuable knowledge structures as a group. Group members share their own opinions by means of structuring or expressing to create such structures (Garrison & Cleveland-Innes, 2005; Heo, Lim, & Kim, 2010). Additionally, shared opinions are examined and discussed by other group members. The value of these opinions is developed in a collaborative environment in which all group members share their cognitive responsibilities. In the process, opinions are put forth as a group in the context of literature and examples, and knowledge structures are created from existing opinions in the environment. The aim is thus for the created knowledge structures to contribute to the learning process. In that sense, there is a close relationship between learning and knowledge building. Hence, studies have stated that knowledge building is a process that contributes to learning and learning may be ensured by developing this process (Bereiter, 2002; Scardamalia, Bereiter, & Lamon, 1994; Scardamalia, 2004). Environments that support the structuring and discussion of shared knowledge for the actualization of knowledge building are required. It has been suggested in the literature that online collaborative learning environments have been designed in accordance with this purpose and that these environments provide students more opportunities than the traditional classroom environment to participate in the knowledge building process (Lipponen, Rahikainen, Lallimo, & Hakkarainen, 2003; Noroozi, Weinberger, Biemans, Mulder, & Chizari, 2013; Stahl, 2006; Stahl, 2003). Primarily, interaction and participation in online collaborative learning environments should be ensured so that the knowledge building process is actualized (Bereiter & Scardamalia, 2003; Lipponen et al., 2003). Interaction and participation can be described as a process in which a mediator plays an important role between learners for the joint structuring of different opinions or common perspectives in the knowledge building process (Hong & Sullivan, 2009; Sing & Khine, 2006). The purpose of the knowledge building process is to encourage students to read, criticize and develop the contributions of all members more in a discussion environment rather than by placing students into groups (Law, Yuen, Wong, & Leng, 2011). The ability to suggest different opinions in a collaborative and social way, to profoundly discuss these opinions and to develop the opinions by allowing for knowledge building can be considered difficult. This challenge entails the necessity of examining the relationship among interaction, participation and learning in online collaborative learning environments (Naranjo,
s, 2012). It is believed that knowledge building can be investigated in a multi-dimensional manner by Onrubia, & Segue considering interaction and participation together. The literature on online collaborative learning environments focuses mainly on the quantity of interaction and participation, while there have been a limited number of analyses on the quality of interaction and participation. However, the quantity does not indicate quality, which cannot be addressed regardless of content. Hence, it has been recommended that database analysis tools that measure participation in online environments could be advanced in the direction of quality (Akçapınar & Aşkar, 2009). Additionally, the necessity of attaching importance to quality in this process has been emphasized (Abrami, Bernard, Bures, Borokhovski, & Tamim, 2011; Bernard, Abrami, Borokhovski, Wade, Tamim, Surkes et al., 2009; Chan & Chan, 2011; Naranjo et al., 2012). From this point of view, the present study focuses on the content, quantity and quality of interaction and participation in knowledge building processes occurring within online collaborative learning environments. 2. Theoretical framework Knowledge building is related to constructivism. A student is allowed to create, structure, criticize and develop knowledge in constructivist education. Constructivism takes the form of an approach moving from the question of how students develop knowledge to how they structure it. It is suggested that constructivism involves knowledge transfer and restructuring by the student rather than the repetition or transfer of knowledge by someone else (Perkins, 1999). Knowledge building principles have been determined so that the knowledge building process can be structured accurately. The aim is to support opinion, suggesting that knowledge building is a constructivist approach, yet there are other aspects to knowledge building (Scardamalia, 2002). One of the principles of knowledge building is epistemic agency, which was construed as the individual and collective responsibility for advancing individual and community knowledge (Russell, 2002; Scardamalia, 2002). The main objectives of the knowledge building process are to encourage students to read and criticize and also to generate greater contributions from all members in the discussion environment. Additionally, it is expected that students will respond not only within their own friend groups but to all group members and even other groups to which the students think they can contribute in discussion (Law et al., 2011). Therefore, students need various features and opportunities that support the knowledge building process (Lee, Chan, & van Aalst, 2006). These include the ability to ensure a collaborative study environment; an exclusive study area in which students can primarily structure their opinions in their respective fields and then present these opinions to group members; a system in which students can assess both themselves and group members and support their opinions; and source support. It is observed that “scaffolds” draw the most attention to these features supporting knowledge building (Scardamalia, Bransford, Kozma, & Quellmalz, 2012). Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 33 2.1. Scaffolds Scaffolds are one type of knowledge building support features that students can use to form and discuss their own opinions in the knowledge building process (Law et al., 2011). The use of scaffolds to reflect students' socio-cognitive (Scardamalia, 2002) and socio-metacognitive (Law, 2005) responsibilities is accepted as an important indicator of the knowledge building process in the literature. Scaffolds are considered important because they help students easily convey their opinions to themselves and group members. The aim is to help students develop and express their own opinions on common tasks using scaffolds (Muukkonen, Hakkarainen, & Lakkala, 2004; Seitamaa-Hakkarainen, Raunio, Raami, Muukkonen, & Hakkarainen, 2001). Students place their research, predictions, analysis and synthesis into the thinking systems of themselves and their friends by means of labeling through scaffolds. Scaffolds keep students who are either writer or reader focused on cognitive processes (van Aalst, 2009). Thus, it can be easier for students to frame their own opinions using scaffolds. Additionally, scaffolds help students establish communication and interact with their friends (Chan & Chan, 2011). Scaffolds can be designed specially according to the aim of the course by the instructor and participants are free to use any of the designed scaffolds to reflect their opinions (Law, Lu, Leng, Yuen, & Lai, 2008). However, the use of scaffolds in accordance with their meaning and aims is an important issue. For example, a scaffold of “my opinion” is designed to show the submitted opinion of an individual student, which should then be criticized, tested and accepted/rejected (Lee et al., 2006). When a student uses a scaffold of “my theory” to structure his/her opinion, it is necessary that he/she be aware of the necessity to mention the type of opinion and to contribute in the case he/she deems it necessary. When readers receive the scaffold of “my theory”, they need to know that they are face to face with a conceptual artifact that should be examined, debated and improved by the community (Sha & van Aalst, 2003; van Aalst, 2009). The aim is to support students in terms of reading notes written by their group friends and writing their own notes using scaffolds. It is thus thought that scaffolds, which have been the subject of much research (Bielaczyc & Ow, 2014; Fujita & re & Allaire, 2005; Pifarre & Cobos, 2010; Tan, Yeo, & Lim, 2005; Zhao & Chan, 2014), are important Teplovs, 2010; Laferrie in the online collaborative knowledge building process. 2.2. Knowledge building process and online collaborative learning Collaborative learning is a pedagogical process that encourages students to discuss information and problems from different perspectives, and to elaborate and refine their understanding to re- and co-construct (new) knowledge or to solve a problem (Veerman, 2000). This kind of learning environment provides students with the opportunity to participate in discussions and gives them the responsibility for their own learning. Computer supported collaborative learning (CSCL) refers to an instructive context that supports collaborative learning. The main purpose of these environments is to support sharing in the knowledge building process that is realized by the learners in developing knowledge society qualifications (Balakrishnan, 2014; Bereiter & Scardamalia, 2006; Jonassen, 2006; Stahl et al., 2006). CSCL provides the possibility to involve students in society completely, and to create a new kind of social knowledge-building structure such as an online collaborative learning environment (Bruckman, 2006). A learning environment in which it is possible for students to participate in collaborative work whenever and wherever they want through an online environment is created outside of the class. Such flexibility gives independence to the learners so that they can participate in the learning process when they are available without being limited by time and place (Lampe, Wohn, Vitak, Ellison, & Wash, 2011). In general, CSCL provides students and teachers with a flexible online collaborative learning environment (Balakrishnan, 2014). It is possible for discussion to continue in depth in an online collaborative learning environment in which it is possible to build knowledge from a distance (Shukor, Tasir, Van der Meijden, & Harun, 2014). The development of these environments can remove deficiencies seen within the learning process. However, it is not guaranteed that all participators within the online collaborative learning environments will contribute to the knowledge building process or that a collaborative structure will be provided (Kreijns, Kirschner, & Jochems, 2003). Interaction and participation within these environments thus need to be examined in detail. 2.3. Relation of the knowledge building process with interaction and participation Interaction and participation are central to the knowledge building process (Scardamalia et al., 1994). In this context, interaction and participation mediate the structuring of personal opinions and common perspectives by learners in the knowledge building community (Sing & Khine, 2006). According to Scardamalia and Bereiter (2006), if a community is unaware of information, then the knowledge building process cannot be realized. This awareness can be ensured by interaction and participation within the community. Interactions and participations in the knowledge building process are important because they provide a reliable and permanent record of the experiences and opinions raised and course activities employed for development (Niu & van Aalst, 2009). 2.4. Interaction and participation in online collaborative learning environments The literature considers that online interaction and participation are key drivers for learning (Hrastinski, 2008). Although they are conceptualized differently by different researchers, interaction and participation are combined in many studies (Chan & Chan, 2011; Hakkarainen & Palonen, 2003; Heo et al., 2010; Hrastinski, 2009; Jeong & Davidson-Shivers, 2006; Jeong, 34 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 2006; Jung, Choi, Lim, & Leem, 2002; Lipponen et al., 2003; Naranjo et al., 2012; Prinsen, Volman, Terwel, & van den Eeden, 2009; Sing & Khine, 2006). These studies consider interaction and participation to relate to each other. Various dimensions of online interaction and similarly online participation have been examined; e.g., the types of interaction and participation and their various effects. The studies consider interaction and participation to be a basic component of learning and the knowledge building process. The realization of participation is difficult without interaction, and similarly, it is expected that interaction occurs in an environment where there is participation (Wenger, 1998). That is, every student participating in an online collaborative learning environment is actually thought of as providing an interaction. Interaction and participation that are closely relevant to each other are thus examined together in the present study. 3. Purpose of the study An online collaborative learning environment was primarily developed in the context of a course in the present study. The aim of the study is to examine the knowledge building processes of the students, the interaction and participation of the students and the relation between knowledge building processes and interaction and participation in the environment. The study thus contributes to the comprehension and development of the knowledge building process in a holistic way. To accomplish this goal, the following sub-problems are posed. 1. What are the knowledge building processes of students according to scaffolds within an online collaborative learning environment in the context of a course? 1.1 What are the knowledge building processes of the students according to their opinion building forms? 1.2 What are the knowledge building processes of the students according to their opinion expression forms? 2. What are the interaction and participation of the students within an online collaborative learning environment in the context of a course? 2.1 What is the quantity of interaction and participation of the students? 2.2 What is the content of interaction and participation of the students? 2.3 What is the quality of interaction and participation of the students? 3. What is the relation between the quantity of interaction and participation of the students and scaffolds as knowledge building process within an online collaborative learning environment in the context of a course? 4. Methods The methodology of the present study has a convergent parallel design, which is a type of mixed research method (Creswell, 2012). In this design, the researcher separately analyzes the quantitative and qualitative data sets, compares or correlates the analysis results of each data set and comments on whether the results support each other. The aim is to illustrate quantitative results with qualitative findings, to synthesize qualitative and quantitative data and thus develop a more complete understanding of phenomena, and to compare multiple levels within a system (Creswell & Plano Clark, 2011). First, both quantitative and qualitative data are collected in this study. These two data collections are concurrent but separate. That is, one does not depend on the results of the other. Additionally, the two data sets hold equal importance in terms of the analysis of research problems. Second, the two data sets are analyzed separately and independently. Results of the two data sets are combined in the third stage through correlation. In the final stage, to what extent and in what ways this relation occurs is interpreted in accordance with the aim of the research. 4.1. Participants The study group consists of 145 prospective teachers taking the course “Instructional Technologies and Material Development Spring 2011e2012” at a private university in Ankara, Turkey. Students from five departmentsdPreschool, Classroom, Elementary Mathematics, Turkish and English Teachingdparticipated in the study. Students in these five departments were distributed into six branches. In creating the branches, an effort was made to distribute the students evenly and a total of 19 groups each comprising six to nine students were formed. 4.2. Procedure An online collaborative learning environment in the context of a course was primarily designed in the procedure process. The program Knowledge Forum was selected as the procedure environment. The groups were formed, the subject titles were prepared and the enrolment of students was ensured in this environment. Scaffolds were introduced in the procedure environment. A syllabus that contained the discussion topics and details for each week of the course was prepared for the instructor and students. The students were provided with information on how to use the Knowledge Forum as the procedure environment, and the use of the program was described in detail. The knowledge building process in the context of a course was then carried out in this environment for 14 weeks. Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 35 4.2.1. Learning environment Online collaborative learning environments having various features were examined in selecting the procedure environment for the present study. The examined environments were open-source learning environments that offer various tools that ensure interaction and participation and that allow students to easily interact with each other and their teachers. Consideration was given to ensuring online collaboration, messaging and discussion, the creation and sharing of content, and the formation and correction of new opinions. In addition, compatibility with database analysis tools and the possibility of detailed log analysis were considered. The Knowledge Forum developed at the University of Toronto was ultimately chosen as the learning environment as it provides an online collaborative learning and knowledge building environment, has a Turkish language option, supports the knowledge building process with various features such as scaffolds, key words, and additional explanations, has a log analysis tool and is a program open for improvement. The use of the Knowledge Forum was discussed with its developers and a permit to use the program for two terms was obtained. The literature considers the Knowledge Forum to be a collaborative knowledge building environment that organizes collections of knowledge and ensures the formation of new opinions. In this environment supporting the collaborative learning process, users can support their opinions by adding graphic, audio, and video media. Features of the Knowledge Forum are designed to help students to refine, reframe and advance ideas (Lee et al., 2006). This environment, in which users take on the learning responsibilities for both themselves and other users, has a Turkish language option and an Analytic Toolkit (ATK) log analysis tool that can perform database analysis by determining various indexes such as Database Overview, Basic Knowledge Building Measures, Use of Scaffold Supports, Single Author Report, Activity Note Creation/Note Reading, Who's Read Whose Notes, Who's Coauthored Notes with Whom, Who's Built on Whom, Who's Referenced Whom, Who's Annotated Whose Notes, Who's Linked to Whom, Group Interaction, Reading/Building on/Linking to Other Work, Growth of Database, Word Trace/Vocabulary Analyzer, Custom Report/Custom Participation Score, Build-on Trees, Scaffold Combinations. ATK is a Web-based analysis tool that was separately designed for the Knowledge Forum and developed for the purpose of recording log data of interactions and participations in the knowledge building process (Burtis, 2002). 4.2.2. Course procedure The course lasted 14 weeks. In the first three weeks, Knowledge Forum subject titles were made available, students enrolled in the environment, students were instructed how to use the environment and groups were formed. The knowledge building process was carried out for an example discussion topic in the fourth week and the process was disclosed to the students in an applied way. The knowledge building process involving students at the forefront was actualized in the subsequent 10 months. Responsibility and authority were appointed to the students in the knowledge building process. Knowledge building was carried out through the Knowledge Forum and the planning process was carried out through the Moodle Learning Management System (LMS) in the procedure. The Moodle LMS was used in this process because the planning process was performed by the instructor prior to the procedure process. The Knowledge Forum includes default scaffolds such as those for theory building and opinion. The students are free to decide which and how many scaffolds to use in knowledge building process (Law et al., 2011). In the present study, default scaffolds of the knowledge building environment were deemed appropriate for the purpose of the course and thus used without modification. Necessary information was provided to students within the scope of the study before they began to use the scaffolds so that they were aware of what is described by the concepts used and what they will be expressing when they use these concepts. The students are told the purpose of each scaffold and examples are given. The students were allowed to freely choose which scaffolds to use. Thus, the students will be able to advance the frontiers of knowledge in knowledge building process. Students participated in the knowledge building process by creating, reading and editing notes in the online collaborative learning environment and they interacted with each other, the content and the interface in this process. The procedure process thus involved studentestudent, studentecontent and studenteinterface interactions. There was studenteteacher interaction at the beginning of the procedure process where the knowledge building environment and lesson plan were introduced to the students. Fig. 1 presents the framework and flow of the process. 4.3. Data collection tools The following data collection tools were used within the scope of this study. ▪ Online collaborative learning environment (Knowledge Forum) log records ▪ Coding scheme created by Hou and Wu (2011) for the content analysis of knowledge building and social interaction in synchronous discussion-based learning activities ▪ A rubric for the assessment of the quality of interaction and participation within the knowledge building process (Avci Yucel & Usluel, 2013) Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 36 Interaction and participation quality Reflecting metacognitive opinions Revealing cognitive products Interaction and participation content Collaborative knowledge building Interaction and participation quantity End of period Discussing perspectives Student–student Student–content More intensive Weeks 4–14 Procedure process Developing a common perspective Orientation to different perspectives Student–interface Student–teacher Weeks 1–3 More intensive Environment examination and preprocedure process Discussion, criticism and justification Beginning of period Interaction and d participation ti i ti process 1. 2. Individual opinions Scaffolds Opinion building forms Opinion expression forms Individual opinions of others Online collaborative knowledge building process Fig. 1. Online collaborative knowledge building. 4.3.1. Log recordings of the online collaborative learning environment (Knowledge Forum) The Knowledge Forum was examined using the Analytic Toolkit (ATK) for log records. It is possible to access log records of the learning environment with ATK by entering the database address, user name and password. General information on interaction and participation in the database can be obtained through ATK. Additionally, search queries such as desired report date, group information, discussion topic, and time term may be input, allowing desired information to be easily and quickly accessed. Selected data on the desired group or groups may be listed in a desired time term through ATK. Additionally, obtained reports may be saved in text format. Data on the scaffolds and the quantity of interaction and participation of the student base were obtained employing ATK within the scope of this study. The varieties and explanations of scaffolds used in the study are given in Table 1 and the quantities of interaction and participation are given in Table 2. Scaffolds obtained from ATK are given in Attachment A together with examples of log data showing interaction and participation quantities. 4.3.2. Coding scheme for the content analysis of knowledge construction and social interaction in synchronous discussion-based learning activities The coding scheme created by Hou and Wu (2011) for the content analysis of social interaction and knowledge building in synchronous discussion-based learning activities has four indicators: academically related, task coordination, social Table 1 Scaffolds. Description Types 1. Opinion building forms (Theory building): forms used by students not to create a new theory but to associate their opinions with the literature and to build and support their opinions with a theoretical structure. My theory I need to understand This theory cannot explain A better theory New information Putting our knowledge together My opinion Elaboration Example Evidence Reason Conclusion Different opinion 2. Opinion expression forms (Opinion): forms used by students to express their opinions. Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 37 Table 2 Quantity of interaction and participation. Type Description Notes created Notes edited Notes read Build-ons created Gives the number of notes created by students. Gives the number of corrections made by the students for either their own notes or the notes created by their group friends. Gives the number of notes read by students, or in other words, how many times the notes in a database were opened. Gives the number of build-on notes of the students. Students can build on each other's notes in the knowledge building environment. interaction and off-topic indicators. Researchers have adapted the social knowledge building coding scheme of Gunawardena, Lowe, and Anderson (1997) to understand the social knowledge building process of learners. Additionally, they have added new articles to the coding scheme for social interaction features of synchronous discussions; e.g., characterization (Thirunarayanan, 2000) and socialization (Isaacs, Walendowski, & Ranganathan, 2002; Nardi Whittaker & Bradner, 2000). Thus, the coding scheme can be used for the content analysis of social interaction and knowledge building in synchronous discussion-based learning activities. To ensure the validity of the coding scheme, Rourke and Anderson (2004) suggested that a coding scheme based on literature reviews or widely adopted in previous research be used. In the present study, each note was treated as a unit and coded, and the codes were then arranged in chronological order. A total of 3112 codes were obtained as a result of discussion among 40 students. The entire record was then recoded by another coder with the same expertise to ensure inter-rater consistency. The kappa coefficient was 0.67, which reached the 0.001 level of significance. All indicators were viewed as a type of interaction indicator, whose original name “social interaction” was changed to “social content”. Thus, the interaction and participation content was analyzed within the scope of academically related, task coordination, social and off-topic indicators. 4.3.3. Rubric for the quality assessment of interaction and participation in the knowledge building process The present study used a rubric developed by Avci Yucel and Usluel (2013) to assess the interaction and participation quality in the online collaborative learning environment. This rubric has four indicators: the contribution of discussion messages to the task, the relevance of discussion messages to the topic, the continuity of discussion messages, and the direction of relations between discussion messages. Discussion messages of each group were scored 0e3 by the instructor (i.e., subject specialist) when examining the contribution of discussion messages to the task and the relevance of discussion messages to the topic. Scores of 3, 2, 1 and 0 correspond to high, medium, low and no contributions of discussion messages to the task and high, medium, low and no relevance to the topic, respectively. Messages were analyzed on the basis of the group in terms of the continuity of discussion messages and the direction of relations between discussion messages. The smallest acceptable message clusters for analysis had two notes and consisted of at least two different group members. Note lengths were scored 1e4 in terms of message continuity. A score of 1 corresponds to small message clusters with note length between 2 and 5, a score of 2 to medium message clusters with note length between 6 and 20, a score of 3 to large message clusters with note length between 21 and 40, and a score of 4 to larger message clusters with note length of 40 or more. The direction of relations between discussion messages for each message cluster was scored 1e4. Messages of each group had a one-way relation (1 point), two-way relation (2 points), three-way relation (3 points) or multi-way relation (4 points). Researchers analyzed and scored each note that groups contributed to the discussion topic in accordance with rubric indicators in their procedure involving 145 students. Total points were obtained for the message cluster, discussion topic and group. Another coder with the same expertise re-scored the messages to ensure inter-rater consistency. The kappa coefficient was 0.78. 4.4. Data analysis Descriptive statistics, one-way repeated-measures analysis of variance (ANOVA), content analysis and correlation analysis were adopted in data analysis. Statistical analysis was conducted using SPSS version 16.0. Table 3 gives the analysis used according to the sub-problem. The process of data analysis is described as follows. A GreenhouseeGeisser correction (Greenhouse & Geisser, 1959) was used when the sphericity assumption did not hold; the p-value was less than 0.05 except for note creation in Mauchly's € ztürk, 2007; King, Rosopa, & Minium, 2010) required for one-way repeated-measures ANOVA. The psphericity test (Büyüko value exceeded 0.05 (p ¼ 0.135) and repeated-measures ANOVA was valid in the case of note creation. The effect size was obtained with the (h2) correlation coefficient. According to Cohen (1988), h2 ¼ 0.01 corresponds to a “small” effect, h2 ¼ 0.06 to a “medium” effect, and h2 ¼ 0.14 or higher to a “large” effect. In this research, the size of an effect was interpreted in the same way. The contents of messages were analyzed as follows. Each note that a group contributed to the discussion topic was examined and correlated with the codes determined in the coding scheme. Multiple codes were used for a note having different contents. The instances of each code were counted and the total number of codes related to a message cluster was obtained. In the analysis of message quality, each note contributed by a group to the discussion topic was examined. Message clusters formed by these notes were examined in accordance with the indicators of the quality assessment rubric. Scoring was performed according to the determined scoring scheme and final scores were obtained by summing individual scores. 38 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 Table 3 Sub-problems and data analyses. Sub problems Data analyses What are the knowledge building processes of students What are the knowledge building according to scaffolds within an online collaborative processes of the students according to their opinion building forms? learning environment in the context of a course? What are the knowledge building processes of the students according to their opinion expression forms? What are the interaction and participation of students What is the quantity of interaction and within an online collaborative learning environment participation of the students? in the context of a course? What is the content of interaction and participation of the students? Improvement at times T1 (beginning of the term), T2 (mid-term) and T3 (end of the term) One-way ANOVA for repeated measurements What is the quality of interaction and participation of the students? What is the relation between the quantity of interaction and participation of the students and scaffolds as knowledge building process within an online collaborative learning environment in the context of a course? One-way ANOVA for repeated measurements Content analysis Coding scheme for the content analysis of knowledge construction and social interaction in synchronous discussion-based learning activities Content analysis Rubric for the assessment of the quality of interaction and participation in knowledge building process Correlation analysis Cohen's kappa analysis was employed to determine the inter-rater consistency between encoders. A total of 8066 notes obtained from six branches and 19 groups in time periods T1, T2 and T3 of the 14-week procedure process were coded. A secondary coder coded 2601 notes from six groups including one group randomly selected from each branch in three different time periods. Inter-rater reliability was calculated, and the kappa coefficient was 0.76 for the content of interaction and participation and 0.78 for the quality of interaction and participation. The kappa coefficients indicate significant correlation between the two coders. The Spearman correlation coefficient was used to find the correlation between variables because a KolmogoroveSmirnov test identified that the distribution was not normal (KS- 0.000, p < 0.05). In the present study, a correlation coefficient between 0 and 0.20 indicates a “very weak relation”, that between 0.21 and 0.40 a “weak relation”, that between 0.41 and 60 a “moderate relation”, that between 0.61 and 0.80 a “strong relation”, and that between 0.81 and 1.00 a “very strong relation” (Fraenkel, Wallen, & Hyun, 2012). Data of all students for the beginning of term (T1) and end of term (T3) relating to scaffolds and the quantity of interaction and participation were primarily collected for correlation analysis. The difference obtained by subtracting data collected at the end of term from the data collected at the beginning of term demonstrated improvements over the course of the term. The correlation between improvements was thus examined (Fig. 2). Fig. 2. Process followed in correlation analysis. Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 39 5. Results and discussion 5.1. Findings on the knowledge building process and quantities of interaction and participation of students Table 4 reveals that there was a significant difference with large effect size among time periods T1, T2 and T3 for opinion building, opinion expression forms, note creation and note build-on of students. Estimated marginal mean values at time periods T1, T2 and T3 reveal a continuous improvement (Fig. 3). However, it is worth noting that there is an insignificant change in the means of note correction of students. There is a significant decrease in the note reading means of students. Students need various support methods such as scaffolds to create and discuss their own opinions (Law et al., 2011). Additionally, Chan and Chan (2011) found that students frame their opinions and use their opinions to guide others through scaffolds. Students thus use scaffolds that benefit information sharing and criticism in the beginning, middle and end of the process. Furthermore, it is thought that this development relates to the quality of interaction and participation. Quality can be evaluated using indicators of the continuity of discussion messages, the relation of the messages to the subject and the contribution of the messages to the task. It is thought that scaffolds make an important contribution in terms of realizing these indicators. For example, scaffolds that students may use to share their opinions without getting off topic are within easy reach on the screen of note creation in the knowledge building environment. It is thought that students reading each other's notes will state their opinions easily with the help of these scaffolds. There is thus quality interaction and a participation process that does not get off topic and contributes to the task and shows continuity. An increase in note creation by students may be explained by continuous development of the literacy skills of students through the procedure process, gradual adaptation of students to the procedure environment and the knowledge building process. The quality assessment rubric of interaction and participation reveals that 14 of the 19 groups in the present study continuously improved through the beginning, middle and end of the process. This can also be interpreted as students developing their notes in terms of content. Previous studies have revealed that the quantity of note creation is related to the €rvel€ depth of explanation (Hakkarainen, Lipponen, & Ja a, 2002) and basic literacy acquisition (Scardamalia et al., 1994). There are two possible reasons for the variability in the quantity of note correction by students. First, at the beginning of the term, notes created by students are more frequently in need of correction, while students make mistakes (such as using incorrect scaffolds or making simple spelling errors) less frequently and increasingly check their own notes towards mid- and end-term. Second, students can revise and correct their notes in the procedure environment, yet data on corrected notes obtained from ATK might also include the re-registry of a note by the student owing to a technical problem. That is, it is possible that a student can save a note that he/she opens and reads but does not correct (Chan, Lee, & van Aalst, 2001). The decrease in the quantity of note reading by students can be explained by the students steadily focusing more on the titles of the notes, an increase in the students' vocabulary awareness, and the students gradually becoming more familiar with each other's way of thinking such that the students can determine what is being communicated in a note without reading the content of the note. Similarly, group friends improve in their ability to predict the content of notes by correctly determining the titles of notes in a summative and descriptive manner. Furthermore, Chan et al. (2001) found that it is possible to implement a successful knowledge building process with less reading in the event that students intensively use the search features of the Knowledge Forum. For instance, a student may search for notes containing “theories” sent the previous week instead of opening many notes. Similarly, students may find the notes that they wish to contribute to or correct according to their area of interest by searching and thus may not feel obliged to open all notes. A student thus reads fewer notes on average toward the end of term. Table 4 One-way repeated-measures ANOVA results. Indicator Sub-category of indicator Time period N X sd Sum of squares df Mean square F p h2 Bonferroni Scaffold Opinion building forms T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3 T1 T2 T3 145 145 145 145 145 145 145 145 145 145 145 145 145 145 145 145 145 145 4.35 4.62 11.23 5.54 15.54 18.25 15.01 20.77 27.63 6.74 4.72 5.4828 5.45 3.92 2.94 6.25 17.17 19.88 5.005 7.970 15.540 6.113 14.401 19.316 15.813 14.186 21.806 12.604 5.664 7.70651 9.065 5.645 2.592 5.693 12.509 17.879 4407.370 1.605 2745.625 30.894 0.000 0.177 1-3, 2-3 12981.641 1.751 7412.758 54.176 0.000 0.273 2-1, 3-1 11578.097 2 5789.048 35.524 0.000 0.198 1-2, 1e3, 2-3 304.014 1.573 193.324 2.722 0.081 0.019 e 461.549 1.793 257.486 9.960 0.000 0.065 1e3 15108.124 1.770 8534.006 64.274 0.000 0.309 2-1, 3-1 Opinion expression forms Quantity Note creation Note correction Note reading Note build-on 1: T1 (beginning of term); 2: T2 (mid-term); 3: T3 (end of term). 40 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 Fig. 3. T1, T2 and T3 means on knowledge building processes and quantities of interaction and participation of students. There are multiple reasons for a greater increase in the number of notes that students build on at the middle and end of the term than at the beginning of the term. One reason is that the students feel the need to answer their group friends because the group friends of students add notes to the notes they write and this cycle will build in strength continuously. Another reason is that students try hard to make their group discussion more successful than the discussions of other groups. Similarly, Cho, Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 41 Table 5 Analyzed indicators and sub-categories of indicators. Indicator Sub-categories of indicators Content 1. Academically related 2. Task coordination 3. Social 4. Quality 5. Note creation 6. Note correction 7. Note reading 8. Note build-on 9. Build-on trees 10. Opinion building forms of students 11. Opinion expression forms of students Quality Quantity Scaffolds Quality and Academically Related Content indicators showing the highest increase in T1, T2, and T3 (14 groups) Social Content increase in T1 and T2, decrease in T2 and T3 (15 groups) Task coordination content (13 groups) Opinion building forms of students (11 groups) Note Correction indicator increasing most in T1, T2, and T3 (one group) Fig. 4. Overview of analysis in terms of indicators. Chen, and Chung (2010) found that participants' social factors of mutual interaction and belonging to a group affect the participants' intent to share knowledge. It has been found that participants like to respond to the contributions of other members of a group to which they think they belong and believe they will similarly receive a response to their contributions. These social factors directly affect knowledge-building intentions. Notes are built on for many reasons such as to provide more detail, to support or reject a note, to propose a better note, and to correct a note. As each student builds on notes for these reasons, a tree of notes is seen to grow. 5.2. Findings on the content and quality of the interaction and participation of students The content and quality of the interaction and participation of students were analyzed by content analysis in the online collaborative learning environment and group scores (19 groups) were obtained. Group scores for all indicators were obtained to determine the position of content and quality among other indicators. Notes were evaluated as the beginning (T1), middle (T2) and end (T3) of the 14-week course. Indicators were divided into four groups, namely content, quality, quantity and scaffolds, and had 11 sub-categories (Table 5). The analysis (Fig. 4) revealed that indicators of academically related content and quality increased the most over the 14 weeks. There were increases in quality and academically related content in 14 of the 19 groups in time periods T1, T2 and T3. There was a continuous decrease only in note correction, which is one of the interaction and participation quantities, in time periods T1, T2 and T3 and that was for only one of the 19 groups. Social content, which is one of the interaction and participation contents, varied the most. 5.2.1. Findings on the content of interaction and participation of students Analysis revealed that academically related content is the most frequently observed content, followed by social content, task coordination content and off-topic content (Table 6). Among academic content, information shared and compared with support opinions and observations accounted for 50% of all interaction and participation content (AK1); questions and the search for answers in exploring and solving dissonance or inconsistency accounted for 26% (AK2); discussions that built knowledge together and determined meaning accounted for 6% (AK3); testing of knowledge built together based on experience and other resources accounted for 6% (AK4); and reflection on meta-cognitive opinions and summarizing agreements at a level above the newly constructed meaning accounted for only 1% (AK5) (Fig. 5). The results show that AK5 (rise-above principle) is realized at a low level from the 12 knowledge building principles that are the key words of knowledge building behavior. In the present study, it is important to realize the Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 42 Table 6 Total numbers of instances of interaction and participation content. Sub-indicators of content Code Phase T1 T2 T3 Total Academically related (AK) AK AK AK AK AK Sharing/comparing information Discovery and exploration of dissonance or inconsistency among participants Negotiation of meaning/co-construction of knowledge Testing and modification of proposed synthesis or co-construction Agreement statement(s)/application of newly constructed meaning 917 202 86 22 64 1291 10 20 4 34 0 17 2 19 0 0 909 1257 207 421 16 2810 41 54 22 117 0 364 41 405 0 0 2681 896 253 93 35 3958 97 75 11 183 0 104 4 108 35 35 4507 2355 546 536 115 8059 148 149 37 334 0 485 47 532 35 35 Total Task coordination (GO) Total Social (S) Total Off topic (KD) Total 1 2 3 4 5 GO1 GO2 GO3 Leading or giving orders Comment on task coordination Questions on task coordination S1 S2 S3 Interaction of identifying each other Social interaction relevant to learning topics Interactions regarding technical issues, such as Internet connection KD1 Messages irrelevant to the discussion task Fig. 5. Code distributions and percentages of interaction and participation content. knowledge building process primarily within the course period. As mentioned by Hou and Wu (2011), Chen, Chen, and Tsai (2009) and Branon and Essex (2001), there may be a deficiency in the ability to reflect the students' opinions immediately in the study period because of the nature of synchronous interactions. Cognitive and meta-cognitive restrictions occur in synchronous interactions. Hou and Wu (2011) found that synchronous interactions necessitate more supporting mechanisms to ease the knowledge building process and to develop cognitive abilities. Additionally, students in the present study are only in the second year of their program and do not have experience in reflecting meta-cognitive opinions. Social content accounted for a portion of the interaction and participation of students in the online learning environment. Students did not interact to identify each other (S1). Students mostly interacted socially to learn topics (5%) and subsequently for technical issues that arose during knowledge building (0.5%). Another portion of the interaction and participation of students related to task coordination. Students mostly guided or gave orders about task coordination (2%), presented their opinions and recommendations on task coordination (2%) and questioned the current task coordination (0.4%). The proportion of off-topic content (KD) accounted for only 0.3% of all interaction and participation content. It is remarkable that the proportions of social and task coordination content were similar to each other but lower than the proportion of academic content. The result can be explained in that while students created only academic content within the knowledge building period, they interacted and participated to prevent getting off topic or to prevent misunderstandings. That is, social and task coordination content is dependent on the academic content. Therefore, because task coordination becomes clear after a short period of time, social interaction and participation might cease after a certain time. 5.2.2. Findings on the quality of the interaction and participation of students Previous studies have revealed that a large number of discussions seem to shut down in online collaborative learning environments (Hewitt, 2005) and get off topic (Hou, Chang, & Sung, 2007). However, in the present study, the relevance of Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 43 Table 7 Total scores of quality assessment indicators and branches in time periods T1, T2 and T3. T1 T2 T3 Total Quality assessment indicators Relevance of messages Contribution of messages to the task Continuity of messages Correlation direction of messages Total 395 269 284 385 1333 958 580 610 710 2858 1171 844 756 996 3767 2524 1693 1650 2091 7958 Branches Branch 1 Branch 2 Branch 3 Branch 4 Branch 5 Branch 6 Total 265 101 273 326 214 154 1333 593 448 368 787 367 227 2790 804 305 848 987 602 289 3835 1662 854 1489 2100 1183 670 7958 Table 8 Correlations between interaction and participation quantities of students and scaffolds in the online collaborative learning environment. Opinion building forms Opinion expression forms Note creation Note correction 0.61** 0.54** 0.16* 0.07 Note reading 0.20* 0.00 Note build-on 0.53* 0.59** Correlation significance level *p < 0.05, **p < 0.01. The Spearman correlation coefficient was used. the discussion message to the topic was rated highest among the quality assessment indicators in determining the quality of interaction and participation. The direction of relations between discussion messages was the second indicator of the quality of interaction and participation. This indicator represents quality by showing how students respond to the contributions of other students (Coll, Engel, & Bustos, 2009; Naranjo et al., 2012). It is also seen that indicators of task contribution and continuity of discussion messages play an important role in determining quality. The indicator of the contribution of messages to the task shows the focus and contribution of groups in knowledge building tasks (Hou & Wu, 2011). There are more small and medium message clusters than large and very large message clusters in the online collaborative knowledge building environment because the indicator of the continuity of discussion messages has a lower score than other indicators. The groups having the highest quality interaction and participation were gathered in branch 4. This branch comprised mixed group of students of Elementary Mathematics and Classroom Teaching. Additionally, both departments had their own branches (3 and 2). Therefore, it can be interpreted as this mixed group of students examined the other branches in detail because they had friends in those branches, and they participated more so as not to fall behind their friends. Quality in terms of each indicator increased across the time periods T1, T2 and T3 in most branches. There was a notable increase from the beginning of term to the end of term in the quality of interaction and participation in the knowledge building processes of students within the scope of this study. Data for the abovementioned findings are given in Table 7. 5.3. Findings on the relation between the interaction and participation of students and the knowledge building process Correlation analysis reveals a positive, significant and strong relation between the use of opinion building forms and note creation and a positive, significant and very weak relation between the use of opinion building forms and note correction and note reading of students. A positive, significant and moderate relation was found between the use of opinion building forms and the note build-on of students. No significant relation was found between the use of opinion expression forms and the note correction and note reading of students while there was a positive, significant and moderate relation between the use of opinion expression forms and note creation. A positive, significant and moderate relation was found between the use of opinion expression forms and the note build-on of students (Table 8). The findings reveal that the note creation and note build-on of students increased if more scaffolds were used by students to build their opinions. The strong or moderate relation between scaffolds used by students to build their opinions and the note creation and note build-on of students indicates that improvements in these fields are collective. There was a similar relation between scaffolds used by students to express their opinions and note creation, and the note build-on of students. Additionally, the moderate strength of many of the relations between indicators suggests that students also create notes or build on notes without benefiting from the scaffolds. This can be explained by the usage 44 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 structure of the knowledge building environment. Students can give their opinions within the knowledge building environment in various ways such as by the creation, correction, criticism and development of notes. Additionally, they can use the scaffolds to build or express their opinions more clearly. It might therefore be a meaningful and positive relation among these indicators. The weak or insignificant relation between scaffolds used by students to build or express their opinions and note reading and note correction suggests that students either benefited from scaffolds very little or not at all while they corrected their own notes or read notes written by their friends. The analysis of the content of scaffolds used by students to build and express opinions reveals that the scaffolds are used more to create a new note or build-on notes of their friends. Additionally, it is thought that the significant but very weak relation between note reading and note correction with the scaffolds used for opinion building is due to the students' need to benefit from a conceptual infrastructure while building their opinions. Sha and van Aalst (2003) stated that the use of opinion building scaffolds necessitates more conceptual information with respect to the use of opinion expressing scaffolds; e.g., when a student uses the “My theory” scaffold to build his/her opinion, he/she needs to be aware of the theory and contributes if he/she deems it necessary. It is thus thought that students might feel it necessary to read or correct notes in this period. 6. Conclusion and recommendations An increasing improvement in the quality of student participation and interaction from the beginning to the end of the term was observed in scaffolds in the knowledge building process within the scope of this study. Scaffolds provide information on how the improvement occurred in the knowledge building process. It can thus be said that students need scaffolds that will guide them and support their opinions in collaborative learning environments. In this way, scaffolds are representations of a teacher in the knowledge building process. Scaffolds play the role of a teacher by supporting and leading students in expressing and building their opinions in a knowledge building process without a teacher; i.e., a process involving intense studentestudent, studentecontent and studenteinterface interactions. It is thus suggested that the number and scope of scaffolds in these environments be increased. Scaffolds are one of the most important support tools for the epistemic agency of students in the knowledge building process in which students follow their improvements. The use of an online collaborative learning environment by students encourages the students to create notes and to follow the note creation process of their friends. In this knowledge building process, it was observed that a student who previously researched discussion topics and studied the course material created more notes in a shorter time. Students having no information about the course were observed to conduct research and attempt to adhere to the speed of their friends after being influenced by those students. This situation occasionally led to the creation of notes with off-topic content. However, other group members either developed and corrected this content or warned the owner of notes so as not to prevent the improvement of their groups. Note build-on thus indicated improvement until the end of term through the creation, criticism or elaboration of notes and the support of reasoning and conclusion scaffolds. Previous studies found that interaction and participation quantities sometimes affect quality (Naranjo et al., 2012), student success (Lee et al., 2006) and levels and discussions of interaction and participation (Cacciamani et al., 2012). They are therefore among frequently referenced indicators although there are ongoing debates whether they are indicators of improvement. For instance, Lee et al. (2006) stated that relating the number of notes that are created, read and corrected by students in their studies to the scaffolds used by students may explain the high rate of change in the success of students. Cacciamani et al. (2012) examined how participation levels affect online course discussions. They found that a higher level of participation can be ensured with higher interaction and participation quantities such as the number of created notes or build-on notes. They also found that creating a discussion environment with a higher level of participation is important in encouraging epistemic agency. Sing and Khine (2006) found that active and wide participation is necessary for successful knowledge building and this situation requires a higher level of sent messages. The present study also found that an increase in the quantities of note creation and build-on is an effective indicator of knowledge building. The results obtained for the interaction and participation quantities of students are actually indicators of improvement. This improvement seen throughout the term could be an indicator that students were positively influenced by the knowledge building process within the online collaborative learning environment. It can be recommended that this improvement that affects many indicators such as quality and success is important, and what is influenced by this improvement and what influences this improvement should thus be examined. Increases in academically related content and quality may be indicators that the interaction and participations of students improved in terms of both academically related content and quality. The first goal of the student when he/she creates a new note or builds on existing notes is to create academically related content. It is believed that scaffolds are also involved in creating academically related content in interaction and participations within an online collaborative learning environment. It is thought that scaffolds keep the student focused on academically related content because they support students to express or build their opinions. It was observed that students contributing off-topic or social content were warned by their friends with task coordination content. It is thus concluded that content is a significant indicator within an interaction and participation process in which students have a significant effect on each other. Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 45 There have been various studies on off-topic and academically related content (Hou & Wu, 2011; Hou et al., 2007, 2008). For instance, Hou and Wu (2011) reported proportions of 58% for off-topic content and 30% for sharing/comparing information as the dimensions of academically related content in their study. They found that the high-quality discussion groups performed better than the low-quality discussion groups in terms of participation, diversity in knowledge building, and coordination. However, they also had more off-topic discussions. In the present study, off-topic content was encountered less and sharing/comparing information of academically related content was encountered most. This difference between the two studies may be due to the students' perspective on the procedure process, the students' prior knowledge, the students' experience with similar programs, which classes the students were in and forms of assessment. It is important to examine or control the effects of these factors. In the knowledge building process, students are expected to reflect their meta-cognitive opinions. Such reflection is difficult and requires synthesis, but improves the knowledge building process by providing more creative and learning supportive content. Students need to gain experience and familiarity by engaging in the knowledge building process in their further studies to reflect their meta-cognitive opinions. The difference can be seen in the proportions of academically related content indicators for similar groups in future studies. The groups for which there was an increase in the knowledge building process and the quantity of interaction and participation also improved in terms of quality. Therefore, improvements of the quantity and quality indicators supported each other in the present study. It is suggested that the knowledge building process lasting 14 weeks in the present study contributed to learning. It is also believed that the present study contributed to the analysis of the knowledge building process in a more profound and holistic way by considering multiple dimensions of quality. Hou and Wu (2011) stated that it is possible to comprehend the limitations of knowledge building by examining the differences between students participating in discussions of high and low quality. Therefore, a different perspective on the knowledge building process would be realized by examining the differences between the groups that showed the greatest improvement in terms of quality and the other groups. Suggestions on how instructors should plan the process can then be made. The results obtained in the present study may help teachers in guiding online collaborative knowledge building activities and designing the environments. Quality is usually viewed as an important process that is difficult to assess. Most studies on quality give recommendations for developing quality (Abrami et al., 2011; Bernard et al., 2009; Hou & Wu, 2011; Kanuka, 2011; Law et al., 2008; Moskaliuk, Kimmerle, & Cress, 2012; Naranjo et al., 2012; Noroozi et al., 2013; Sing & Khine, 2006). In this respect, it is necessary to sustain the development of quality to make better quality assessments. It is suggested that different studies on quality assessments would benefit researchers in revealing the different aspects of the quality development process and assist educators in designing procedures and tests. The relation between the use of scaffolds in knowledge building processes and the quantities of note creation and note build-on shows that the interaction and participation quantity is an important component of the online collaborative knowledge building process. It can thus be stated that these indicators support and trigger each other. Improvements of students related to knowledge building processes will be seen in the students' interaction and participation. Similarly, Niu and van Aalst (2009) found a significant relation among data of Analytic Toolkit Productivity (Notes Created, Notes Read) and Analytic Toolkit Knowledge Building (Percentage of Notes with Links, Note Revisions, Scaffold Uses) in their study. They stated that this productivity (notes created and percentage of notes read are productivity measures) can contribute to knowledge building but it needs to be productivity aimed at constructing explanations or theories. They expressed this situation as evidence for an explanation-seeking discourse creating a conceptual change. The above results can be used to improve the structure of the learningeteaching process and thus increase learning and direct the designs of online collaborative learning environments. The correlation among analyzed indicators may be strengthened according to the properties of each tool introduced to the environments, and expected improvements of the learning of students might be made in multiple directions. Therefore, desired improvements can be ensured by introducing simple tools to the learning environment. The present study revealed that results on scaffolds used by students and the quantities, contents and qualities of interaction and participation support each other. Additionally, the scaffolds used by students affect the quantities and contents of interaction and participation and qualities. Using similar environments in which students are so influenced by group friends and selecting current topics that might be of interest to discuss in these environments could simultaneously ensure many expected improvements. The present study showed that there were different but intertwined indicators that affect each other. The present study contributes to various fields in that it examined the knowledge building process and the interaction and participation of students in an online collaborative learning environment using various indicators. Interaction and participation were discussed in terms of quantity, content and quality because quantity, which has been examined in past research, is not sufficient to explain interaction and participation. This study contributes to learning in a knowledge building process in which students (i) are completely active and are responsible for their learning and assessments both individually and as a group, (ii) attempt to prevent incorrect learning by means of discussion and researching new resources, and (iii) continue research to determine the accuracy of the knowledge they shared. The present study revealed that a course conducted within an online collaborative knowledge building environment can contribute to the opinion building and expression of students; improve the quantity, content and quality of the interaction and participation of students; and thus the learning of students. Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 46 Appendix A. Examples of log data obtained from ATK Example 1: Example of log data showing the total number of scaffolds used by each student and the group Report date: 5 May 2013 Group information: sixth branch, second group Discussion topics: All Time period: 10 February 2012e18 September 2012 (14 weeks) Student No. Total number of scaffolds used by the student Total number of scaffolds used by the group 20993900 20994118 20995167 21094569 21095561 20895283 17 41 81 97 6 19 261 Example 2: Example of log data showing the total number of scaffolds used by each student and the group Report date: 5 May 2013 Group information: sixth branch, second group Discussion topic: Effective presentation techniques Time period: 19 April 2012 (10th week) Branch and group Scaffold Sixth branch, second group Opinion building forms Sixth branch, second group Opinion expression forms Total My Theory I need to understand This theory cannot explain A better theory New information Putting our knowledge together Total My opinion Elaboration Example Evidence Reason Conclusion Different opinion Total 2 5 1 2 1 8 19 4 1 2 4 6 2 8 27 Example 3: Example of log data showing the interaction and participation quantities for each group Report date: 5 May 2013 Group information: sixth branch, second group Discussion topic: Effective presentation techniques Time period: 19 April 2012 (10th week) Interaction and participation quantities Sub-indicator Note creation Note correcting Note reading Note build-on Build-on trees Total 264 53 38 78 21 Example 4: Example of log data showing the quantity of interaction and participation for each student and the group Report date: 5 May 2013 Group information: sixth branch, second group Discussion topic: Effective presentation techniques Time period: 19 April 2012 (10th week) Student No. Note creation Note correcting Note reading Note build-on 20993900 20994118 20995167 21094569 21095561 20895283 Total 16 45 80 95 9 19 264 2 1 17 27 1 5 53 4 10 8 12 1 3 38 14 13 13 11 12 15 78 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 47 References van Aalst, J. (2009). Distinguishing knowledge-sharing, knowledge-construction, and knowledge-creation discourses. International Journal of ComputerSupported Collaborative Learning, 4(3), 259e287. Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. M. (2011). Interaction in distance education and online learning: using evidence and theory to improve practice. Journal of Computing in Higher Education, 23, 82e103. Akçapınar, G., & Aşkar, P. (2009). Measuring author contributions to the Mediawiki. In The IADIS international conference,WWW/Internet 2009, Roma, Italy. Avci Yucel, U., & Usluel, Y. K. (2013). The quality of interaction and participation in knowledge building processes within the online collaborative learning environments. In International academic conference on education, teaching and e-learning (IAC-ETeL 2013), Prague, Czech Republic. Balakrishnan, B. (2014). Online computer supported collaborative learning (CSCL) for engineering students: a case study in Malaysia. Computer Applications in Engineering Education, 1e11. Bereiter, C. (2002). Education and mind in the knowledge age. London: Lawrence Erlbaum. €nboer (Eds.), Bereiter, C., & Scardamalia, M. (2003). Learning to work creatively with knowledge. In E. De Corte, L. Verschaffel, N. Entwistle, & J. van Merrie Unravelling basic components and dimensions of powerful learning environments. EARLI Advances in Learning and Instruction Series. Bereiter, C., & Scardamalia, M. (2006). Education for the knowledge age: design-centered models of teaching and instruction. In P. A. Alexander, & P. H. Winne (Eds.), Handbook of educational psychology (pp. 695e713). Mahwah, NJ: Lawrence Erlbaum Associates. Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, A., Tamim, R., Surkes, M. A., et al. (2009). A meta-analysis of three interaction treatments in distance education. Review of Educational Research, 79(3), 1243e1289. Bielaczyc, K., & Ow, J. (2014). Multi-player epistemic games: guiding the enactment of classroom knowledge-building communities. International Journal of Computer-Supported Collaborative Learning, 9(1), 33e62. Branon, R. F., & Essex, C. (2001). Synchronous and asynchronous communication tools in distance education: a survey of instructors. TechTrends, 45, 36e42. Bruckman, A. (2006). Learning in online communities. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 461e472). Cambridge: Cambridge University Press. Burtis, J. (2002). Analytic toolkit for knowledge forum (Version 4.0). Toronto, ON: Institute for Knowledge Innovation and Technology, Ontario Institute for Studies in Education/University of Toronto. € ztürk, Ş. (2007). Sosyal bilimler için veri analizi el kitabı: istatistik, araştırma deseni, SPSS uygulamaları ve yorum (7. Basım). Ankara: Pegem A Yayıncılık. Büyüko Cacciamani, S., Donatella, C., Francesca, M., Tiziana, F., & Nobuko, F. (2012). Influence of participation, facilitator styles, and metacognitive reflection on knowledge building in online university courses. Computers and Education, 58(3), 874e884. Chan, C. K. K., & Chan, Y. Y. (2011). Students' views of collaboration and online participation in Knowledge Forum. Computers and Education, 57, 1445e1457. Chan, C. K. K., Lee, E. Y. C., & van Aalst, J. (2001). Assessing and fostering knowledge building inquiry and discourse. In Symposium “social and cognitive assesment of computer-supported inquiry” at the 9th Biennial meeting of the European association for learning and instruction, Fribourg, 2e29, Switzerland. Chen, Y., Chen, N. S., & Tsai, C. C. (2009). The use of online synchronous discussion for web-based professional development for teachers. Computers and Education, 53(4), 1155e1166. Cho, H., Chen, M., & Chung, S. (2010). Testing an integrative theoretical model of knowledge-sharing behavior in the context of Wikipedia. Journal of the American Society for Information Science and Technology, 61(6), 1198e1212. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. Coll, C., Engel, A., & Bustos, A. (2009). Distributed teaching presence and participants' activity profiles: a theoretical approach to the structural analysis of asynchronous learning networks. European Journal of Education, 44(4), 521e538. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston: Pearson. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). New York: McGraw-Hill. Fujita, N., & Teplovs, C. (2010). Software-based scaffolding: supporting the development of knowledge building discourse in online courses. In K. Gomez, L. Lyons, & J. Radinsky (Eds.), Learning in the disciplines: Proceedings of the 9th international conference of the learning sciences (ICLS 2010) (pp. 1056e1062). Chicago, IL: International Society of the Learning Sciences. Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: interaction is not enough. The American Journal of Distance Education, 19(3), 133e148. Greenhouse, S., & Geisser, S. (1959). On methods in the analysis of profile data. Psychonomics, 24, 95e112. Gunawardena, C., Lowe, C., & Anderson, T. (1997). Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397e431. Hakkarainen, K., Lipponen, L., & J€ arvel€ a, S. (2002). Epistemology of inquiry and computer-supported collaborative learning. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL 2: Carrying forward the conversation (pp. 11e41). Mahwah, NJ: Lawrence Erlbaum Associates. Hakkarainen, K., & Palonen, T. (2003). Patterns of female and male students' participation in peer interaction in computer-supported learning. Computers and Education, 40, 327e342. Heo, H., Lim, K. Y., & Kim, Y. (2010). Exploratory study on the patterns of online interaction and knowledge co-construction in project-based learning. Computers and Education, 55, 1383e1392. Hewitt, J. (2005). Toward an understanding of how threads die in asynchronous computer conference. The Journal of Learning Science, 14(4), 567e589. Hong, H. Y., & Sullivan, F. R. (2009). Towards an idea-centered, principle-based design approach to support learning as knowledge creation. Educational Technology Research and Development, 57(5), 613e627. Hou, H. T., Chang, K. E., & Sung, Y. T. (2007). An analysis of peer assessment online discussions within a course that uses project-based learning. Interactive Learning Environments, 15(3), 237e251. Hou, H. T., Chang, K.-E., & Sung, Y. T. (2008). Analysis of problem-solving-based online asynchronous discussion pattern. Educational Technology and Society, 11(1), 17e28. Hou, H. T., & Wu, S. Y. (2011). Analyzing the social knowledge construction behavioral patterns of an online synchronous collaborative discussion instructional activity using an instant messaging tool: a case study. Computers and Education, 57(2), 1459e1468. Hrastinski, S. (2008). What is online learner participation? A literature review. Computers and Education, 51, 1755e1765. Hrastinski, S. (2009). A theory of online learning as online participation. Computers and Education, 52(1), 78e82. Isaacs, E., Walendowski, A., & Ranganathan, D. (2002). Hubbub: a sound-enhanced mobile instant messenger that supports awareness and opportunistic interactions. In Proceedings of CHI 2002 (pp. 179e186). Minneapolis, MN: ACM Press. Jeong, A. (2006). Gender interaction patterns and gender participation in computer-supported collaborative argumentation. American Journal of Distance Education, 20(4), 195e210. Jeong, A., & Davidson-Shivers, G. (2006). The effects of gender interaction patterns on student participation in computer-supported collaborative argumentation. Educational Technology, Research, and Development, 54(6), 543e568. Jonassen, D. (2006). Modeling with technology: Mindtools for conceptual change (3rd ed.). Upper Saddle River, NJ: Prentice Hall. Jung, I., Choi, S., Lim, C., & Leem, J. (2002). Effects of different types of interaction on learning achievement, satisfaction and participation in Web-based instruction. Innovations in Education and Teaching International, 39(2), 153e162. Kanuka, H. (2011). Interaction and the online distance classroom: do instructional methods effect the quality of interaction? Journal of Computing in Higher Education, 23, 143e156. King, B. M., Rosopa, P. J., & Minium, E. M. (2010). Statistical reasoning in the behavioral sciences (6th ed.). New York: John Wiley & Sons. 48 Ü.A. Yücel, Y.K. Usluel / Computers & Education 97 (2016) 31e48 Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer supported collaborative learning environments: a review of the research. Computers in Human Behavior, 19, 335e353. re, T., & Allaire, S. (2005). Scaffolding student teachers' online discourse for knowledge building purposes. Technology and Teacher Education Annual, Laferrie 2, 939. Lampe, C., Wohn, D. Y., Vitak, J., Ellison, N. B., & Wash, R. (2011). Student use of Facebook for organizing collaborative classroom activities. International Journal of Computer-Supported Collaborative Learning, 6(3), 329e347. Law, N. (2005). Assessing learning outcomes in CSCL settings. In T. Koschmann, T.-W. Chan, & D. D. Suthers (Eds.), Computer supported collaborative learning conference (CSCL) (pp. 373e377). Taipei, Taiwan: Lawrence Erlbaum Associates. Law, N., Lu, J., Leng, J., Yuen, J., & Lai, M. (2008). Understanding knowledge building from multiple perspectives. In Paper presented in the workshop on common framework for CSCL interaction analysis, international conference for the learning sciences 2008, Utrecht, Netherlands. Law, N., Yuen, J., Wong, W. O. W., & Leng, J. (2011). Understanding learners' knowledge building trajectory through visualizations of multiple automated analyses. In S. Puntambekar, G. Erkens, & C. E. Hmelo-Silver (Eds.), Computer-supported collaborative learning series: 12. Analyzing interactions in CSCL: Methods, approaches and issues (pp. 47e82) (Part 1). Springer. Lee, E. Y. C., Chan, C. K. K., & van Aalst, J. (2006). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning, 1, 277e307. Lipponen, L., Rahikainen, M., Lallimo, J., & Hakkarainen, K. (2003). Patterns of participation and discourse in elementary students' computer-supported collaborative learning. Learning and Instruction, 13, 487e509. Moskaliuk, J., Kimmerle, J., & Cress, U. (2012). Collaborative knowledge building with wikis: the impact of redundancy and polarity. Computers and Education, 58, 1049e1057. Muukkonen, H., Hakkarainen, K., & Lakkala, M. (2004). Computer-mediated progressive inquiry in higher education. In T. S. Roberts (Ed.), Online collaborative learning: Theory and practice (pp. 28e53). Hershey, PA: Information Science Publishing.
s, M. T. (2012). Participation and cognitive quality profiles in an online discussion forum. British Journal of Educational Naranjo, M., Onrubia, J., & Segue Technology, 43(2), 282e294. Nardi, B. A., Whittaker, S., & Bradner, E. (2000). Interaction and outeraction: instant messaging in action. In Proceedings of CSCW 2000 (pp. 79e88). Philadelphia, PA: ACM Press. Niu, H., & van Aalst, J. (2009). Participation in knowledge-building discourse: an analysis of online discussions in mainstream and honours social studies. Canadian Journal of Learning and Technology, 35(1), 1e23. Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M., & Chizari, M. (2013). Facilitating argumentative knowledge construction through a transactive discussion script in CSCL. Computers and Education, 61, 59e76. Perkins, D. N. (1999). The many faces of constructivism. Educational Leadership, 57(2), 354e371. Pifarre, M., & Cobos, R. (2010). Promoting metacognitive skills through peer scaffolding in a CSCL environment. International Journal of Computer-Supported Collaborative Learning, 5(2), 237e253. Prinsen, F. R., Volman, M. L. L., Terwel, J., & van den Eeden, P. (2009). Effects on participation of an experimental CSCL-programme to support elaboration: do all students benefit? Computers and Education, 52, 113e125. Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis, educational technology. Research and Development, 52(1), 5e18. Russell, A. (2002). The role of epistemic agency and knowledge building discourse to foster interprofessional practice in a Canadian hospital. In American education and research association annual conference, New Orleans. Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In B. Smith (Ed.), Liberal education in a knowledge society (pp. 67e98). Chicago, IL: Open Court. Scardamalia, M. (2004). CSILE/Knowledge forum. In A. Kovalchick, & K. Dawson (Eds.), Education and technology: An encyclopedia (pp. 183e192). Santa Barbara, CA: ABCCLIO. Scardamalia, M., & Bereiter, C. (2003). Knowledge building. In J. W. Guthrie (Ed.), Encyclopedia of education (2nd ed.). New York: Macmillan Reference, USA. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: theory, pedagogy, and technology. In The Cambridge handbook of the learning sciences (pp. 97e115). Cambridge: Cambridge University Press. Scardamalia, M., Bereiter, C., & Lamon, M. (1994). The CSILE project: trying to bring the classroom into World 3. In K. McGilly (Ed.), Classroom lessons: Integration cognitive theory and classroom practice (pp. 201e228). Cambridge: MIT Press. Scardamalia, M., Bransford, J., Kozma, R., & Quellmalz, E. (2012). New assessments and environments for knowledge building. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 231e300). Dordrecht: Springer. Seitamaa-Hakkarainen, P., Raunio, A. M., Raami, A., Muukkonen, H., & Hakkarainen, K. (2001). Computer-support for collaborative designing. International Journal of Technology and Design Education, 11, 181e202. Sha, L., & van Aalst, J. (2003). An application of social network analysis to knowledge building. Probing individual, social, and cultural aspects of knowledge building. In The annual meeting of the American educational research association, 21e25 April, Chicago. Shukor, N. A., Tasir, Z., Van der Meijden, H., & Harun, J. (2014). Exploring students' knowledge construction strategies in computer-supported collaborative learning discussions using sequential analysis. Educational Technology and Society, 17(4), 216e228. Sing, C. C., & Khine, M. S. (2006). An analysis of interaction and participation patterns in online community. Educational Technology and Society, 9(1), 250e261. So, H. J., Seah, L. H., & Toh-Heng, H. L. (2010). Designing collaborative knowledge building environments accessible to all learners: impacts and design challenges. Computers and Education, 54(2), 479e490. Stahl, G. (2003). Knowledge negotiation in asynchronous learning networks. In Hawaii international conference on system sciences (HICSS 2003), January 6e9, 2003, Hawaii, USA. Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. Cambridge, MA: MIT Press. Stahl, G., Koschmann, T. D., & Suthers, D. (2006). Computer-supported collaborative learning: an historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409e426). New York: Cambridge University Press. Tan, S. C., Yeo, A. C. J., & Lim, W. Y. (2005). Changing epistemology of science learning through inquiry with computer-supported collaborative learning. Journal of Computers in Mathematics and Science Teaching, 24(4), 367. Thirunarayanan, M. O. (2000). Cutting down on chat confusion: a proposal for managing instructor-controlled chat systems. Ubiquity, 1(38). http://www. acm.org/ubiquity/views/m_thirunarayanan_1.html. Retrieved August 29, 2010. Veerman, A. L. (2000). Computer-supported collaborative learning through argumentation. Unpublished doctoral dissertation. Enschede: Print Partners Ipskamp. Wenger, E. (1998). Communities of practice: Learning. Meaning, and identity. Cambridge: Cambridge University Press. Zhao, K., & Chan, C. K. K. (2014). Fostering collective and individual learning through knowledge building. International Journal of Computer-Supported Collaborative Learning, 9(1), 63e95.