Abstract
This paper presents a novel method for the evaluation of students’ use of asynchronous discussions in online learning environments. In particular, the paper shows how students’ cognitive development across different course topics can be examined using the combination of natural language processing and graph-based analysis techniques. Drawing on the theoretical foundation of the community of inquiry model, we show how topic modeling and epistemic network analysis can provide qualitatively new insight into students’ development of critical and deep thinking skills. We also show how the same method can be used to investigate the effectiveness of instructional interventions and its effect on student learning. The results of this study and its practical implications are further discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blei, D.M.: Topic modeling in digital humanities. J. Digit. Humanit. 2(1) (2012). Special Issue
Akyol, Z., Garrison, D.R.: Assessing metacognition in an online community of inquiry. Internet High. Educ. 14(3), 183–190 (2011). https://doi.org/10.1016/j.iheduc.2011.01.005
Anderson, T., Dron, J.: Three generations of distance education pedagogy. Int. Rev. Res. Open Distance Learn. 12(3), 80–97 (2010). http://www.irrodl.org/index.php/irrodl/article/view/890/
Anderson, T., Rourke, L., Garrison, D.R., Archer, W.: Assessing teaching presence in a computer conferencing context. J. Asynchronous Learn. Netw. 5, 1–17 (2001)
Arbaugh, J., Cleveland-Innes, M., Diaz, S.R., Garrison, D.R., Ice, P., Richardson, J.C., Swan, K.P.: Developing a community of inquiry instrument: testing a measure of the community of inquiry framework using a multi-institutional sample. Internet High. Educ. 11(3–4), 133–136 (2008). https://doi.org/10.1016/j.iheduc.2008.06.003
Azevedo, R., Moos, D.C., Greene, J.A., Winters, F.I., Cromley, J.G.: Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia? Educ. Technol. Res. Dev. 56(1), 45–72 (2008). https://doi.org/10.1007/s11423-007-9067-0
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). http://dl.acm.org/citation.cfm?id=944919.944937
Cohen, J.: The analysis of variance. In: Statistical Power Analysis for the Behavioral Sciences, pp. 273–406. L. Erlbaum Associates, Hillsdale (1988)
Dewey, J.: My pedagogical creed. Sch. J. 54(3), 77–80 (1897)
Feldman, R., Sanger, J.: The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data. Cambridge University Press, Cambridge (2007)
Fischer, F., Kollar, I., Mandl, H., Haake, J.M.: Scripting Computer-Supported Collaborative Learning: Cognitive. Computational and Educational Perspectives. Springer Science & Business Media, New York (2007). https://doi.org/10.1007/978-0-387-36949-5
Garrison, D.R.: Cognitive presence for effective asynchronous online learning: the role of reflective inquiry, self-direction and metacognition. Elem. Qual. Online Educ. Pract. Dir. 4(1), 47–58 (2003)
Garrison, D.R., Anderson, T., Archer, W.: Critical inquiry in a text-based environment: computer conferencing in higher education. Internet High. Educ. 2(2–3), 87–105 (1999). https://doi.org/10.1016/S1096-7516(00)00016-6
Garrison, D.R., Anderson, T., Archer, W.: Critical thinking, cognitive presence, and computer conferencing in distance education. Am. J. Distance Educ. 15(1), 7–23 (2001). https://doi.org/10.1080/08923640109527071
Garrison, D.R., Anderson, T., Archer, W.: The first decade of the community of inquiry framework: a retrospective. Internet High. Educ. 13(1–2), 5–9 (2010). https://doi.org/10.1016/j.iheduc.2009.10.003
Gašević, D., Adesope, O., Joksimović, S., Kovanović, V.: Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. Internet High. Educ. 24, 53–65 (2015). https://doi.org/10.1016/j.iheduc.2014.09.006
Heo, H., Lim, K.Y., Kim, Y.: Exploratory study on the patterns of online interaction and knowledge co-construction in project-based learning. Comput. Educ. 55(3), 1383–1392 (2010). https://doi.org/10.1016/j.compedu.2010.06.012
Kovanović, V., Joksimović, S., Gašević, D., Hatala, M.: Automated cognitive presence detection in online discussion transcripts. In: Proceedings of the Workshops at the LAK 2014 Conference Co-located with 4th International Conference on Learning Analytics and Knowledge (LAK 2014), Indianapolis, IN (2014). http://ceur-ws.org/Vol-1137/
Kovanović, V., Joksimović, S., Gašević, D., Hatala, M., Siemens, G.: Content analytics: the definition, scope, and an overview of published research. In: Lang, C., Siemens, G., Wise, A., Gašević, D. (eds.) Handbook of Learning Analytics and Educational Data Mining, pp. 77–92. SoLAR, Edmonton (2017). https://doi.org/10.18608/hla17.007
Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., Hatala, M.: What public media reveals about MOOCs: a systematic analysis of news reports. Br. J. Educ. Technol. 46(3), 510–527 (2015). https://doi.org/10.1111/bjet.12277
Kovanović, V., Joksimović, S., Poquet, O., Hennis, T., Čukić, I., de Vries, P., Hatala, M., Dawson, S., Siemens, G., Gašević, D.: Exploring communities of inquiry in massive open online courses. Comput. Educ. 119, 44–58 (2018). https://doi.org/10.1016/j.compedu.2017.11.010
Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., Siemens, G.: Towards automated content analysis of discussion transcripts: a cognitive presence case. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK 2016), pp. 15–24. ACM, New York (2016). https://doi.org/10.1145/2883851.2883950
Lipman, M.: Thinking in Education. Cambridge University Press, Cambridge (1991)
Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing, vol. 999. MIT Press, Cambridge (1999)
Nash, P., Shaffer, D.W.: Mentor modeling: the internalization of modeled professional thinking in an epistemic game. J. Comput. Assist. Learn. 27(2), 173–189 (2011). https://doi.org/10.1111/j.1365-2729.2010.00385.x
Ramage, D., Rosen, E., Chuang, J., Manning, C.D., McFarland, D.A.: Topic modeling for the social sciences. In: NIPS 2009 Workshop on Applications for Topic Models: Text and Beyond, Whistler, Canada (2009)
Reich, J., Tingley, D., Leder-Luis, J., Roberts, M.E., Stewart, B.: Computer-assisted reading and discovery for student generated text in massive open online courses. J. Learn. Analytics 2(1), 156–184 (2014). http://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4138
Rourke, L., Anderson, T., Garrison, D.R., Archer, W.: Assessing social presence in asynchronous text-based computer conferencing. J. Distance Educ. 14(2), 50–71 (1999). http://www.ijede.ca/index.php/jde/article/view/153
Shaffer, D.W.: Epistemic frames for epistemic games. Comput. Educ. 46(3), 223–234 (2006). https://doi.org/10.1016/j.compedu.2005.11.003
Shaffer, D.W.: Epistemic frames and islands of expertise: learning from infusion experiences. In: Proceedings of the 6th International Conference on Learning Sciences, ICLS 2004, pp. 473–480. International Society of the Learning Sciences, Santa Monica (2004). http://dl.acm.org/citation.cfm?id=1149126.1149184
Shaffer, D.W., Collier, W., Ruis, A.R.: A tutorial on epistemic network analysis: analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Analytics 3(3), 9–45 (2016). https://doi.org/10.18608/jla.2016.33.3
Shaffer, D.W., Hatfield, D., Svarovsky, G.N., Nash, P., Nulty, A., Bagley, E., Frank, K., Rupp, A.A., Mislevy, R.: Epistemic network analysis: a prototype for 21st-century assessment of learning. Int. J. Learn. Media 1(2), 33–53 (2009). https://doi.org/10.1162/ijlm.2009.0013
Stahl, G., Koschmann, T., Suthers, D., Sawyer, R.K.: Computer-supported collaborative learning: a historical perspective. In: The Cambridge Handbook of the Learning Sciences, pp. 409–426. Cambridge University Press, Cambridge, New York (2006)
Wallach, H.M.: Topic modeling: beyond bag-of-words. In: Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, pp. 977–984. ACM, New York (2006). https://doi.org/10.1145/1143844.1143967
Waters, Z., Kovanović, V., Kitto, K., Gašević, D.: Structure matters: adoption of structured classification approach in the context of cognitive presence classification. In: Zuccon, G., Geva, S., Joho, H., Scholer, F., Sun, A., Zhang, P. (eds.) AIRS 2015. LNCS, vol. 9460, pp. 227–238. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-28940-3_18
Winne, P.H., Hadwin, A.F.: Studying as self-regulated learning. In: Hacker, D.J., Dunlosky, J., Graesser, A.C. (eds.) Metacognition in educational theory and practice. The Educational Psychology Series, pp. 277–304. Lawrence Erlbaum Associates Publishers, Mahwah (1998)
Yang, D., Wen, M., Rose, C.: Towards identifying the resolvability of threads in MOOCs. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 21–31 (2014). http://www.aclweb.org/anthology/W/W14/W14-41.pdf#page=28
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Ferreira, R., Kovanović, V., Gašević, D., Rolim, V. (2018). Towards Combined Network and Text Analytics of Student Discourse in Online Discussions. In: Penstein Rosé, C., et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham. https://doi.org/10.1007/978-3-319-93843-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-93843-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93842-4
Online ISBN: 978-3-319-93843-1
eBook Packages: Computer ScienceComputer Science (R0)