Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Automated Analysis of Cognitive Presence in Online Discussions Written in Portuguese

  • Conference paper
  • First Online:
Lifelong Technology-Enhanced Learning (EC-TEL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11082))

Included in the following conference series:

Abstract

This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s \(\kappa \) of .72, which represents a “substantial” agreement, and it is above the Cohen’s \(\kappa \) threshold of .70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://spacy.io.

  2. 2.

    https://spacy.io.

  3. 3.

    https://spacy.io.

References

  1. Akyol, Z., Arbaugh, J.B., Cleveland-Innes, M., Garrison, D.R., Ice, P., Richardson, J.C., Swan, K.: A response to the review of the community of inquiry framework. Int. J. E-Learn. Distance Educ. 23(2), 123–136 (2009)

    Google Scholar 

  2. Anderson, T., Rourke, L., Garrison, D.R., Archer, W.: Assessing teaching presence in a computer conferencing context. J. Asynchronous Learn. Netw. 5, 1–17 (2001)

    Google Scholar 

  3. de Araújo, E.M., de Oliveira Neto, J.D.: Avaliação do pensamento crítico e da presença cognitiva em fórum de discussão online utilizando a análise estatística textual. In: Proceedings of International Conference on Engineering and Computer Education, vol. 8, pp. 113–117 (2013)

    Google Scholar 

  4. Arbaugh, J., Cleveland-Innes, M., Diaz, S.R., Garrison, D.R., Ice, P., Richardson, J.C., Swan, K.P.: Developing a community of inquiry instrument: testing a measure of the community of inquiry framework using a multi-institutional sample. Internet High. Educ. 11(3–4), 133–136 (2008). https://doi.org/10.1016/j.iheduc.2008.06.003

    Article  Google Scholar 

  5. Bauer, M.W.: Content analysis. An introduction to its methodology-by Klaus Krippendorff from words to numbers. Narrative, data and social science-by roberto franzosi. Br. J. Sociol. 58(2), 329–331 (2007)

    Article  Google Scholar 

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  7. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  MATH  Google Scholar 

  8. Corich, S., Hunt, K., Hunt, L.: Computerised content analysis for measuring critical thinking within discussion forums. J. E-learn. Knowl. Soc. 2(1), 1–8 (2006)

    Google Scholar 

  9. Dowell, N.M., Skrypnyk, O., Joksimovic, S., Graesser, A.C., Dawson, S., GaĹevic, D., Hennis, T.A., de Vries, P., Kovanovic, V.: Modeling learners’ social centrality and performance through language and discourse. International Educational Data Mining Society (2015)

    Google Scholar 

  10. Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems. J. Mach. Learn. Res 15(1), 3133–3181 (2014)

    MathSciNet  MATH  Google Scholar 

  11. Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning. Springer Series in Statistics, vol. 1. Springer, New York (2001). https://doi.org/10.1007/978-0-387-21606-5

    Book  MATH  Google Scholar 

  12. Gašević, D., Adesope, O., Joksimović, S., Kovanović, V.: Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. Internet High. Educ. 24, 53–65 (2015). https://doi.org/10.1016/j.iheduc.2014.09.006

    Article  Google Scholar 

  13. Gašević, D., Kovanović, V., Joksimović, S.: Piecing the learning analytics puzzle: a consolidated model of a field of research and practice. Learn. Res. Pract. 3(1), 63–78 (2017). https://doi.org/10.1080/23735082.2017.1286142

    Article  Google Scholar 

  14. Garrison, D.R., Anderson, T., Archer, W.: Critical thinking, cognitive presence, and computer conferencing in distance education. Am. J. Distance Educ. 15(1), 7–23 (2001). https://doi.org/10.1080/08923640109527071

    Article  Google Scholar 

  15. Garrison, D.R., Anderson, T., Archer, W.: The first decade of the community of inquiry framework: a retrospective. Internet High. Educ. 13(1–2), 5–9 (2010)

    Article  Google Scholar 

  16. Heo, H., Lim, K.Y., Kim, Y.: Exploratory study on the patterns of online interaction and knowledge co-construction in project-based learning. Comput. Educ. 55(3), 1383–1392 (2010). https://doi.org/10.1016/j.compedu.2010.06.012

    Article  Google Scholar 

  17. Hew, K.F., Cheung, W.S.: Attracting student participation in asynchronous online discussions: a case study of peer facilitation. Comput. Educ. 51(3), 1111–1124 (2008)

    Article  Google Scholar 

  18. Holsti, O.R.: Content Analysis for the Social Sciences and Humanities. Addison-Wesley Pub. Co., Reading (1969)

    Google Scholar 

  19. Joksimovic, S., Gasevic, D., Kovanovic, V., Adesope, O., Hatala, M.: Psychological characteristics in cognitive presence of communities of inquiry: a linguistic analysis of online discussions. Internet High. Educ. 22, 1–10 (2014)

    Article  Google Scholar 

  20. Joksimović, S., Kovanović, V., Jovanović, J., Zouaq, A., Gašević, D., Hatala, M.: What do cMOOC participants talk about in social media?: a topic analysis of discourse in a cMOOC. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 156–165. ACM (2015)

    Google Scholar 

  21. Kovanović, V., Gašević, D., Hatala, M.: Learning analytics for communities of inquiry. J. Learn. Anal. 1(3), 195–198 (2014)

    Article  Google Scholar 

  22. Kovanović, V., Joksimović, S., Gašević, D., Hatala, M.: Automated cognitive presence detection in online discussion transcripts. In: Proceedings of the Workshops at the LAK 2014 Conference Co-Located with 4th International Conference on Learning Analytics and Knowledge (LAK 2014), Indianapolis, IN (2014). http://ceur-ws.org/Vol-1137/

  23. Kovanović, V., Joksimović, S., Gašević, D., Hatala, M., Siemens, G.: Content analytics: the definition, scope, and an overview of published research. In: Lang, C., Siemens, G., Wise, A., Gašević, D. (eds.) Handbook of Learning Analytics and Educational Data Mining, pp. 77–92. SoLAR, Edmonton (2017). https://doi.org/10.18608/hla17.007

    Chapter  Google Scholar 

  24. Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., Siemens, G.: Towards automated content analysis of discussion transcripts: a cognitive presence case. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK 2016), pp. 15–24. ACM, New York (2016)

    Google Scholar 

  25. Kuhn, M., Wing, J., Weston, S., Williams, A., Keefer, C., Engelhardt, A., et al.: Caret: classification and regression training. R package version 4 (2017)

    Google Scholar 

  26. Kusner, M., Sun, Y., Kolkin, N., Weinberger, K.: From word embeddings to document distances. In: International Conference on Machine Learning, pp. 957–966 (2015)

    Google Scholar 

  27. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33, 159–174 (1977)

    Article  MATH  Google Scholar 

  28. Liaw, A., Wiener, M., et al.: Classification and regression by random forest. R News 2(3), 18–22 (2002)

    Google Scholar 

  29. Lipman, M.: Thinking in Education. Cambridge University Press, New York (1991)

    Google Scholar 

  30. McGill, T.J., Klobas, J.E.: A task technology fit view of learning management system impact. Comput. Educ. 52(2), 496–508 (2009)

    Article  Google Scholar 

  31. Mcklin, T.E.: Analyzing Cognitive Presence in Online Courses Using an Artificial Neural Network. Ph.D. thesis, Atlanta, GA, USA (2004). aAI3190967

    Google Scholar 

  32. McNamara, D.S., Graesser, A.C., McCarthy, P.M., Cai, Z.: Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge University Press, Cambridge (2014)

    Book  Google Scholar 

  33. Park, C.L.: Replicating the use of a cognitive presence measurement tool (2009)

    Google Scholar 

  34. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  35. Rosé, C., Wang, Y.C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., Fischer, F.: Analyzing collaborative learning processes automatically: exploiting the advances of computational linguistics in computer-supported collaborative learning. Int. J. Comput. Support. Collab. Learn. 3(3), 237–271 (2008)

    Article  Google Scholar 

  36. Rourke, L., Anderson, T., Garrison, D.R., Archer, W.: Assessing social presence in asynchronous text-based computer conferencing. J. Distance Educ. 14(2), 50–71 (1999). http://www.ijede.ca/index.php/jde/article/view/153

    Google Scholar 

  37. Rourke, L., Anderson, T., Garrison, D.R., Archer, W.: Methodological issues in the content analysis of computer conference transcripts. Int. J. Artif. Intell. Educ. (IJAIED) 12, 8–22 (2001)

    Google Scholar 

  38. Rozenfeld, C.C.D.F.: Fóruns online na formação crítico-reflexiva de professores de línguas estrangeiras: uma representação do pensamento crítico em fases na/pela linguagem. Alfa Rev. Linguíst. (São José do Rio Preto) 1, 35–62 (2014)

    Article  Google Scholar 

  39. Scarton, C., Gasperin, C., Aluisio, S.: Revisiting the readability assessment of texts in Portuguese. In: Kuri-Morales, A., Simari, G.R. (eds.) IBERAMIA 2010. LNCS (LNAI), vol. 6433, pp. 306–315. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-16952-6_31

    Chapter  Google Scholar 

  40. Stone, P.J., Dunphy, D.C., Smith, M.S.: The general inquirer: a computer approach to content analysis (1966)

    Google Scholar 

  41. Strijbos, J.W.: Assessment of (computer-supported) collaborative learning. IEEE Trans. Learn. Technol. 4(1), 59–73 (2011)

    Article  Google Scholar 

  42. Strijbos, J.W., Martens, R.L., Prins, F.J., Jochems, W.M.: Content analysis: what are they talking about? Comput. Educ. 46(1), 29–48 (2006)

    Article  Google Scholar 

  43. Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010)

    Article  Google Scholar 

  44. Waters, Z., Kovanović, V., Kitto, K., Gašević, D.: Structure matters: adoption of structured classification approach in the context of cognitive presence classification. In: Zuccon, G., Geva, S., Joho, H., Scholer, F., Sun, A., Zhang, P. (eds.) AIRS 2015. LNCS, vol. 9460, pp. 227–238. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-28940-3_18

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Computação, Universidade Federal Rural de Pernambuco, Recife, Brazil

    Valter Neto, Vitor Rolim, Rafael Ferreira, Rafael Dueire Lins & Rodrigo Lins

  2. University of Edinburgh, Old College, South Bridge, Edinburgh, EH8 9YL, UK

    Rafael Ferreira & Dragan Gašević

  3. University of South Australia, 160 Currie St, Adelaide, SA, 5000, Australia

    Vitomir Kovanović

  4. Monash University, 19 Ancora Imparo Way, Clayton, VIC, 3800, Australia

    Dragan Gašević

Authors
  1. Valter Neto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vitor Rolim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafael Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vitomir Kovanović
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dragan Gašević
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rafael Dueire Lins
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rodrigo Lins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafael Ferreira .

Editor information

Editors and Affiliations

  1. Graz University of Technology, Graz, Austria

    Viktoria Pammer-Schindler

  2. Pontificia Universidad Católica de Chile, Providencia, Santiago de Chile, Chile

    Mar Pérez-Sanagustín

  3. DIPF | Leibniz Institute for Research and Information in Education, Frankfurt, Germany

    Hendrik Drachsler

  4. RayCom BV, Utrecht, Utrecht, The Netherlands

    Raymond Elferink

  5. Open University Netherlands, Heerlen, The Netherlands

    Maren Scheffel

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Neto, V. et al. (2018). Automated Analysis of Cognitive Presence in Online Discussions Written in Portuguese. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds) Lifelong Technology-Enhanced Learning. EC-TEL 2018. Lecture Notes in Computer Science(), vol 11082. Springer, Cham. https://doi.org/10.1007/978-3-319-98572-5_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-98572-5_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-98571-8

  • Online ISBN: 978-3-319-98572-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation