research-article

An evaluation of learning analytics to identify exploratory dialogue in online discussions

Authors:

Rebecca Ferguson,

Simon Buckingham ShumAuthors Info & Claims

LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge

Pages 85 - 93

https://doi.org/10.1145/2460296.2460313

Published: 08 April 2013 Publication History

Abstract

Social learning analytics are concerned with the process of knowledge construction as learners build knowledge together in their social and cultural environments. One of the most important tools employed during this process is language. In this paper we take exploratory dialogue, a joint form of co-reasoning, to be an external indicator that learning is taking place. Using techniques developed within the field of computational linguistics, we build on previous work using cue phrases to identify exploratory dialogue within online discussion. Automatic detection of this type of dialogue is framed as a binary classification task that labels each contribution to an online discussion as exploratory or non-exploratory. We describe the development of a self-training framework that employs discourse features and topical features for classification by integrating both cue-phrase matching and k-nearest neighbour classification. Experiments with a corpus constructed from the archive of a two-day online conference show that our proposed framework outperforms other approaches. A classifier developed using the self-training framework is able to make useful distinctions between the learning dialogue taking place at different times within an online conference as well as between the contributions of individual participants.

References

[1]

SoLAR, Open Learning Analytics: An Integrated & Modularized Platform. White Paper, Society for Learning Analytics Research. 2011.

[2]

Sfard, A., On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27, 2, (1998), 4--13.

[3]

Goodman, P. S. and A, D. L., Methodological issues in measuring group learning. Small Group Research, 42, 4, (2011), 379--404.

[4]

Koch, J., Does distance learning work? A large sample, control group study of student success in distance learning. E-Journal of Instructional Science and Technology, 8, 1, (2005).

[5]

Axelson, R. D. and Flick, A., Defining student engagement. Change: The Magazine of Higher Learning, 43, 1, (2010), 38--43.

[6]

Littleton, K. and Whitelock, D., The negotiation and co-construction of meaning and understanding within a postgraduate online learning community. Learning, Media and Technology, 30, 2, (2005), 147--164.

[7]

Vygotsky, L. S., The instrumental method in psychology. In: R. W. Rieber and J. Wollock (Eds.), The Collected Works of L. S. Vygotsky. Plenum Press. (Original work written 1924--1934), New York, 1997.

[8]

Wegerif, R. and Mercer, N., Computers and reasoning through talk in the classroom. Language and Education, 10, 1, (1996), 47--64.

[9]

Mercer, N. and Wegerif, R., Is 'exploratory talk' productive talk? In: P. Light and K. Littleton (Eds.), Learning with Computers: Analysing Productive Interaction. Routledge, London and New York, 1999.

Digital Library

[10]

Mercer, N., Littleton, K. and Wegerif, R., Methods for studying the processes of interaction and collaborative activity in computer-based educational activities. Technology, Pedagogy & Education, 13, 2, (2004), 195--212.

[11]

Mercer, N. and Littleton, K., Dialogue and the Development of Children's Thinking. Routledge, London, 2007.

[12]

Ferguson, R., The Construction of Shared Knowledge through Asynchronous Dialogue. PhD, The Open University, Milton Keynes. http://oro.open.ac.uk/19908/2009.

[13]

Ferguson, R., Whitelock, D. and Littleton, K., Improvable objects and attached dialogue: new literacy practices employed by learners to build knowledge together in asynchronous settings. Digital Culture and Education, 2, 1, (2010), 116--136.

[14]

Ferguson, R. and Buckingham Shum, S., Social Learning Analytics: Five Approaches. In: LAK12: 2nd International Conference on Learning Analytics and Knowledge (30 April - 2 May) (Vancouver, Canada, 2012). ACM.

Digital Library

[15]

Ferguson, R. and Buckingham Shum, S., Learning analytics to identify exploratory dialogue within synchronous text chat. In: Proc. 1st Int. Conf. on Learning Analytics and Knowledge (27 Feb -- 1 Mar) (Banff, Canada, 2011). ACM.

Digital Library

[16]

Buckingham Shum, S. and Ferguson, R., Social learning analytics. Educ. Technology & Society, 15, 3, (2012), 3--26.

[17]

Mercer, N., Words & Minds: How We Use Language To Think Together. Routledge, London, 2000.

[18]

Mercer, N., Developing dialogues. In: G. Wells and G. Claxton (Eds.), Learning for Life in the 21st Century. Blackwell Publishers, Oxford, 2002.

[19]

Mercer, N., Wegerif, R. and Dawes, L., Children's talk and the development of reasoning in the classroom. British Educational Research Journal, 25, 1, (1999), 95--111.

[20]

Wegerif, R., Using computers to help coach exploratory talk across the curriculum. Computers & Education, 26, 1--3, (1996), 51--60.

[21]

Rojas-Drummond, S. and Mercer, N., Scaffolding the development of effective collaboration and learning. Int. Jnl. Education Research, 39, (2003), 99--111.

[22]

Rojas-Drummond, S., Perez, V., Velez, M., Gomez, L. and Mendoza, A., Talking for reasoning among Mexican primary school children. Learning and Instruction, 13, (2003), 653--670.

[23]

Wegerif, R., A dialogic understanding of the relationship between CSCL and teaching thinking skills. Computer-Supported Collaborative Learning, 1, (2006), 143--157.

[24]

Anderson, R. C., Chinn, C., Waggoner, M. and Nguyen, K., Intellectually stimulating story discussions. In: J. Osborn and F. Lehr (Eds.), Literacy for All. Guildford Press, New York, 1998.

[25]

Mezirow, J., Transformative learning: theory to practice. New Directions for Adult and Continuing Education, 74, (1997), 5--12.

[26]

Michaels, S., O'Connor, C. and Resnick, L. B., Deliberative discourse idealized and realized: accountable talk in the classroom and civic life. Studies in the Philosophy of Education, 27, (2008), 283--297.

[27]

Weinberger, A. and Fischer, F., A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education, 46, (2006), 71--95.

Digital Library

[28]

Mercer, N., Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1, 2, (2004), 137--168.

[29]

Austin, J. L., How To Do Things with Words. Clarendon Press, Oxford, 1962.

[30]

Stolcke, A., Ries, K., Coccaro, N., Shriberg, E., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Ess-Dykema, C. and Meteer, M., Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26, 3, (2000), 339--373.

Digital Library

[31]

Kral, P. and Cerisara, C., Dialogue act recognition approaches. Computing and Informatics, 29, (2010), 227--250.

[32]

Cover, T. and Hart, P., Nearest neighbor pattern classification. EEE Transactions on Information Theory, 13, 1, (1967), 21--27.

Digital Library

[33]

Zhu, X., Semi-supervised learning literature survey. University of Wisconsin. Available online from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.103.1693, Madison, 2005.

[34]

Domingos, P. and Pazzani, M., On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 29, 2--3, (1997), 103--130.

Digital Library

[35]

Cortes, C. and Vapnik, V., Support-vector networks. Machine Learning, 20, 3, (1995), 273--297.

[36]

Nigam, K., Lafferty, J. and McCallum, A., Using maximum entropy for text classification. Proc. IJCAI-99 Workshop on Machine Learning for Information Filtering, (Stockholm, Sweden, 1999), 61--67.

[37]

Druck, G., Mann, G. and McCallum, A., Learning from labeled features using generalized expectation criteria. In: Proc. 31st ACM SIGIR Conf. (Singapore, 2008). ACM. 595--602.

Digital Library

[38]

Carletta, J., Assessing agreement on classification tasks: the kappa statistic. Computational Linguistics, 22, 2, (1996), 249--254.

Digital Library

[39]

Cavnar, W. B. and Trenkle, J. M. N-gram-based text categorization, Symposium on Document Analysis and Information Retrieval (Las Vegas, 1994), 161--175.

[40]

Pistilli, M. D., Arnold, K. and Bethune, M., Using Academic Analytics to Promote Student Success. EDUCAUSE Review Online, July/Aug, (2012).

[41]

ARG, Assessment for Learning: 10 Principles. Assessment Reform Group (assessment-reform-group.org). 2002.

[42]

Govaerts, S., Duval, E., Verbert, K. and Pardo, A., The Student Activity Meter for awareness and self-reflection. ACM. 2012.

[43]

Ong, W. J., Orality and Literacy: The Technologizing of the Word. Methuen, London, 1982.

Cited By

Shan LZhao HFeng Y(2024)Research on discourse role recognition in task-oriented collaborative dialogueJournal of Intelligent & Fuzzy Systems10.3233/JIFS-23526346:3(5709-5721)Online publication date: 5-Mar-2024
https://doi.org/10.3233/JIFS-235263
Zade HWilliams STran TSmith CVenkatagiri SHsieh GStarbird K(2023)To Reply or to Quote: Comparing Conversational Framing Strategies on TwitterACM Journal on Computing and Sustainable Societies10.1145/3625680Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3625680
Kitto KManly CFerguson RPoquet OHilliger IKhosravi HRienties BDawson S(2023)Towards more replicable content analysis for learning analyticsLAK23: 13th International Learning Analytics and Knowledge Conference10.1145/3576050.3576096(303-314)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3576050.3576096
Show More Cited By

Index Terms

An evaluation of learning analytics to identify exploratory dialogue in online discussions
1. Applied computing
  1. Education

Recommendations

Classifying student dialogue acts with multimodal learning analytics
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge

Supporting learning with rich natural language dialogue has been the focus of increasing attention in recent years. Many adaptive learning environments model students' natural language input, and there is growing recognition that these systems can be ...
Social learning analytics: five approaches
LAK '12: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge

This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social ...
Learning analytics to identify exploratory dialogue within synchronous text chat
LAK '11: Proceedings of the 1st International Conference on Learning Analytics and Knowledge

While generic web analytics tend to focus on easily harvested quantitative data, Learning Analytics will often seek qualitative understanding of the context and meaning of this information. This is critical in the case of dialogue, which may be employed ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge

April 2013

300 pages

ISBN:9781450317856

DOI:10.1145/2460296

Editors:
Dan Suthers
University of Hawaii, USA
,
Katrien Verbert
Eindhoven University of Technology, The Netherlands
,
Erik Duval
University of Leuven, Belgium
,
Xavier Ochoa
Escuela Superior Politécnica del Litoral, Ecuador

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
K.U. Leuven: K.U. Leuven
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '13

Sponsor:

SIGWEB
K.U. Leuven
SIGCHI

LAK '13: Third Conference on Learning Analytics and Knowledge

April 8 - 13, 2013

Leuven, Belgium

Acceptance Rates

LAK '13 Paper Acceptance Rate 16 of 58 submissions, 28%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
606
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shan LZhao HFeng Y(2024)Research on discourse role recognition in task-oriented collaborative dialogueJournal of Intelligent & Fuzzy Systems10.3233/JIFS-23526346:3(5709-5721)Online publication date: 5-Mar-2024
https://doi.org/10.3233/JIFS-235263
Zade HWilliams STran TSmith CVenkatagiri SHsieh GStarbird K(2023)To Reply or to Quote: Comparing Conversational Framing Strategies on TwitterACM Journal on Computing and Sustainable Societies10.1145/3625680Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3625680
Kitto KManly CFerguson RPoquet OHilliger IKhosravi HRienties BDawson S(2023)Towards more replicable content analysis for learning analyticsLAK23: 13th International Learning Analytics and Knowledge Conference10.1145/3576050.3576096(303-314)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3576050.3576096
Gomez-Vasquez LRomero-Hall EJaramillo Cherrez NGhani SRodriguez ARipine C(2022)Keeping Citizens Informed and Engaged During the COVID-19 Pandemic Using #YoMeInformoPMA: A Case from Latin AmericaHealth Communication10.1080/10410236.2022.203550938:9(1813-1820)Online publication date: 12-Feb-2022
https://doi.org/10.1080/10410236.2022.2035509
Chen XZou DXie H(2022)A decade of learning analytics: Structural topic modeling based bibliometric analysisEducation and Information Technologies10.1007/s10639-022-11046-z27:8(10517-10561)Online publication date: 18-Apr-2022
https://doi.org/10.1007/s10639-022-11046-z
Farrow EMoore JGasevic DScheffel MDowell NJoksimovic SSiemens G(2021)A network analytic approach to integrating multiple quality measures for asynchronous online discussionsLAK21: 11th International Learning Analytics and Knowledge Conference10.1145/3448139.3448163(248-258)Online publication date: 12-Apr-2021
https://dl.acm.org/doi/10.1145/3448139.3448163
Gruzd AKumar PAbul-Fottouh DHaythornthwaite C(2020)Coding and Classifying Knowledge Exchange on Social Media: a Comparative Analysis of the #Twitterstorians and AskHistorians CommunitiesComputer Supported Cooperative Work (CSCW)10.1007/s10606-020-09376-yOnline publication date: 29-Jun-2020
https://doi.org/10.1007/s10606-020-09376-y
Zuo MXu YWang ZZhao RLi X(2019)Analysis of Forum Interaction Behavior Based on Cloud ClassTechnology in Education: Pedagogical Innovations10.1007/978-981-13-9895-7_12(131-145)Online publication date: 12-Jul-2019
https://doi.org/10.1007/978-981-13-9895-7_12
Cobos ROlmos L(2018)A Learning Analytics Tool for Predictive Modeling of Dropout and Certificate Acquisition on MOOCs for Professional Learning2018 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)10.1109/IEEM.2018.8607541(1533-1537)Online publication date: Dec-2018
https://doi.org/10.1109/IEEM.2018.8607541
Liu MLiu LPardo A(2017)Using Learning Analytics to Support Engagement in Collaborative WritingInternational Journal of Distance Education Technologies10.4018/IJDET.201710010515:4(79-98)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.4018/IJDET.2017100105
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents