research-article

Integrated Closed-loop Learning Analytics Scheme in a First Year Experience Course

Authors:

Trunojoyo Anggara,

G. Alex Ambrose,

Nitesh V. ChawlaAuthors Info & Claims

LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge

Pages 521 - 530

https://doi.org/10.1145/3303772.3303803

Published: 04 March 2019 Publication History

Abstract

Identifying non-thriving students and intervening to boost them are two processes that recent literature suggests should be more tightly integrated. We perform this integration over six semesters in a First Year Experience (FYE) course with the aim of boosting student success, by using an integrated closed-loop learning analytics scheme that consists of multiple steps broken into three main phases, as follows: Architecting for Collection (steps: design, build, capture), Analyzing for Action (steps: identify, notify, boost), and Assessing for Improvement (steps: evaluate, report). We close the loop by allowing later steps to inform earlier ones in real-time during a semester and iteratively year to year, thereby improving the course from data-driven insights. This process depends on the purposeful design of an integrated learning environment that facilitates data collection, storage, and analysis. Methods for evaluating the effectiveness of our analytics-based student interventions show that our criterion for identifying non-thriving students was satisfactory and that non-thriving students demonstrated more substantial changes from mid-term to final course grades than already-thriving students. Lastly, we make a case for using early performance in the FYE as an indicator of overall performance and retention of first-year students.

References

[1]

{n. d.}. Basic Overview of How LTI Works. ({n. d.}). https://www.imsglobal.org/basic-overview-how-lti-works

[2]

{n. d.}. Caliper Analytics. ({n. d.}). http://www.imsglobal.org/activity/caliper

[3]

{n. d.}. What is the Experience API? ({n. d.}). https://xapi.com/overview/

[4]

2014. (2014). https://web.archive.org/web/20141021223329/http://facultyhandbook.nd.edu:80/assets/137497/undergraduate_academic_code_final_for_7.1.2014.docx

[5]

G. Alex Ambrose, Kevin Abbott, and Alison Lanski. 2017. Under the Hood of a Next Generation Digital Learning Environment in Progress. (Jul 2017). https://er.educause.edu/articles/2017/7/under-the-hood-of-a-next-generation-digital-learning-environment-in-progress

[6]

Apereo-Learning-Analytics-Initiative. 2018. Apereo Learning Analytics Initiative - OpenLRW. (Jul 2018). https://github.com/Apereo-Learning-Analytics-Initiative/OpenLRW

[7]

Kimberly E. Arnold, Matthew D. Pistilli, and Kimberly E. Arnold. 2012. Course Signals at Purdue: Using Learning Analytics to Increase Student Success. 2nd International Conference on Learning Analytics and Knowledge May (2012), 2--5. arXiv:0709.1706v2

Digital Library

[8]

Jonathan Bergmann and Aaron Sams. 2012. Flip your classroom: Reach every student in every class every day. (2012).

[9]

Jacob Lowell Bishop, Matthew A Verleger, et al. 2013. The flipped classroom: A survey of the research. In ASEE national conference proceedings, Atlanta, GA, Vol. 30. 1--18.

[10]

Malcolm Brown, Joanne Dehoney, and Nancy Millichap. 2015. The Next Generation Digital Learning Environment: A Report on Research. (Apr 2015). https://library.educause.edu/resources/2015/4/the-next-generation-digital-learning-environment-a-report-on-research

[11]

Samuel PM Choi, Sze Sing Lam, Kam Cheong Li, and Billy TM Wong. 2018. Learning Analytics at Low Cost: At-risk Student Prediction with Clicker Data and Systematic Proactive Interventions. Journal of Educational Technology & Society 21, 2 (2018), 273--290.

[12]

Doug Clow. 2012. The learning analytics cycle. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge - LAK '12. 134.

Digital Library

[13]

Linda Corrin and Paula De Barba. 2014. Exploring students' interpretation of feedback delivered through learning analytics dashboards. Proceedings of ascilite Dunedin 2014 November (2014), 629--633.

[14]

Angel Del Blanco, Angel Serrano, Manuel Freire, Ivan Martinez-Ortiz, and Baltasar Fernandez-Manjon. 2013. E-Learning standards and learning analytics. Can data collection be improved by using standard data models?. In IEEE Global Engineering Education Conference, EDUCON. 1255--1261.

[15]

Beth Dietz-Uhler and Janet E Hurn. 2013. Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning 12, 1 (2013), 17--26.

[16]

Rebecca Ferguson. 2012. Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning 4, 5--6 (2012), 304--317.

Digital Library

[17]

Rebecca Ferguson and Doug Clow. 2017. Where is the evidence?. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference on - LAK '17. ACM, 56--65.

Digital Library

[18]

Douglas Fuchs and Lynn S Fuchs. 2006. Introduction to response to intervention: What, why, and how valid is it? Reading research quarterly 41, 1 (2006), 93--99.

[19]

Kathleen Goodman. 2006. First-year seminars increase persistence and retention. First-Year Programs (2006).

[20]

Deryl K Hatch and Crystal E Garcia. 2017. Academic advising and the persistence intentions of community college students in their first weeks in college. The Review of Higher Education 40, 3 (2017), 353--390.

[21]

Eric Jones, Travis Oliphant, Pearu Peterson, et al. 2001. SciPy: Open source scientific tools for Python. (2001). http://www.scipy.org/

[22]

Cindy A Kilgo, Jessica K Ezell Sheets, and Ernest T Pascarella. 2015. The link between high-impact practices and student learning: Some longitudinal evidence. Higher Education 69, 4 (2015), 509--525.

[23]

George D. Kuh. 2008. High-impact educational practices: what they are, who has access to them, and why they matter. Association of American Colleges and Universities. https://www.aacu.org/publications-research/publications/high-impact-educational-practices-what-they-are-who-has-access-0

[24]

George D. Kuh. 2009. What Student Affairs Professionals Need to Know About Student Engagement. Journal of College Student Development 50, 6 (2009), 683--706.

[25]

Karen D Mattingly, Margaret C Rice, and Zane L Berge. 2012. Learning analytics as a tool for closing the assessment loop in higher education. Knowledge Management & E-Learning: An International Journal (KM&EL) 4, 3 (2012), 236--247.

[26]

Patrick Miller and Xiaojing Duan. 2018. NGDLE Learning Analytics: Gaining a 360-Degree View of Learning. (Jan 2018). https://er.educause.edu/blogs/2018/1/ngdle-learning-analytics-gaining-a-360-degree-view-of-learning

[27]

Sunny Moon. 2013. High-Impact Educational Practices as Promoting Student Retention and Success. (2013), 203--221.

[28]

Yishay Mor, Rebecca Ferguson, and Barbara Wasson. 2015. Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology 46, 2 (2015), 221--229.

[29]

Paul A Murtaugh, Leslie D Burns, and Jill Schuster. 1999. Predicting the Retention of University Students. Research in Higher Education 40, 3 (1999), 355--371.

[30]

Abelardo Pardo, Roberto Martínez-Maldonado, Simon Buckingham Shum, Jurgen Schulte, Simon McIntyre, Dragan Gašević, Jing Gao, and George Siemens. 2017. Connecting data with student support actions in a course: A Hands-on Tutorial. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference on - LAK '17. 522--523.

Digital Library

[31]

Yeonjeong Park and Il-Hyun Jo. 2015. Development of the Learning Analytics Dashboard to Support Students' Learning Performance Learning Analytics Dashboards (LADs). Journal of Universal Computer Science 21, 1 (2015), 110--133. arXiv:110-133

[32]

Stephen R Porter and Randy L Swing. 2006. Understanding how first-year seminars affect persistence. Research in Higher Education 47, 1 (2006), 89--109.

[33]

Jose Luis Santos, Katrien Verbert, Sten Govaerts, and Erik Duval. 2013. Addressing learner issues with StepUp! Proceedings of the Third International Conference on Learning Analytics and Knowledge - LAK '13 (2013), 14.

Digital Library

[34]

George Siemens. 2013. Learning analytics: The emergence of a discipline. American Behavioral Scientist 57, 10 (2013), 1380--1400.

[35]

George Siemens and Phil Long. 2011. Penetrating the fog: Analytics in learning and education. EDUCAUSE review 46, 5 (2011), 30.

[36]

Hadyn K Swecker, Matthew Fifolt, and Linda Searby. 2013. Academic advising and first-generation college students: A quantitative study on student retention. NACADA Journal 33, 1 (2013), 46--53.

[37]

Jörg Vianden and Patrick J Barlow. 2015. Strengthen the bond: Relationships between academic advising quality and undergraduate student loyalty. The Journal of the National Academic Advising Association 35, 2 (2015), 15--27.

[38]

Anthony J Viera, Joanne M Garrett, et al. 2005. Understanding interobserver agreement: the kappa statistic. Fam Med 37, 5 (2005), 360--363.

[39]

Alyssa Friend Wise. 2014. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the fourth international conference on learning analytics and knowledge. ACM, 203--211.

Digital Library

[40]

Annika Wolff, Zdenek Zdrahal, Andriy Nikolov, and Michal Pantucek. 2013. Improving retention: Predicting at-risk students by analysing clicking behaviour in a virtual learning environment. LAK '13 Proceedings of the Third International Conference on Learning Analytics and Knowledge (2013).

Digital Library

Cited By

Rocha JHernández-Leal ERamos VMunoz RCechinel CPrimo TQueiroga E(2024)Implementing a Learning Record Warehouse for Different Interoperability Specifications in Moodle LMS2024 International Symposium on Computers in Education (SIIE)10.1109/SIIE63180.2024.10604489(1-6)Online publication date: 19-Jun-2024
https://doi.org/10.1109/SIIE63180.2024.10604489
Duan XPei BAmbrose GHershkovitz ACheng YWang C(2023)Towards transparent and trustworthy prediction of student learning achievement by including instructors as co-designers: a case studyEducation and Information Technologies10.1007/s10639-023-11954-829:3(3075-3096)Online publication date: 17-Jun-2023
https://doi.org/10.1007/s10639-023-11954-8
Ahmad AYordanov IYau JSchneider JDrachsler H(2023)A Trusted Learning Analytics Dashboard for Displaying OERDistributed Learning Ecosystems10.1007/978-3-658-38703-7_15(279-303)Online publication date: 21-Feb-2023
https://doi.org/10.1007/978-3-658-38703-7_15
Show More Cited By

Recommendations

Open academic analytics initiative: initial research findings
LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge

This paper describes the results on research work performed by the Open Academic Analytics Initiative, an on-going research project aimed at developing an early detection system of college students at academic risk, using data mining models trained ...
Advising the whole student: eAdvising analytics and the contextual suppression of advisor values

Institutions are applying methods and practices from data analytics under the umbrella term of "learning analytics" to inform instruction, library practices, and institutional research, among other things. This study reports findings from interviews ...
New Entrants' Expectations of the First Year Computer Science Experience in the Context of a New National High School Curriculum
ITiCSE '19: Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education

Computer science (CS) has the highest first year dropout rate amongst UK undergraduates. In this context, understanding the expectations of new students is a first step to designing a CS curriculum that will improve their first year experience. The aim ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge

March 2019

565 pages

ISBN:9781450362566

DOI:10.1145/3303772

General Chairs:
Sharon Hsiao
Arizona State University, USA
,
Jim Cunningham
Arizona State University, USA
,
Katie McCarthy
Georgia State University, USA
,
Grace Lynch
Society for Learning Analytics Research, Australia
,
Program Chairs:
Christopher Brooks
University of Michigan, USA
,
Rebecca Ferguson
The Open University, UK
,
Ulrich Hoppe
University of Duisburg-Essen, Germany

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
SoLAR: The Society for Learning Analytics Research
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

LAK19

LAK19: The 9th International Learning Analytics & Knowledge Conference

March 4 - 8, 2019

AZ, Tempe, USA

Acceptance Rates

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
354
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rocha JHernández-Leal ERamos VMunoz RCechinel CPrimo TQueiroga E(2024)Implementing a Learning Record Warehouse for Different Interoperability Specifications in Moodle LMS2024 International Symposium on Computers in Education (SIIE)10.1109/SIIE63180.2024.10604489(1-6)Online publication date: 19-Jun-2024
https://doi.org/10.1109/SIIE63180.2024.10604489
Duan XPei BAmbrose GHershkovitz ACheng YWang C(2023)Towards transparent and trustworthy prediction of student learning achievement by including instructors as co-designers: a case studyEducation and Information Technologies10.1007/s10639-023-11954-829:3(3075-3096)Online publication date: 17-Jun-2023
https://doi.org/10.1007/s10639-023-11954-8
Ahmad AYordanov IYau JSchneider JDrachsler H(2023)A Trusted Learning Analytics Dashboard for Displaying OERDistributed Learning Ecosystems10.1007/978-3-658-38703-7_15(279-303)Online publication date: 21-Feb-2023
https://doi.org/10.1007/978-3-658-38703-7_15
Ahmad ASchneider JGriffiths DBiedermann DSchiffner DGreller WDrachsler H(2022)Connecting the dots – A literature review on learning analytics indicators from a learning design perspectiveJournal of Computer Assisted Learning10.1111/jcal.12716Online publication date: 26-Jul-2022
https://doi.org/10.1111/jcal.12716
Karimi-Haghighi MCastillo CHernández-Leo D(2022)A Causal Inference Study on the Effects of First Year Workload on the Dropout Rate of UndergraduatesArtificial Intelligence in Education10.1007/978-3-031-11644-5_2(15-27)Online publication date: 27-Jul-2022
https://doi.org/10.1007/978-3-031-11644-5_2
Israel‐Fishelson RHershkovitz A(2021)Micro‐persistence and difficulty in a game‐based learning environment for computational thinking acquisitionJournal of Computer Assisted Learning10.1111/jcal.1252737:3(839-850)Online publication date: 31-Jan-2021
https://doi.org/10.1111/jcal.12527
Ueda HNguyen HBui HTran THatakeyama HHasnine M(2021)Briefing and Geovisualizing on International Practices of Learning Analytics in Higher Education2021 International Conference on Advanced Learning Technologies (ICALT)10.1109/ICALT52272.2021.00109(342-344)Online publication date: Jul-2021
https://doi.org/10.1109/ICALT52272.2021.00109
Lim HKim SChung KLee KKim THeo J(2021)Is college students’ trajectory associated with academic performance?Computers & Education10.1016/j.compedu.2021.104397178:COnline publication date: 29-Dec-2021
https://dl.acm.org/doi/10.1016/j.compedu.2021.104397
Israel-Fishelson RHershkovitz A(2020)Shooting for the StarsEarly Warning Systems and Targeted Interventions for Student Success in Online Courses10.4018/978-1-7998-5074-8.ch012(239-258)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-5074-8.ch012
Knobbout JVan Der Stappen E(2020)Where is the Learning in Learning Analytics? A Systematic Literature Review on the Operationalization of Learning-Related Constructs in the Evaluation of Learning Analytics InterventionsIEEE Transactions on Learning Technologies10.1109/TLT.2020.299997013:3(631-645)Online publication date: 1-Jul-2020
https://doi.org/10.1109/TLT.2020.2999970
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents