Cited By
View all- Morrison KSpitzer PTurri VFeng MKühl NPerer A(2024)The Impact of Imperfect XAI on Human-AI Decision-MakingProceedings of the ACM on Human-Computer Interaction10.1145/36410228:CSCW1(1-39)Online publication date: 26-Apr-2024
ML-Based Teaching Systems: A Conceptual Framework
Article No.: 348, Pages 1 - 25
Abstract
As the shortage of skilled workers continues to be a pressing issue, exacerbated by demographic change, it is becoming a critical challenge for organizations to preserve the knowledge of retiring experts and pass it on to novices. While this knowledge transfer has traditionally occurred through personal interaction, it lacks scalability and requires significant resources and time. IT-based teaching systems have addressed this scalability issue, but their development is still tedious and time-consuming. In this work, we investigate the potential of machine learning (ML) models to facilitate knowledge transfer in an organizational context, leading to more cost-effective IT-based teaching systems. Through a systematic literature review, we examine key concepts, themes, and dimensions to understand better and design ML-based teaching systems. To do so, we capture and consolidate the capabilities of ML models in IT-based teaching systems, inductively analyze relevant concepts in this context, and determine their interrelationships. We present our findings in the form of a review of the key concepts, themes, and dimensions to understand and inform on ML-based teaching systems. Building on these results, our work contributes to research on computer-supported cooperative work by conceptualizing how ML-based teaching systems can preserve expert knowledge and facilitate its transfer from SMEs to human novices. In this way, we shed light on this emerging subfield of human-computer interaction and serve to build an interdisciplinary research agenda.
References
[1]
Azad Abad, Moin Nabi, and Alessandro Moschitti. 2017. Autonomous crowdsourcing through human-machine collaborative learning. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 873--876.
[2]
Benjamin M Abdel-Karim, Nicolas Pfeuffer, Gernot Rohde, and Oliver Hinz. 2020. How and what can humans learn from being in the loop? KI-Künstliche Intelligenz, Vol. 34, 2 (2020), 199--207.
[3]
Alaa N Akkila, Abdelbaset Almasri, Adel Ahmed, Naser Al-Masri, Yousef Abu Sultan, Ahmed Y Mahmoud, Ihab Zaqout, and Samy S Abu-Naser. 2019. Survey of Intelligent Tutoring Systems up to the end of 2017. International Journal of Academic Information Systems Research (IJAISR), Vol. 3, 4 (2019).
[4]
Yasar Akyuz. 2020. Effects of intelligent tutoring systems (ITS) on personalized learning (PL). Creative Education, Vol. 11, 6 (2020), 953--978.
[5]
Maryam Alavi and Dorothy E Leidner. 2001. Knowledge management and knowledge management systems: Conceptual foundations and research issues. MIS quarterly (2001), 107--136.
[6]
John R Anderson, C Franklin Boyle, and Brian J Reiser. 1985. Intelligent tutoring systems. Science, Vol. 228, 4698 (1985), 456--462.
[7]
Thushari Atapattu, Katrina Falkner, and Nickolas Falkner. 2015. Educational question answering motivated by question-specific concept maps. In International Conference on Artificial Intelligence in Education. Springer, 13--22.
[8]
Sumit Basu and Janara Christensen. 2013. Teaching classification boundaries to humans. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 27. 109--115.
[9]
Dianne C Berry. 1987. The problem of implicit knowledge. Expert systems, Vol. 4, 3 (1987), 144--151.
[10]
Kenneth G Brown. 2001. Using computers to deliver training: Which employees learn and why? Personnel Psychology, Vol. 54, 2 (2001), 271--296.
[11]
Anne Burmeister and Jürgen Deller. 2016. Knowledge retention from older and retiring workers: What do we know, and where do we go from here? Work, Aging and Retirement, Vol. 2, 2 (2016), 87--104.
[12]
Andrew C Butler, Elizabeth J Marsh, JP Slavinsky, and Richard G Baraniuk. 2014. Integrating cognitive science and technology improves learning in a STEM classroom. Educational Psychology Review, Vol. 26, 2 (2014), 331--340.
[13]
Maya Cakmak and Manuel Lopes. 2012. Algorithmic and human teaching of sequential decision tasks. In Twenty-Sixth AAAI Conference on Artificial Intelligence.
[14]
Heloisa Candello, Claudio Pinhanez, Michael Muller, and Mairieli Wessel. 2022. Unveiling Practices of Customer Service Content Curators of Conversational Agents. Proceedings of the ACM on Human-Computer Interaction, Vol. 6, CSCW2 (2022), 1--33.
[15]
Rui Castro, Charles Kalish, Robert Nowak, Ruichen Qian, Tim Rogers, and Jerry Zhu. 2008. Human active learning. Advances in neural information processing systems, Vol. 21 (2008).
[16]
Yuxin Chen, Oisin Mac Aodha, Shihan Su, Pietro Perona, and Yisong Yue. 2018. Near-optimal machine teaching via explanatory teaching sets. In International Conference on Artificial Intelligence and Statistics. PMLR, 1970--1978.
[17]
Fabio Clarizia, Massimo De Santo, Marco Lombardi, and Domenico Santaniello. 2021. E-learning and industry 4.0: A chatbot for training employees. In Proceedings of Fifth International Congress on Information and Communication Technology. Springer, 445--453.
[18]
Juliet Corbin and Anselm Strauss. 2010. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Canadian Journal of University Continuing Education, Vol. 36, 2 (2010).
[19]
Renée E Derouin, Barbara A Fritzsche, and Eduardo Salas. 2005. E-learning in organizations. Journal of management, Vol. 31, 6 (2005), 920--940.
[20]
P Dillenbourg, P Jermann, D Schneider, D Traum, and C Buiu. 1997. The design of MOO agents: Implications from an empirical CSCW study. In Proceedings 8th World Conference on Artificial Intelligence in Education.
[21]
Niklas Engbom et al. 2019. Firm and worker dynamics in an aging labor market. (2019).
[22]
Martha Evens and Joel Michael. 2006. One-on-one tutoring by humans and computers. Psychology Press.
[23]
Jingchao Fang, Yanhao Wang, Chi-Lan Yang, Ching Liu, and Hao-Chuan Wang. 2022. Understanding the Effects of Structured Note-taking Systems for Video-based Learners in Individual and Social Learning Contexts. Proceedings of the ACM on Human-Computer Interaction, Vol. 6, GROUP (2022), 1--21.
[24]
Kurt D Fenstermacher. 2005. The tyranny of tacit knowledge: What artificial intelligence tells us about knowledge representation. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences. IEEE, 243a--243a.
[25]
Jeremiah T Folsom-Kovarik, Gita Sukthankar, and Sae Schatz. 2013. Tractable POMDP representations for intelligent tutoring systems. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 4, 2 (2013), 1--22.
[26]
Claude Frasson and Esma Aïmeur. 1998. Designing a multi-strategic intelligent tutoring system for training in industry. Computers in industry, Vol. 37, 2 (1998), 153--167.
[27]
Bhavya Ghai, Q Vera Liao, Yunfeng Zhang, Rachel Bellamy, and Klaus Mueller. 2021. Explainable active learning (xal) toward ai explanations as interfaces for machine teachers. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW3 (2021), 1--28.
[28]
Dennis A Gioia, Kevin G Corley, and Aimee L Hamilton. 2013. Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational research methods, Vol. 16, 1 (2013), 15--31.
[29]
Anat Goldstein, Lior Fink, Amit Meitin, Shiran Bohadana, Oscar Lutenberg, and Gilad Ravid. 2018. Applying machine learning on sensor data for irrigation recommendations: revealing the agronomist's tacit knowledge. Precision agriculture, Vol. 19, 3 (2018), 421--444.
[30]
Michael E Gorman. 2002. Types of knowledge and their roles in technology transfer. The Journal of Technology Transfer, Vol. 27, 3 (2002), 219--231.
[31]
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Counterfactual visual explanations. In International Conference on Machine Learning. PMLR, 2376--2384.
[32]
Sebastian Gross, Bassam Mokbel, Barbara Hammer, and Niels Pinkwart. 2015. Learning feedback in intelligent tutoring systems. KI-Künstliche Intelligenz, Vol. 29, 4 (2015), 413--418.
[33]
Matej Guid, Martin Movz ina, Matevvz Pavlivc, and Klemen Turvs ivc. 2019a. Learning by arguing in argument-based machine learning framework. In International Conference on Intelligent Tutoring Systems. Springer, 112--122.
[34]
Matej Guid, Matevz Pavlic, and Martin Mozina. 2019b. Automated Feedback Generation for Argument-Based Intelligent Tutoring Systems. In CSEDU (1). 70--77.
[35]
Demetris Hadjimichael and Haridimos Tsoukas. 2019. Toward a better understanding of tacit knowledge in organizations: Taking stock and moving forward. Academy of Management Annals, Vol. 13, 2 (2019), 672--703.
[36]
Nile W Hatch and Jeffrey H Dyer. 2004. Human capital and learning as a source of sustainable competitive advantage. Strategic management journal, Vol. 25, 12 (2004), 1155--1178.
[37]
Judith A Holton. 2007. The coding process and its challenges. The Sage handbook of grounded theory, Vol. 3 (2007), 265--289.
[38]
Shiu-Li Huang and Jung-Hung Shiu. 2012. A user-centric adaptive learning system for e-learning 2.0. Journal of Educational Technology & Society, Vol. 15, 3 (2012), 214--225.
[39]
Axel Hund, Heinz-Theo Wagner, Daniel Beimborn, and Tim Weitzel. 2021. Digital innovation: Review and novel perspective. The Journal of Strategic Information Systems, Vol. 30, 4 (2021), 101695.
[40]
Edward Johns, Oisin Mac Aodha, and Gabriel J Brostow. 2015. Becoming the expert-interactive multi-class machine teaching. In proceedings of the IEEE conference on computer vision and pattern recognition. 2616--2624.
[41]
Gerald C Kane and Robert G Fichman. 2009. The shoemaker's children: Using wikis for information systems teaching, research, and publication. MIS quarterly (2009), 1--17.
[42]
Laewoo Kang and Steven Jackson. 2021. Tech-Art-Theory: Improvisational Methods for HCI Learning and Teaching. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--25.
[43]
Jihyun Kim, Kelly Merrill Jr, Kun Xu, and Deanna D Sellnow. 2020. My teacher is a machine: Understanding students' perceptions of AI teaching assistants in online education. International Journal of Human-Computer Interaction, Vol. 36, 20 (2020), 1902--1911.
[44]
Jihyun Kim, Kelly Merrill Jr, Kun Xu, and Deanna D Sellnow. 2021. I like my relational machine teacher: An AI instructor's communication styles and social presence in online education. International Journal of Human-Computer Interaction, Vol. 37, 18 (2021), 1760--1770.
[45]
Ekaterina Kochmar, Dung Do Vu, Robert Belfer, Varun Gupta, Iulian Vlad Serban, and Joelle Pineau. 2022. Automated data-driven generation of personalized pedagogical interventions in intelligent tutoring systems. International Journal of Artificial Intelligence in Education, Vol. 32, 2 (2022), 323--349.
[46]
Harvey J Krahn, Andrea L Howard, and Nancy L Galambos. 2015. Exploring or floundering? The meaning of employment and educational fluctuations in emerging adulthood. Youth & Society, Vol. 47, 2 (2015), 245--266.
[47]
Dominik Kreuzberger, Niklas Kühl, and Sebastian Hirschl. 2023. Machine learning operations (mlops): Overview, definition, and architecture. IEEE Access (2023).
[48]
Niklas Kühl, Max Schemmer, Marc Goutier, and Gerhard Satzger. 2022. Artificial intelligence and machine learning. Electronic Markets (2022), 1--10.
[49]
Kibum Kwon and Soebin Jang. 2021. There is no good war for talent: a critical review of the literature on talent management. Employee Relations: The International Journal (2021).
[50]
Niklas Kühl and Jonas Lehner. 2016. Programming for refugees - an active learning approach for teaching Java to heterogeneous groups. In Informatik 2016, Heinrich C. Mayr and Martin Pinzger (Eds.). Gesellschaft für Informatik e.V., Bonn, 1175--1178.
[51]
Alice Lam. 2000. Tacit knowledge, organizational learning and societal institutions: An integrated framework. Organization studies, Vol. 21, 3 (2000), 487--513.
[52]
Dorothy E Leidner. 2018. Review and theory symbiosis: An introspective retrospective. Journal of the Association for Information Systems, Vol. 19, 6 (2018), 1.
[53]
Nadège Levallet and Yolande E Chan. 2018. Organizational knowledge retention and knowledge loss. Journal of Knowledge Management (2018).
[54]
Moria Levy. 2011. Knowledge retention: minimizing organizational business loss. Journal of Knowledge Management (2011).
[55]
Lichess. 2022.
[56]
Ke Liu and Johan Awang bin Othman. 2022. Design of Experiential Teaching System for Solfeggio in Normal Universities Based on Machine Learning Algorithm. Mobile Information Systems, Vol. 2022 (2022).
[57]
Oisin Mac Aodha, Shihan Su, Yuxin Chen, Pietro Perona, and Yisong Yue. 2018. Teaching categories to human learners with visual explanations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3820--3828.
[58]
Masaki Matsubara, Masaki Kobayashi, and Atsuyuki Morishima. 2018. A Learning Effect by Presenting Machine Prediction as a Reference Answer in Self-correction. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, 3522--3528.
[59]
Martin Movz ina, Matej Guid, Aleksander Sadikov, Vida Groznik, and Ivan Bratko. 2012. Goal-oriented conceptualization of procedural knowledge. In International Conference on Intelligent Tutoring Systems. Springer, 286--291.
[60]
Michael Muller, Melanie Feinberg, Timothy George, Steven J Jackson, Bonnie E John, Mary Beth Kery, and Samir Passi. 2019. Human-centered study of data science work practices. In Extended abstracts of the 2019 CHI conference on human factors in computing systems. 1--8.
[61]
Fiona Fui-Hoon Nah and Izak Benbasat. 2004. Knowledge-based support in a group decision making context: An expert-novice comparison. Journal of the Association for Information Systems, Vol. 5, 3 (2004), 5.
[62]
Takumi Nakayama, Masaki Matsubara, and Atsuyuki Morishima. 2021. Crowd-Worker Skill Improvement with AI Co-Learners. In Proceedings of the 9th International Conference on Human-Agent Interaction. 316--322.
[63]
Hannele Niemi. 2021. AI in learning: Preparing grounds for future learning. Journal of Pacific Rim Psychology, Vol. 15 (2021), 18344909211038105.
[64]
Ikujiro Nonaka. 1994. A dynamic theory of organizational knowledge creation. Organization science, Vol. 5, 1 (1994), 14--37.
[65]
Ikujir=o Nonaka and Hirotaka Takeuchi. 2007. The knowledge-creating company. Harvard business review, Vol. 85, 7/8 (2007), 162.
[66]
Jussi Okkonen, Vilma Vuori, and Miikka Palvalin. 2019. Digitalization changing work: Employees' view on the benefits and hindrances. In Information Technology and Systems: Proceedings of ICITS 2019. Springer, 165--176.
[67]
Fidelia A Orji and Julita Vassileva. 2021. Modelling and Quantifying Learner Motivation for Adaptive Systems: Current Insight and Future Perspectives. In International Conference on Human-Computer Interaction. Springer, 79--92.
[68]
Kaustubh R Patil, Jerry Zhu, Łukasz Kopeć, and Bradley C Love. 2014. Optimal teaching for limited-capacity human learners. Advances in neural information processing systems, Vol. 27 (2014).
[69]
Selwyn Piramuthu. 2005. Knowledge-based web-enabled agents and intelligent tutoring systems. IEEE transactions on education, Vol. 48, 4 (2005), 750--756.
[70]
Michael Polanyi. 2009. The tacit dimension. In Knowledge in organizations. Routledge, 135--146.
[71]
Fen Qin, Kai Li, and Jianyuan Yan. 2020. Understanding user trust in artificial intelligence-based educational systems: Evidence from China. British Journal of Educational Technology, Vol. 51, 5 (2020), 1693--1710.
[72]
Jan Joris Roessingh, Gerald Poppinga, Joost van Oijen, and Armon Toubman. 2019. Application of artificial intelligence to adaptive instruction-combining the concepts. In International Conference on Human-Computer Interaction. Springer, 542--556.
[73]
Gilbert Ryle. 1945. Knowing how and knowing that: The presidential address. In Proceedings of the Aristotelian society, Vol. 46. JSTOR, 1--16.
[74]
Sanjay K Sahay, Nihita Goel, Murtuza Jadliwala, and Shambhu Upadhyaya. 2021. Advances in secure knowledge management in the artificial intelligence era. Information Systems Frontiers, Vol. 23, 4 (2021), 807--810.
[75]
Kjeld Schmidt and Kjeld Schmidt. 2011. Cooperative work and coordinative practices. Springer.
[76]
Achim Schmitt, Stefano Borzillo, and Gilbert Probst. 2012. Don't let knowledge walk away: Knowledge retention during employee downsizing. Management Learning, Vol. 43, 1 (2012), 53--74.
[77]
Isabella Seeber, Lena Waizenegger, Stefan Seidel, Stefan Morana, Izak Benbasat, and Paul Benjamin Lowry. 2020. Collaborating with technology-based autonomous agents: Issues and research opportunities. Internet Research (2020).
[78]
Cathrine Seidelin, Yvonne Dittrich, and Erik Grönvall. 2018. Data Work in a Knowledge-Broker Organization: How Cross-Organizational Data Maintenance shapes Human Data Interactions. In British Human Computer Interaction Conference 2018. BCS Learning and Development Ltd, 1--12.
[79]
Iulian Vlad Serban, Varun Gupta, Ekaterina Kochmar, Dung D Vu, Robert Belfer, Joelle Pineau, Aaron Courville, Laurent Charlin, and Yoshua Bengio. 2020. A large-scale, open-domain, mixed-interface dialogue-based ITS for STEM. In International Conference on Artificial Intelligence in Education. Springer, 387--392.
[80]
Zeyuan Shang, Emanuel Zgraggen, Benedetto Buratti, Ferdinand Kossmann, Philipp Eichmann, Yeounoh Chung, Carsten Binnig, Eli Upfal, and Tim Kraska. 2019. Democratizing data science through interactive curation of ml pipelines. In Proceedings of the 2019 international conference on management of data. 1171--1188.
[81]
Yu-Shan Shih, Ting-Ting Lee, Chieh-Yu Liu, and Mary Etta Mills. 2013. Evaluation of an online orientation program for new healthcare employees. CIN: Computers, Informatics, Nursing, Vol. 31, 7 (2013), 343--350.
[82]
Patrice Y Simard, Saleema Amershi, David M Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et al. 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017).
[83]
Adish Singla, Ilija Bogunovic, Gábor Bartók, Amin Karbasi, and Andreas Krause. 2014. Near-optimally teaching the crowd to classify. In International Conference on Machine Learning. PMLR, 154--162.
[84]
Gary Stein, Avelino J Gonzalez, and Clayton Barham. 2013. Machines that learn and teach seamlessly. IEEE transactions on learning technologies, Vol. 6, 4 (2013), 389--402.
[85]
Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret Burnett, Thomas Dietterich, Erin Sullivan, and Jonathan Herlocker. 2009. Interacting meaningfully with machine learning systems: Three experiments. International journal of human-computer studies, Vol. 67, 8 (2009), 639--662.
[86]
Shihan Su, Yuxin Chen, Oisin Mac Aodha, Pietro Perona, and Yisong Yue. 2017. Interpretable machine teaching via feature feedback. (2017).
[87]
Wei Sun, Yunzhi Li, Feng Tian, Xiangmin Fan, and Hongan Wang. 2019. How presenters perceive and react to audience flow prediction in-situ: An explorative study of live online lectures. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--19.
[88]
Christos Troussas, Konstantina Chrysafiadi, and Maria Virvou. 2018. Machine learning and fuzzy logic techniques for personalized tutoring of foreign languages. In International Conference on Artificial Intelligence in Education. Springer, 358--362.
[89]
Maike Vollstedt and Sebastian Rezat. 2019. An introduction to grounded theory with a special focus on axial coding and the coding paradigm. Compendium for early career researchers in mathematics education, Vol. 13 (2019), 81--100.
[90]
Edward Wakelam, Amanda Jefferies, Neil Davey, and Yi Sun. 2015. The potential for using artificial intelligence techniques to improve e-learning systems. In ECEL 2015 Conference proceedings.
[91]
Emily Wall, Soroush Ghorashi, and Gonzalo Ramos. 2019. Using expert patterns in assisted interactive machine learning: A study in machine teaching. In IFIP Conference on Human-Computer Interaction. Springer, 578--599.
[92]
Thiemo Wambsganss, Tobias Kueng, Matthias Soellner, and Jan Marco Leimeister. 2021. ArgueTutor: An adaptive dialog-based learning system for argumentation skills. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1--13.
[93]
Thiemo Wambsganss, Matthias Söllner, and Jan Marco Leimeister. 2020. Design and evaluation of an adaptive dialog-based tutoring system for argumentation skills. In International Conference on Information Systems (ICIS).-Hyderabad, India.
[94]
Dakuo Wang, Justin D Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019b. Human-AI collaboration in data science: Exploring data scientists' perceptions of automated AI. Proceedings of the ACM on human-computer interaction, Vol. 3, CSCW (2019), 1--24.
[95]
Ming-Chen Wang, Vahid Golderzahi, and Hsing-Kuo Pao. 2019a. Extracting Explainable Deep Representation for Machine Tutoring. In 2019 IEEE International Conference on Big Data (Big Data). IEEE, 4651--4658.
[96]
Pei Wang, Kabir Nagrecha, and Nuno Vasconcelos. 2021a. Gradient-based algorithms for machine teaching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1387--1396.
[97]
Pei Wang and Nuno Vasconcelos. 2020. Scout: Self-aware discriminant counterfactual explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8981--8990.
[98]
Pei Wang and Nuno Vasconcelos. 2021. A Machine Teaching Framework for Scalable Recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4945--4954.
[99]
Xu Wang, Meredith Thompson, Kexin Yang, Dan Roy, Kenneth R Koedinger, Carolyn P Rose, and Justin Reich. 2021b. Practice-based teacher questioning strategy training with ELK: A role-playing simulation for eliciting learner knowledge. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--27.
[100]
Jane Webster and Richard T Watson. 2002. Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly (2002), xiii--xxiii.
[101]
Amy Wells, Shaan Patel, Jason B Lee, and Kiran Motaparthi. 2021. Artificial intelligence in dermatopathology: Diagnosis, education, and research. Journal of Cutaneous Pathology, Vol. 48, 8 (2021), 1061--1068.
[102]
Joseph B Wiggins, Kristy Elizabeth Boyer, Alok Baikadi, Aysu Ezen-Can, Joseph F Grafsgaard, Eun Young Ha, James C Lester, Christopher M Mitchell, and Eric N Wiebe. 2015. JavaTutor: an intelligent tutoring system that adapts to cognitive and affective states during computer programming. In Proceedings of the 46th acm technical symposium on computer science education. 599--599.
[103]
Karl M Wiig, Robert De Hoog, and Rob Van der Spek. 1997. Supporting knowledge management: a selection of methods and techniques. Expert systems with applications, Vol. 13, 1 (1997), 15--27.
[104]
Hans Friedrich Witschel, Prajakta Diwanji, and Knut Hinkelmann. 2021. A Dialog-Based Tutoring System for Project-Based Learning in Information Systems Education. In New Trends in Business Information Systems and Technology. Springer, 81--95.
[105]
Christine T Wolf. 2019. Conceptualizing care in the everyday work practices of machine learning developers. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion. 331--335.
[106]
Joost F Wolfswinkel, Elfi Furtmueller, and Celeste PM Wilderom. 2013. Using grounded theory as a method for rigorously reviewing literature. European journal of information systems, Vol. 22, 1 (2013), 45--55.
[107]
Kexin Bella Yang, Tomohiro Nagashima, Junhui Yao, Joseph Jay Williams, Kenneth Holstein, and Vincent Aleven. 2021. Can Crowds Customize Instructional Materials with Minimal Expert Guidance? Exploring Teacher-guided Crowdsourcing for Improving Hints in an AI-based Tutor. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--24.
[108]
Yetkin Yildirim and Akif Celepcikay. 2021. Artificial Intelligence and Machine Learning Applications in Education. Eurasian Journal of Higher Education, Vol. 2, 4 (2021), 1--11.
[109]
Alexey Zagalsky, Dov Te'eni, Inbal Yahav, David G Schwartz, Gahl Silverman, Daniel Cohen, Yossi Mann, and Dafna Lewinsky. 2021. The design of reciprocal learning between human and artificial intelligence. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW2 (2021), 1--36.
[110]
Matej Zapuvs ek, Martin Movz ina, Ivan Bratko, Jovz e Rugelj, and Matej Guid. 2014. Designing an interactive teaching tool with ABML knowledge refinement loop. In International Conference on Intelligent Tutoring Systems. Springer, 575--582.
[111]
Jing Zhang, Huihui Wang, Shunmei Meng, and Victor S Sheng. 2020. Interactive learning with proactive cognition enhancement for crowd workers. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 540--547.
[112]
Yao Zhou, Arun Reddy Nelakurthi, and Jingrui He. 2018. Unlearn what you have learned: Adaptive crowd teaching with exponentially decayed memory learners. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2817--2826.
[113]
Jerry Zhu. 2013. Machine teaching for bayesian learners in the exponential family. Advances in Neural Information Processing Systems, Vol. 26 (2013).
[114]
Xiaojin Zhu. 2015. Machine teaching: An inverse problem to machine learning and an approach toward optimal education. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 29.
[115]
Xiaojin Zhu, Adish Singla, Sandra Zilles, and Anna N Rafferty. 2018. An overview of machine teaching. arXiv preprint arXiv:1801.05927 (2018).
Index Terms
- ML-Based Teaching Systems: A Conceptual Framework
Recommendations
An experiment in agent teaching by subject matter experts
This paper presents a successful knowledge-acquisition experiment in which subject matter experts who did not have any prior knowledge-engineering experience succeeded in teaching the Disciple- COA agent how to critique courses of action, a challenge ...
Generation, testing and recommendation of teaching materials using classification learning
The study focuses on problems related to the application of Artificial Intelligence (AI) in Human-Computer Interaction (HCI) for Educational Adaptive Hypermedia (EAH). It analyses how to apply Machine Learning (ML) techniques to support teachers in ...
Architecting ML-enabled systems: Challenges, best practices, and design decisions
Abstract Context:Machine learning is increasingly used in a wide set of applications ranging from recommendation engines to autonomous systems through business intelligence and smart assistants. Designing and developing machine learning systems is a ...
Highlights- This work describes a mixed method study on architecting ML-enabled systems.
- This work includes a systematic literature review and expert interviews.
- This work elicits 35 design challenges, 42 best practices and 27 design ...
Comments
Information & Contributors
Information
Published In
October 2023
4055 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 October 2023
Published in PACMHCI Volume 7, Issue CSCW2
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 178Total Downloads
- Downloads (Last 12 months)178
- Downloads (Last 6 weeks)16
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
Cited By
View all- Morrison KSpitzer PTurri VFeng MKühl NPerer A(2024)The Impact of Imperfect XAI on Human-AI Decision-MakingProceedings of the ACM on Human-Computer Interaction10.1145/36410228:CSCW1(1-39)Online publication date: 26-Apr-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in