Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Automated discovery of symbolic laws governing skill acquisition from naturally occurring data

Nature Computational Science volume 4, pages 334–345 (2024)Cite this article

Subjects

A preprint version of the article is available at arXiv.

Abstract

Skill acquisition is a key area of research in cognitive psychology as it encompasses multiple psychological processes. The laws discovered under experimental paradigms are controversial and lack generalizability. This paper aims to unearth the laws of skill learning from large-scale training log data. A two-stage algorithm was developed to tackle the issues of unobservable cognitive states and an algorithmic explosion in searching. A deep learning model is initially employed to determine the learner’s cognitive state and assess the feature importance. Symbolic regression algorithms are then used to parse the neural network model into algebraic equations. Experimental results show that the algorithm can accurately restore preset laws within a noise range in continuous feedback settings. When applied to Lumosity training data, the method outperforms traditional and recent models in fitness terms. The study reveals two new forms of skill acquisition laws and reaffirms some previous findings.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overall model architecture diagram.
Fig. 2: Results of the simulated data experiment.
Fig. 3: The skill acquisition patterns discovered by the proposed method.

Similar content being viewed by others

Data availability

All data generated or analyzed during this study are available via GitHub at https://github.com/ccnu-mathits/ADM (ref. 58) and Zenodo at https://doi.org/10.5281/zenodo.10938670 (ref. 59). Source Data are provided with this paper.

Code availability

The source code of this study is freely available on Github at https://github.com/ccnu-mathits/ADM (ref. 58), and via Zenodo at https://doi.org/10.5281/zenodo.10938670 (ref. 59).

References

  1. VanLehn, K. Cognitive skill acquisition. Ann. Rev. Psychol. 47, 513–539 (1996).

    Article  Google Scholar 

  2. DeKeyser, R. in Skill Acquisition Theory 83–104 (Routledge, 2020).

  3. Tabibian, B. et al. Enhancing human learning via spaced repetition optimization. Proc. Natl Acad. Sci. USA 116, 3988–3993 (2019).

    Article  MathSciNet  Google Scholar 

  4. Evans, N. J., Brown, S. D., Mewhort, D. J. & Heathcote, A. Refining the law of practice. Psychol. Rev. 125, 592 (2018).

    Article  Google Scholar 

  5. Heathcote, A., Brown, S. & Mewhort, D. J. The power law repealed: the case for an exponential law of practice. Psychon. Bull. Rev. 7, 185–207 (2000).

    Article  Google Scholar 

  6. Shrager, J., Hogg, T. & Huberman, B. A. A graph-dynamic model of the power law of practice and the problem-solving fan-effect. Science 242, 414–416 (1988).

    Article  Google Scholar 

  7. Wixted, J. T. The enigma of forgetting. Proc. Natl Acad. Sci. USA 119, e2201332119 (2022).

    Article  Google Scholar 

  8. Averell, L. & Heathcote, A. The form of the forgetting curve and the fate of memories. J. Math. Psychol. 55, 25–35 (2011).

    Article  MathSciNet  Google Scholar 

  9. Roediger III, H. L. Relativity of remembering: why the laws of memory vanished. Annu. Rev. Psychol. 59, 225–254 (2008).

    Article  Google Scholar 

  10. Chiaburu, D. S. & Marinova, S. V. What predicts skill transfer? An exploratory study of goal orientation, training self-efficacy and organizational supports. Int. J. Train. Dev. 9, 110–123 (2005).

    Article  Google Scholar 

  11. Sturm, L. P. et al. A systematic review of skills transfer after surgical simulation training. Ann. Surgery 248, 166–179 (2008).

    Article  Google Scholar 

  12. Logan, G. D. Toward an instance theory of automatization. Psycho. Rev. 95, 492 (1988).

    Article  Google Scholar 

  13. Logan, G. D. Shapes of reaction-time distributions and shapes of learning curves: a test of the instance theory of automaticity. J. Exp. Psychol. 18, 883 (1992).

    Google Scholar 

  14. Anderson, J. R. Acquisition of cognitive skill. Psychol. Rev. 89, 369 (1982).

    Article  Google Scholar 

  15. Tenison, C. & Anderson, J. R. Modeling the distinct phases of skill acquisition. J. Exp. Psychol. 42, 749 (2016).

    Google Scholar 

  16. Tenison, C., Fincham, J. M. & Anderson, J. R. Phases of learning: how skill acquisition impacts cognitive processing. Cognitive Psychol. 87, 1–28 (2016).

    Article  Google Scholar 

  17. Jordan, M. I. Serial order: a parallel distributed processing approach. Adv. Pyschol. 121, 471–495 (1997).

  18. McClelland, J. L. et al. Parallel Distributed Processing Vol. 2 (MIT Press, 1986).

  19. Young, R. M. & Lewis, R. L. in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control 224–256 (Cambridge Univ. Press, 1999).

  20. Anderson, J. R., Matessa, M. & Lebiere, C. in Human–Computer Interaction Vol. 12, 439–462 (Lawrence Erlbaum Associates, 1997).

  21. Ritter, F. E., Tehranchi, F. & Oury, J. D. ACT-R: a cognitive architecture for modeling cognition. WIREs Cogn. Sci. 10, e1488 (2019).

    Article  Google Scholar 

  22. Goldstone, R. L. & Lupyan, G. Discovering psychological principles by mining naturally occurring data sets. Topics Cogn. Sci. 8, 548–568 (2016).

    Article  Google Scholar 

  23. Jenkins, J. J., Cermak, L. & Craik, F. in Levels of Processing in Human Memory 429–446 (Pyschology, 1979).

  24. Udrescu, S.-M. & Tegmark, M. AI Feynman: a physics-inspired method for symbolic regression. Sci. Adv. 6, eaay2631 (2020).

    Article  Google Scholar 

  25. Wang, H. et al. Scientific discovery in the age of artificial intelligence. Nature 620, 47–60 (2023).

    Article  Google Scholar 

  26. Schmidt, M. & Lipson, H. Distilling free-form natural laws from experimental data. Science 324, 81–85 (2009).

    Article  Google Scholar 

  27. Cranmer, M. et al. Discovering symbolic models from deep learning with inductive biases. In 34th Conference on Neural Information Processing Systems 17429–17442 (NeurIPS, 2020).

  28. Rudy, S. H., Brunton, S. L., Proctor, J. L. & Kutz, J. N. Data-driven discovery of partial differential equations. Sci. Adv. 3, e1602614 (2017).

    Article  Google Scholar 

  29. Chen, Z., Liu, Y. & Sun, H. Physics-informed learning of governing equations from scarce data. Nat. Commun. 12, 6136 (2021).

    Article  Google Scholar 

  30. Margraf, J. T., Jung, H., Scheurer, C. & Reuter, K. Exploring catalytic reaction networks with machine learning. Nat. Catal. 6, 112–121 (2023).

    Article  Google Scholar 

  31. Han, Z.-K. et al. Single-atom alloy catalysts designed by first-principles calculations and artificial intelligence. Nat. Commun. 12, 1833 (2021).

    Article  Google Scholar 

  32. Wang, Y., Wagner, N. & Rondinelli, J. M. Symbolic regression in materials science. MRS Commun. 9, 793–805 (2019).

    Article  Google Scholar 

  33. He, M. & Zhang, L. Machine learning and symbolic regression investigation on stability of mxene materials. Comput. Mater. Sci. 196, 110578 (2021).

    Article  Google Scholar 

  34. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  35. Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems 30 (NIPS, 2017).

  36. Baker, F. B. The Basics of Item Response Theory (ERIC, 2001).

  37. Hambleton, R. K., Swaminathan, H. & Rogers, H. J. Fundamentals of Item Response Theory Vol. 2 (Sage, 1991).

  38. Swaminathan, H. & Gifford, J. A. Bayesian estimation in the three-parameter logistic model. Psychometrika 51, 589–601 (1986).

    Article  MathSciNet  Google Scholar 

  39. Maris, G. & Bechger, T. On interpreting the model parameters for the three parameter logistic model. Measurement 7, 75–88 (2009).

    Google Scholar 

  40. Hornik, K. Approximation capabilities of multilayer feedforward networks. Neural Netw. 4, 251–257 (1991).

    Article  Google Scholar 

  41. Raghu, M., Poole, B., Kleinberg, J., Ganguli, S. & Sohl-Dickstein, J. On the expressive power of deep neural networks. In Proc. 34th International Conference on Machine Learning 2847–2854 (PMLR, 2017).

  42. Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303–314 (1989).

    Article  MathSciNet  Google Scholar 

  43. Spearman, C. “General intelligence,” objectively determined and measured. Am. J. Psychol. 15, 201–293 (1904).

    Article  Google Scholar 

  44. Eid, M., Geiser, C., Koch, T. & Heene, M. Anomalous results in g-factor models: explanations and alternatives. Psychol. Methods 22, 541 (2017).

    Article  Google Scholar 

  45. Steyvers, M. & Schafer, R. J. Inferring latent learning factors in large-scale cognitive training data. Nat. Hum. Behav. 4, 1145–1155 (2020).

    Article  Google Scholar 

  46. Simons, D. J. et al. Do “brain-training” programs work?. Psychol. Sci. Public Interest 17, 103–186 (2016).

    Article  Google Scholar 

  47. Kievit, R. A. et al. Mutualistic coupling between vocabulary and reasoning supports cognitive development during late adolescence and early adulthood. Psychol. Sci. 28, 1419–1431 (2017).

    Article  Google Scholar 

  48. Kievit, R. A., Hofman, A. D. & Nation, K. Mutualistic coupling between vocabulary and reasoning in young children: a replication and extension of the study by Kievit et al. (2017). Psychol. Sci. 30, 1245–1252 (2019).

    Article  Google Scholar 

  49. Fisher, R. A. Design of experiments. Brit. Med. J. 1, 554 (1936).

    Article  Google Scholar 

  50. Kumar, A., Benjamin, A. S., Heathcote, A. & Steyvers, M. Comparing models of learning and relearning in large-scale cognitive training data sets. NPJ Sci. Learn. 7, 24 (2022).

    Article  Google Scholar 

  51. Liu, R. & Koedinger, K. R. Towards reliable and valid measurement of individualized student parameters. In Proc.10th International Conference on Educational Data Mining 135–142 (International Educational Data Mining Society, 2017).

  52. Koedinger, K. R., Carvalho, P. F., Liu, R. & McLaughlin, E. A. An astonishing regularity in student learning rate. Proc. Natl Acad. Sci. USA 120, e2221311120 (2023).

    Article  Google Scholar 

  53. Neath, A. A. & Cavanaugh, J. E. The bayesian information criterion: background, derivation, and applications. WIREs Comput. Stats 4, 199–203 (2012).

    Article  Google Scholar 

  54. Vrieze, S. I. Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychol. Methods 17, 228 (2012).

    Article  Google Scholar 

  55. Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. In Proc. 33rd International Conference on Neural Information Processing Systems 8026–8037 (Curran Associates Inc., 2019).

  56. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2015).

  57. Steyvers, M. & Benjamin, A. S. The joint contribution of participation and performance to learning functions: exploring the effects of age in large-scale data sets. Behav. Res. Methods 51, 1531–1543 (2019).

    Article  Google Scholar 

  58. Liu, S. et al. ccnu-mathits/ADM. GitHub https://github.com/ccnu-mathits/ADM (2024).

  59. Liu, S. et al. ccnu-mathits/ADM: source code. Zenodo https://doi.org/10.5281/zenodo.10938670 (2024).

Download references

Acknowledgements

This work was jointly supported by the National Science and Technology Major Project (grant no. 2022ZD0117103 to J.S. and Z.Y.), the National Natural Science Foundation of China (grant no. 62293554 to J.S. and Z.Y., 62107017 to X.S. and 62077021 to J.S.), the Higher Education Science Research Program of China Association of Higher Education (grant no. 23XXK0301 to J.S.), the China Postdoctoral Science Foundation (grant no. 2023T160256 to X.S.), Hubei Provincial Natural Science Foundation of China (grant no. 2023AFA020 to S.L. and 2022CFB414 to J.S.), and Fundamental Research Funds for the Central Universities (grant no. CCNU23XJ007 to X.S. and CCNU22LJ005 to S.L.).

Author information

Authors and Affiliations

  1. National Engineering Research Center of Educational Big Data, Central China Normal University, Wuhan, China

    Sannyuya Liu, Qing Li, Xiaoxuan Shen, Jianwen Sun & Zongkai Yang

  2. Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, China

    Sannyuya Liu, Qing Li, Xiaoxuan Shen, Jianwen Sun & Zongkai Yang

  3. National Engineering Research Center for E-learning, Central China Normal University, Wuhan, China

    Sannyuya Liu & Zongkai Yang

Authors
  1. Sannyuya Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoxuan Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianwen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zongkai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.L. and X.S. conceptualized the work. S.L. and X.S. designed the methodology. Q.L. performed investigations and curated the data, whereas S.L., Q.L. and Z.Y. performed the data validation. J.S. and Z.Y. administered the project. J.S. and Z.Y. acquired funding. X.S., J.S. and Z.Y. supervised the project. S.L. and X.S. wrote the original draft, whereas Q.L. and J.S. reviewed and edited it.

Corresponding authors

Correspondence to Xiaoxuan Shen, Jianwen Sun or Zongkai Yang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks Martijn Meeter, Konstantina Sokratous and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Ananya Rastogi, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Partial results on the Lumosity dataset.

More specifically, (a) the change curve of mean fitting absolute error during the iteration process of the deep learning regressor; (b) the change curve of the value of the regularization term during the iteration processes; (c) the prediction error distribution of 1000 randomly selected records from the trained H1000+R1 model. The box plot displays the interquartile range (IQR) with the median line, while the whiskers extend to the minimum and maximum values or a multiple of the IQR from the quartiles. Outliers are depicted as individual points beyond the whiskers; (d) the proportion of the number of practice times for each skill to the total number of practice times; (e) the distribution of feature importance for each skill in H1000+R1.

Source data

Extended Data Fig. 2 Schematic representation of the simulation experiment.

(a) Flowchart depicting the process of generating simulated data and the data format. (b) Schematic diagram illustrating the evaluation procedure of the simulation experiment. (c) Sample representation of the simulated data.

Source data

Extended Data Fig. 3 The feature set utilized in the real-world dataset (Lumosity) experiment.

The feature set comprises three components: user (U), exercise (E), and scheduling (S). The features consist of two categories: discrete and continuous. Discrete features are encoded through one-hot encoding, while continuous features are encoded using real values. Game features are two-dimensional, consisting of skill and subskill, both of which are discrete features and characterize the relationship between the game and cognitive skills. Subskill is a subdivision feature of skill.

Source data

Extended Data Fig. 4 Analysis of the difference between the mastery level of skills calculated by the deep learning regressor and the discovered governing laws.

Here, we present the changes in skill proficiency at each time point during 2000 practice sessions for five learners. The blue line represents the mastery curve computed by the deep regressor, while the red line represents the curve computed by the discovered governing law. The governing law is the same for all learners, but due to differences in their choices and order of practice, there are variations in the independent variable of the governing law for each learner.

Source data

Supplementary information

Supplementary Information

Supplementary Figs. 1 and 2.

Reporting Summary

Source data

Source Data Fig. 1

Image source data for Fig. 1.

Source Data Fig. 2

Numerical source data for Fig. 2.

Source Data Fig. 3

Numerical source data for Fig. 3.

Source Data Extended Data Fig. 1

Numerical source data for Extended Data Fig. 1.

Source Data Extended Data Fig. 2

Image source data for Extended Data Fig. 2.

Source Data Extended Data Fig. 3

Image source data for Extended Data Fig. 3.

Source Data Extended Data Fig. 4

Numerical source data for Extended Data Fig. 4.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Li, Q., Shen, X. et al. Automated discovery of symbolic laws governing skill acquisition from naturally occurring data. Nat Comput Sci 4, 334–345 (2024). https://doi.org/10.1038/s43588-024-00629-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s43588-024-00629-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing