Abstract
Category theory has been successfully applied in various domains of science, shedding light on universal principles unifying diverse phenomena and thereby enabling knowledge transfer between them. Applications to machine learning have been pursued recently, and yet there is still a gap between abstract mathematical foundations and concrete applications to machine learning tasks. In this paper we introduce DisCoPyro as a categorical structure learning framework, which combines categorical structures (such as symmetric monoidal categories and operads) with amortized variational inference, and can be applied, e.g., in program learning for variational autoencoders. We provide both mathematical foundations and concrete applications together with comparison of experimental performance with other models (e.g., neuro-symbolic models). We speculate that DisCoPyro could ultimately contribute to the development of artificial general intelligence.
Supported by the JST Moonshot Programme on AI Robotics (JPMJMS2033-02).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Also called a “hypersignature”.
References
Ambrogioni, L., et al.: Automatic structured variational inference. In: Proc. AISTATS, pp. 676–684 (2021)
Bingham, E., et al.: Pyro: deep universal probabilistic programming. JMLR 20, 973–978 (2019)
Bradley, T.D.: What is applied category theory? arXiv preprint arXiv:1809.05923 (2018)
Coecke, B., Sadrzadeh, M., Clark, S.: Mathematical foundations for a compositional distributional model of meaning. arXiv preprint arXiv:1003.4394 (2010)
Feinman, R., Lake, B.M.: Learning task-general representations with generative neuro-symbolic modeling. In: Proc. ICLR (2021)
de Felice, G., Toumi, A., Coecke, B.: DisCoPy: monoidal categories in python. In: Applied Category Theory Conference, pp. 1–20. EPTCS (2020). http://arxiv.org/abs/2005.02975arXiv: 2005.02975
Fong, B., Spivak, D.I.: Seven Sketches in Compositionality: An Invitation to Applied Category Theory. Cambridge University Press, Cambridge (2019)
Fritz, T.: A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Adv. Math. 370, 107239 (2020)
He, J., et al.: Variational autoencoders with jointly optimized latent dependency structure. In: Proc. ICLR 2019, pp. 1–16 (2019)
Hermida, C., Makkaiy, M., Power, J.: Higher dimensional multigraphs. In: Proceedings - Symposium on Logic in Computer Science, pp. 199–206 (1998)
Hewitt, L.B., et al.: The variational homoencoder: learning to learn high capacity generative models from few examples. Proc. UAI 2018, 988–997 (2018)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Kirsch, L., Kunze, J., Barber, D.: Modular networks: learning to decompose neural computation. In: Proc. NeurIPS 2018, pp. 2408–2418 (2018)
Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: The Omniglot challenge: a 3-year progress report. Curr. Opin. Behav. Sci. 29, 97–104 (2019)
Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017)
Latouche, G., Ramaswami, V.: Introduction to matrix analytic methods in stochastic modeling. In: Society for Industrial and Applied Mathematics (1999)
Milnor, J.: On spaces having the homotopy type of \(CW\)-complex. Trans. Am. Math. Soc. 90, 272–280 (1959)
Phillips, S., Wilson, W.H.: Categorial compositionality: a category theory explanation for the systematicity of human cognition. PLoS Comput. Biol. 6, e1000858 (2010)
Rezende, D.J., et al.: One-shot generalization in deep generative models. In: International Conference on Machine Learning, vol. 48 (2016)
Rosenbaum, C., Klinger, T., Riemer, M.: Routing networks: adaptive selection of non-linear functions for multi-task learning. In: Proc. ICLR 2018, pp. 1–10 (2018)
Shiebler, D., Gavranović, B., Wilson, P.: Category theory in machine learning. In: Proc. ACT 2021 (2021)
Spivak, D.I.: The operad of wiring diagrams: formalizing a graphical language for databases, recursion, and plug-and-play circuits (2013). arxiv: 1305.0297
Webb, S., et al.: Faithful inversion of generative models for effective amortized inference. In: Proc. NIPS 2018, pp. 3074–3084 (2018)
Zhao, S., Song, J., Ermon, S.: Learning hierarchical features from generative models. In: International Conference on Machine Learning (2017)
Acknowledgements
The authors would like to thank the anonymous reviewers for their constructive feedback and encouragement. This work was supported by the Japan Science and Technology Agency (JST JPMJMS2033) and National Science Foundation of the United States of America (NSF 2047253).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sennesh, E., Xu, T., Maruyama, Y. (2023). Computing with Categories in Machine Learning. In: Hammer, P., Alirezaie, M., Strannegård, C. (eds) Artificial General Intelligence. AGI 2023. Lecture Notes in Computer Science(), vol 13921. Springer, Cham. https://doi.org/10.1007/978-3-031-33469-6_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-33469-6_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33468-9
Online ISBN: 978-3-031-33469-6
eBook Packages: Computer ScienceComputer Science (R0)