Abstract
As there is a need for interpretable classification models in many application domains, symbolic, interpretable classification models have been studied for many years in the literature. Rule-based models are an important class of such models. However, most of the common algorithms for learning rule-based models rely on heuristic search strategies developed for specific rule-learning settings. These search strategies are very different from those used in neural forms of machine learning, where gradient-based approaches are used. Attempting to combine neural and symbolic machine learning, recent studies have therefore explored gradient-based rule learning using neural network architectures. These new proposals make it possible to apply approaches for learning neural networks to rule learning. However, these past studies focus on unordered rule sets for classification tasks, while many common rule-learning algorithms learn rule sets with an order. In this work, we propose RL-Net, an approach for learning ordered rule lists based on neural networks. We demonstrate that the performance we obtain on classification tasks is similar to the state-of-the-art algorithms for rule learning in binary and multi-class classification settings. Moreover, we show that our model can easily be adapted to multi-label learning tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aoga, J.O.R., Nijssen, S., Schaus, P.: Modeling pattern set mining using Boolean circuits. In: Schiex, T., de Givry, S. (eds.) CP 2019. LNCS, vol. 11802, pp. 621–638. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30048-7_36
Beck, F., Fürnkranz, J.: An empirical investigation into deep and shallow rule learning. Front. Artif. Intell. 4 (2021)
Beck, F., Fürnkranz, J.: An investigation into mini-batch rule learning. arXiv preprint arXiv:2106.10202 (2021)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and regression trees. Routledge (2017)
Cohen, W.W.: Fast effective rule induction. In: Twelfth International Conference on Machine Learning, pp. 115–123. Elsevier (1995)
Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Fischer, J., Vreeken, J.: Differentiable pattern set mining. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 383–392 (2021)
Ganter, B., Wille, R.: Formal Concept Analysis: Mathematical Foundations. Springer Science & Business Media (2012). https://doi.org/10.1007/978-3-642-59830-2
Guns, T., Nijssen, S., De Raedt, L.: Evaluating pattern set mining strategies in a constraint programming framework. In: Huang, J.Z., Cao, L., Srivastava, J. (eds.) PAKDD 2011. LNCS (LNAI), vol. 6635, pp. 382–394. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20847-8_32
Guns, T., Nijssen, S., De Raedt, L.: K-pattern set mining under constraints. IEEE Trans. Knowl. Data Eng. 25(2), 402–418 (2011)
Li, W., Han, J., Pei, J.: CMAR: accurate and efficient classification based on multiple class-association rules. In: Proceedings 2001 IEEE International Conference on Data Mining, pp. 369–376. IEEE (2001)
Liu, B., Hsu, W., Ma, Y., et al.: Integrating classification and association rule mining. In: KDD, vol. 98, pp. 80–86 (1998)
Louizos, C., Welling, M., Kingma, D.P.: Learning sparse neural networks through L0 regularization. In: 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings (2018)
Qiao, L., Wang, W., Lin, B.: Learning accurate and interpretable decision rule sets from neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4303–4311 (2021)
Shi, S., Xie, Y., Wang, Z., Ding, B., Li, Y., Zhang, M.: Explainable Neural Rule Learning. In: WWW 2022 - Proceedings of the ACM Web Conference 2022, pp. 3031–3041. Association for Computing Machinery, Inc (2022)
Yang, Y., Morillo, I.G., Hospedales, T.M.: Deep neural decision trees. In: ICML Workshop on Human Interpretability in Machine Learning. arXiv preprint arXiv:1806.06988 (2018)
Acknowledgments
This work was supported by Service Public de Wallonie Recherche under grant n\(^{\circ }\)2010235 - ARIAC by DIGITALWALLONIA4.AI, and under grant n\(^{\circ }\)2110107 - SERENITY2 by WIN2WAL. We would also like to thank FAPESP, Brazil (Grants No. 2017/21174-8 and 2020/00123-9) for the financial support.
Computational resources have been provided by the supercomputing facilities of the Université Catholique de Louvain (CISM/UCL) and the Consortium des Équipements de Calcul Intensif en Fédération Wallonie Bruxelles (CÉCI) funded by the Fond de la Recherche Scientifique de Belgique (F.R.S.-FNRS) under convention 2.5020.11 and by the Walloon Region.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dierckx, L., Veroneze, R., Nijssen, S. (2023). RL-Net: Interpretable Rule Learning with Neural Networks. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13935. Springer, Cham. https://doi.org/10.1007/978-3-031-33374-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-33374-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33373-6
Online ISBN: 978-3-031-33374-3
eBook Packages: Computer ScienceComputer Science (R0)