ARM-Net: Adaptive Relation Modeling Network for Structured Data

Relational databases are the de facto standard for storing and querying structured data, and extracting insights from structured data requires advanced analytics. Deep neural networks (DNNs) have achieved super-human prediction performance in particular data types, e.g., images. However, existing DNNs may not produce meaningful results when applied to structured data. The reason is that there are correlations and dependencies across combinations of attribute values in a table, and these do not follow a simple geometric pattern that can be mimicked by a DNN. The number of possible such ``cross features'' is combinatorial, making them computationally prohibitive to model. Furthermore, the deployment of learning models in real-world applications has highlighted the need for interpretability, especially for high-stakes applications, which remains a major drawback for many DNNs.In this paper, we present ARM-Net, an adaptive relation modeling network tailored for structured data, and a lightweight framework ARMOR based on ARM-Net for relational data analytics, which is designed to be accurate, efficient and interpretable. The key idea is to model feature interactions with cross features selectively and dynamically, by first transforming the input features into exponential space, and then determining interaction weights and the interaction order adaptively for each cross feature. We propose a novel sparse attention mechanism to dynamically generate the interaction weights given the input tuple, so that we can model cross features of arbitrary orders and selectively filter noisy features. Then during model inference, ARM-Net can identify the most informative cross features in an input-aware manner for more accurate prediction and better interpretability. Our extensive experiments on real-world datasets show that ARM-Net consistently outperforms existing models and provides interpretable predictions for data-driven decision making.