Abstract
Uplift modeling aims to estimate Conditional Average Treatment Effects (CATE) for a given factor, such as a marketing intervention or a medical treatment. Given covariates of a single subject under different treatment indicators, most of existing approaches, especially those based on deep learning, either learn the same representation from a single model or learn different representations from two separate models. Thus, these methods could not learn discriminative representations or could not utilize both information of control and treatment group. In this paper, we develop an attentive neural uplift model to alleviate the above shortcomings by utilizing attention mechanisms to map the original covariate space \(\mathcal {X}\) into a latent space \(\mathcal {Z}\) in a single model. Given covariates of a subject, the learned representations in space \(\mathcal {Z}\) which we called after-treatment representation are discriminative under different treatment indicators, thus can model potential outcomes more effectively. Moreover, the model is trained on a single neural network so that the information shared by treatment and control group is utilised. Experiments on synthetic and real-world datasets show our proposed method is competitive with the state-of-the-art.
G. Xu and C. Yin—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Athey, S., Imbens, G.W.: Machine learning methods for estimating heterogeneous causal effects. Stat 1050, 1–26 (2015)
Betlei, A., Diemert, E., Amini, M.R.: Uplift modeling with generalization guarantees. In: Proceedings of KDD (2021)
Cai, T., Tian, L., Wong, P.H., Wei, L.: Analysis of randomized comparative clinical trial data for personalized treatment selections. Biostatistics 12, 270–282 (2011)
Chen, H., Harinen, T., Lee, J.Y., Yung, M., Zhao, Z.: Causalml: Python package for causal machine learning. arXiv preprint arXiv:2002.11631 (2020)
Cho, K., Courville, A., Bengio, Y.: Describing multimedia content using attention-based encoder-decoder networks. IEEE Trans. Multimedia 17, 1875–1886 (2015)
Devriendt, F., Moldovan, D., Verbeke, W.: A literature survey and experimental evaluation of the state-of-the-art in uplift modeling: a stepping stone toward the development of prescriptive analytics. Big Data 6, 13–41 (2018)
Devriendt, F., Van Belle, J., Guns, T., Verbeke, W.: Learning to rank for uplift modeling. IEEE Trans. Knowl. Data Eng. (2020)
Diemert, E., Betlei, A., Renaudin, C., Amini, M.R.: A large scale benchmark for uplift modeling. In: Proceedings of the KDD (2018)
D’Amour, A., Ding, P., Feller, A., Lei, L., Sekhon, J.: Overlap in observational studies with high-dimensional covariates. J. Econom. 221, 644–654 (2021)
Galassi, A., Lippi, M., Torroni, P.: Attention in natural language processing. IEEE Trans. Neural Netw. Learn. Syst. 32, 4291–4308 (2020)
Grimmer, J., Messing, S., Westwood, S.J.: Estimating heterogeneous treatment effects and the effects of heterogeneous treatments with ensemble methods. Polit. Anal. 25, 413–434 (2017)
Gubela, R., Bequé, A., Lessmann, S., Gebert, F.: Conversion uplift in e-commerce: a systematic benchmark of modeling strategies. Int. J. Inf. Technol. Decis. Mak. 18, 747–791 (2019)
Holland, P.W.: Statistics and causal inference. J. Am. Stat. Assoc. 81, 945–960 (1986)
Jaskowski, M., Jaroszewicz, S.: Uplift modeling for clinical trial data. In: ICML Workshop on Clinical Data Analysis (2012)
Künzel, S.R., Sekhon, J.S., Bickel, P.J., Yu, B.: Metalearners for estimating heterogeneous treatment effects using machine learning. In: Proceedings of the National Academy of Sciences (2019)
Lo, V.S.: The true lift model: a novel data mining approach to response modeling in database marketing. ACM SIGKDD Explor. Newsl 4, 78–86 (2002)
Louizos, C., Shalit, U., Mooij, J., Sontag, D., Zemel, R., Welling, M.: Causal effect inference with deep latent-variable models. arXiv preprint arXiv:1705.08821 (2017)
Nie, X., Wager, S.: Quasi-oracle estimation of heterogeneous treatment effects. Biometrika 108, 299–319 (2021)
Pearl, J.: Causal inference in statistics: an overview. Stat. Surv. 3, 96–146 (2009)
Radcliffe, N.J.: Hillstrom’s MineThatData Email Analytics Challenge: An Approach Using Uplift Modelling. Stochastic Solutions Limited, Edinburgh (2008)
Radcliffe, N.J., Surry, P.D.: Real-world uplift modelling with significance-based uplift trees. White Paper TR-2011-1, Stochastic Solutions (2011)
Rubin, D.B.: Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66, 688 (1974)
Rubin, D.B.: Causal inference using potential outcomes: design, modeling, decisions. J. Am. Stat. Assoc. 100, 322–331 (2005)
Shi, C., Blei, D.M., Veitch, V.: Adapting neural networks for the estimation of treatment effects. arXiv preprint arXiv:1906.02120 (2019)
Splawa-Neyman, J., Dabrowska, D.M., Speed, T.: On the application of probability theory to agricultural experiments. Essay on principles. section 9. Statistical Science (1990)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the NeurIPS (2017)
Wager, S., Athey, S.: Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. 113, 1228–1242 (2018)
Wang, F., Tax, D.M.: Survey on the attention based RNN model and its applications in computer vision. arXiv preprint arXiv:1601.06823 (2016)
Zhang, W., Li, J., Liu, L.: A unified survey of treatment effect heterogeneity modelling and uplift modelling. ACM Comput. Surv. (CSUR) 54, 1–36 (2021)
Zhao, Y., Fang, X., Simchi-Levi, D.: A practically competitive and provably consistent algorithm for uplift modeling. In: Proceedings of the ICDM (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, G. et al. (2022). Learning Discriminative Representation Base on Attention for Uplift. In: Gama, J., Li, T., Yu, Y., Chen, E., Zheng, Y., Teng, F. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2022. Lecture Notes in Computer Science(), vol 13282. Springer, Cham. https://doi.org/10.1007/978-3-031-05981-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-05981-0_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05980-3
Online ISBN: 978-3-031-05981-0
eBook Packages: Computer ScienceComputer Science (R0)