research-article

Detecting statistical interactions with additive groves of trees

Authors:

Daria Sorokina,

Mirek Riedewald,

Daniel FinkAuthors Info & Claims

ICML '08: Proceedings of the 25th international conference on Machine learning

Pages 1000 - 1007

https://doi.org/10.1145/1390156.1390282

Published: 05 July 2008 Publication History

Abstract

Discovering additive structure is an important step towards understanding a complex multi-dimensional function because it allows the function to be expressed as the sum of lower-dimensional components. When variables interact, however, their effects are not additive and must be modeled and interpreted simultaneously. We present a new approach for the problem of interaction detection. Our method is based on comparing the performance of unrestricted and restricted prediction models, where restricted models are prevented from modeling an interaction in question. We show that an additive model-based regression ensemble, Additive Groves, can be restricted appropriately for use with this framework, and thus has the right properties for accurately detecting variable interactions.

References

[1]

Camacho, R. (1998). Inducing models of human control skills. Proc. ECML'98.

Digital Library

[2]

Caruana, R., Elhawary, M., Fink, D., Hochachka, W. M., Kelling, S., Munson, A., Riedewald, M., & Sorokina, D. (2006). Mining citizen science data to predict prevalence of wild bird species. KDD'06.

Digital Library

[3]

Christensen, R. (1996). Plane answers to complex questions, the theory of linear models. Springer.

Digital Library

[4]

Friedman, J. (2005). RuleFit with R. http://www-stat.stanford.edu/~jhf/R-RuleFit.html.

[5]

Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of Statistics, 29, 1189--1232.

[6]

Friedman, J. H., & Popescu, B. E. (2005). Predictive learning via rule ensembles (Technical Report). Stanford University.

[7]

Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. JMLR, 3.

Digital Library

[8]

Hooker, G. (2004). Discovering ANOVA structure in black box functions. Proc. ACM SIGKDD.

Digital Library

[9]

Hooker, G. (2007). Generalized functional ANOVA diagnostics for high dimensional functions of dependent variables. JCGS.

[10]

Jakulin, A., & Bratko, I. (2004). Testing the significance of attribute interactions. Proc. ICML'04.

Digital Library

[11]

Pace, R. K., & Barry, R. (1997). Sparse spatial autoregressions. Statistics and Probability Letters, 33.

[12]

Rasmussen, C. E., Neal, R. M., Hinton, G., van Camp, D., Revow, M., Ghahramani, Z., Kustra, R., & Tibshirani, R. (2003). Delve. University of Toronto. http://www.cs.toronto.edu/~delve.

[13]

Ruppert, D., Wand, M. P., & Carroll, R. J. (2003). Semiparametric regression. Cambridge.

[14]

Sorokina, D., Caruana, R., & Riedewald, M. (2007). Additive Groves of regression trees. Proc. ECML.

Digital Library

[15]

Torgo, L. (2007). Regression DataSets. www.liacc.up.pt/~ltorgo/Regression/DataSets.html.

Cited By

Minotto TRobert PHobæk Haff ISandve G(2024)Assessing the feasibility of statistical inference using synthetic antibody-antigen datasetsStatistical Applications in Genetics and Molecular Biology10.1515/sagmb-2023-002723:1Online publication date: 3-Apr-2024
https://doi.org/10.1515/sagmb-2023-0027
Liang PZadeh AMorency L(2024)Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open QuestionsACM Computing Surveys10.1145/365658056:10(1-42)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3656580
Xu LHan ZWang ZZhao D(2024)Finding Component Relationships: A Deep-Learning-Based Anomaly Detection InterpreterIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.336043511:3(4149-4162)Online publication date: Jun-2024
https://doi.org/10.1109/TCSS.2024.3360435
Show More Cited By

Index Terms

Detecting statistical interactions with additive groves of trees

Recommendations

Modeling additive structure and detecting interactions with groves of trees
The expected additive weight of trees

We consider a general additive weight of random trees which depends on the structure of the subtrees, on weight functions defined on the number of internal and external nodes and on the degrees of the nodes appearing in the tree and its subtrees. ...
Cohen's weighted kappa with additive weights

Cohen's weighted kappa is a popular descriptive statistic for summarizing interrater agreement on an ordinal scale. An agreement table with $$n\in \mathbb N _{\ge 3}$$ ordered categories can be collapsed into $$n-1$$ distinct $$2\times 2$$ tables by combining adjacent categories. Weighted kappa with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '08: Proceedings of the 25th international conference on Machine learning

July 2008

1310 pages

ISBN:9781605582054

DOI:10.1145/1390156

General Chair:
William Cohen
Carnegie Mellon University
,
Program Chairs:
Andrew McCallum
University of Massachusetts Amherst
,
Sam Roweis
University of Toronto and Google

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Pascal
University of Helsinki
Xerox
Federation of Finnish Learned Societies
Google Inc.
NSF
Machine Learning Journal/Springer
Microsoft Research: Microsoft Research
Intel: Intel
Yahoo!
Helsinki Institute for Information Technology
IBM: IBM

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

ICML '08

Sponsor:

Microsoft Research
Intel
IBM

ICML '08: The 25th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

July 5 - 9, 2008

Helsinki, Finland

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
281
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Minotto TRobert PHobæk Haff ISandve G(2024)Assessing the feasibility of statistical inference using synthetic antibody-antigen datasetsStatistical Applications in Genetics and Molecular Biology10.1515/sagmb-2023-002723:1Online publication date: 3-Apr-2024
https://doi.org/10.1515/sagmb-2023-0027
Liang PZadeh AMorency L(2024)Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open QuestionsACM Computing Surveys10.1145/365658056:10(1-42)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3656580
Xu LHan ZWang ZZhao D(2024)Finding Component Relationships: A Deep-Learning-Based Anomaly Detection InterpreterIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.336043511:3(4149-4162)Online publication date: Jun-2024
https://doi.org/10.1109/TCSS.2024.3360435
Liu QFeng Y(2024)Machine collaborationStat10.1002/sta4.66113:1Online publication date: 29-Feb-2024
https://doi.org/10.1002/sta4.661
Torop MMasoomi AHill DKose KIoannidis SDy JOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)SmoothHessProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668328(50697-50729)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668328
Liang PCheng YFan XLing CNie SChen RDeng ZAllen NAuerbach RMahmood FSalakhutdinov RMorency LOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Quantifying & modeling multimodal interactionsProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667314(27351-27393)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667314
Li MZhang QKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Does a neural network really encode symbolic concepts?Proceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619251(20452-20469)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619251
Chen DYe WKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)How to address monotonicity for model risk management?Proceedings of the 40th International Conference on Machine Learning10.5555/3618408.3618616(5282-5295)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3618616
Yao ZZhang XNie PLv HYang YZou WYang L(2023)Identification of Milk Adulteration in Camel Milk Using FT-Mid-Infrared Spectroscopy and Machine Learning ModelsFoods10.3390/foods1224451712:24(4517)Online publication date: 18-Dec-2023
https://doi.org/10.3390/foods12244517
Liang PCheng YSalakhutdinov RMorency L(2023)Multimodal Fusion Interactions: A Study of Human and Automatic QuantificationProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614151(425-435)Online publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1145/3577190.3614151
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents