article

Active Learning with Multiple Localized Regression Models

Authors:

Meghana Deodhar,

Maytal Saar-Tsechansky,

Vineet KeshariAuthors Info & Claims

INFORMS Journal on Computing, Volume 29, Issue 3

Pages 503 - 522

Published: 01 August 2017 Publication History

Abstract

Oftentimes businesses face the challenge of requiring costly information to improve the accuracy of prediction tasks. One notable example is obtaining informative customer feedback e.g., customer-product ratings via costly incentives to improve the effectiveness of recommender systems. In this paper, we develop a novel active learning approach, which aims to intelligently select informative training instances to be labeled so as to maximally improve the prediction accuracy of a real-valued prediction model. We focus on large, heterogeneous, and dyadic data, and on localized modeling techniques, which have been shown to model such data particularly well, as compared to a single, "global" model. Importantly, dyadic data with covariates is pervasive in contemporary big data applications such as large-scale recommender systems and search advertising. A key benefit from incorporating dyadic information is their simple, meaningful representation of heterogeneous data, in contrast to alternative local modeling techniques that typically produce complex and incomprehensible predictive patterns. We develop a computationally efficient active learning policy specifically tailored to exploit multiple local prediction models to identify informative acquisitions. Existing active learning policies are often computationally prohibitive for the setting we explore, and our policy makes the application of active learning computationally feasible for this setting. We present comprehensive empirical evaluations that demonstrate the benefits of our approach and explore its performance in real world, challenging domains.

References

[1]

Abe N, Mamitsuka H (1998) Query learning strategies using boosting and bagging. Proc. ICML '98 (Morgan Kaufmann Publishers, San Francisco), 1-9.

[2]

Agarwal D, Merugu S (2007) Predictive discrete latent factor models for large scale dyadic data. Proc. 13th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (ACM, New York), 26-35.

[3]

Baumann T, Germond AJ (1993) Application of the Kohonen network to short-termload forecasting. Proc. ANNPS '93 (IEEE Computer Society, Washington, DC), 407-412.

[4]

Bilgic M, Bennett PN (2012) Active query selection for learning rankers. Proc. 35th Internat. ACM SIGIR Conf. Res. Development Inform. Retrieval (ACM, New York), 1033-1034.

[5]

Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and Regression Trees (Wadsworth, Belmont, CA).

[6]

Burbidge R, Rowland JJ, King RD (2007) Active learning for regression based on query by committee. Proc. Intelligent Data Engrg. Automated Learn. (IDEAL) (Springer, New York), 209-218.

[7]

Cohn D, Atlas L, Ladner R (1994) Improving generalization with active learning. Machine Learn. 15(2):201-221.

Digital Library

[8]

Cohn D, Ghahramani Z, Jordan M (1996) Active learning with statistical models. J. Artificial Intelligence Res. 4:129-145.

Digital Library

[9]

Deodhar M, Ghosh J (2008) Simultaneous co-segmentation and predictive modeling for large, temporal marketing data. Proc. Data Mining Design Marketing, ICDM 2008 Workshop (IEEE Computer Society, Washington, DC), 806-815.

[10]

Deodhar M, Ghosh J (2010) SCOAL: A framework for simultaneous co-clustering and learning from complex data. J. ACM Trans. Knowledge Discovery from Data 4(3):Article no. 10.

[11]

Djukanovic M, Babic B, Sobajic D, Pao Y (1993) Unsupervised/ supervised learning concept for 24-hour load forecasting. IEE Proc.-Generation, Transmission Distribution 140(4): 311-318.

[12]

Fedorov V (1972) Theory of Optimal Experiments (Academic Press, New York).

[13]

Fukumizu K (2000) Statistical active learning in multilayer perceptrons. IEEE Trans. Neural Networks 11(1):17-26.

Digital Library

[14]

Gill P, Murray W, Wright M (1981) Practical Optimization (Academic Press, London).

[15]

Hastie T, Tibshirani R, Friedman J (2001) The Elements of Statistical Learning (Springer, New York).

[16]

Huang Z (2007) Selectively acquiring ratings for product recommendation. Proc. 9th Internat. Conf. Electronic Commerce (ICEC '07) (ACM, New York), 379-388.

[17]

Kanamori T, Shimodaira H (2003) Active learning algorithm using the maximum weighted log-likelihood estimator. J. Statist. Planning Inference 116(1):149-162.

[18]

Kiefer J (1959) Optimum experimental designs. J. Roy. Statist. Soc. 21(2):272-304.

[19]

Kim B, Sullivan M (1998) The effect of parent brand experience on line extension trial and repeat purchase. Marketing Lett. 9(2): 181-193.

[20]

Kohavi R, Longbotham R, Sommerfield D, Henne1 R (2009) Controlled experiments on the web: Survey and practical guide. Data Mining Knowledge Discovery 18(1):140-181.

Digital Library

[21]

Koren Y (2008) Factorization meets the neighborhood: A multi-faceted collaborative filtering model. Proc. 14th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (ACM, New York), 426-434.

[22]

Lewis DD, Gale WA (1994) A sequential algorithm for training text classifiers. Croft BW, van Rijsbergen CJ, eds. Proc. SIGIR '94 (Springer, London), 3-12.

[23]

Liu TY (2011) Learning to Rank for Information Retrieval (Springer, New York).

[24]

Long B, Bian J, Chapelle O, Zhang Y, Inagaki Y, Chang Y (2015) Active learning for ranking through expected loss optimization. IEEE Trans. Knowledge Data Engrg. 27(5):1180-1191.

[25]

Melville P, Saar-Tsechansky M, Provost F, Mooney R (2005) An expected utility approach to active feature-value acquisition. Proc. ICDM '05 (IEEE Computer Society, New York).

[26]

Menon AK, Jian X, Kim J, Vaidya J, Ohno-Machado L (2013) Detecting inappropriate access to electronic health records using collaborative filtering. Machine Learn. 95(1):87-101.

Digital Library

[27]

Quinlan JR (1992) Learning with continuous classes. Proc. AI '92 (World Scientific, Singapore), 343-348.

[28]

RayChaudhuri T, Hamey LGC (1995) Minimisation of data collection by active learning. Proc. ICNN '95 (IEEE, Piscataway, NJ), 1338-1341.

[29]

Roy N, McCallum AK (2001) Toward optimal active learning through sampling estimation of error reduction. Proc. ICML '01 (Morgan Kaufmann Publishers, San Francisco), 441-448.

[30]

Rubens N, Sugiyama M (2007) Influence-based collaborative active learning. Proc. RecSys '07 (ACM, New York), 145-148.

[31]

Saar-Tsechansky M, Provost F (2004) Active sampling for class probability estimation and ranking. Machine Learn. 54(2): 153-178.

[32]

Saar-Tsechansky M, Melville P, Provost F (2009) Active feature-value acquisition. Management Sci. 55(4):664-684.

Digital Library

[33]

Seetharaman PB, Ainslie A, Chintagunta PK (1999) Investigating household state dependence effects across categories. J. Marketing Res. 36(4):488-500.

[34]

Settles B (2012) Active Learning (Synthesis Lectures on Artificial Intelligence and Machine Learning) (Morgan and Claypool Publishers, San Rafael, CA).

[35]

Seung HS, Opper M, Smopolinsky H (1992) Query by committee. Proc. COLT '92 (ACM, New York), 287-294.

[36]

Sugiyama M (2006) Active learning in approximately linear regression based on conditional expectation of generalization error. J. Machine Learn. Res. 7:141-166.

Digital Library

[37]

Sugiyama M, Nakajima S (2009) Pool-based active learning in approximate linear regression. Machine Learn. 75(3):249-274.

Digital Library

[38]

Sugiyama M, Rubens N (2008) Active learning with model selection in linear regression. Proc. SIAM Internat. Conf. Data Mining (SIAM, Philadelphia), 518-529.

[39]

Wang Y, Witten IH (1997) Inducing model trees for continuous classes. van Someren M, Widmer G, eds. Proc. ECML '97, Prague.

[40]

Wedel M, Steenkamp J (1991) A clusterwise regression method for simultaneous fuzzy market structuring and benefit segmentation. J. Marketing Res. 28(4):385-396.

[41]

Wiens D (2000) Robust weights and designs for biased regression models: Least squares and generalized m-estimation. J. Statist. Planning Inference 83(2):395-412.

[42]

Zhang C, Chen T (2002) An active learning framework for content-based information retrieval. IEEE Trans. Multimedia 4(2): 260-268.

Digital Library

[43]

Zheng Z, Padmanabhan B (2006) Selectively acquiring customer information: A new data acquisition problem and an active learning-based solution. Management Sci. 52(5): 697-712.

Digital Library

Cited By

Bi XYang MAdomavicius G(2024)Consumer Acquisition for Recommender SystemsInformation Systems Research10.1287/isre.2023.122935:1(339-362)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1287/isre.2023.1229
Unger MLi PSen STuzhilin A(2023)Don’t Need All Eggs in One Basket: Reconstructing Composite Embeddings of Customers from Individual-Domain EmbeddingsACM Transactions on Management Information Systems10.1145/357871014:2(1-30)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3578710
Lin STang SWang YWang D(2022)Toward Efficient Ensemble Learning with Structure ConstraintsINFORMS Journal on Computing10.1287/ijoc.2022.122434:6(3096-3116)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1287/ijoc.2022.1224

Index Terms

Active Learning with Multiple Localized Regression Models

Index terms have been assigned to the content through auto-classification.

Recommendations

Cost‐effective multi‐instance multilabel active learning
Abstract
Multi‐instance multi‐label (MIML) Active Learning (M2AL) aims to improve the learner while reducing the cost as much as possible by querying informative labels of complex bags composed of diverse instances. Existing M2AL solutions suffer high ...
Multiple-Instance Active Learning for Image Categorization
MMM '09: Proceedings of the 15th International Multimedia Modeling Conference on Advances in Multimedia Modeling

Both multiple-instance learning and active learning are widely employed in image categorization, but generally they are applied separately. This paper studies the integration of these two methods. Different from typical active learning approaches, the ...
Transfer active learning
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

Active learning traditionally assumes that labeled and unlabeled samples are subject to the same distributions and the goal of an active learner is to label the most informative unlabeled samples. In reality, situations may exist that we may not have ...

Comments

Information & Contributors

Information

Published In

cover image INFORMS Journal on Computing

INFORMS Journal on Computing Volume 29, Issue 3

Summer 2017

201 pages

ISSN:1526-5528

Issue’s Table of Contents

Publisher

INFORMS

Linthicum, MD, United States

Publication History

Published: 01 August 2017

Accepted: 20 June 2015

Received: 24 January 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bi XYang MAdomavicius G(2024)Consumer Acquisition for Recommender SystemsInformation Systems Research10.1287/isre.2023.122935:1(339-362)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1287/isre.2023.1229
Unger MLi PSen STuzhilin A(2023)Don’t Need All Eggs in One Basket: Reconstructing Composite Embeddings of Customers from Individual-Domain EmbeddingsACM Transactions on Management Information Systems10.1145/357871014:2(1-30)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3578710
Lin STang SWang YWang D(2022)Toward Efficient Ensemble Learning with Structure ConstraintsINFORMS Journal on Computing10.1287/ijoc.2022.122434:6(3096-3116)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1287/ijoc.2022.1224

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents