Abstract
We study the problem of learning from positive and unlabeled examples. Although several techniques exist for dealing with this problem, they all assume that positive examples in the positive set P and the positive examples in the unlabeled set U are generated from the same distribution. This assumption may be violated in practice. For example, one wants to collect all printer pages from the Web. One can use the printer pages from one site as the set P of positive pages and use product pages from another site as U. One wants to classify the pages in U into printer pages and non-printer pages. Although printer pages from the two sites have many similarities, they can also be quite different because different sites often present similar products in different styles and have different focuses. In such cases, existing methods perform poorly. This paper proposes a novel technique A-EM to deal with the problem. Experiment results with product page classification demonstrate the effectiveness of the proposed technique.
Chapter PDF
Similar content being viewed by others
References
Basu, S., Banerjee, A., Mooney, R.: Semi-supervised clustering by seeding. In: ICML 2002 (2002)
Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT 1998 (1998)
Bockhorst, J., Craven, M.: Exploiting relations among concepts to acquire weakly labeled training data. In: ICML 2002 (2002)
Crammer, K., Chechik, G.: A needle in a haystack: local one-class optimization. In: ICML 2004 (2004)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society (1977)
Denis, F.: PAC learning from positive statistical queries. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 112–126. Springer, Heidelberg (1998)
Goldman, S., Zhou, Y.: Enhancing supervised learning with unlabeled data. In: ICML 2000 (2000)
Koppel, M., Schler, J.: Authorship Verification as a one-class classification problem. In: ICML 2004 (2004)
Lee, W., Liu, B.: Learning with positive and unlabeled examples using weighted logistic regression. In: ICML 2003 (2003)
Lewis, D., Gale, W.: A sequential algorithm for training text classifiers. In: SIGIR 1994 (1994)
Li, X., Liu, B.: Learning to classify text using positive and unlabeled data. In: IJCAI 2003 (2003)
Liu, B., Lee, W., Yu, P., Li, X.: Partially supervised classification of text documents. In: ICML 2002 (2002)
Liu, B., Dai, Y., Li, X., Lee, W., Yu, P.: Building text classifiers using positive and unlabeled examples. In: ICDM 2003 (2003)
McCallum, A.: Multi-label text classification with a mixture model trained by EM. In: AAAI 1999 Workshop on Text Learning (1999)
Muggleton, S.: Learning from the positive data. Machine Learning (2001)
Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM. Machine Learning (2000)
Rocchio, J.: Relevant feedback in information retrieval. In: Salton, G. (ed.) The smart retrieval system: experiments in auto-matic document processing, Englewood Cliffs (1971)
Sarawagi, S., Chakrabarti, S., Godbole, S.: Cross-training: learning probabilistic mappings between topics. In: KDD 2003 (2003)
Scholkopf, B., Platt, J., Shawe, J., Smola, A., Williamson, R.: 1999. Estimating the support of a high-dimensional distribution. Technical Report MSR-TR-99-87, Microsoft Research (1999)
Vapnik, V.: The nature of statistical learning theory (1995)
Wu, P., Dietterich, T.: Improving SVM accuracy by training on auxiliary data sources. In: ICML 2004 (2004)
Yang, Y., Liu, X.: A re-examination of text categorization methods. In: SIGIR 1999 (1999)
Yu, H., Han, J., Chang, K.: PEBL: Positive example based learning for Web page classification using SVM. In: KDD 2002 (2002)
Yu, H.: General MC: Estimating boundary of positive class from small positive data. In: ICDM 2003 (2003)
Zelikovitz, S., Hirsh, H.: Improving short text classification using unlabeled background knowledge to assess document similarity. In: ICML 2000 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, XL., Liu, B. (2005). Learning from Positive and Unlabeled Examples with Different Data Distributions. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_24
Download citation
DOI: https://doi.org/10.1007/11564096_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)