From theories to queries: Active learning in practice

B Settles - … learning and experimental design workshop in …, 2011 - proceedings.mlr.press
Active learning and experimental design workshop in conjunction …, 2011proceedings.mlr.press
This article surveys recent work in active learning aimed at making it more practical for real-
world use. In general, active learning systems aim to make machine learning more
economical, since they can participate in the acquisition of their own training data. An active
learner might iteratively select informative query instances to be labeled by an oracle, for
example. Work over the last two decades has shown that such approaches are effective at
maintaining accuracy while reducing training set size in many machine learning …
Abstract
This article surveys recent work in active learning aimed at making it more practical for real-world use. In general, active learning systems aim to make machine learning more economical, since they can participate in the acquisition of their own training data. An active learner might iteratively select informative query instances to be labeled by an oracle, for example. Work over the last two decades has shown that such approaches are effective at maintaining accuracy while reducing training set size in many machine learning applications. However, as we begin to deploy active learning in real ongoing learning systems and data annotation projects, we are encountering unexpected problems–due in part to practical realities that violate the basic assumptions of earlier foundational work. I review some of these issues, and discuss recent work being done to address the challenges.
proceedings.mlr.press