Multitask Learning is an approach to inductive transfer that improves learning for one task by using the information contained in the training signals of other related tasks. It does this by learning tasks in parallel while using a shared representation; what is learned for each task can help other tasks be learned better. In this thesis we demonstrate multitask learning for a dozen problems. We explain how multitask learning works and show that there are many opportunities for multitask learning in real domains. We show that in some cases features that would normally be used as inputs work better if used as multitask outputs instead. We present suggestions for how to get the most out of multitask learning in artificial neural nets, present an algorithm for multitask learning with case-based methods like k-nearest neighbor and kernel regression, and sketch an algorithm for multitask learning in decision trees. Multitask learning improves generalization performance, can be applied in many different kinds of domains, and can be used with different learning algorithms. We conjecture there will be many opportunities for its use on real-world problems.
Cited By
- Segev N, Harel M, Mannor S, Crammer K and El-Yaniv R (2017). Learn on Source, Refine on Target: A Model Transfer Learning Framework with Random Forests, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39:9, (1811-1824), Online publication date: 1-Sep-2017.
- Bahrampour S, Nasrabadi N, Ray A and Jenkins W (2016). Multimodal Task-Driven Dictionary Learning for Image Classification, IEEE Transactions on Image Processing, 25:1, (24-38), Online publication date: 1-Jan-2016.
- Wang X, Zheng W, Li X and Zhang J (2016). Cross-Scenario Transfer Person Reidentification, IEEE Transactions on Circuits and Systems for Video Technology, 26:8, (1447-1460), Online publication date: 1-Aug-2016.
- Bueno-Crespo A, SáNchez-GarcíA A and Sancho-GóMez J (2012). Improving learning by using artificial hints, Neurocomputing, 79, (18-25), Online publication date: 1-Mar-2012.
- Menke J and Martinez T (2009). Artificial neural network reduction through oracle learning, Intelligent Data Analysis, 13:1, (135-149), Online publication date: 1-Jan-2009.
- García-Laencina P, Serrano J, Figueiras-Vidal A and Sancho-Gómez J Multi-task Neural Networks for Dealing with Missing Inputs Proceedings of the 2nd international work-conference on The Interplay Between Natural and Artificial Computation, Part I: Bio-inspired Modeling of Cognitive Tasks, (282-291)
- Heckerman D, Kadie C and Listgarten J Leveraging information across HLA alleles/supertypes improves epitope prediction Proceedings of the 10th annual international conference on Research in Computational Molecular Biology, (296-308)
- Eaton E Multi-resolution learning for knowledge transfer proceedings of the 21st national conference on Artificial intelligence - Volume 2, (1908-1909)
- Caruana R (1997). Multitask Learning, Machine Language, 28:1, (41-75), Online publication date: 1-Jul-1997.
- Yu Q, Liu P, Wu Z, Ang S, Meng H and Cai L Learning cross-lingual information with multilingual BLSTM for speech synthesis of low-resource languages 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (5545-5549)
Recommendations
Multitask Learning
Special issue on inductive transferMultitask Learning is an approach to inductive transfer that improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias. It does this by learning tasks in parallel while using a shared ...
Learning Task Grouping using Supervised Task Space Partitioning in Lifelong Multitask Learning
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge ManagementLifelong multitask learning is a multitask learning framework in which a learning agent faces the tasks that need to be learnt in an online manner. Lifelong multitask learning framework may be applied to a variety of applications such as image ...
A Regularization Approach to Learning Task Relationships in Multitask Learning
Multitask learning is a learning paradigm that seeks to improve the generalization performance of a learning task with the help of some other related tasks. In this article, we propose a regularization approach to learning the relationships between ...