[PDF][PDF] Natural language processing (almost) from scratch

R Collobert, J Weston, L Bottou, M Karlen… - Journal of machine …, 2011 - jmlr.org
Journal of machine learning research, 2011jmlr.org
We propose a unified neural network architecture and learning algorithm that can be applied
to various natural language processing tasks including part-of-speech tagging, chunking,
named entity recognition, and semantic role labeling. This versatility is achieved by trying to
avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead
of exploiting man-made input features carefully optimized for each task, our system learns
internal representations on the basis of vast amounts of mostly unlabeled training data. This …
Abstract
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements.
jmlr.org