[PDF][PDF] Hierarchical attention networks for document classification

Z Yang, D Yang, C Dyer, X He, A Smola… - Proceedings of the …, 2016 - aclanthology.org
Proceedings of the 2016 conference of the North American chapter of …, 2016aclanthology.org
We propose a hierarchical attention network for document classification. Our model has two
distinctive characteristics:(i) it has a hierarchical structure that mirrors the hierarchical
structure of documents;(ii) it has two levels of attention mechanisms applied at the wordand
sentence-level, enabling it to attend differentially to more and less important content when
constructing the document representation. Experiments conducted on six large scale text
classification tasks demonstrate that the proposed architecture outperform previous methods …
Abstract
We propose a hierarchical attention network for document classification. Our model has two distinctive characteristics:(i) it has a hierarchical structure that mirrors the hierarchical structure of documents;(ii) it has two levels of attention mechanisms applied at the wordand sentence-level, enabling it to attend differentially to more and less important content when constructing the document representation. Experiments conducted on six large scale text classification tasks demonstrate that the proposed architecture outperform previous methods by a substantial margin. Visualization of the attention layers illustrates that the model selects qualitatively informative words and sentences.
aclanthology.org