- Visual Analysis for Recurent Neural Networks

Recurrent neural networks, and in particular long short-term memory networks (LSTMs), are a remarkably effective tool for sequence processing that learn a dense black-box hidden representation of their sequential input. Researchers interested in better understanding these models have studied the changes in hidden state representations over time and noticed some interpretable patterns but also significant noise.

We present LSTMVis a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics. The tool allows a user to select a hypothesis input range to focus on local state changes, to match these states changes to similar patterns in a large data set, and to align these results with structural annotations from their domain. We provide data for the tool to analyze specific hidden state properties on dataset containing nesting, phrase structure, and chord progressions, and demonstrate how the tool can be used to isolate patterns for further statistical analysis.

@ARTICLE {8017583,
author = {H. Strobelt and S. Gehrmann and H. Pfister and A. M. Rush},
journal = {IEEE Transactions on Visualization & Computer Graphics},
title = {LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks},
year = {2018},
volume = {24},
number = {01},
issn = {1941-0506},
pages = {667-676},
keywords = {tools;recurrent neural networks;visualization;pattern matching;computational modeling;data models},
doi = {10.1109/TVCG.2017.2744158},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
month = {jan}
}