What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
Abstract
Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr. In this paper, we re-examine this problem from a different angle, by looking deep into what defines a realistically-natural and topically-coherent story. To this end, we propose three assessment criteria: relevance, coherence and expressiveness, which we observe through empirical analysis could constitute a "high-quality" story to the human eye. Following this quality guideline, we propose a reinforcement learning framework, ReCo-RL, with reward functions designed to capture the essence of these quality criteria. Experiments on the Visual Storytelling Dataset (VIST) with both automatic and human evaluations demonstrate that our ReCo-RL model achieves better performance than state-of-the-art baselines on both traditional metrics and the proposed new criteria.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2019
- DOI:
- 10.48550/arXiv.1909.05316
- arXiv:
- arXiv:1909.05316
- Bibcode:
- 2019arXiv190905316H
- Keywords:
-
- Computer Science - Computation and Language
- E-Print:
- Accepted paper in Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI) 2020