How Many Words Is a Picture Worth? Automatic Caption Generation for News Images

Abstract

In this paper we tackle the problem of automatic caption generation for news images. Our approach leverages the vast resource of pictures available on the web and the fact that many of them are captioned. Inspired by recent work in summarization, we propose extractive and abstractive caption generation models. They both operate over the output of a probabilistic image annotation model that pre-processes the pictures and suggests key-words to describe their content. Experimental results show that an abstractive model defined over phrases is superior to extractive methods