Web video search by mutual boosting between the inside and outside text of video

This paper proposes a new idea and approach for the web video search. Instead of only using the surrounding text of video in webpage, our approach boosts mutually and utilizes jointly the inside and outside text of video to support the video-based and frame-based search. In our approach, the inside text is the video caption, while the outside text is the surrounding text of video in webpage. In our view, the caption text, although has some wrong characters and words caused by the automatic caption recognition, is a useful indicator for the content of video. While the relevant surrounding texts of video, although is difficult to locate and confirm, have the correct characters and words which usually indicate the video content, especially the title and introduction of video. In this paper, we integrate their advantages and alleviate their disadvantages by the mutual boosting idea, that is, to employ the inside text to confirm the relevance of outside text, and to utilize the relevant outside text to correct the inside text. Mutual boosting not only enhances the query-by-text video search, but also further supports the query-by-text frame search with the corrected caption. Based on the above idea, our approach can be divided into three phases: Firstly, we proposed a new approach for automatic caption detection and extraction. Then, we extract the surrounding text candidates of video. Finally, the mutual boosting approach is employed to get the relevant and accurate text of web video. The experiments show the proposed approach can achieve good performance.