The Library
Deep attentive video summarization with distribution consistency learning
Tools
Ji, Zhong, Zhao, Yuxiao, Pang, Yanwei, Li, Xi and Han, Jungong (2021) Deep attentive video summarization with distribution consistency learning. IEEE Transactions on Neural Networks and Learning Systems, 32 (4). pp. 1765-1775. doi:10.1109/TNNLS.2020.2991083 ISSN 2162-237X.
|
PDF
WRAP-deep-attentive-video-consistency-learning-Han-2020.pdf - Accepted Version - Requires a PDF viewer. Download (3053Kb) | Preview |
Official URL: http://dx.doi.org/10.1109/TNNLS.2020.2991083
Abstract
This article studies supervised video summarization by formulating it into a sequence-to-sequence learning framework, in which the input and output are sequences of original video frames and their predicted importance scores, respectively. Two critical issues are addressed in this article: short-term contextual attention insufficiency and distribution inconsistency. The former lies in the insufficiency of capturing the short-term contextual attention information within the video sequence itself since the existing approaches focus a lot on the long-term encoder-decoder attention. The latter refers to the distributions of predicted importance score sequence and the ground-truth sequence is inconsistent, which may lead to a suboptimal solution. To better mitigate the first issue, we incorporate a self-attention mechanism in the encoder to highlight the important keyframes in a short-term context. The proposed approach alongside the encoder-decoder attention constitutes our deep attentive models for video summarization. For the second one, we propose a distribution consistency learning method by employing a simple yet effective regularization loss term, which seeks a consistent distribution for the two sequences. Our final approach is dubbed as Attentive and Distribution consistent video Summarization (ADSum). Extensive experiments on benchmark data sets demonstrate the superiority of the proposed ADSum approach against state-of-the-art approaches.
Item Type: | Journal Article | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software | ||||||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Engineering > Engineering | ||||||||||||
Library of Congress Subject Headings (LCSH): | Digital video, Video compression -- Standards, MPEG (Video coding standard), Supervised learning (Machine learning), Computational intelligence, Distribution (Probability theory) | ||||||||||||
Journal or Publication Title: | IEEE Transactions on Neural Networks and Learning Systems | ||||||||||||
Publisher: | IEEE | ||||||||||||
ISSN: | 2162-237X | ||||||||||||
Official Date: | April 2021 | ||||||||||||
Dates: |
|
||||||||||||
Volume: | 32 | ||||||||||||
Number: | 4 | ||||||||||||
Page Range: | pp. 1765-1775 | ||||||||||||
DOI: | 10.1109/TNNLS.2020.2991083 | ||||||||||||
Status: | Peer Reviewed | ||||||||||||
Publication Status: | Published | ||||||||||||
Reuse Statement (publisher, data, author rights): | © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | ||||||||||||
Access rights to Published version: | Restricted or Subscription Access | ||||||||||||
Date of first compliant deposit: | 21 May 2020 | ||||||||||||
Date of first compliant Open Access: | 21 May 2020 | ||||||||||||
RIOXX Funder/Project Grant: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year