Discontinuous Seam-Carving for Video Retargeting
Venue
Computer Vision and Pattern Recognition (CVPR 2010)
Publication Year
2010
Authors
Matthias Grundmann, Vivek Kwatra, Mei Han, Irfan Essa
BibTeX
Abstract
We introduce a new algorithm for video retargeting that uses discontinuous
seam-carving in both space and time for resizing videos. We propose a novel
appearance-based temporal coherence formulation that allows for frame-by-frame
processing and results in temporally discontinuous seams, as opposed to
geometrically smooth and continuous seams. This formulation optimizes the
difference in appearance of the resultant retargeted frame to the optimal
temporally coherent one, and allows for carving around fast moving salient regions.
Additionally, we generalize the idea of appearance-based coherence to the spatial
domain by introducing piece-wise spatial seams. Our spatial coherence measure
minimizes the change in gradients during retargeting, which preserves spatial
detail better than minimization of color difference alone. We also show that
retargeting based on per-frame saliency (gradient-based or feature-based) does not
always lead to desirable results and propose a novel automatically computed measure
of spatio-temporal saliency. As needed, the user can also augment the saliency by
interactive region-brushing. Our retargeting algorithm processes the video
sequentially, which allows us to deal with streaming videos. We demonstrate results
over a wide range of video examples and evaluate the effectiveness of each
component of our algorithm.
