| Abstract
|
:
|
In this thesis we address the problem of summarizing visual data in images and video sequences. We show how this problem can be solved by optimizing purely visual criteria, based on comparison of image or video patches. As the first step, we show how the well-studied mosaicing problem (summarizing a video using a single image) can be solved without motion estimation, by direct optimization of the coherence of the target mosaic w.r.t. the source video. Our method is very robust and naturally works for very dynamic scenes. Generalizing this approach, we introduce visual similarity measure which compares two images/videos of arbitrary sizes. This measure is bidirectional and measures distance between the collections of all patches of the two signals (images or videos) at multiple scales. We show that by direct optimization of this measure complex images and videos can be coherently summarized in significantly smaller space/space-time dimensions. In addition to its generality, our approach demonstrates ability to utilize visual redundancy which cannot be achieved by other known methods. We further show how this visual similarity measure and approach can be used for additional tasks, including synthesis of visual data, removal of undesired objects, image collages, automatic image cropping, and more.
|