 |
Scene Completion using millions of images |
 |
 |
|
|
|
|
| Results |
|
| Results with Tiny LAB Descriptor & 1.5 million images in database |
|
| LAB - Set 1 - Lake |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
| LAB - Set 2 - Street |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
| LAB - Set 3 - Forrest |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
| Results with Gist+LAB at approx 300k images in database |
|
| Gist - Set 1 - Forrest |
| Query/Mask |
|
|
| Results |
 |
 |
 |
|
| Gist - Set 2 - Skyline |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
| Gist - Set 3 - Rainbow |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
| Gist - Set 4 - Palace |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
| Gist - Set 5 - Alley |
| Query/Mask |
 |
 |
| Results |
 |
 |
 |
|
|
| Average results and closing thoughts |
|
|
|
|
The above images are more or less randomly picked examples for our average output images. In most cases, the gist descriptor
is able to find images with a matching scene in the database, yet the images do not seamlessly integrate with the query image.
On the one hand, this indicates that our database was too small to find better matches, on the other hand there are different cases
where we cross the limits of the algorithm. In our implementation, the software did not necessarily find the best placement for an
overlying image as we bounded the ssd to a maximum distance of 80 pixels to decrease computation time. This did e.g. lead to cases where
two skyline pictures where matched but the skylines did not overlay correctly.
The descriptor itself takes account of the whole query image, especially including the masked area, when we actually are only interested
in the area around the mask to find a part of any image with content matching the bounding area. In fact we achieved nice results with scenes that
did not exactly match the source, e.g. the Palace shown above generated results where the street was replaced by water which look much more natural
than our results with actual streets. The current behavior becomes more problematic
as the masking area size gets bigger. Think of the example with the street and the car the user wants to replace, discussed in the
Motivation. In a database with unlimited images we would in fact find another car to replace our car with. Of
course such a result is extremly unlikely but time will tell if we will be able to find a better description for the area to be replaced and to
match our scene.
|
|
|
|
|
