Webomator: Bradley W. Schenck's blog
Bradley W. Schenck's books Webomator Blog Topics Archives Retro Sci Fi
Search retro robot art
Subscribe RSS retro future Bradley W. Schenck at Facebook Bradley W. Schenck at Goodreads Bradley W. Schenck on Twitter Bradley W. Schenck at DeviantArt Bradley W. Schenck Also by Bradley W. Schenck I play games.
SIGGRAPH Paper on a new – automated – inpainting technique

Filed under Computer Graphics

digital image healingA paper from this year’s SIGGRAPH by James Hays and Alexei Efros (Carnegie Mellon) takes a new approach to digital image “healing” or hole filling. This is a problem faced in many situations, whether it’s restoring a damaged picture, eliminating flaws like a bug on the lens, or just replacing unwanted areas of an image – picture, for example, a plate for a motion picture backdrop in which the makers of a period film want to eliminate modern details.

What makes this technique unusual is that it doesn’t look for other areas in the image for texture and detail to insert. The software that the authors developed uses a database of 2.3 million unique photographs and – without human intervention – looks for semanitic matches for the area that they chose to replace.

No human tagging or hints were used. The program looked over the entire database for broad matches, then looked more carefully for the best fit. When finished (after about an hour on a single CPU, or five minutes on a cluster of fifteen), the authors were presented with several results to choose from. Their paper shows how completely different those results often were, which is one of the most interesting aspects of the algorithm. They also show us examples of bad matches, in which existing methods provided better results than theirs.

Overall the results are impressive, especially considering that the software has to figure out a lot about the images it’s “looking” at in order for this to work. The larger the database, the better it does.

image processing for hole filling

I think that there are some inherent problems with actually using this sort of technique. Their huge database was acquired from Flickr accounts and so there would obviously be copyright issues; also, because some of the solutions include people – even crowds of them – a commercial use could produce photographs of people without model releases, or even any way of knowing who the heck ended up in the picture. And because it’s hard to figure how else you could assemble a library of millions of source photographs, I think that’s a pretty large hurdle to jump. Still, very interesting.


Leave a Reply

Your email address will not be published. Required fields are marked *

The Webomator Blog is powered by WordPress.
Down in the Basement. Where it Strains Against its Chains and Turns a Gigantic Wheel of Pain, for all Eternity. Muahahahahaha.