This Algorithm Can Create 3D Animations From A Single Still Image

"Photo Wake-Up"

Chung-Yi Weng, PhD student at the University of Washington, and some of his friends created something truly astonishing.

Their software called "Photo Wake-Up" allows character animations to simply "walk out" of a static image frame — without leaving a hole in the picture behind them. The results were published in a recently submitted paper.

Weng's method identifies a 2D subject in a single photo as input, and creates a 3D animated version of that subject. The animation can then "walk out, run, sit, or jump in 3D."

And it could redefine the way we interact with photos. "We believe the method not only enables new ways for people to enjoy and interact with photos, but also suggests a pathway to reconstructing a virtual avatar from a single image," Weng, and his collaborators explain in the paper.

Like Magic

The effect, as seen in the video below, is amazing, albeit jarring. Basketball legend Stephen Curry can be seen jumping into action, jogging straight out of his photo frame. One of Picasso's surrealist creations cuts itself out of its frame perfectly, leaving the painting behind it in tact.

 

For the effect to work properly, all Weng's software needs is a still frame showing a silhouette. It cuts out the 2D shape, and warps it around a 3D skeleton that matches it.

A Big Improvement

Researchers have tried to create a similar effect in the past, but the results have been a lot less impressive. Weng's new approach adds an important new ability: it can identify different body parts like arms and legs, and warp each one individually in a way that matches the 2D cutout exactly.

The software even works in augmented reality, and could redefine the way we interact with two-dimensional pieces of art the next time we visit an art gallery.

READ MORE: Machine vision can create Harry Potter–style photos for muggles [MIT Technology Review]

More on 3D animation: Witness the Remarkable Evolution of Animation in Video Games