Technolgy Rules: Facebook animates photo-realistic avatars to mimic VR users’ faces

Wednesday, May 2, 2018

Facebook animates photo-realistic avatars to mimic VR users’ faces

Facebook wants you to look and move like you in VR, even if you’ve got a headset strapped to your face in the real world. That’s why it’s building a new technology that uses a photo to map someone’s face into VR, and sensors to detect facial expressions and movements to animate that avatar so it looks like you without an Oculus on your head.

CTO Mike Schroepfer previewed the technology during his day 2 keynote at Facebook’s F8 conference. Eventually, this technology could let you bring your real-world identity into VR so you’re recognizable by friends. That’s critical to VR’s potential to let us eradicate the barriers of distance and spend time in the same “room” with someone on the other side of the world. These social VR experiences will fall flat without emotion that’s obscured by headsets or left out of static avatars. But if Facebook can port your facial expressions alongside your mug, VR could elicit similar emotions to being with someone in person.

Facebook has been making steady progress on the avatar front over the years. What began as a generic blue face eventually got personalized features, skin tones and life-like features, and became a polished and evocative digital representation of a real person. Still, they’re not quite photo-realistic.

Facebook is inching closer, though, by using hand-labeled characteristics on portraits of people’s faces to train its artificial intelligence how to turn a photo into an accurate avatar.

Meanwhile, Facebook has tried to come up with new ways to translate emotion into avatars. Back in late 2016, Facebook showed off its “VR emoji gestures,” which let users shake their fists to turn their avatar’s face mad, or shrug their shoulders to adopt a confused expression.

Still, the biggest problem with Facebook’s avatars is that they’re trapped in its worlds of Oculus and social VR. In October, I called on Facebook to build a competitor to Snapchat’s wildly popular Bitmoji avatars, and we’re still waiting.

Facebook seriously needs its own Bitmoji

VR headsets haven’t seen the explosive user adoption some expected, in large part because they lack enough compelling experiences inside. There are zombie shooters and puzzle rooms and shipwrecks to explore, but most tire of them quickly. Games and media lose their novelty in a way social networking doesn’t. Imagine what you were playing or watching 14 years ago, yet we’re still using Facebook.

That’s why the company needs to nail emotion within VR. It’s the key to making the medium impactful and addictive.