Technolgy Rules: A Netflix hack lets you feel the action in a scene by vibrating your phone

Thursday, June 20, 2019

A Netflix hack lets you feel the action in a scene by vibrating your phone

Netflix Hack Day, the company’s internal hackathon, has a habit of producing some amazing gems — like a brain-controlled interface, a Fitbit hack that shuts off Netflix when you fall asleep, a Netflix app for the original NES and a way to navigate the Netflix app with Face ID and ARKit, to name a few. At this year’s Netflix Hack Day, employees ventured into areas like voice technology and haptics — the latter, so your phone could vibrate right along with the on-screen action, among other things.

Project Rumble Pack, as the hack that used haptics was called, takes inspiration from mobile gaming. Some games vibrate, which allows players to feel the action — like a bouncing ball, a car on a race track, an object getting hit or destroyed and so on.

Similarly, Project Rumble lets you feel the action in a scene from a show or movie — like a fight, battle or big explosion. (Imagine a Michael Bay movie with Rumble Pack turned on!) The team behind the hack, Hans van de Bruggen and Ed Barker, demoed haptics in an episode of Voltron where a huge explosion makes the phone shake in your hands.

The hack was created by syncing Netflix content with haptic effects using Immersion Corporation technology.

Another hack, called The Voice of Netflix, taught Netflix to speak using the voice of Netflix’s favorite characters. The team trained a neural net to find words in Netflix’s content, which could then be used to create new sentences on demand.

A third favorite was TerraVision — a practical hack that sounds like a business opportunity.

The hack lets filmmakers drop into an interface a photo of a look they like for a film location, then get back the closest results from a library of location photos. The hack used a computer vision model trained to recognize places for its reverse-image search functionality.

The final highlight was a silly hack that plays “walk-out music” — like the music that kicks in when Oscar speeches go too long — when someone overstays their allotted time in a booked conference room.

Sadly, many of Netflix’s hacks don’t tend to escape the confines of the hackathon itself. But they can inspire real-world projects in other ways, and help to keep the creativity flowing.

An overview of this year’s Netflix Hack Day, which focused on Netflix’s studio efforts, is below.