Deep-fake dubs will mess with your head and finances
Deep-fake dubs can modify or change an
actor’s original film and TV performances.
A new start-up is helping TV shows and films effortlessly reach new markets around the world. Customers feed the company’s software with video from a film or TV show along with dubbed dialogue recorded by humans. Machine learning models then create new lip movements that match the translated speech and pastes them onto the actor’s head. The results aren’t 100% flawless (yet), but they are pretty good. What makes this technology particularly interesting is its potential to scale. These deep-fake dubs offer tremendous value for money: they’re cheap and quick to create, especially when compared to the cost of full remakes. And, with the advent of global streaming platforms like Netflix, Disney Plus, and Amazon Prime Video, it’s easier than ever for such legitimate and illegitimate dubbs to reach international markets.
The Wall Street Journal recently highlighted, reported that the demand for streaming services in the US is saturated + companies are looking abroad for future growth. In the first quarter of 2021, for example, 89 percent of new Netflix users came from outside the US and Canada. To create shows with domestic and international viewer appeal you can now roll out versions with local V/O that seem more authentic because the deep-fake dubs preserve, to some degree, the performance of the original actors. The dubs are more sensitive to the facial expressions of the performers, retaining their emotion and line delivery.
My concerns are these.
Like all other technology that makes something faster or cheaper to make, or replicate, this deep-fake dub technology will quickly get better and cheaper.
The machine learning aspect scares me because the goal of that learning is to make the deep-fake invisible (to the human eye and brain).
Once that happens this technology will be embraced and used by unscrupulous people to coerce, confuse and steal from groups and individuals. What is now being done with robo-calls will be done on video platforms like zoom™.