Disney researchers have developed an artificial intelligence system that seemingly makes it far easier to make an actor appear younger or older in a scene. While artists will still be able to make manual adjustments to make sure the effect looks as realistic as possible, the AI tool could take care of most of the heavy lifting. It’s said to take the AI just five seconds to apply the aging effects to a single frame.
Re-aging an actor is typically an expensive and laborious process that requires artists to go through a scene frame-by-frame to manually change the character’s appearance. Attempts have been made in the past to automate the process with neural networks and machine learning. Disney’s researchers note that, while they might work well for still images, other systems “typically suffer from facial identity loss, poor resolution and unstable results across subsequent video frames.” They claim their solution offers “the first practical, fully-automatic and production-ready method for re-aging faces in video images.”
The team wrote in a paper that it would be impossible to train the FRAN (face re-aging network) neural network on a dataset of real people. That would require pairs of images showing a subject with the same facial expression, pose, lighting and background at two different and known ages. Instead, the researchers created a database of several thousand randomly generated faces. They re-aged those synthetic faces using existing machine learning aging tools, then fed the results of that process into FRAN.
The neural network can analyze a headshot and predict which parts of a face would be affected by aging and then it applies effects like wrinkles or skin smoothing as a layer on top of the original face. As Gizmodo notes, the researchers claim this approach allows FRAN to re-age the performer with their identity and appearance intact, even when their head or face is moving around or the lighting changes in a shot. Unlike with other methods, FRAN doesn’t require an extra face alignment step either.
There are many good reasons why Disney would want to develop such a tool. It could lessen visual effects artists’ workloads and speed up the process, for one thing. In addition to helping productions without blockbuster budgets age their actors up or down, it might help keep ballooning budgets in check. A significant chunk of The Irishman‘s budget, which reports suggest was as much as $200 million, went toward making Robert De Niro, Al Pacino and Joe Pesci look younger.
Disney has been de-aging performers in its own projects, notably with Mark Hamill in Star Wars. When Harrison Ford returns as Indiana Jones next summer, he’ll also look a little younger than you’re used to seeing him — at least for the opening sequence. Disney’s new re-aging tool should make it faster for effects artists to take years off such performers’ faces in the future.