From 2022 to 2023, the film industry underwent a small improvement towards sustainability.
The power of AI in the VFX industry
Over the past year, we have seen a huge amount of development in 2D image generation AI tools using platforms such as Stable Diffusion, Midjourney and Dall-e. These text-to-image AI tools turn text prompts into detailed imagery in a matter of seconds. As production companies look towards 2024, the film industry continues to undergo a transformative shift partly due to the integration of artificial intelligence (AI) and visual effects (VFX).
Tue 19th of Sep 2023
With its ability to learn from data and make predictions, AI is revolutionizing VFX, giving artists powerful tools to push the boundaries of creativity and efficiency. AI technology is advancing and is already being used in the visual effects industry.
While AI is not yet able to generate to create full 4K or 8K shots, it can be used for background replacements and rotoscoping, to name a few. Rotoscoping, such as removing stunt wires from a shot, can be a very time-consuming task which makes any VFX artist frown.
Autonomous rotoscoping is a game-changer
Weta Digital‘s use of AI in rotoscoping for “The Mandalorian“ is a perfect example of this technology’s potential. Their innovative AI-driven approach was able to handle the complex task of rotoscoping the entire film, effectively cutting down the time by an estimated 90%.
AI tools can also handle complex scenarios such as motion blur or overlapping objects, showing their versatility and adaptability. The development possibilities are virtually endless as well.
Realistic facial animation
Besides rotoscoping, creating realistic facial animations is one of the major points for the believability of a film. With AI companies can generate highly accurate and lifelike animations for digital characters by analyzing human facial movements and expressions.
For example, Disney used AI to create the facial animation of Sully in “Monsters University”. AI was able to analyze the movements and expressions of real-life actors and learn human facial expressions to make the character’s performance more realistic and emotionally engaging.
These are just a few examples from the VFX industry. As being said, AI has potential in various aspects from autonomous rotoscoping and realistic facial animation to crowd simulation, set extensions, matte painting, and many more.
All about AI in VFX
According to Scott Squires, a veteran visual effects supervisor who worked for Industrial, Light & Magic (ILM) for 20 years, much of the very time-consuming and tedious hand work VFX done in the past has been at least partially automated. He agrees about new technology enabling VFX artists to make an even higher quality level of work and providing more options for creativity.
Automating VFX tasks, film studios increase the cost-effectiveness and reduce the need for large teams leading to lower productions costs. And last but not least, it enhances creativity by reducing the time spent on repetitive tasks.
Simplified example of NST in action adding style to Ember’s flames. © Pixar
AI-driven films
Pixar released their latest animation film “Elemental” in 2023. The characters are more abstract than ever, and the prospect of making a character who is made of fire, water or air is no easy one. When hand sculpting and animating every detail is not an option, Pixar used “volumetric neural style transfer” (NST).
The creative team took the raw voxels of the “pyro simulation,” or generated flames, and passed it through a style transfer network trained on an artist’s expression of what they wanted the character’s flames to look like.
The resulting voxels have the natural, unpredictable look of a simulation but also the unmistakable cast of the artist’s choice. Pixar’s Paul Kanyuk points out that Pixar didn’t use AI to make Elemental. According to Kanyuk they used volumetric NST to shape her silhouette edges. Still, NST is a machine learning (ML) technique, which generally speaking, falls under the AI umbrella. AI tools were used also in DNEG’s “Nimona” and HBO’s “The Last of Us”.
The future of VFX
As AI technology continues to advance, we can expect AI’s role in visual effects to become even more significant. While we have already seen remarkable applications of AI, the journey has just begun. The future holds exciting possibilities, with AI set to revolutionize the VFX industry, leading to more immersive and visually stunning cinematic experiences. AI’s role in VFX is not about replacing artists though, but empowering them with superior tools.
Besides that, even rendering will become faster and more efficient in the near future due to AI. As technology continue to evolve and become increasingly integrated into the movie industry, we must acknowledge the incredible computational requirements of them.
Sources:
https://modvfx.com/blog/ai-tools-for-vfx
https://studios.disneyresearch.com/2022/11/30/efficient-neural-style-transfer-for-volumetric-simulations/
https://techcrunch.com/2023/08/25/vfx-artists-show-that-hollywood-can-use-ai-to-create-not-exploit/
RELATED
- All Post
- News
Finland's well-developed film industry infrastructure, including state-of-the-art studios and skilled professionals, makes it an attractive choice for productions of all…
Sustainable filming equipment are here to stay. Read more about how ARRI and Red are keeping their cameras on the…
An international big-budget film can still generate tons of waste which increases its carbon emissions.
The production incentive has played a major role in getting and keeping the productions in Finland. Read more here!
The international environmental sustainability project "albert" will also be introduced at Kajawood Studios in Sotkamo.