By: Fernando Gallo
In a perfect world, AI technology would make things easier for people without impinging on their creativity. However, many of the concerns of AI’s disruption across all industries are that it is indeed taking human creativity and making it into something soulless. Not to mention the ethical concerns of AI being trained on existing art without credit to the artists it is stealing from.
It’s a messy situation, and the visual effects industry hasn’t been left out. While the fear makes sense, the truth is that AI is still quite slow in VFX. For now, AI can generate images and videos like stock footage—good for references, ideation, and quick concepts, but not much else. AI, in its current form, cannot correctly address the needs of artists and directors on improvements and changes, often just giving random or nonsensical solutions. And the quality is lacking – the high-quality standards of VFX are beyond AI’s current capabilities. You’ve seen the memes about the hands that AI creates. It’s the calling card of AI in 2024 to create things that are close to reality, but once you start looking at the details, it falls apart.
The VFX industry is no stranger to big disruptions. In 1993, Steven Spielberg and his team forever changed the industry with their pioneering computer-generated imagery (CGI) effects that brought dinosaurs to life on the big screen.
While motion capture technology like rotoscoping began in the 1920s, it didn’t become mainstream until the 2000s, when films like the animated Polar Express, one of the first films made entirely with motion capture tech, and perfected by James Cameron in Avatar.
AI is set to change the industry once again…or is it? Each change had people wondering how it would affect the industry and jobs, but they turned out to be useful tools occupying specific niches, and people learned how to use them and specialize in them.
AI as a Creative Ally
The automation of repetitive tasks is the biggest benefit of AI in VFX. Tasks like rotoscoping, and object tracking can be done by AI, freeing up artists for more creative work. When animation changed from the traditional hand-drawing method to the 2D or 3D computer-generated, animators could achieve more in less time. With AI it’s reasonable to assume that AI will speed this up even further, making things faster, filling gaps between key frames better, leaving artists to focus on scripts and programming instead of repetitive labor tasks.
AI’s enhanced efficiency and productivity could have big implications for departments that deal with realism effects like lighting, physics, liquids, and final compositing. In VFX, we usually work with specific software for those elements that simulate and build everything in 3D. AI could take shortcuts to build details, crafting faster, lighter, and consuming less hardware.
Mundane or time consuming tasks, AI has got it in the bag. But are storytelling or more creative tasks like character design at risk of being outsourced? Many creators want the power and control over these main ideas, so outsourcing them doesn’t make sense for their workflows. However, AI is a powerhouse at brainstorming sessions, providing detail to stories and characters. It can also help diversify characters, models, or textures. For example, when creating a city in 3D, AI can take existing building types and details and generate hundreds or even thousands of different buildings and elements to make the city come alive without painstakingly creating every detail.
Some AI tools like AI-driven rotoscoping make it much quicker to isolate elements from footage, allowing for more focus on creative aspects. Using machine learning algorithms, pioneer software has created highly accurate and realistic facial expressions and body movements. However, humans still need to supervise and clean the results.
When combined with AI, tools like Houdini can procedurally generate intricate landscapes, urban settings, and natural phenomena, providing a robust foundation for VFX artists to build upon.
Real World Examples
You don’t need to look hard for a recent example boosted by AI. Netflix’s “Love, Death & Robots” series employed AI to enhance animation quality and streamline production workflows. AI-driven tools were used for procedural content generation, creating highly detailed and visually stunning environments that set the series apart.
Old-school filmmakers aren’t poo-pooing the tech, either. Martin Scorsese’s film The Irishman utilized AI-powered de-aging technology to make Robert DeNiro look younger. Machine learning algorithms analyzed and modified facial features, providing a seamless and believable transformation that traditional VFX techniques could not have achieved as effectively.
Major franchises aren’t immune, either. The virtual production techniques used in “The Mandalorian” have set a new standard in the industry. By combining real-time rendering with AI-driven tools, the production team created immersive and dynamic virtual environments that interacted with live-action elements in real-time. This enhanced visual quality meant greater flexibility and creativity in storytelling.
The Ethics of It All
This is the main question on everyone’s mind. Is AI on track to replace human artists, leading to job losses, industry disruption and the death of creativity?
Hold on. Let’s go back to a moment in history when people worried about something similar. When traditional animation was replaced by 2D and 3D software, alarm bells rang. Before we had computers and digital tools, animation was done by hand drawing or artisanal techniques like stop motion. Even inserting VFX was done manually by manipulating film frame by frame.
When things started moving to digital, animators were worried—but what ended up happening was they traded in their flipbooks for a computer mouse. Many of them learned the software, embraced new methods, and created more art faster. The volume of work actually increased, disrupting the industry to have more jobs, projects, and art.
That motion capture tech I talked about earlier? It gave animators the same worries. Actors had to learn how to get rigged up, wearing weird suits, sensors and marks all over their bodies and faces, but would animators and VFX departments no longer be needed? Quite the contrary. Motion capture results still need finessing, with artistic inputs like exaggeration, harmonization, silhouette adjustment and a lot of fixes. Digital characters usually do stuff that actors can’t, like big jumps or really fast movements, and animators still have to create those. Not to mention that sometimes the actors act as something that isn’t human – you’ve surely seen the behind-the-scenes clips of Benedict Cumberbatch doing his best dragon impression in The Hobbit. In the end, it simply became a new tool that artists had to learn to work with.
My best is that AI won’t replace VFX crews – but people who know AI will get more work.
Copyright, Legalities, and AI
Generative AI learns by looking at existing work – things created originally by a photographer or artist, in the case of VFX. It uses the material as a reference, rebuilding everything to the point that we often can’t identify the source.
Copyright, one of the primary legal protections for creative work, is being heavily scrutinized. Several high-profile lawsuits are being waged against companies that have pioneered and popularized generative AI.
Beyond copyright, integrating AI in the VFX industry necessitates the establishment of ethical guidelines and potentially government regulations to ensure responsible use. Industry-wide standards should be developed to address data privacy, intellectual property rights, and the potential for job displacement. These guidelines should promote transparency, accountability, and fairness, ensuring AI technologies are used ethically and sustainably. Government regulations may also be needed to oversee AI deployment, providing a framework that protects the workforce and the integrity of creative work in the VFX industry.
Bias and Representation
Because AI algorithms learn from what we’ve already created, biases present in the training data can lead to skewed or discriminatory outputs. It uses the whole internet as a source, and if you’ve been on it long enough, you’re aware of the dark side of the web. Echo chambers, racism, sexism, and homophobia abound, sometimes more openly than offline. For now, AI companies have implemented post-processing filters and even human intervention in the results to deal with this problem.
Moving Forward
The VFX industry is rapidly adopting AI technologies and will eventually automate tasks like rotoscoping, color grading, and motion tracking. AI-driven procedural generation will allow for the autonomous creation of complex environments, characters, and animations, especially in gaming. Beyond that, integrating AI with augmented and virtual reality will revolutionize the VFX industry by enabling new forms of interactive and immersive storytelling, blurring the line between the real and the virtual.
When AI is used for the right tasks, artists will have time and can pour their skills into creativity, focusing on the more nuanced and artistic elements of their work. This synergy between AI and human creativity can lead to unprecedented advancements in the quality and complexity of visual effects. VFX artists must stay ahead of technological trends and continuously learn to ensure their relevance and value in an AI-powered industry.
The unique human touch, creativity, and emotional understanding inherent in VFX artistry remain irreplaceable. Human artists bring a depth of emotion and intuition to VFX and storytelling that AI cannot replicate, at least not for now. The ability to convey subtle emotions, understand cultural nuances, and create compelling narratives is a distinctly human trait crucial to visual effects. While AI can enhance and streamline VFX production’s technical aspects, the artistry’s heart and soul will always rely on human creativity and insight.
About the author
Fernando Gallo is a renowned Lead 3D Animator based in Toronto, Canada, contributing significantly to the VFX industry through his work on acclaimed feature films and TV shows. Some of his recent and notable projects as a Lead Animator include “John Wick 4”, Del Toro’s “Cabinet of Curiosities”, “The Reacher”, and “From”. His animation expertise is also evident in productions such as “The Boys”, “Halo”, “FUBAR”, “Star Trek: Discovery”, “The Fall of The House of Usher”, and many others for Amazon, Netflix, Sony, and Paramount. In addition to his professional work, Fernando enjoys hand drawing, writing, photography, music, and character design. Visit his website here: fernandogallo3d.com
Published by: Martin De Juan
