The film industry continue to be imbued with more complex action, and invariably more complex visual effects, filmmakers are turning to new production techniques to imagine these scenes virtually even before they’ve been shot, and then to scout sets, interact live with CG assets and characters, and to shoot, revise and iterate virtual scenes on the fly.

As VFX have grown to be a greater part of movies and television today, there’s a growing divide between what the filmmakers can see through the camera on the live-action set, and what they have to imagine will be added digitally many months later. Virtual Production attempts to unite those two worlds in real-time using game engines like Unreal and Unity, combined with high-power graphics cards, camera tracking, as well as VR and AR, filmmakers are now able to create scenes across the physical and digital worlds, together.

A new type of filmmaking

The virtual production empowers the filmmakers to create shots based on what “feels” right, rather than looking at a computer and trying to guess where things should go when it’s not feeling right. And they can do it quickly without having to spend hours moving heavy equipment. 

The technology puts the tools of storytelling back in the hands of the filmmakers, rather than an army of technicians.This lets them explore ideas a lot faster with intuition and a stronger creative vision.

New worlds

Several films with hugely fantastical worlds and characters have taken advantage of virtual production techniques to get their images on the screen. 

Avatar represented a huge leap forward in virtual production (and there are several sequels on the way). Ready Player One relied on virtual production to bring its OASIS world to life. It’s in the OASIS where avatars of human characters interact. That virtual world was ‘filmed’ with motion captured actors using a game engine-powered real-time simul-cam set-up that allowed director Steven Spielberg to ‘find’ the right shots, since the sets the actors occupied had been pre-made in rough form digitally. He could also re-visit scenes afterwards, re-framing and re-taking scenes even after the performance was done.

Epic Games Unreal Engine software is one of the significant players in this virtual production environment. It allowed for example to not only film a scene with motion captured actors and a virtual camera, but also to introduce raytracing into the real-time rendered output. I personally find the result incredibly insane.

The two major game engine producers, Unreal and Unity, are certainly pushing virtual production. It’s their engines that are the backbone of many real-time rendering and virtual set production environments. It’s important to note for independent filmmakers that these engines are generally available for free, at least in terms of getting started.

Virtual Camera

The Virtual Camera enables a user to drive a Cine Camera plug-in like Arri or RED in Unreal Engine using an iPad in a virtual production environment. With ARKit or an optical motion capture system such as Vicon or Optitrack, the position and rotation of the iPad is broadcast wirelessly to the PC, with the PC sending video back to the iPad.

Camera settings such as Focal Length, Aperture, Focus Distance, and Stabilization can be adjusted using touch input. Additionally, the Virtual Camera can be used for taking high-res screenshots , setting waypoints, recording camera motion and other tasks related to virtual production.

The physical cinematography equipment on a film set is replicated in the virtual world, and tracked so they match exactly. This allows the director of photography to put their hands on the gear and react more naturally as the performances unfold.

The assets of the film is made with the game engine, instead of creating a movie set.

The history of Virtual Production:

Its earliest use and iterations can be traced to advancements and innovations in filmmaking technologies. Peter Jackson’s The Lord of the Rings: The Fellowship of the Ring (2001)used virtual reality (VR) goggles and virtual cameras to plan camera moves.

James Cameron took it a step farther with Avatar (2009), as seen in Image 1, to create a bioluminescent species and exotic environments with a motion capture stage and simulcam. Simulcam is a VP tool used to “superimpose virtual characters over the live-action in real-time and aid in framing and timing for the crew” (Kadner 2019).

Jon Favreau continues to lead the charge with ground-breaking film and television projects such as The Jungle Book, The Lion King (2019), and The Mandalorian (2019), by designing and altering photorealistic environments in real-time.

A short analysis of the game from the initial vision

Uncharted is an action-adventure game series developed by Naughty Dog and published by Sony Interactive Entertainment for PlayStation consoles. The main series of games follows Nathan Drake, a treasure hunter who travels across the world to uncover various historical mysteries.

Locations are usually unexplored remote places, rich in uncontaminated nature, if not by the presence of ancient temples and hidden treasures.

The main character is Nathan Drake as mentioned before, the kind of successful middle-aged man that conceals his reprehensible behavior behind good looks, buckets of charm, and moral gymnastics. Nathan is the contemplative man torn between the adventures of his past and the domesticity of his present.

The peculiarity and uniqueness of this game, in addition to the adventurous plots full of twists, is the graphics in general and the landscape choice that accompany the protagonist throughout all the saga. Starting from a very realistic vision of landscapes and characters, the game develops its own characteristics: a combination of cinematic design and theatricality, like a movie story, but in the experience of being caught up in an adventure so compelling that you feel, if only for a moment, that you’re there.

Naughty Dog’s concept artists, character artists, environment artists, modellers, UI artists, lighting artists, shading artists and technical artists have all posted their Uncharted work in ArtStation portfolios, here some examples:

In these concepts, which I personally think contain the essence of the game, they reflect the principles mentioned in the previous lesson: grouping, focal area, balance and rhythm.

Technical achievement mixes with impeccable art design to give a warmth feeling of a William Turner painting. Uncharted artists have learned a fine and unique touch that sets them apart. This unique touch allowed them to create landscapes and locations full of details, cinematic light that give life to a theatricality environment. This is why Uncharted doesn’t look like most movies; it looks better.

1. MINIATURE EFFECT

Different ways to create impossible worlds and locations in movies

A miniature effect, a special effect created for motion pictures and television programs using scale models. Miniature set designers design and build miniature props and sets of motion pictures. They build models used for visual effects that meet the look and requirements of the production.

While the use of computer generated imagery (CGI) has largely overtaken their use, there are still films being made that use stop motion animation with very elaborate miniaturized sets and fully articulated characters, especially for projects requiring physical interaction with fire, explosions or water, but the result It’s always a combination of both, model and vfx. Titanic (1997), Godzilla (1998), The Lord of the Rings trilogy (2001-3), Casino Royale (2006), Inception (2010), and Interstellar (2014) are examples of highly successful films that have utilized miniatures for a significant component of their visual effects work.

Will there be further development of the miniature effect or have we reached the end of their use?

2. WHAT IS RHYTHM?

Analyzing through edits and beat

Editing a video or a film is one of the most vital aspects of storytelling. The right rhythm and pace of the edit can determine the progression of the narrative, on the other hand it can also change how the audience receives the message of the film entirely. Weak editing is not only visually irritating, it can also drive away the attention of the spectator from the video itself.

But is there a specific rule to respect in order to create a rhythm?

3. AERIAL MOTION CONTROL TECHNIQUE

Aerial VFX w/ timelapse

Filmmaker Rufus Blackwell recently shared with the internet his latest project. His techniques combine motion control through planned waypoint missions, and then manipulating the footage in a visual effects software to mirror the ground in an almost “Inception”-like effect.

He has continued to experiment with techniques that involve aerial time-lapse, motion control, and visual effects to find new and unique uses for drones that can capture video.

“I started with basic aerial time-lapse. Then started working on aerial motion control techniques. Using preplanned waypoint missions it is possible to set the drone up like an motion control rig in the sky. You can repeat the same set of camera moves at a different times of day to create a beautiful day to night transition.”

Blackwell explained

Blackwell also need some compositing and animation skills to track, smooth, and simulate additional movements to create the unique motion you see in the video.

” Then on top of that there are various techniques that allow me to reposition the camera in post and create camera moves on top of the drones automated movement. Finally by stabilizing the source material and flipping the image with a soft matte you can create the mirrorworld effect, sometimes changing the vertical scale of the images to give a different perspective.”

Blackwell explained

The shooting techniques are many and different from each other, in this case we talk about hyperlapse of aerial shots. If we combine new types of film like these with the art of vfx we can create a completely different visual experience, and this work is the result.

4. WHAT IS CINEMATIC LIGHTING?

Borrowed from French cinéma, shortening of cinématographe (term coined by the Lumière brothers in the 1890s), from Ancient Greek κίνημα (kínēma, “movement”) + Ancient Greek -γράφειν (-gráphein, “write(record)“.

Cinematic lighting is a film lighting technique that goes beyond the standard three-point lighting setup to add drama, depth, and atmosphere to the story. Cinematic lighting utilizes lighting tricks like bouncing light, diffusing light, and adjusting color temperatures.

Lighting is a fundamental to film because it creates a visual mood, atmosphere, and sense of meaning for the audience. Whether it’s dressing a film set or blocking actors, every step of the cinematic process affects the lighting setup, and vice-versa.

• Tells the audience where to look.
• Reflects the psychology of characters.
• Defines and supports the genre of the film

Main roles for the Lighting Setup for a Scene?

Firstly the director, who shares visual inspirations and ideas for cinematic lighting, the director of photography or cinematographer creates then the lighting plan with input from the director. The gaffer designs and executes the cinematographer’s lighting plan and oversees the crew that brings the lighting plan to life.

With excellent light it is possible to obtain beautiful and powerful shots even if the location is not the best, but what are the rules to be respected to achieve a good level of cinematic light?

5. Why Retopology is used?

Retopology is the process of converting high-resolution models into something much smaller that can be used for animation. It can be a difficult process, but the basic idea is to create another mesh that simplifies the original HD asset.

In the past a 3D artist would painstakingly build a mesh polygon-by-polygon. The problem with this approach is that it’s very technical and difficult.

Sculpting is a more intuitive process that lends itself better to how artists think and work.

The downside is that you have to create a lot of polygons to sculpt the curves you need. And too much detail will slow down even the most powerful computer.

That’s where retopology comes in. By overlaying a low-polygon mesh onto a high-polygon mesh you can get something that’s perfect for animating and easy on your processor.