Speaker 1: While filming this scene from Disney's The Mandalorian, the actors could see their surroundings, but the surroundings weren't actually there. All of this is just LED screens, displaying backgrounds pre-made in a video game engine. Compare that with this fight scene from Avengers Endgame, where actors jumped around in a sea of green imagining how VFX artists would make this planet look once filming had ended. The Mandalorian is one of the first major productions to choose LED walls over green screens, and the benefits for the actors are just the tip of the iceberg. LED walls make the lighting better, filming smoother, and in certain cases cost a lot less than using green screens. But to understand why the team behind The Mandalorian chose these LED screens, we have to understand just how they work. Mandalorian showrunner Jon Favreau revolutionised virtual production while directing The Lion King and The Jungle Book. However, the process for these two remakes still relied heavily on blue screen and post-production work. For The Mandalorian, LED wall technology seemed like the next logical step, given the show's production budget and time frame. Now, you may be thinking, this isn't so new, I've seen something like this before. And you're right, kind of. The predecessor to what we see on The Mandalorian is a driving scene like this one from Doctor No. You've got the actor in the car, and behind them, a screen with footage of the road they've travelled. But the technology was limited. Say you want to move the camera angle during the scene, that projected footage can't move with the camera. But by using Unreal Engine, tech borrowed from the video game field, that problem is solved. Artists can create a photorealistic 3D background that moves strictly with the camera's field of view, known as the frustum. So if the camera swings around and changes angles, the background shifts in precisely the same way. This allows motion-tracked cameras to execute traditional cinematography techniques within the virtual set. Achieving cinematic movements like the parallax effect, where an object in the foreground moves at a different speed than the background, amplifies the illusion of filming at an actual location. Now that LED screens can move with the camera's eye, virtual sets can solve a bunch of green screen problems. The biggest one? Probably being lighting.

Speaker 2: We've all seen movies, TV shows, where the lighting on the day didn't necessarily match the post-production work that was added many, many months later.

Speaker 1: That's Richard Bluff, Industrial Light & Magic's VFX supervisor for The Mandalorian. He says lighting is one of the key benefits of working with virtual sets. The light coming from the LEDs provide realistic colours and reflections on the actors and props, something you simply can't achieve with green screen. They are also able to completely avoid the problem of spill. To better understand that, we talked to Kim Libreri at Epic Games, creators of Unreal Engine, which was used to build the virtual sets for The Mandalorian.

Speaker 3: The problem with a green screen is that you have to have a green screen behind you. And what that does is that instead of projecting beautiful lighting from what the environment would be behind you, it basically puts a lot of green light on you. And we call that spill. If you wrap an actor with a big 360 LED wall, you can light in a way that you would never be able to do on a green screen. So you can really make it feel like the characters are embedded in their environments.

Speaker 1: This green screen spill would have been an even bigger problem than usual for The Mandalorian, whose main character gets a shiny new suit of armour early on in the season. So realistic lighting from the LEDs makes the show look better, but virtual sets help with practical concerns too. Take this scene in the office of The Client, played by Werner Herzog. In a scene like this, surely they just built a simple set, right? Wrong. All of this is that same LED screen. According to Richard, only the desk, the floor and one or two columns were really there.

Speaker 2: Everything else was displayed on the screen, but this allowed us to limit the amount of stage space that we used in shooting season one of Mandalorian, which has understandably a big impact on time and budget.

Speaker 1: To be fair, the initial investment in a virtual set like this is huge. However, the creators claim that money saved from travelling to locations, building new sets and costly post-production can make the investment worth it. Along with saving time and money, the team developed a system that allowed artists to make changes and control the world on the day of shooting. Settings like exposure, colour, animation playback and fill lighting are available to the filmmakers at a moment's notice. If they want to move a mountain from one side of the virtual set to the other, they can, right there on set. Not only is it helpful for the actor, who no longer has to imagine where that mountain is, but it's a game changer for the director of photography.

Speaker 2: Because in the distance, if there is a mountain range that is a story point, then the DP is no longer guessing where those elements are. He can frame up to them.

Speaker 1: The virtual set constructed for season one was 75 feet in diameter, 21 feet high and also had a roof composed of LEDs. The cast and crew refer to this space as the volume.

Speaker 2: And we had doors that can kind of close in to almost give you full 360-degree coverage of LED screens minus the floor. It's driven by a small group of machines running Unreal and our StageCraft technologies.

Speaker 1: At the end of the day, the big question is, do virtual sets really mean the end for green screens? Well, even The Mandalorian, with its technological advances, used a version of a green screen, but via the LED wall. Another capability of virtual sets is to choose pinpointed areas of the wall and turn those green.

Speaker 2: Now, we now have the ability to limit the amount of green screen that's visible behind the character. So now it doesn't wrap around the entire stage. It's just a little piece, but also they gain the lighting from that environment, which gives us a more seamless result in post-production.

Speaker 1: Additionally, scenes involving explosives are best kept to traditional methods. This is to prevent LEDs from getting damaged. So will green screens survive this wave of tech innovation?

Speaker 2: Eventually, of course, we hope never to use green screen. But I still see that there will be a future for it in the short term, because there's likely always a need to remove people to add additional action behind them. But we are getting to the point where the amount of green screen that's being used is massively reduced, and the sky's the limit right now.