Solo Part 2: Technical On Set Immersive Environments

Featured article on

Article BY MIKE SEYMOUR ON JUNE 6, 2018
for FXGuide

Immersive environments are the practical use of lighting display technology in blending the lines between traditional VFX and in camera VFX. On Solo, the team used advanced projection technology to provide in camera effects and believable lighting.

For sequences such as the speeder chase and the infamous Kessel Run, the visual effects team had an immense challenge in creating the over 90 minutes of visual effects.

Rob Bredow, Visual Effects Supervisor, decided to deploy a version of old-school techniques combined with cutting-edge technology to film as much as possible on camera. This provided authentic contact lighting on the actors when they are either in the speeder chase on Corellia or piloting the Millennium Falcon. Bredow and the team at ILM worked in close collaboration with Cinematographer Bradford Young, the special effects team led by supervisor Dominic Tuohy, veteran creature effects supervisor Neal Scanlan and the team at Lux Machina to make the set as realistic as possible for filming.

Rob on set in the Falcon

Millennium Falcon

For the interior Millennium Falcon cockpit set, rear projection screens were utilized so that the actors could see and react to pre-designed animations of flying and entering hyperspace. Philip Galler and his team at Lux Machina, who had worked previously on Rogue One, joined the team to help develop the on-set giant projection system.

“When you see Han walk into the cockpit of the Millennium Falcon for the first time when they’re in space, the stars you see out the windows are actually all digitally projected and caught in camera. Or when you see him sitting down in the cockpit with Lando and they take the jump to the hyperspace, that shot is all in camera with just a little color correction,” explains Bredow.

The Falcon cockpit design and pre-viz

The on set projection system

Production frame from the final film

In many respects shooting rear projection is a very old school visual effect approach, but this application of the technique was anything but low tech. “I think we pulled out every trick in the book on this film, and developed a few new ones of our own,” explains Bredow. “We took some of the oldest visual effects techniques, such as front- and rear-screen projection, and updated them with the latest technology. This allowed us to film 360-degree environments on the stunning Dryden’s Yacht set. We also used the latest laser projection technology to surround the Falcon cockpit with screens. When we immersed the cast into hyperspace, they were actually experiencing it like you would on a simulator ride, only at feature-film quality that worked on-camera.”

The team tested both LEDs as well as front and rear projection and settled on rear projection for a variety of reasons.  One of the key aspects was that DOP Bradford Young likes working at fairly low light levels. “He prefers that to having less light and not stopping the camera down,” explained Bredow. “He always wanted the cameras stopped wide open and he prefers to not use a lot of ND.”  The team tested LED screens and found them too bright.  While it is possible to mix or fade down the LED screens, Galler points out that for LED screens, as you reduce the LEDs, the dynamic range drops and the team would have reduced “their effective  useful range on the LEDs to less than an equivalent 8 bit source file”.

For the Falcon sequences, the team built a 30 foot high screen that was about 30 feet in diameter, with a 180 degree cylinder around the cockpit of the Millennium Falcon. “So pretty much every direction you looked out, the windows were filled with our projections,” explained Bredow. The team used multiple projectors. For this set, there were five 4K projectors turned on their side, in portrait mode. This gave the team a 4K tall by 8K wide, seamless screen outside the ship. “This give us a nice bright, super high resolution projection surface that provided both the light for what happened inside the cockpit, but it also provided final quality imagery out the window,” Bredow explained.

The team uses specialist projection mapping software called Disguise, which handles the projection (more below). But there were two other problems the team had to overcome. The first was balancing the projection light color profile. The ILM color scientists measured the projected images from the server when viewed through the Arri 65 production cameras. Then using real time software Notch, Lux Machina could adjust the color profile of the server footage to make sure the recorded images outside the Falcon windows was both the correct color temperature and color gamut. The Lux Machina team had this calibration software set up so they could run it on a laptop and give immediate results.

The second issue was frame syncing the projectors to the Arri-65 cameras, which do not have a traditional broadcast sync. Lux Machina solved this, “so that we could have frame accurate playback with no blended frames in camera, even though the Alexa 65 has no Genlock,” explains Galler.

The main rear projection screens were run from a set of servers. Normally the Disguise software can run multiple 4K screens, but in the case of Solo, Rob Bredow wanted to make sure they had the absolute best image quality. If the screen were only providing contact lighting onto the actors, it would not have been so critical, but Bredow wanted to be able to capture the screens as part of the main camera filming, thus avoiding any keying or need for green screens. Lux Machina set the servers up to use ILM’s RLE encoded Animation Codec footage of the exterior star fields and nebulas. “Even with all the horsepower those machines had, we split up our files across multiple servers” explained Galler. For the main Falcon set up , “there was five 4K outputs, which really, honestly, is within the scope of what are the machines could normally do, but we had them split over three separate machines.”  The machine’s were producing one gigabyte, a second per file, per server, which was about the maximum threshold of the RAID arrays of the servers (they could do a maximum of 1.2 gigabytes a second). “We were actually running at a really, really high CPU load and drive load,” he added.

In addition to the main back projection rig, the team also used a smaller rig with a floating LED screen for closeups, (see the Speeder Section below). The large projection rig with its laser projectors, was on set for 6 weeks.

The Falcon cockpit was built on a hydraulic gimbal that was “driven more in a manual ‘mantronix’ sort of way. That was triggered with more traditional actuators by the special effects team,” recalls Bredow. “We did have a bunch of interactive elements but we didn’t try to do a lot of 3D rendering in real time.” The crew did have a set of elements available to be triggered in real time as separate 3D elements. “For example, as the Falcon was racing through the Kessel Run in one of the really cloudy environments, we had pre-rendered an 8K by 4K media segment and we also made it looping, so ILM made multiple 25 second looping 8K x 4K sequences for that particular section of the Kessel Run”. In all, ILM made more than 20 minutes of media for the Falcon Projection set up, at full resolution and at a quality so that the team use it to capture in camera. “Then on top of that we would layer in interactively blaster fire coming from the tie fighters. We had a trigger system so we could add into different areas of the giant screen, as an overlay. We had an interactive system so that either Ron Howard, Bradford or I could cue when we needed, say, more blaster fire,” he adds. The team actually built the entire Kessel Run as a giant interactive sequence. “We had some 160 cues throughout the sequence that I wrote into the script and that the team programmed into the system. So at any time onto payback, if Ron wanted to do something again, we could.” This gave the director the ability to run the entire Kessel, run top to bottom, with any of the team interacting and making changes as needed, or “we could just do a scene over and over again, and we could cue up the whole thing in just 10 seconds and be ready to roll if Ron wanted to work on a particular performance with a particular actor.”

Dryden’s Star Yacht

Dryden’s Star Yacht was constructed on sound stages at Pinewood.  Lamont and set decorator Lee Sandales used the great ocean liners of the 1930s as their inspiration for Dryden’s Star Yacht’s interior architecture and dressing, and the Art Deco influences in the design are clear.

In Dryden’s yacht there are display cabinets, and among the treasures are some from another film franchises such as Indiana Jones, in addition to just the Star Wars universe.

The creative teams took the opportunity to create, build and dress sets with an opulent finish, with beautiful gold and silver metallic luster to the elements. Dryden’s Private Study is also an enchanting space, a museum of glass cases filled with ancient artifacts, old weapons, gems and other invaluable oddities and treasures. The treasures were garnered from the environments and indigenous populations that Dryden and the Crimson Dawn syndicate have plundered. If audiences look carefully, they will see familiar items from around the galaxy.

All the light coming in the windows is provided by a 375 ft long and 24 ft high projection screen, lit by 15 HD projectors.

The design for the yacht

Production final frame

Dryden’s ship was given a real view out into the world, like the Millennium Falcon cockpit sequences, this was done with rear projection by the visual effects department again in concert with Lux Machina. In order to create this illusion “in camera,” the team first shot 360-degree photography from a helicopter. To get a clear image, they photographed with the doors open and the crew strapped in for safety. Bredow then explains that, “we took the final quality images and projected them onto a 370-foot-long surface, comprised of 10 blended laser projectors with special lenses to accommodate the set. The final resolution was over 16,000 pixels wide, or over eight times the resolution of HDTV.”

 One of the innovations on the Yacht set  used for this very large rear projection systems was a new lens system. Unlike a traditional projector lens used on the Falcon set, the image here bounced from the special lens back to project behind the actual projector. “The light travels over the projector body, which is obviously unlike most projectors. It is really fascinating how the new lenses work,” comments Galler. This was critical as it reduced the throw distance from the projector to the screen. Without this, the studio would have needed to have been vastly bigger to allow for the projection distances to the screens.

What allows this to work is both the new special lens the team utilized and the warping adjustment to the source material that meant that the image looks correct to the camera eye position, even when projected at such acute angles.

On Dryden Yacht set the team were able to work with just an approximate 11 foot throw to our the screens from the projectors. By comparison using the most traditional projector lenses for the Falcon cockpit, the team needed had a projection throw off closer to 30 feet.

Outside the actual projection screens on set.

Speeder

For the Speeder chase at the beginning of the film, the crew filmed real cars driving around an abandoned power plant, a couple hours outside of Pinewood studios. The ‘speeder’ had wheels which ILM erased in post, to give the illusion that it was floating.

The various Corellian environments of the chase were built across Pinewood Studios and at Fawley Power Station, an oil-fired, disused power station built in the late ’60s and located in Hampshire, in the south of England. Its utilitarian boiler houses and turbine halls hugely inform the overall look and design of Corellia. The VFX team measured and mapped the interior and exterior of Fawley Power Station site, around 200 acres in total, with LIDAR laser scanning equipment.

“Some of those shots were captured heavily in camera with just some background extensions and the wheels removed,” explained Bredow. In addition, some full CG speeder replacements were done, if the lighting wasn’t matching for some reason.

“We did a lot of different testing, both animation testing and filming with cars with actual stunt drivers.” Bredow decided himself to just try painting out the wheel to see what the speeder might look like. “And it turned out that the most satisfying thing was that kind of stunt, four wheel drifting action that we’ve seen a lot in films. That seems to be the most fun.” The special effects team was led by Dominic Tuohy built the real cars. “They were 550 horsepower, and rear wheel drive or a front wheel drive cars that could be switched on the fly,” explained Bredow. “The stunt drivers really put them through their paces to pull off some of the stuff you see in the film”.

Fawley Power Station

The development of Han’s speeder was an extended process and an interdepartmental collaboration among the art, action vehicles and special effects teams.  Many concepts, designs and models by teams working in both the U.K. and the U.S. were produced, and after a long process the team arrived at a design that made everyone happy. The filmmakers had decided the speeder should be blue with white racing stripes, and a little beaten up.

Han’s speeder is dressed with items as diverse as a Morris Minor bonnet hinge, a set of baguette warmers and a refuelling nozzle from a Vickers VC10 aircraft.  The 1960s-style chrome bezels and switchgear for the dashboard were purchased at a second-hand auto parts sale. The final design of the white stripes on Han’s speeder were the result of almost 100 variations of width, shape, length, color and position!

For stunt work, two of Han’s speeders were built on a custom-built chassis with a V8 engine, allowing them to reach over 100 mph!  During the shoot, more than 80 tires were used on the two stunt-driving speeders due to wear and tear.

Special effects supervisor Dominic Tuohy explains, “The speeder chase really excited me about this project. We’ve seen a lot of chases within the Star Wars franchise but we have never seen a speeder chase.” Tuohy’s team constructed the speeder as a special all wheel-drive vehicle. “The real challenge was making it look like a traditional vehicle, without wheels, as obviously the Star Wars speeders don’t have them, and it was an opportunity for us to do some fantastic engineering, building the speeder to endure fast corners, different surfaces, tarmac, concrete, all the things a standard car couldn’t do,” says Tuohy. “We fitted a combination of rigs and pod-rigs to fit both cast and stunt drivers at the same time in a way in which the stunt driver could not be seen, and for the camera crew to mount their equipment.”

Describing the chase, Tuohy says, “Part of the car chase was showing Han’s driving skills as well as his flaws, so we had him deliberately hitting things that would explode to keep the audiences guessing whether he’s in control or not.”  For the special effects team, this meant lots of pyrotechnics and atmospheric effects, which bolster the authenticity of the chase.

The crew made every effort to capture as much in-camera as possible, including the environments and also the incredible vehicles in the film,” says Bredow. “Those are real 550 horsepower speeders where VFX removed the wheels and enhanced the world around it”. When shooting the reverse shots of the cast in the speeders the team once again built a system similar to the one for the Falcon, but this time with a movable top LED screen.

Bottom of frame is Lux Machina’s Phil Galler and associate Wyatt Bartel with the 4×4 GUI on the right.

This rig is very similar to one that Lux Machina had built several times before, for car commercials. The team used Touch Designer driving Disguise software to allow the scenes to be moved and adjusted quickly on set, depending on the angle of the shot and how the DOP wanted the reflections to play and the contact lighting on the cast to appear. Disguise allows real time 3D visualization and control . The team can make an entire set in 3D or import assets from ILM and the pre-viz team. Then the same program allows use right throughout the entire process. It is based on the three simple stages of Design, Sequence and Control.

The  Disguise software was formerly known as D3 and was developed originally for concerts. The developers  first project was a Massive attack concert and then a vast U2 concert and it is still used today on concert tours such as Katy Perry and most recently a major Metallica Tour.

Lux Machina extensively used Disguise’s timeline tool sets, as well as the 3D workflow to move the screens around in 3D to represent what was needed for the Speeder sequence. “We used the projection mapping content onto the screens and used the parallel mapping function (onto the screens) as well, so we could represent where the screens were in the physical space of the Speeder.”  The Disguise software has various tools for virtualization of the screen relative to the content and the projectors, “I like to think of it as the difference between perspective and orthographic mapping” explains Galler. “And most of my clients understand that because they deal with drafting tools where they sometimes represent drawings orthographically and sometimes in perspective.  That is the way I described the use of Disguise in this area, it has tools that give you a point of view of the content versus just directly shooting content flat onto a screen”. In the 3D mode the operator sees a full 3D model of the set, the screens and the projectors, but the content appears as it will to the camera, without the necessary distortions that are needed to make the lensing and projections look correct in reality.

The Lux Machina crew do large scale concerts and award shows. As large and as important as the Solo project was to the team, it was only an average size projection project for them. One of the first film projects Phil Galler and the team ever did was the elaborate projection system for the cloud home of Tom Cruise in Oblivion. Comparing how are the technology has changed since that film, just four years ago, Galler commented that the difference is vast, “the latest laser projectors are so much better. They’re so much nicer. The technology has come so far. Brightness was one of the things that we battled with on Oblivion but there were other things – back then you couldn’t put a lot of projectors in portrait mode as the lamps would flicker, with the laser projectors, you just don’t have that problem. Plus today you have a wider range of lensing options”.

A focus puller operating beside the projection system

The other big difference today is how quickly the team can set up. On Oblivion, it took Lux Machina two and a half weeks to load in. On Solo, “we did the entire 375 feet Yacht set in just four days. We came in on a Monday, and by Thursday the crew came in to do camera tests. That represents a massive increase in deployment speed, due to the tools that we’ve developed using the Disguise software and just overall improvements in projector quality”, Galler explained.

The final illusion of the speeder escape was completed with a seamless integration of the environment between the on-set props designed and installed by the Art Department and the digital set extensions created by ILM. This close collaboration between the departments was a theme throughout the making of the film.

Read the article here