Riot Games and Partners on How They Are Integrating XR Broadcasting Technology in Worlds 2020

October 25, 2020

2020 has been an unusual but painful year for the global entertainment and sports industry due to an unpredictable pandemic (COVID-19) that has plagued the world since February. Though esports has the advantage to put itself online and continue running, plenty of competition cancelations and no large-scale offline events have made esports enthusiasts and players disappointed in the past eight months.

Fortunately, after overcoming various pressure and difficulties, the world has embraced the League of Legends World Championship (Worlds 2020), the only large-scale international offline esports event this year in Shanghai.

The last several weeks of Worlds 2020 offered significant moments in esports broadcasting history, as Riot Games applied a technology called Extended Reality (XR) during Worlds 2020 broadcast.

XR is a term referring to virtual and reality combined environments, and human-machine interaction generated by computer science. Simply, it is a combination of Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) technology. XR is typically used in large-cost film or TV series, such as Disney’s Star Wars: The Mandalorian.

In order to better present a film-level XR watching experience in Worlds 2020 to audiences, Riot Games teamed up with long-term partner Possible Productions and Lux Machina, who was also part of the team behind The Mandalorian.

“Riot Games assembled a team of more than 40 specialists, content creators, and artists to enable the mixed reality components of the broadcast.” Wyatt Bartel, senior technical director at Lux Machina told The Esports Observer.

In past editions of the World Championships, the event applied a “city to city” strategy for promoting the event and competing in the region. However, due to the COVID-19 restrictions in China, the previous strategy became impossible this year.

“The decision to use XR to the extent it is being deployed in the broadcast today was an evolution of conversations going back to May of 2020. ” Nick Troop, executive producer for Worlds 2020 at Riot Games told TEO, “We recognized this technology would enable us to bring fans closer to the action and allow them to dramatically increase engagement with the broadcast.”

Pictured: XR Technology in Disney’s Star Wars: The Mandalorian. Credit: Lux Machina

Troop also mentioned that Riot Games has been integrating XR technology into broadcasts since 2016. Most notably, Riot Games applied AR technology to feature the Elder Dragon character, which flew around Bird’s Nest Stadium at the Worlds 2017 Grand Final opening ceremony. The design helped Riot Games win the award for Outstanding live Graphic Design at the 39th Annual Sports Emmy Awards in 2018.

While XR was applied in every match of Worlds 2020, it has caused challenges and a larger workload for the XR team. The Worlds XR team actually built a 360-degree virtual space around the stage so that audiences could see Shanghai landmarks such as Oriental Pearl, as well as the effects of four in-game element dragons.

“The most challenging part of the process was that many of the tools and technology we needed to achieve the vision did not yet exist. Never before has XR used live game data to extend the audience’s experience and drive on-screen effects,” Bartel said, “Combined with a never-before-seen scale, live broadcast, and multiple cameras, every day we went to work we realized we were defining the edge of what is possible.

“Beneath the hood of the XR stage, there are many custom software elements and hardware combinations running on the stage that were designed by Lux Machina over the past few months,” Bartel added.

In addition, Bartel also detailed how the XR operation team made special practical effects during certain moments like Ban&Pick and player deaths.

“I don’t think any team has done as deep an integration between technologies as the Riot production team did this year at Worlds 2020.” Bartel said.

“The special effects you see at Ban&Pick are powered by more original code that we developed,” Bartel said. “This code allows game data to pass back and forth between the Lux Machina modified Unreal Engine powering the stage and the battle taking place on the Rift. Certain moments will trigger the stage to automatically begin our special effects, making what’s happening in the game into the real word dynamically.”

In the special effect process of the traditional film industry, actors normally perform behind the green screens. While players are not professional actors, it could be a strange place for players and influence their in-game performance.

Pictured: Special effects during Bank&Pick. Riot Games

“Due to the nature of XR, the players can actually see much of what the viewing audience at home sees, something that wouldn’t be possible with a traditional green screen. This enables them to experience the elements on stage in a natural and engaging way,” Bartel explained. “We help provide a sense of ‘place’ for League of Legends players by integrating both physical and digital elements into the players’ environment. Everything below a certain height on our stage is a physical build-out.”

After European team G2 Esports won a 3:0 against South Korean team Gen.G Esports at the knockout stage of Worlds 2020, G2 Esports celebrated its victory by trying to throw one of its players Rasmus “Caps” Borregaard Winther into the virtual pool. That moment has become one of the iconic memes in the League of Legends esports this year. After another South Korean team, Damwon Gaming (DWG) defeated G2 Esports at the semi-finals, the team did the same thing to celebrate the victory. Bartel clarified that it was not planned, and a really fun moment for the production team.

“It was a natural response from the players to the realism that our mixed reality environment was able to bring to the competition. It also shows how important it is to have a production design that can be engaged to bring people’s creativity out in ways that we never expected,” Bartel said. “When done correctly, an XR set becomes a sandbox to play and interact within in entirely new ways every day.”

Now that all the production work before the final has completed, Bartel and Troop’s production team will embrace the most challenging part of this Worlds 2020 – The offline final at Shanghai Pudong Soccer Stadium Oct. 31. Around 6,312 live audience members on-site and millions of online viewers will witness the most important best-of-five match between Chinese team Suning Esports and South Korean team DWG.

“We wanted to make sure every viewer, as soon as they joined the stream, was blown away and felt completely immersed in Worlds, the same way they would if they were physically in the audience,” said Troop.