Liv partners with Meta to capture mixed reality videos on Meta Quest

Liv, a pioneer in mixed reality capture, has partnered with Meta to become the official capture solution for Meta Quest wireless virtual reality headsets.

Founded by AJ “DrDoom” Shewki, Liv has inked a multi-year partnership with Meta to bring Liv’s mixed reality capture and virtual camera solutions to developers publishing on Meta Quest and also creators who wish to use those features in Meta Rift and Quest apps. Developers are adding the capability now for standalone headsets and it should be available for consumers to use in the coming weeks in VR games.

A big part of game marketing today combines trailers and video content made by the creator and player community inside games, and Liv wants to continue supporting developers in their mission to reach as many creators, players and fans as possible in XR. If you think about it, Liv is probably going to be the way that we’re going to record our first experiences in the metaverse.

“The first and most important thing is that it’s going to run natively on a Quest,” Shewki said in an interview with GamesBeat. “This will work on a standalone device, on the Quest — meaning the Quest, Quest 2 and Quest 3 and all the upcoming Quest devices. It’s part of our partnership with Meta.”

Shewki added, “We spent seven years building technologies. Our mission has always been to empower people in VR and AR to play and share their favorite moments with their friends, family and fans. This will be the first time that we can do this directly on standalone devices like the Quest.”

The Liv SDK, the Liv Creator Kit is already being integrated into Gorilla Tag by Another Axiom; Penguin Paradise (and their new game Skelly), by Sava Studios; Scary Baboon, by Flixzy, and a large list of other VR apps.

“Another Axiom builds fully realized spaces that are meant to be shared together, like in our popular game Gorilla Tag,” said David Yee, COO at Another Axiom, in a statement. “We’re always looking at new ways to give our players and creators a great experience they can share with their family and friends. This partnership between Liv and Meta provides access to best-in-class capture and virtual camera technology, introducing new ways to capture and share in-headset experiences. We can’t wait to see what the community does with these new tools.”

“It’s going to be a massive boost to user generated content inside the top VR applications,” Shewki said.

As part of this partnership, Meta is deprecating its own mixed reality capture (MRC) tooling and Liv will take over as the official solution.

With Liv, VR players can do things like take a virtual selfie, operate first-person and third-person cameras and perform real-time mixed reality capture with physical cameras.

Background

Liv has served the VR game dev community since 2016 and will continue to do so now with the help and support of Meta, Shewki said. But for most of that time, the Liv technology for capturing videos only worked with wired VR headsets that connect to a PC. The tech started there because it was easier to have a recording application running alongside an actual game using the power of a PC. But now the VR wireless headsets have become more powerful, as has the Liv tech.

Now, the difference is that it will work with the most popular VR headsets, the Meta Quest standalone wireless headsets, which don’t need to be connected to a PC. The company has 24 people and they’ve been working on this part for about a year.

“We spent the last seven years building camera technologies for app developers in VR, and content creators and players in VR. Historically, we’ve primarily been on Steam. When we started seven years ago, open VR, or Steam VR, was the only platform available,” Shewki said. “We made a whole bunch of assumptions back then about how the technology ought to work and how it ought to integrate with games. So what we’re releasing is effectively taking those seven years worth of learnings that we have learned alongside developers as we’ve been building this and we’re releasing a new SDK.”

So far, roughly 50 to 100 developers have been downloading the SDK every month. Most of them are making games, and they’re developing for VR systems that are connected to the PC. Many of these are for educational users at schools and universities.

The Quest market, for wireless standalone headsets, is an order of magnitude bigger than the PCVR market, thanks to games like Gorilla Tag.

“We expect our monthly creator numbers to go up,” he said. “We are going to roll out with tons of games. Our goal has been to be on every device on every platform.”

And there are some games where this works now.

“As part of this announcement, we’re also excited to share that we’ve got tons of new games getting Liv support, including Gorilla Tag by Another Axiom, and Racket Club by Resolution Games,” Shewki said.

The rush to the Quest

With the Liv mixed reality capture (MRC) SDK, you can film real people composited into the game world, which is primarily used for high-end company production and VFX work. And Liv also has a trailer production studio in Australia that uses its own tooling, and Liv makes trailers for some of the biggest game developers and platforms in the world.

And then there is the Liv App on Steam, which people use for mixed reality capture and a virtual camera. But the limitation has been it had to be wired to a PC. Now, it will be available on standalone devices without the need for a PC or high-powered GPU in your PC, and will be natively available in the game.

“You won’t need to download a Liv app and run the Liv app in parallel. You will spawn the camera (in-game, film and save the content). and soon also stream the concept natively from the device without ever leaving the headset,” Shewki said. “So it is a solution built for people who are primarily in Quests and don’t have additional tooling on their PC.”

To clarify, he said that if you’re on a Quest and you’re primarily playing Quest games without a PC, you can finally create really high-quality and rich video content from your favorite applications using a combination of selfie cameras, third person cameras, drone cameras and first-person view cameras, with all the bells and whistles that you need to make great content, he said.

The rollout plan for the new SDK

The beta release of Liv’s new software development kit (Liv Creator Kit, or LCK) for Unity-based apps is available now. The goal is to unlock content creation for developers who publish on Quest by empowering their players with social camera tooling to put in the hands of their communities.

All Liv Creator Kit’s features are included in V1.0 SDK that is released today the first person, third person, and drone cameras.

Beyond 2024, Liv is looking at improving the tooling for creators on the Liv App. Liv’s backlog has grown big over the years, Shewki said.

As for trailers, the company used Liv for the raw captures, and with a bunch of editing and post-production magic courtesy of Liv Productions.

Sava, the creator at Sava Studios, said in a statement, “I am proud to say we added LCK into Penguin Paradise and our new game Skelly because we want to give our community a way to make the best and high quality content in our game.”

“I am proud to say we added LCK into Penguin Paradise and our new game Skelly because we want to give our community a way to make the best and high quality content in our game!”

Creators will be able to immediately use the new camera in their favorite apps as Liv integrates more and more games, starting with Gorilla Tag in November.

Shewki said he wants these cameras to feel like they belong in the game world.

“We specifically want to avoid people thinking of it as, ‘I have to go download an additional tool.’ I’m going to have a rich camera available to me that I can spawn at any time and record my favorite moments without ever leaving the headset. And that’s what this will unlock,” Shewki said. “We’re going to be rolling out with Gorilla Tag and some other big titles initially. Once we roll out, the SDK will be publicly accessible.”

In the app now, only one camera can be running at any given time. Rather than monetize game developers, Shewki said the company will monetize directly with Meta. The company will make sure its tech can work on all upcoming popular AR and VR devices when they launch, he said.

Streaming will be the next thing that Liv will work on. At that point, players will be able to upload and stream directly to Liv. But that’s not ready yet.