Weird Camera Rig Captures Volumetric Video That Can Be Manipulated in 3D

Earlier this year, filmmaker Josh Gladstone showed off an incredible volumetric video rig he built using GoPro action cameras. Since then, he has been refining the hardware and software, delivering an even more impressive light field array and workflow.

At its most basic, field volumetric video is three-dimensional video content you can move around. “Volumetric video is moving images that you can move around and look at from various angles, sort of like holograms,” explains Gladstone.

There are a few different ways to capture volumetric video, but many require a lot of expensive and sophisticated camera hardware and a controlled studio environment. Even a “sparse” light field array can include as many as 50 cameras. Gladstone built a volumetric video array that can be used in the real world and relies on just five inexpensive action cameras. So, if Google’s 48-camera light field array is “sparse,” Gladstone says his modest array is “super sparse.”

When Gladstone spoke with PetaPixel earlier this year, he explained the importance of his software, which enabled great results from only five regular cameras. The software “takes the images from each camera, computes their camera poses, and then uses AI to render the Layered Depth Images (LDI). This LDI consists of 8 layers of RGB + Alpha + Depth. It’s similar to the Multiplane Images implementation. It stretches each layer into the z-axis using per-layer depth information. In the multiplane implementation, I was using 32 layers of flat images. This layered depth implementation is eight layers, so it’s more efficient,” he said.

Gladstone has continued working with light field volumetric video and has recently released the new video seen above that shows off his new rig, explains layer depth video processing, demonstrates how people can use volumetric video, and details Gladstone’s latest software, Cake_player.

Cake_player is the “only way to view the delicious futuristic media format ‘Light Field Layered Depth video,'” per Gladstone. The app is available for macOS and Windows and works alongside the Meta Quest 2 and Quest Pro (in beta) and Looking Glass volumetric displays.

As Gladstone’s new video and software show, he has been hard at work revising his camera rig, which features a new 3D-printed frame and is very sleek. Like before, Gladstone uses five GoPro cameras, although they are much closer together this time and the entire structure is significantly more compact.

While the ability to pan around a moving scene is one of the niftier benefits of light field volumetric video, another great feature that Gladstone shows is that because the video file is realistically layered and playing back as three-dimensional media, graphics and visual effects artists can place 3D objects inside the scene and manipulate them in real-time.

Alongside the GIF above, Gladstone sent PetaPixel numerous other GIFs showing his advances since this past summer, including ways that users can consume volumetric video content, which is still a relatively new technology.

People can keep up with Josh Gladstone’s work, including his continued development of volumetric video capture and processing technology, on his website and by following him on Instagram.

Image credits: Images courtesy of Josh Gladstone