Ep. 13 – VR and 360 Video
With Alx Klive (360 Designs)
360 Video and Virtual Reality are often seen as two sides of the same coin, though in practice, different techniques and skills are required for both. Technological advancements have allowed companies to really push the boundaries in these two industries, even more dramatic developments on the horizon.
For years Virtual Reality has fascinated both companies and consumers, around 75% of Forbes World’s Most Valuable Brands have created some form of virtual reality for customers or employees. Social Media has not been far behind the curve, with all the major platforms experimenting with ways in which they can become involved in this new and exciting trend.
In this episode we discuss 360-degree video; an immersive type of video recording where the view in every direction is recorded at the same time, filmed using an omnidirectional camera or a collection of cameras.
Our guest, Alx Klive, formed 360 Designs in 2015; a company that produces, sells and rents professional tools for 360 video and virtual reality, whilst also producing a host of live 360 video productions.
I caught up with Alx at the Broadcast Video Expo in London where he was fresh from giving a keynote speech…
Alx Kline
SL: Thanks for joining me today Alx, let’s start with the basics by defining what 360 video and VR actually is?
AK: For me, there is a lot of debate over this. Is it VR if you’re watching video in mono, or is it 360 video? From my perspective it’s VR if you’re watching it with a headset, it’s the delivery that defines it. If you’re listening to something on a radio, it’s radio; if you’re watching something on television, it’s television. My personal view is that if you’re watching it in a headset it’s VR, doesn’t matter whether you’re watching in mono, stereo. We’re currently at the AM radio stage with VR, it’s going to get better, the technology will improve. If you’re watching it on YouTube, that’s 360 video. You’re swiping around the screen, it’s in the browser and you’re using your mouse to move around. That for me is 360 video, though both terms are used interchangeably.
SL: What are it’s best uses and applications? What sort of video works really well in 360?
AK: It’s a good question, for me and our business, we are really hot on the live side. Live VR, live 360 video. If we’re not talking about games and the CGI side, we’re talking about video in VR – unquestionably to me, the biggest part of that industry will be live VR. It’s going to change the world, when the quality gets to the point where the phone’s in our pockets have 8K screens, 16K, 32K screens. That’s retina grade VR, and I’ve experienced 8K live in a headset, you don’t see the pixels anymore. It’s teleportation! That’s going to be a very powerful thing, in five to ten years. That’s where the money will be too from a business perspective.
“The biggest part of that industry will be live VR,
and it’s going to change the world.”
SL: What are it’s best uses and applications? What sort of video works really well in 360?
AK: It’s a good question, for me and our business, we are really hot on the live side. Live VR, live 360 video. If we’re not talking about games and the CGI side, we’re talking about video in VR – unquestionably to me, the biggest part of that industry will be live VR. It’s going to change the world, when the quality gets to the point where the phone’s in our pockets have 8K screens, 16K, 32K screens. That’s retina grade VR, and I’ve experienced 8K live in a headset, you don’t see the pixels anymore. It’s teleportation! That’s going to be a very powerful thing, in five to ten years. That’s where the money will be too from a business perspective.
SL: We’re talking about immersive events, like concerts, sports events. So you could be doing other industries out of work in the future, because why would they bother going?
AK: Sports is going to be massive, music festivals – anywhere you have fans. Fans pay money, fans for a football team or a band. It’s mega, because of the size of the potential audience. They’re talking 8 billion smartphone owners by 2025, that’s everyone on the planet. That’s actually a very interesting point about this medium, and it is a medium. You think about previous mediums, like radio, you had to go out a buy a radio. This, the device you need to consume it, is already in your pocket. That’s what makes this different from anything that’s come before.
SL: Do you think that VR is being driven by the market in the way that 3D wasn’t? Skeptics would say that 3D was forced upon audiences, but a 360, VR experience at something like a concert has great market value, is it being pushed forward naturally by the market?
AK: It’s definitely not a gimmick, or a fad. This thing is here to stay. It’s not like 3D, I have quite strong feelings about this. 3D took off again around 2006 through to 2010/11. At the time I was a trade journalist, and I was quite opinionated on the fact that I didn’t think 3D was going to take off, the reason being that 3D to do in the home, is really difficult to set up. I’m pretty technical, I had a 3D TV and 3D Blu-ray, the two times in a year that we wanted to use it, it was the learning curve of setting it all up. You’d spend 20 minutes trying to get it to work, finding the glasses.
The bottom line is that 3D was too much work for the consumer to set up for too little an improvement in the experience. Most people didn’t care about 3D. When you walk into a movie screen to watch a film and they hand you 3D glasses, what is your reaction? Mine is indifference. That’s why it didn’t work. The only time people accepted it, is when it’s laid on for them by a projectionist – a professional who sets it up for you, hands you the glasses, otherwise people aren’t interested. However, VR headsets are intrinsically 3D capable, by definition. I’ve been speaking a lot to the broadcast industry, and I’ve been telling broadcasters and content owners that have libraries of 3D films that they’ve invested in, that 3D is going to have a bit of renaissance. You can consume it very easily, like if you’re on a plane. It’s going to have a renaissance because of VR.
SL: Do you think 360 Video, VR is on the horizon then?
AK: Yes and no. There are fundamental architectural issues that make 3D in VR for video extremely challenging, and they can’t be solved for at least five years. Right now, 3D in VR is single axis, you turn your head left to right, but if you look up or tilt your head it isn’t in 3D. If you imagine you’re wearing a VR headset, and you look straight up, the headset doesn’t know that you’ve looked straight up. You could have turned around and looked straight up, now your eyes are reversed. This is why right now, anyone doing 3D and VR blends it to mono at the top (zenith) and the bottom (nadir) because there is no way of handling it. So to do what I call free axis VR in stereo you require a completely different architectural approach – you have to deliver all of the information to a headset that will be extremely powerful. More powerful than live-stitching PCs at the moment, that can generate those left and right eye points on the fly. I don’t see that happening for a few years yet.
Top 5 VR Headsets
SL: What differentiates between a low end DIY camera rig to a more high quality professional one?
AK: A lot of things, a GoPro rig for example has been very popular for a few years. Popular because they’re cheap, and small. That has allowed people to build low, parallax, tightly bundled together rigs. But you can’t gen-lock them. You need to sync the cameras, so the sensors are firing 30 times a second at the same time. If you’ve got seven cameras, they need to be firing at the same time. This is really important for shooting VR, any movement you have that crosses a stitch line from one camera to the next, if they’re not in sync – it causes major problems. That’s the first thing that distinguishes a professional VR camera from a prosumer type camera. After that you’re talking sensor size, dynamic range, colour science that goes behind the sensor. It comes back to what is good in a normal camera, a good sensor, and they don’t come cheap.
SL: Talk us through the production workflow from filming with a VR camera.
AK: Our very high-end VR cameras using ‘Blackmagic’, we are developing VR rigs. We have made a name for ourselves around these cameras called Mini EYE. We have two cameras, one is a three camera rig, three of these Blackmagic cameras with special lens’ on them, and one is a four camera rig. So using the three camera rig as an example. To shoot VR is quite simple in theory, you need multiple cameras that overlap and shoot in all directions. We have three cameras that have a 190 degree lens but the sensor is rectangular so it’s 190 on the long side of the sensor. 150 degrees on the short side. So we turn the cameras on their sides, which is 150 + 150 + 150 which totals 450 degrees, obviously giving us the overlap. Top and bottom, each one is 190 which crosses, allowing you to capture the whole scene with just three cameras.
How to make 360 video: Basic Workflow
“Everything that we’re now using – the same approaches –
were all invented in four years in the 19th century.”
SL: In Post Production it’s a case of stitching together and blending the image?
AK: It’s called equire rectangular, which is the most popular format for this. It’s taking a fully spherical capture from all angles and squeezing it into a rectangle so it can be sent through the normal pipelines like a normal video. You capture on three separate cameras, you’re recording three separate files – you then need to stitch those together. This idea of stitching is like creating a panoramic picture, there’s some distortion you apply, and there is software you use. Photography itself was invented in 1840, two years after that 3D photography was created, two years after that panoramic photography was invented, all in the space of four years in the 1840s! Everything that we’re now using, the same approaches were all invented in four years in the 19th century. It’s very interesting.
SL: In a production sense, what are the limitations of the technology?
AK: The first problem is, where do you hide the crew or the cameraman. But, it’s quite easy to solve. You do have to think about it, if it’s a narrative or cinematic production, you have to light the whole room for example. But these are fascinating challenges to address, can you do extreme camera angles? Turns out you can! We’ve recently completed a short 360 film called ‘Knives’ which has won some awards, we have another one coming up called ‘Rose Coloured’, directed by a brilliant new director called Adam Cosco, could be the Stanley Kubrick of VR. It looks fantastic! I was skeptical about narrative VR but now I see it, it can be really interesting.
SL: Could audio cues be the thing that drives this forward in a narrative sense?
AK: Definitely, there are directors experimenting with this very idea, in a horror for example, if you’re in a forest and you hear a twig snap you’re going to look over there. But this can also be done with lighting as well, push people the way you want. That’s what makes this fascinating, young directors experimenting with this new medium, that’s what makes it exciting.
Interview with Alx Klive at 2016 Cinegear Expo LA
“The industry is moving at an incredible pace.
This is not a fad.”
SL: Finally, how do you see the technology developing over the next few years?
AK: The industry is moving at an incredible pace. This is not a fad. We’re based in the United States, and there is a great deal of activity going on, the big companies are investing in this. But if you go to a production company level, a broadcast level and agencies and brands there is a lot of investment going into this. The technology is moving quite quickly but it will take time. We’re still at the AM radio stage, it’ll be another five to ten years before this goes mainstream and people are regularly tuning into live VR. Episodic VR content is starting to happen, and I think it’ll grow steadily. There is a trough of disillusion with some journalists, who say that VR is over. Now people are starting to come out with the numbers of the sales of the Sony Playstation VR, Samsung Gear VR, Google Daydream Headset. It’s actually moving quite quickly, but it will take a long time to develop everything and to become mainstream.
SL: Thanks Alx, that’s been fascinating.