While some consider them "not VR", viewing 360x180 panoramas on a VR headset is quite cool. Consumer 360 cameras are cheap and software like Hugin and PTGui let you turn normal photographs into panoramas.
There is a problem with panoramas though: depth.
Assuming you are using only one eye to view a panorama in VR, the image will be correct in all directions. Of course there won't be accomodation (focus), consumer VR headsets don't do that yet. The problem comes in when you have stereo vision.
Look around in a panorama using both eyes, everything looks ok. It's fairly flat, but ok. Now look at the ground. Oh crap, I'm flying in the air!
The standard way a panorama is rendered is by projecting it onto the inside of a sphere. This sphere is probably around 30m radius (that's where our stereo depth perception breaks down). This makes things around us look far away, which tends to suit most panoramas. But it also means the ground below us is 30m away! With stereo vision, we feel like we are way up in the air, even though the panorama was captured at head height.
One way to fix this is to use 3d panoramas (stereo images). This captures depth for the entire view. But these have issues too: you can't tilt your head sideways (otherwise your eye alignment won't match the panorama) and it's harder/more expensive to capture.
The technique I'm using in this demo is to add in ground plane geometry and project the panorama onto it, in addition to the usual sky sphere. From a fixed 3DOF view, a ground plane with correct projection will look the same, except you can see the depth is correct. You can tilt your head, the depth is dynamic instead of baked.
This ground plane does magnify another issue with panoramas: they are 3 Degrees Of Freedom only. Well, they don't have to be. In this demo you can change to a 6DOF mode. You can now walk around the panorama. Of course this distorts the view quite a bit. In particular any vertical surfaces projected onto the ground plane look distorted when your head isn't in the exact centre of the panorama space. But it still looks cool (especially if you are looking down, since the nearby floor is less distorted).
I've also added a 4DOF mode. In this mode, you can look around (3DOF), but also move up and down. Horizontal movement is disabled. You can bend down and get a closer look at the ground, with less distortion than the full 6DOF mode.
The demo includes 4 panoramas. Three are from the European Southern Observatory (their website has many awesome panoramas at crazy resolutions). The other is a Nvidia Ansel capture from the game The Witness.
A - Toggle ground plane
B - Cycle to the next panorama
X - Cycle to the next degree of freedom. There are 3, 4 and 6 DOF available.
Right Thumbstick click - Reset position/orientation
Index Triggers - Spawn a cube
Grip Triggers - Grab a cube
No, I didn't add cube juggling just for the demo. This was built on a base project for my students, so it already had some hand/physics stuff in there.
Rift/Rift-S version. Would probably work on SteamVR with a Vive too, but is untested.https://github.com/rajetic/DepthPanorama/releases/download/v0.1/Depth.Panorama.Test.v0.1.Rift.zip
Quest version. You can upload it using Side Quest
How does it feel? Which degree of freedom mode do you like best? Did you ignore the panoramas and just juggle cubes?