cancel
Showing results for 
Search instead for 
Did you mean: 

Is it possible to access the passthrough camera feed programmatically on the Rift S?

SparkyTD
Honored Guest
I know that the feeds are only b/w and they are full of random artifacts, but it would still be convenient if I could use them for an OpenCV program running in my VR app. Is this possible with the Oculus SDK?
8 REPLIES 8

MikeF
Trustee
no, asked 100 times over. not going to happen.

JeffNik
MVP
MVP

MikeF said:

no, asked 100 times over. not going to happen.


@MikeF - Why are so confident that they wont eventually make it available?

Jame1s
Honored Guest
Is there a way to turn on the pass through camera for a bit to reorient myself in my room or to grab something really quickly?

SparkyTD
Honored Guest

Jame1s said:

Is there a way to turn on the pass through camera for a bit to reorient myself in my room or to grab something really quickly?

@Jame1s On the Rift S, you can double press the menu button on the right controller and it will do exactly that. I'm not sure about other HMDs.

JeffNik
MVP
MVP
@Jame1s, @SparkyTD: Note that, in version 14 at least, that feature is off by default - you have to go to settings in the headset, under Experiments, and set "Passthrough Shortcut" to "on".

Quest is supposed to have passthrough on demand in version 15, (experimental setting, too), and its toggled by tapping on the headset itself twice. I'm very much looking forward to that update!

MatrixPaul
Protege
I would also like to access the pass-through camera output for Unity or indeed any other way. Perhaps even just taking a screenshot of the scene with passthrough enabled.

Hey, any update on whether you were able to use passthrough with unity? I am currently working on a project that would greatly benefit from this. It won't stream to my phone too. Also, I have the oculus quest 2 so not sure if there have been any recent updates that may have changed this. 

pabramsor
Honored Guest

Now it is possible to enable passthrough in Unity, there is an excellent tutorial on the official webpage: https://developer.oculus.com/documentation/unity/unity-passthrough/?locale=es_ES

 

But for now (and it seems that it will be forever), the only thing you can do is underlay or overlay the camera projection. The images cannot be touched/accessed from the code. It is neither possible to post-process them because the overlay/underlay are combined in a safe and separate stage by the compositor (As far as I understood). At least for the Quest 2 devices. 

 

What I am not 100% sure about is, and it is the reason why I reached this post, if it is possible to access/postprocess the images using the Oculus Rift S. I did it with Valve Index headset using SteamVR and OpenVR SDK. Oculus Rift is stated to be compatible with SteamVR, but I don't know if the passthrough is hindered somewhere in the low-level driver.

 

Any update on this by mid-2022?

 

UPDATE:

Sadly, it seems that passthrough won't work on Oculus Rift https://forums.oculusvr.com/t5/Unity-VR-Development/When-will-the-passthrough-v31-SDK-be-available-f...