cancel
Showing results for 
Search instead for 
Did you mean: 

Access the Passthrough Camera via API

Arghzival
Protege
Itd be great if we could utilize the passthrough camera to create lightweight AR experiences. Even something like having the passthrough camera on as the backdrop while in Oculus Home. This would allow me to have multiple windows open at once (now possible in v15) giving me a seemingly limitless AR desktop experience!
Would also be great to allow developers to access and use the passthrough camera within apps.
11 REPLIES 11

Braydar
Explorer
I would be horrified if they ever gave developers access to the entire video contents of my home. 

Umpa_PC
Rising Star
I think they are the wrong type of cameras for that application - they are tracking cameras, that's why the picture from them is lousy.
Oculus Rift S - Oculus Quest 128GB
MSI trident 3 7RB-200UK Intel Core i5-7400 3 Ghz x2
MSI GTX 1050 Ti (4GB) & MSI Aero GTX 1060 OC (6GB) & MSI Aero GTX 1070 OC 8GB
16 GB RAM x2, 1TB HDD x2, 1TB SSD x2
Windows 10 Home Edition Version 10.0.18363 Build 18363, Oculus version 17, Quest Version 17
Fan Cooling by Zotac FireStorm - AfterBurner cause me problems.

kojack
MVP
MVP
Unfortunately they won't allow it, as it's a privacy issue. People freak out enough about having cameras on (or around) a headset that only Oculus has access to.
If we had access to the cameras, it would be possible to do fiducial marker tracking, which would allow multiple Quests to do co-location games using a common coordinate system

For passthrough AR it's better to use a dedicated addon like a Zed Mini (colour stereoscopic cameras that stick to the front of your headset). But it requires a Nvidia GPU to work, so can't run on a Quest.
(Well, if you had two link cables it would work as a PC headset with AR, the Zed Mini also needs a USB-C cable as long as the headset cable)

Author: Oculus Monitor,  Auto Oculus Touch,  Forum Dark Mode, Phantom Touch Remover,  X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2

Umpa_PC
Rising Star
And after all that, the quality would be crap right ?
Oculus Rift S - Oculus Quest 128GB
MSI trident 3 7RB-200UK Intel Core i5-7400 3 Ghz x2
MSI GTX 1050 Ti (4GB) & MSI Aero GTX 1060 OC (6GB) & MSI Aero GTX 1070 OC 8GB
16 GB RAM x2, 1TB HDD x2, 1TB SSD x2
Windows 10 Home Edition Version 10.0.18363 Build 18363, Oculus version 17, Quest Version 17
Fan Cooling by Zotac FireStorm - AfterBurner cause me problems.

frx089
Honored Guest
@Braydar I do not understand the privacy concerns, is not the same of having an android phone and a camera? don't you have a camera app on your phone? 

JeffNik
MVP
MVP
I just want to use the passthough background for an AR-like app ... I don't need accurate 3D data about my playspace, or even high quality video... I just want to see my real environment behind my 3D generated content that floats in the air in front of the user. In my opinion, the video distortion actually ADDs a "virtually generated" feel to the app. I understand the privacy concerns, but programmers are eventually going to get access to passthough video for AR apps on other hardware.

wuzp
Rising Star
Doesn't this forum have a "developer's side," with more qualified people that can answer the question?

Brizbane
Honored Guest

JeffNik said:

I just want to use the passthough background for an AR-like app ... I don't need accurate 3D data about my playspace, or even high quality video... I just want to see my real environment behind my 3D generated content that floats in the air in front of the user. In my opinion, the video distortion actually ADDs a "virtually generated" feel to the app. I understand the privacy concerns, but programmers are eventually going to get access to passthough video for AR apps on other hardware.



I'd just like to have a way to pause what I'm doing, go to pass through and take a sip of coffee, smoke a cig, etc.  Without having to take the headset off. 

wuzp
Rising Star

Brizbane said:
I'd just like to have a way to pause what I'm doing, go to pass through and take a sip of coffee, smoke a cig, etc.  Without having to take the headset off. 


Not sure if the cigar[ette] being sucked into the unit by the cooling fan will be very helpful [or not].