cancel
Showing results for 
Search instead for 
Did you mean: 

Scan QR code with passthrough camera?

sean_yang0498
Honored Guest

Hello, Oculus community,

 

I am an internship student and I got a task to figure out if it's possible to get images from the passthrough cameras on Quest 2. The purpose is to scan a QR code for creating AR experiences. I don't have any development experience with Oculus and AR, but only some experience in Unity.

 

I googled it and saw some results. But most of them are posted months or even years before. So I am not sure how is it now in the latest update. I also noticed there are some project showcases that used the passthrough cameras for AR experiences, but none of them used the camera for scanning QR codes.

 

Is there a solution to my question? I really appreciate any help.

1 ACCEPTED SOLUTION

Accepted Solutions

kojack
MVP
MVP

Sadly this is not possible.

The Oculus SDK doesn't provide any direct access to the cameras, there is no way to get pass through video. (This is for privacy reasons, imagine an unethical VR dev getting a feed of your cameras)

 

The passthrough API that you would have seen (with cool games like Hauntify) doesn't actually give the passthrough feed to the game. It stays in the runtime. The game can control where passthrough or game gfx are rendered, but it can't see what was rendered (it's not sent to Unity as a texture).

 

I want fiducial marker (like QR code) recognition for large area tracking (to correct for drift and sync multiple headsets to the same tracking space) but the only way to do it is stick a webcam to the Quest and use that (software support is going to be tricky though).

 

Author: Oculus Monitor,  Auto Oculus Touch,  Forum Dark Mode, Phantom Touch Remover,  X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2

View solution in original post

14 REPLIES 14

james_griffin_ibm
Honored Guest

I am also looking for a solution to this in Unity3d. This would help enable enterprise cloud app authentication.

kojack
MVP
MVP

Sadly this is not possible.

The Oculus SDK doesn't provide any direct access to the cameras, there is no way to get pass through video. (This is for privacy reasons, imagine an unethical VR dev getting a feed of your cameras)

 

The passthrough API that you would have seen (with cool games like Hauntify) doesn't actually give the passthrough feed to the game. It stays in the runtime. The game can control where passthrough or game gfx are rendered, but it can't see what was rendered (it's not sent to Unity as a texture).

 

I want fiducial marker (like QR code) recognition for large area tracking (to correct for drift and sync multiple headsets to the same tracking space) but the only way to do it is stick a webcam to the Quest and use that (software support is going to be tricky though).

 

Author: Oculus Monitor,  Auto Oculus Touch,  Forum Dark Mode, Phantom Touch Remover,  X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2

Alright, thank you anyway for giving me a clear answer to this!

the privacy answer makes no sense, and certainly could be a setting that let's the user decide.  you're disabling functionality that would actually enable Hololens/Magic Leap type apps for no useful reason. 

Chetpih
Honored Guest

Hi! As an intern exploring the Oculus community, I understand the challenges of getting up to speed quickly on new technology. For your task, explore OCR (optical character recognition) technology and see if any solutions would allow you to scan QR codes using the passthrough cameras on Quest 2. I've heard promising things about a company called Smart Engines, which specializes in this area, but I recommend researching and checking out their latest updates to see if they have what you're looking for. Good luck with your internship project!

aniyatcervantes94
Honored Guest

Have you tried reaching out to the Oculus community or Unity forums for advice and tips on how to implement this?

kojack
MVP
MVP

As I said above, this is NOT possible. No access to the images from the passthrough camera is provided by the SDK.

Author: Oculus Monitor,  Auto Oculus Touch,  Forum Dark Mode, Phantom Touch Remover,  X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2

meredithtbeasley94
Honored Guest

It's possible to access the passthrough cameras on Quest 2 and use them for AR experiences.
As for scanning QR codes with the passthrough cameras, it's definitely worth looking into.
Also, consider integrating a document verification and scanning solution like https://www.idanalyzer.com/products/prime-id-scanner.html into your AR experience. It offers advanced features like ID verification, biometric facial recognition, and document scanning that could be a great fit for your project.

 

To read a QR code, the app needs access to the image data coming from the cameras.

The passthrough API does not provide app access to the image data coming from the cameras.

 

What the passthrough API does is allow you to add a passthrough layer to the rendering stack, then adjust the transparency of the layers to control where the passthrough appears. You can render VR stuff on top of the passthrough (like the guardian setup). You can render passthrough on top of VR (like a window into the real world). What you can't do is actually access any pixel of the passthrough to do things like QR code processing or OCR. All 3rd party AR apps for Quests (1,2,Pro) just use these features. Meta themselves can access the data to do things like detecting certain keyboards, but 3rd party developers can't do that.

 

 

Author: Oculus Monitor,  Auto Oculus Touch,  Forum Dark Mode, Phantom Touch Remover,  X-Plane Fixer
Hardware: Threadripper 1950x, MSI Gaming Trio 2080TI, Asrock X399 Taich
Headsets: Wrap 1200VR, DK1, DK2, CV1, Rift-S, GearVR, Go, Quest, Quest 2, Reverb G2