New to the forums? Click here to read the How To guide. -- Developers click here.

What Can the DK2 IR Camera See?

InteriorDInteriorD Posts: 83
edited November 2014 in General
Hello OculusVR,

I'm a DK2 owner who would like to know what sort of images the DK2 camera could see. A picture of what the DK2 can 'see' would be super helpful. This is for security purposes. I'm sure we would all feel a bit more secure if we know for a fact that the DK2 camera can't see the pudgy/scrawny/well-toned bodies under our clothing while we're standing around with our mouths agape.

On a side note, I'm trying to strap the Leap Motion Controller onto the DK2 but despite listening to the community's advice, I'm unable to locate the IR light emitters on the DK2. It would be very helpful if you can let me know where they are so I won't obstruct it.

Thanks in advance.
«134

Comments

  • achachiachachi Posts: 43
    edited July 2014
    Open up you're webcam and look at it through that. you'll see the Ir emitters lit up.

    But you do make a good point about privacy, I'm sure someone from Oculus could clear it up, because I guess they don't want to build an archive of us in our underwear and flabby waistlines. There's one or two with the body of an Adonis , me included ;), that might tickle their fancy, but I would prefer to be in the knowledge that camera data is not being transferred away from the computer.
  • InteriorDInteriorD Posts: 83
    achachi wrote:
    Open up you're webcam and look at it through that. you'll see the Ir emitters lit up.

    I don't have a webcam... the only cameras I have are the ones on my iPad and iPhone
  • DreamwriterDreamwriter Posts: 1,101
    3Jane
    It can only see light sources, nothing else, otherwise it would be useless. It needs to track the position of the IR lights it sees, and for that to be fast and simple, it should only see them as white dots. Anything else and it would have to interpret the entire scene, which would require heavy computational power and tricky programming. The Wii Remote does the exact same thing (that's how it senses where you are pointing it/if you are moving it towards/away from the screen), and its calibration screen actually shows what it sees, it translates every light source into white dots of varying sizes based on brightness.
  • DeVisioNDeVisioN Posts: 40
    Total speculation on my side, but anyways.

    I think raw camera output is something like this, but with a bit more points.
    65764-208706-calibratejpg-620x.jpg

    To keep latency down, Oculus devs most likely moved a lot of image filtering to the camera, so only needed and relevant info goes down the cable. There shouldn't be any real images going to the rendering pipe.

    About IR emitters on DK2:
    crystal_cove_front.png

    See those white dots? These are IR emitters, but on final DK2 version they are covered for aesthetic and protective purposes.

    EDIT: off-topic, but how do you resize images in the post?
    Think about today and tomorrow. There is nothing you can do about the past.
  • InteriorDInteriorD Posts: 83
    DeVisioN wrote:
    Total speculation on my side, but anyways.

    I think raw camera output is something like this, but with a bit more points.
    65764-208706-calibratejpg-620x.jpg

    To keep latency down, Oculus devs most likely moved a lot of image filtering to the camera, so only needed and relevant info goes down the cable. There shouldn't be any real images going to the rendering pipe.

    About IR emitters on DK2:
    crystal_cove_front.png

    See those white dots? These are IR emitters, but on final DK2 version they are covered for aesthetic and protective purposes.

    EDIT: off-topic, but how do you resize images in the post?

    Thanks for the detailed reply. I'm wondering more about the potential ability of the camera rather than what it's used for. I'm probably just being unnecessarily paranoid though.

    I don't know how to resize images, sorry.
  • DragonbaitDragonbait Posts: 40
    edited July 2014
    It's an infrared camera, all it can see are infrared light sources.
    If you look at the photos you will see it doesn't even have a clear lens on the front, it has an opaque white cover specifically to prevent visible light (other than IR) getting through.
    DeVisions photo from the Wii setup is exactly what this camera sees.

    IMG_3335-640x426.jpg
  • bp2008bp2008 Posts: 256
    Hiro Protagonist
    I'm also sure it is just like the wii's camera, although perhaps higher resolution so it can be more precise. The goal of this camera was to obtain low latency, high reliability tracking data, and that means they want a black frame with little light dots that they can easily and reliably track. So most likely the camera has filters to block all light wavelengths except that which the LEDs on the DK2 use (which is most likely 850nm as it is outside the spectrum of human-visible light, yet most camera sensors are still fairly sensitive to this). Even then, sunlight includes a lot of infrared at this wavelength, and I'm sure they accommodated for this by increasing the shutter speed and reducing sensitivity to minimize the effect of this light pollution.

    So in the end, the camera most likely is tuned so that it sees only the LEDs on the DK2, and as little else as possible.
  • menionemenione Posts: 45
    I too would like confirmation on this. I could imagine that if your system got hacked that the hacker could potentially gain access to all hardware, including the use of the camera. So it would be nice to get confirmation of what the camera is capable of doing.
  • InteriorDInteriorD Posts: 83
    edited July 2014
    Dragonbait wrote:
    It's an infrared camera, all it can see are infrared light sources.
    If you look at the photos you will see it doesn't even have a clear lens on the front, it has an opaque white cover specifically to prevent visible light (other than IR) getting through.
    DeVisions photo from the Wii setup is exactly what this camera sees.

    I'm aware of that since I sit in front of it all day and it's always pointed at me. The "cover" you're talking about is the semi-translucent reflective coating (I can faintly make out what's behind the coating). The wavelengths of the visible spectrum should be able to reach the sensor in decent quantities since there's a stronger light source in front of the coating than behind it.

    Whether or not the camera can make use of this input is what I'm trying to figure out in this thread.

    Infrared is right next to the visible spectrum. Because of that, even without an infrared-specific light source, everything we can see should be visible to an infrared camera under regular light (maybe with a lot of blooming, but still visible none-the-less). The DK2 camera's threshold for filtering might have been set artificially high for low-latency when processing the input, but that doesn't mean that the camera is unable to "see" with some degree of normality.

    menione wrote:
    I too would like confirmation on this. I could imagine that if your system got hacked that the hacker could potentially gain access to all hardware, including the use of the camera. So it would be nice to get confirmation of what the camera is capable of doing.

    This is what I'm worried about. I know for a fact that there are hackers out there capable of accessing hardware such as microphones and webcams, and that they're also capable of disconnecting the indicator light from activation.

    Edit: Massive rework of post for grammar/legibility. Consider this v2.0
  • mungewellmungewell Posts: 51
    With more and more cameras pointing at 'us', I think that you are right to be concerned about what the camera can see. Perhaps Oculus should address/comment on this...

    First step to finding out ourselves is to get the USB-IDs and USB report, to see if this conforms to a known camera device. If the camera uses specific Oculus USB-ID this might not be too easy, but under Linux you can re-map IDs so that common camera modules recognize it.

    I would suspect that it's a standard camera chip, and that the dot tracking is done in software on PC. However it is possible that tracking is performed in camera chip (same as Wii-Mote).
    Simon.
  • InteriorDInteriorD Posts: 83
    mungewell wrote:
    With more and more cameras pointing at 'us', I think that you are right to be concerned about what the camera can see. Perhaps Oculus should address/comment on this...

    First step to finding out ourselves is to get the USB-IDs and USB report, to see if this conforms to a known camera device. If the camera uses specific Oculus USB-ID this might not be too easy, but under Linux you can re-map IDs so that common camera modules recognize it.

    I would suspect that it's a standard camera chip, and that the dot tracking is done in software on PC. However it is possible that tracking is performed in camera chip (same as Wii-Mote).
    Simon.

    I'd love to do this if I had the technical proficiency....
  • AuxAux Posts: 8
    I'll save you the trouble!

    As long as the IR filter is in place, the camera cannot see visible light. Most light though- sun, fluorescent, etc- has an IR component to it, so a normal camera feed + lens would look a little like white night vision (think Ghost Hunters). So we drop the exposure on the camera very, very, very low until all that comes through is extremely bright point sources like our LEDs. It would also probably pick up the sun if you pointed the camera directly at it.

    Here is a screenshot of one of our internal tools (IIRC this was from Crystal Cove):

    pXMKOLa.png?2

    This was taken in a very bright office setting. Note that you can't see me at all, and I am standing next to the headset holding it up with one hand. The point off to the side of the headset is the reflection of an LED off of probably my glasses, and the streak on the top left is a fluorescent office light.

    TL;DR: The camera can't see you unless you are standing on the sun, and if that is the case you have probably melted so the point is moot.
  • InteriorDInteriorD Posts: 83
    Aux wrote:
    I'll save you the trouble!

    As long as the IR filter is in place, the camera cannot see visible light. Most light though- sun, fluorescent, etc- has an IR component to it, so a normal camera feed + lens would look a little like white night vision (think Ghost Hunters). So we drop the exposure on the camera very, very, very low until all that comes through is extremely bright point sources like our LEDs. It would also probably pick up the sun if you pointed the camera directly at it.

    Here is a screenshot of one of our internal tools (IIRC this was from Crystal Cove):

    pXMKOLa.png?2

    This was taken in a very bright office setting. Note that you can't see me at all, and I am standing next to the headset holding it up with one hand. The point off to the side of the headset is the reflection of an LED off of probably my arm, and the streak on the top left is a fluorescent office light.

    TL;DR: The camera can't see you unless you are standing on the sun, and if that is the case you have probably melted so the point is moot.

    An official response! Thanks for taking the time to address this question, Aux!

    You mentioned that you've reduced the exposure of the camera. Unless this was a hardware reduction, it still leaves open the question of whether someone could make the software increase the exposure, thus turning the DK2 camera into some sort of IR-based webcam.

    P.S. (Beginner at programming here) Since I'm working on strapping a Leap Motion Controller to my DK2, it might be very helpful if I can gain access to the camera tool you used to take the above picture so I could efficiently tweak the LMC IR output. Is that a possibility?
  • mungewellmungewell Posts: 51
    Thanks for the official response, and confirming that it's a 'true' imaging device.

    From another posting I see that the USBID is Oculus specific ('VID_2833&PID_0201'), so this is likely not (yet) recognized by any Linux drivers. The most 'common' drivers are UVC or the GSPCA driver (which supports a lot of driver/sensor combinations).

    http://linuxtv.org/wiki/index.php/UVC_Webcam_Devices
    http://linuxtv.org/wiki/index.php/Gspca_devices

    Any chance we can get confirmation that one of these works/is suitable for the camera?
    Simon
  • lazydodolazydodo Posts: 76
    in the inf there's a second vid/pid pair (VID_1E4E&PID_0109) which appears to be some kind of webcam, when using those drivers i was able to get this out

    qfbjp7Ql.jpg
  • InteriorDInteriorD Posts: 83
    lazydodo wrote:
    in the inf there's a second vid/pid pair (VID_1E4E&PID_0109) which appears to be some kind of webcam, when using those drivers i was able to get this out

    qfbjp7Ql.jpg

    Well done! That's enough to make me keep it covered with a post-it.
  • ChaossChaoss Posts: 276
    Hiro Protagonist
    When I accessed the camera (one of the first things I tried after plugging it in) I got a similar image however it was pretty much solid green even with my lights on or off with only a few weird artifacts as you see in the left side of the image posted. I was also unable to access the camera without the little blue LED coming on which may offer you some comfort.
    "In the future we will be designing dreams, worlds & experiences rather than games."
  • racerx3racerx3 Posts: 160
    lazydodo wrote:
    in the inf there's a second vid/pid pair (VID_1E4E&PID_0109) which appears to be some kind of webcam, when using those drivers i was able to get this out

    qfbjp7Ql.jpg

    If you indeed got this image from the DK2's camera, I agree that this is a privacy liability that at the very least needs to be emphasized officially.
  • mungewellmungewell Posts: 51
    Chaoss wrote:
    I was also unable to access the camera without the little blue LED coming on which may offer you some comfort.

    In most cases the LED is driven by a GPIO from the Camera's controller chip, and this is just activated by the driver or firmware when the camera is set into streaming mode. Obviously the driver/firmware could be 'tinkered' with to prevent this happening. Whether this is a real concern is yet to be determined (although some would be especially concerned given the ownership of Oculus by Facebook).

    That other USBID appears to be a camera from Etron Technology.
    http://sourceforge.net/p/linux-uvc/mailman/message/32461880/

    Another post on the forums pointed to a presentation which explains how Oculus uses the camera to sense/compute position. They seem to use only video (ie. no other data stream) from camera.
    https://www.youtube.com/watch?v=dxbh-TM5yNc#t=27m

    Simon.
  • lazydodolazydodo Posts: 76
    mungewell wrote:
    That other USBID appears to be a camera from Etron Technology.
    I used their drivers to get the image, can confirm they work after you mess around with the .inf files a bit, naturally breaking positional tracking until you restore the oculus ones.
  • InteriorDInteriorD Posts: 83
    lazydodo wrote:
    mungewell wrote:
    That other USBID appears to be a camera from Etron Technology.
    I used their drivers to get the image, can confirm they work after you mess around with the .inf files a bit, naturally breaking positional tracking until you restore the oculus ones.

    Maybe we should hold a competition to see who could turn the DK2 camera into a functional webcam? :P
  • arconarcon Posts: 97
    InteriorD wrote:
    lazydodo wrote:
    mungewell wrote:
    That other USBID appears to be a camera from Etron Technology.
    I used their drivers to get the image, can confirm they work after you mess around with the .inf files a bit, naturally breaking positional tracking until you restore the oculus ones.

    Maybe we should hold a competition to see who could turn the DK2 camera into a functional webcam? :P


    Here: 1gMRuBRcMLtDqq1W.medium

    Fully Functional Webcam. Just need the Software:

    Source: Step 20
    Signature_zps6b6399d4.png
    DK2 Info: Order ID: FM-1042XXX | Date: Mar 19, 2014 09:53 AM PDT | Status: Delivered (04.08)
  • Freebirth001Freebirth001 Posts: 73
    Brain Burst
    InteriorD wrote:
    lazydodo wrote:
    in the inf there's a second vid/pid pair (VID_1E4E&PID_0109) which appears to be some kind of webcam, when using those drivers i was able to get this out

    qfbjp7Ql.jpg

    Well done! That's enough to make me keep it covered with a post-it.

    seriously? i mean isn't that a little paranoid? i understand security concerns. but not only would a person with malicious intent gain access to your computer, they would then have to gain access to the camera, assuming it is attached. and then they would see hardly anything, and virtually nothing and if you have your window blinds down literally nothing. its not getting a clear picture, even if it did IR doesnt see through clothing or anything like that. i mean this is even less that the "security concerns" people raise with the xbox one and its camera. when there hasnt been a single succsesfull attempt at hacking one yet. it just seems like a lot of effort for almost no gaim for the person with malicious intent. when your chat logs would be far more valuable, and easier, for someone to access.
  • mrgreen72mrgreen72 Posts: 683
    Art3mis
    edited July 2014
    InteriorD wrote:
    Well done! That's enough to make me keep it covered with a post-it.
    I'm sure our good friends at Oculus have humanity has better to do than watch you jerk off on the internet. ;)
  • menionemenione Posts: 45
    seriously? i mean isn't that a little paranoid? i understand security concerns. but not only would a person with malicious intent gain access to your computer, they would then have to gain access to the camera, assuming it is attached. and then they would see hardly anything, and virtually nothing and if you have your window blinds down literally nothing. its not getting a clear picture, even if it did IR doesnt see through clothing or anything like that. i mean this is even less that the "security concerns" people raise with the xbox one and its camera. when there hasnt been a single succsesfull attempt at hacking one yet. it just seems like a lot of effort for almost no gaim for the person with malicious intent. when your chat logs would be far more valuable, and easier, for someone to access.

    Malicious intent to gain access to your computer is always a threat, this is not a far-fetched scenario at all. And if they had access to your computer it's pretty easy to figure out you have a camera attached.

    The xbox security concern is different. Most of the fuss raised with that was with people thinking Microsoft would spy on them. The fact that xbox is in a walled garden environment protects it from a lot of hacking attempts. This is not the case with a PC.
  • InteriorDInteriorD Posts: 83
    InteriorD wrote:
    lazydodo wrote:
    in the inf there's a second vid/pid pair (VID_1E4E&PID_0109) which appears to be some kind of webcam, when using those drivers i was able to get this out

    qfbjp7Ql.jpg

    Well done! That's enough to make me keep it covered with a post-it.

    seriously? i mean isn't that a little paranoid? i understand security concerns. but not only would a person with malicious intent gain access to your computer, they would then have to gain access to the camera, assuming it is attached. and then they would see hardly anything, and virtually nothing and if you have your window blinds down literally nothing. its not getting a clear picture, even if it did IR doesnt see through clothing or anything like that. i mean this is even less that the "security concerns" people raise with the xbox one and its camera. when there hasnt been a single succsesfull attempt at hacking one yet. it just seems like a lot of effort for almost no gaim for the person with malicious intent. when your chat logs would be far more valuable, and easier, for someone to access.

    I understand how it seems, but keep in mind that the camera pictures in this thread from were captured with minimal tweaking.

    The more widespread a technology becomes, the more vested interest there is in creating "unauthorized-use solutions" for it. Yes, it may seem extremely unlikely that anyone would bother hacking the DK2 camera at this point in time, but when the technology matures and there's widespread adoption, it becomes exponentially more likely. It's basically the market-share problem packaged differently (# of malware for Windows vs OS X).

    I think that the Kinect hasn't reached the point where it's widespread enough to justify pouring time into properly hacking it, and might never. The Rift, though, is much more versatile and much more likely to pass that point. This is just preparation.

    Also, it may be easier to access chat logs, but keep in mind that chat logs and visual images contain completely different kinds of information.
  • InteriorDInteriorD Posts: 83
    mrgreen72 wrote:
    InteriorD wrote:
    Well done! That's enough to make me keep it covered with a post-it.
    I'm sure our good friends at Oculus have better to do than watch you jerk off on the internet. ;)

    I never said that OVR/FB will be responsible for the hacking. Why would they? I'm sure they have more lucrative and rewarding things to do, as you said.

    Your statement is sort of like saying Sony (or any other laptop manufacturer) is directly responsible for capturing images of high school teenagers undressed through their webcam in this actual case: http://en.wikipedia.org/wiki/Robbins_v._Lower_Merion_School_District

    They're just the ones who provided the tools. It's the people maliciously using them we should be wary of.
  • Freebirth001Freebirth001 Posts: 73
    Brain Burst
    menione wrote:
    seriously? i mean isn't that a little paranoid? i understand security concerns. but not only would a person with malicious intent gain access to your computer, they would then have to gain access to the camera, assuming it is attached. and then they would see hardly anything, and virtually nothing and if you have your window blinds down literally nothing. its not getting a clear picture, even if it did IR doesnt see through clothing or anything like that. i mean this is even less that the "security concerns" people raise with the xbox one and its camera. when there hasnt been a single succsesfull attempt at hacking one yet. it just seems like a lot of effort for almost no gaim for the person with malicious intent. when your chat logs would be far more valuable, and easier, for someone to access.

    Malicious intent to gain access to your computer is always a threat, this is not a far-fetched scenario at all. And if they had access to your computer it's pretty easy to figure out you have a camera attached.

    The xbox security concern is different. Most of the fuss raised with that was with people thinking Microsoft would spy on them. The fact that xbox is in a walled garden environment protects it from a lot of hacking attempts. This is not the case with a PC.

    yes someone gaining access to your computer is a threat. however it is a threat that already exists. because ithe dks camera is not a low band IR camera (it is unable to see through most materials) as opposed to ones used in airport screening booths. and with the addition of the visible light filter and the lack of its own infrared source to illuminate the area. the dk2 camera just doesnt seem like a target for someone who is intentionally getting into your computer. not when it would be more profitable to go after your web history, install a keylogger to get your passwords, or if they are specifically going after you. your chat history to try and find something embarasing.
  • menionemenione Posts: 45
    yes someone gaining access to your computer is a threat. however it is a threat that already exists. because ithe dks camera is not a low band IR camera (it is unable to see through most materials) as opposed to ones used in airport screening booths. and with the addition of the visible light filter and the lack of its own infrared source to illuminate the area. the dk2 camera just doesnt seem like a target for someone who is intentionally getting into your computer. not when it would be more profitable to go after your web history, install a keylogger to get your passwords, or if they are specifically going after you. your chat history to try and find something embarasing.
    Yes, I doubt that someone would target you specifically because you have a DK2 camera, but if you get hacked you should be aware of this possibility. It's just something should be aware of, just like when you buy a normal webcam.

    And you are talking about getting access to password history and access to your webcam as two mutually exclusive things, but I'm sure a hacker could easily access both if they're already in your system.
  • InteriorDInteriorD Posts: 83
    Reddit user MaglevNL posted a picture he captured on his DK2 camera through his Skype when it's sunny outside and the lights are off.

    FjwUvJ8.jpg

    http://www.reddit.com/r/oculus/comments/2c8paq/this_is_what_the_dk2_camera_can_see/

    Edit: and with the lights on

    KCZXhhw.jpg
«134
Sign In or Register to comment.