cancel
Showing results for 
Search instead for 
Did you mean: 

MS Kinect as Input Device

knchaffin
Explorer
I have some MS Kinect for Windows devices along with the SDK. So far I have only used and studied the sample programs, but from what I see, I think the Kinect could make a very capable gesture control input to Oculus apps. I think it could be programmed to recognize arm and hand positions and movements in 3D space.
22 REPLIES 22

LaurentMT
Honored Guest
Hi,

I'm also interested in investigating this kind of usage (Rift + Kinect).
I know Kinect is not very popular in the VR community because of the latency but I still want to test it farther.

I already started a project on the subject during my spare time. First results with Kinect alone are quite positive to my own opinion. Now, I have to wait for the arrival of my dev kit (surely not before may or june, since I'm a preorder) but I try to anticipate the eventual problems to do fusion of data coming from Kinect and Rift.

If I can get some good results, I'll share work and sources with everybody.

I'll keep you in touch.

Laurent

daloots
Honored Guest
Hi,

+1 on being interested.

I've done some stuff with the Kinect in the past. Latency has gotten better, but skeleton detection is still pretty slow. I think the depth map itself has much lower latency. Perhaps that could be used for relatively fast positional tracking in a restricted setup. Something like using the closest point as position.
I'm a final-day kickstarter backer from Europe, so will be waiting some time for my dev kit as well...

Cheers,
Davy

tbowren
Honored Guest
I recently got my Kinect working in Ogre, and my Ogre scenes to read the Oculus tracking data. Once I have the shader warping and viewport frustum adjustments working I will be able to put my oculus head on top of the kinect body. I am sure its horribly limited as to what I can do but I'm very anxious to try this out.

tobby
Honored Guest
This guy has already done it:

http://www.youtube.com/watch?v=R0-dsbeasgA

leelovell
Honored Guest
That video is cool.

So much potential.

Cheshyr
Honored Guest
That was amazing. It's not that far off either, except perhaps the omni-treadmill. He had some good points.

LaurentMT
Honored Guest
Thanks for the video tobby ! I've had already seen a few things done by this guy (okreylos) via the openkinect community and for sure he's a good source of inspiration for VR.

LaurentMT
Honored Guest
I just did a quick test of a virtual body in Unity 4 with body tracking done by kinect.
For this test, I've used this library. Basically, the library is a Unity wrapper for microsoft kinect sdk and it provides a class which wraps the skeleton tracking done by kinect.

The result is fun.

But to be honest this simple solution (kinect skeleton tracking) won't provide very good results in current state (instability in hands detection, no rotation of arms and hands, ...). Anyway, I'll pursue my tests.

I've also worked on a lean based navigation system using Kinect. The principle is quite similar to this demo using a wii balance board. Results seem quite promising but I still have to check with dev kit that it doesn't induce simulator sickness. :roll:

For now, my thoughts are that a good (and affordable) set could be :
- the rift dev kit for display and angular tracking of head,
- a razer hydra for positional tracking of head (I think I'll test this solution)
- a lean based navigation system :
    - wii balance board (cheaper)
    - kinect (more expensive but could provide some additional data for body tracking)
- additional razer hydra used as a hand tool.

knchaffin
Explorer
Michaeljohn, I have no idea what you are saying here.