New to the forums? Click here to read the "How To" Guide.

Developer? Click here to go to the Developer Forums.

Did Oculus just standardize 5-finger tracking and make hand-controllers obsolete??

ZenbaneZenbane Posts: 15,148 Valuable Player
edited September 2019 in General
One of my favorite announcements from Facebook and Oculus today is the new controller-free hand tracking.




This is quite amazing since it gives us 5-finger tracking while removing any bulky controllers from our hands entirely, thus allowing us to use our mouse, keyboard, and interact with any other real world objects while keeping our hands and fingers tracked. An added bonus is the way this works seamlessly with Passthrough technology; and of course the future we will see with Facebook's vision for Augmented Reality and Mixed Reality.

The only downside (aside from perfecting tracking accuracy) is the lack of haptic feedback. However, I never cared for haptic feedback at all (it adds battery drain and doesn't accurately reflect what happens in the real world!) so it's a worthy sacrifice considering the huge benefits and possibilities.

And this technology is going to be open-sourced. So... is this the new universal path forward?
Are you a fan of the Myst games? Check out my Mod at http://www.mystrock.com/
Catch me on Twitter: twitter.com/zenbane
«1

Comments

  • Protocol7Protocol7 Posts: 333
    Trinity
    Controllers still have a place for many games where you use thumbsticks for locomotion or triggers for shooting etc. The tactile experience of holding something is often desirable. I think haptic feedback is valuable. But I very much look forward to using the finger tracking.

  • ZenbaneZenbane Posts: 15,148 Valuable Player
    Protocol7 said:
    Controllers still have a place for many games where you use thumbsticks for locomotion or triggers for shooting etc. The tactile experience of holding something is often desirable. I think haptic feedback is valuable. But I very much look forward to using the finger tracking.


    It seems like we can just use any real world object to function as a gun, sword, shield, etc. Grab a Nerf gun! For example. No more using duct tape and pipes to create makeshift weaponry all for the sake of accommodating hand controllers. Now we can use any real world representation.

    And for those who need haptic feedback, we could likely see that in the form of VR Gloves soon as well.
    Are you a fan of the Myst games? Check out my Mod at http://www.mystrock.com/
    Catch me on Twitter: twitter.com/zenbane
  • nalex66nalex66 Posts: 5,431 Volunteer Moderator
    This is a very impressive and interesting development, although some games will still benefit from physical controllers (locomotion input).

    I can see many games working really well with just hands, though. They could come up with UI methods like tapping your wrist to open a menu, allowing more detailed interaction without needing physical buttons. The possibilities are endless!
    i7 5820K @ 4.25GHz | EVGA GTX 1080 SC | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
    SSDs: Intel 660p M.2 2TB, Samsung 860 Evo 1TB, 850 Evo 1TB, 840 Evo 1TB | Startech 4 controller PCIe USB 3.0
  • inovatorinovator Posts: 2,270 Valuable Player
    Personally I love haptic in the controls. Especially for palvov and gun games. If given a choice I would use whatever is best for the games. I hope they figure how to have it on the hand tracking
  • snowdogsnowdog Posts: 7,493 Valuable Player
    Well it's going to have a lot of uses for non-gaming applications but the vast majority of gamers will ALWAYS prefer to hold something in their hands and use sticks for locomotion.

    The gloves that were on Tested years ago that provided not only haptic feedback but also physically stopped your fingers from moving when you gripped things would be useful but even then you'd still need some way of having players move during gameplay.
    "This you have to understand. There's only one way to hurt a man who's lost everything. Give him back something broken."

    Thomas Covenant, Unbeliever
  • inovatorinovator Posts: 2,270 Valuable Player
    snowdog said:
    Well it's going to have a lot of uses for non-gaming applications but the vast majority of gamers will ALWAYS prefer to hold something in their hands and use sticks for locomotion.

    The gloves that were on Tested years ago that provided not only haptic feedback but also physically stopped your fingers from moving when you gripped things would be useful but even then you'd still need some way of having players move during gameplay.
    Agreed
  • dburnedburne Posts: 3,523 Valuable Player
    Yeah there will be a place for controllers like Touch for some time to come.
    Hand tracking will be pretty neat for certain experiences/games designed for it.
    Don

    EVGA Z390 Dark MB | I9 9900k| EVGA 2080Ti FTW3 Ultra |32 GB G Skill 3200 cl14 ram | Warthog Throttle | VKB Gunfighter Pro/MCG Pro grip | Crosswind Pedals | EVGA DG 87 Case| Rift S | Quest |
  • MorgrumMorgrum Posts: 1,712 Valuable Player
    Whelp here comes hand tracked VR Porn mark my words!
    WAAAGH!
  • SpuzzumSpuzzum Posts: 843
    3Jane
    Protocol7 said:
    Controllers still have a place for many games where you use thumbsticks for locomotion or triggers for shooting etc. The tactile experience of holding something is often desirable. I think haptic feedback is valuable. But I very much look forward to using the finger tracking.

    Completely agree...until they figure out a way to make it feel like you're actually holding an object without actually holding an object. If their haptic glove could do that, even feel like you're gripping a pistol, then that changes everything. Except for their being no weight to the object.
  • SpuzzumSpuzzum Posts: 843
    3Jane
    Morgrum said:
    Whelp here comes hand tracked VR Porn mark my words!
    At least now you can point and select in the stickiest of situations. :P
  • MowTinMowTin Posts: 1,916 Valuable Player

    I would love this in a flight sim where I can use my hands to flip switches and turn dials while holding a joystick. 

    i7 9700k 2080ti   CV1, Rift-S, Index
  • KenSniperKenSniper Posts: 192
    Art3mis
    I hope it comes to the Rift S as well, they technically use the same tracking tech right? 
  • inovatorinovator Posts: 2,270 Valuable Player
    Not announcing it for the rift as well is treating the quest like it's the main dog. Oh wait it is now. I'm ok with that.
  • SpuzzumSpuzzum Posts: 843
    3Jane
    There's still another day of OC6 to go, but it seems it's all about the Quest as well.

    https://www.oculusconnect.com/#schedule
  • ZenbaneZenbane Posts: 15,148 Valuable Player
    edited September 2019
    Question for those who responded so far,

    What if the Hand Controller in games is purely virtual?

    Do we really need a physical controller if we can simply conjure up a virtual controller?
    Are you a fan of the Myst games? Check out my Mod at http://www.mystrock.com/
    Catch me on Twitter: twitter.com/zenbane
  • inovatorinovator Posts: 2,270 Valuable Player
    Zenbane said:
    Question for those who responded so far,

    What if the Hand Controller in games is purely virtual?

    Do we really need a physical controller if we can simply conjure up a virtual controller?
    Maybe not. Maybe move a finger or hand forward or backward or sideways for locomotion.  I'm sure it could be done. Risking sounding like a broken record for shooting games I'd like haptics built in. Maybe they will be. I dont remember them saying they wont be.
     
  • nalex66nalex66 Posts: 5,431 Volunteer Moderator
    From what I've read of hands-on accounts (heh), it's not as fast and accurate as Touch controllers, so some games might work fine without controllers, while others will be better with.
    i7 5820K @ 4.25GHz | EVGA GTX 1080 SC | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
    SSDs: Intel 660p M.2 2TB, Samsung 860 Evo 1TB, 850 Evo 1TB, 840 Evo 1TB | Startech 4 controller PCIe USB 3.0
  • kojackkojack Posts: 6,414 Volunteer Moderator
    edited September 2019
    This is definitely very cool.

    I've had a Leap Motion for years. I've got a Leap mount on both my DK2 and CV1. I've got a couple of Leaps at work too. The hand tracking is great (after the Orion update), all 5 fingers tracked really well with very wide FOV. There's even SteamVR support for using Leap Motion instead of controllers to control current games.
    Some hand tracking experiences are awesome.
    However it is absolutely not a complete replacement for good physical controllers. A great addition for certain experiences, but I'd hate to give up things like touch.

    One area that interests me with hand tracking is calibration of physical game controls. For example playing a driving game with a real wheel. The problem is games may (well, will) place the player and the in game wheel at positions that don't match your real controller. By using the hand tracking, you can calibrate the actual wheel position and adjust the game to match, so there's a 1:1 match.
    Here was a video I did as a test.
    Imagine if the game asked you to grab the wheel at the 9 and 3 positions, then it moved the in game wheel to match. To avoid the occlusion glitches, you could detect proximity to the wheel and morph to a grip pose.

    Remember that currently Quest is using high speed IMU data to compensate for the slower camera tracking and occlusion issues, plus direct input controls. With hand tracking there's no button presses or motion at all when not in view.



  • inovatorinovator Posts: 2,270 Valuable Player
    For a moment when hand tracking was anounced I was thinking replacement. after some member posts it makes me think its just going to be additional for certain games.
  • SpuzzumSpuzzum Posts: 843
    3Jane
    The biggest issue I can think of, is tracking when they're out of sight of the cameras. It's hard enough with controllers when they're behind your head, and they have all that 3DoF tech inside, and can guesstimate when the cameras lose sight of them. If you paired that with haptic gloves that can still be tracked, then that would work.
  • kojackkojack Posts: 6,414 Volunteer Moderator
    Spuzzum said:
    The biggest issue I can think of, is tracking when they're out of sight of the cameras. It's hard enough with controllers when they're behind your head, and they have all that 3DoF tech inside, and can guesstimate when the cameras lose sight of them. If you paired that with haptic gloves that can still be tracked, then that would work.
    Yep, for example in Roborecall you wouldn't be able to reach over your shoulder and grab a shotgun, because grabbing must be done in camera view.

    I really liked this Leap Motion demo:
    I put some non VR people through it, they picked it up very fast and had a lot of fun.

    This will be great for things like virtual cockpit controls in simulators. But I'd still want a HOTAS and rudder pedals.

  • SpuzzumSpuzzum Posts: 843
    3Jane
    edited September 2019
    Also, besides the hand being visible, they need a way to register you actually grabbed something when out of view. They did mention that wristband doohickey thingamabob that interacts with the muscles, but that's still a few years away. If that thing can make it seem like you're actually gripping something, that would be cool.
  • bigmike20vtbigmike20vt Posts: 4,153 Valuable Player
    edited September 2019
    MowTin said:

    I would love this in a flight sim where I can use my hands to flip switches and turn dials while holding a joystick. 

    indeed. One of my main concerns for when elite dangerous ever gets space legs is how the transition will happen between being in a cockpit to 1st person in VR.
    I had resigned myself to just having my xbox pad on hand for getting out of my ship but something like this could work really well..... of course ED is not a pure VR game so there is no guarantee that such controllers would be supported, but it is on the oculus store so there is hope.

    also means NMS can have the best of all worlds as well... proper hotas and all the motion stuff as well (although i must admit the current touch implementation is very good already)

    most games that i play i suspect hand tracking would be an extra however on top of having a controller.  i do just like having something in my hand to play with..... but again this could just be a for instance plastic toy gun  which i could pick up when getting out of my flight seat and letting go of my hotas.

    It does need to work well however, it was not so long ago microsoft were advertising the same thing with kinect....... I used kinect and as a novelty it was ok, but there was no way it was able to replace a traditional controller.
    Fiat Coupe, gone. 350Z gone. Dirty nappies, no sleep & practical transport incoming. Thank goodness for VR :)
  • WildtWildt Posts: 2,264 Valuable Player
    Unless Oculus' finger tracking tech is A LOT better than Leap Motion, you guys might be disappointed. Several updates after Orion, it was still so poor that I just gave it away.
    PCVR: CV1 || 4 sensors || TPcast wireless adapter || MamutVR Gun stock V3
    PSVR: PS4 Pro || Move Controllers || Aim controller
    WMR: HP Reverb
  • ElusiveMarlinElusiveMarlin Posts: 293
    Nexus 6

    I am loving the advances made in this area, and feel that this does indeed have a place, but feel it is a very niche place.

    I feel that most users will want/need to "feel" like they are holding an item, especially in FPS/Racing/FlightSim games. (Which covers the majority!)

    Giving them the ability to use anything as an object in game is great, but I wonder how well the tracking will cope once the “everyday item” occludes the fingers from the sensors.  Will slight finger twitches for trigger pulling be noticed? for example.

    Will hand placement match where the real-world hands are and if not, the immersion will be broken.





    RIFT CV1
    RIFT S
    Leeds, UK.

    Intel Core i7 7700K 3.7GHz | RAM 32GB DDR4 2666MHz | GeForce GTX 1080 8GB |

    "Behind every mask there is a face, and behind that a story...."
  • Shadowmask72Shadowmask72 Posts: 4,072 Valuable Player
    edited September 2019
    Yep, I had Leap for a while stuck to my Rift and it was nice tech but not something I would use often. For specific actions though very nice. In a way, reminds me a little of Microsoft's Kinect where you can do hand motions to control what's happening. Hopefully FB provides enough software incentives to make use of it beyond it being a novelty action.  I mean right now look at the Index knuckles there isn't really any decent application that requires finger movements. Other than typing, playing a musical instrument, playing chess and making gestures, what real world application is there moving your fingers around translated to a game/VR experience?


    System Specs: RTX 2080 ti , i9 9900K CPU, 16 GB DDR 4 RAM, Win 10 64 Bit OS.
  • kojackkojack Posts: 6,414 Volunteer Moderator
    One of the attendees wrote this on twitter:
    After playing with the Quest’s hand tracking at #OC6:
    I’d put it at ~60% Leap Motion’s fidelity
    ~200ms latency was most distracting
    Finger occlusion was comparable
    Pinch interactions very solid
    Any fast movements immediately freeze at the hand model’s last known location

    Given that it’s an early-2020 beta there’s time to continue honing Most of the affordances in the mad scientist’s lair demo were pinch or poke interactions, with no graspable objects, so I can’t speak to more active manipulations

    The tracking volume was sufficiently wide, reaching the edge of the FOV fine On a couple of occasions the hand models swapped left and right hands facing backwards which happens on Leap v4 as well






  • dburnedburne Posts: 3,523 Valuable Player
    MowTin said:

    I would love this in a flight sim where I can use my hands to flip switches and turn dials while holding a joystick. 

    Yeah that would be very cool in DCS.
    One can already use Touch to manipulate switches, using straight hands would seem awesome.
    But yeah, I can't see myself losing my joystick hardware anytime soon. 
    Don

    EVGA Z390 Dark MB | I9 9900k| EVGA 2080Ti FTW3 Ultra |32 GB G Skill 3200 cl14 ram | Warthog Throttle | VKB Gunfighter Pro/MCG Pro grip | Crosswind Pedals | EVGA DG 87 Case| Rift S | Quest |
  • WildtWildt Posts: 2,264 Valuable Player
    kojack said:
    One of the attendees wrote this on twitter:
    After playing with the Quest’s hand tracking at #OC6:
    I’d put it at ~60% Leap Motion’s fidelity
    ~200ms latency was most distracting
    Finger occlusion was comparable
    Pinch interactions very solid
    Any fast movements immediately freeze at the hand model’s last known location

    Given that it’s an early-2020 beta there’s time to continue honing Most of the affordances in the mad scientist’s lair demo were pinch or poke interactions, with no graspable objects, so I can’t speak to more active manipulations

    The tracking volume was sufficiently wide, reaching the edge of the FOV fine On a couple of occasions the hand models swapped left and right hands facing backwards which happens on Leap v4 as well






    Someone still has a lot of work to do apparently :)
    PCVR: CV1 || 4 sensors || TPcast wireless adapter || MamutVR Gun stock V3
    PSVR: PS4 Pro || Move Controllers || Aim controller
    WMR: HP Reverb
  • MradrMradr Posts: 3,753 Valuable Player
    kojack said:
    One of the attendees wrote this on twitter:
    After playing with the Quest’s hand tracking at #OC6:
    I’d put it at ~60% Leap Motion’s fidelity
    ~200ms latency was most distracting
    Finger occlusion was comparable
    Pinch interactions very solid
    Any fast movements immediately freeze at the hand model’s last known location

    Sounds good for UI though into media applications. 
Sign In or Register to comment.