New to the forums? Click here to read the "How To" Guide.

Developer? Click here to go to the Developer Forums.

Vive to Get Eye-tracking Add-on

nalex66nalex66 Posts: 5,401 Volunteer Moderator
Article on Road to VR.

I thought this was interesting, considering the various discussions that we've had here recently about screen resolution, foveated rendering, and eye tracking.

A third-party hardware developer is releasing an add-on kit for the Vive that integrates eye-tracking. When paired with an Nvidia GPU, it can do foveated rendering to reduce the load on the video card for processing VR scenes. This is something that is functional now, using current technology, and can be used with any VR app without needing any modification.

I think this demonstrates that we'll definitely see higher resolution screens in second-gen VR HMDs without increased GPU horsepower required. If anything, this technology could make high-end VR more accessible than ever--imagine running 8K VR on a GTX 960!
i7 5820K @ 4.25GHz | EVGA GTX 1080 SC | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
SSDs: Intel 660p M.2 2TB, Samsung 860 Evo 1TB, 850 Evo 1TB, 840 Evo 1TB | Startech 4 controller PCIe USB 3.0
«13

Comments

  • nalex66nalex66 Posts: 5,401 Volunteer Moderator
    Obviously as a Rift owner, the add-on kit does nothing for me, but I'm very interested in the underlying technology. The fact that they can put together a working, affordable dev kit to add eye-tracking within the confines of an existing HMD is very promising.

    Even if the eye-tracking isn't super-accurate, we can only see high detail in a cone of a few degrees. If the foveated rendering covers a 10-15 degree zone if high detail, it should provide enough buffer to account for imperfect eye tracking. That would presumably still reduce the rendering overhead significantly.
    i7 5820K @ 4.25GHz | EVGA GTX 1080 SC | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
    SSDs: Intel 660p M.2 2TB, Samsung 860 Evo 1TB, 850 Evo 1TB, 840 Evo 1TB | Startech 4 controller PCIe USB 3.0
  • ZoomieZoomie Posts: 1,777 Valuable Player
    If nothing else this is a step in the right direction, just like the wireless kits.  
    I find it interesting that they describe Foveated rendering as way of running the Vive (or Rift) on lower spec machines.  I suppose this is because they are trying to define their consumer based on existing HMD's - as an add-on manufacturer.

    I'm more interested in the idea that high spec machines will be able to surpass current limitations with this technology.  Foveated rendering means better quality with whatever GPU you use; The gains should be observed at both the high and low ends of consumer GPU's.
    Any sufficiently advanced technology is indistinguishable from magic.
    - Arthur C Clarke
  • RedRizlaRedRizla Posts: 7,010 Valuable Player
    edited April 2017
    So will this tech be used for the Oculus Rift 2, is there any mention of something in the pipe line for Oculus eye tracking like we see here for the Vive?
  • ZoomieZoomie Posts: 1,777 Valuable Player
    Sadly I doubt it.  The Vive seems to have a much more active add-on & hardware development community than the Rift - especially in China where a lot of these add-ons seem to originate. 
    Any sufficiently advanced technology is indistinguishable from magic.
    - Arthur C Clarke
  • RedRizlaRedRizla Posts: 7,010 Valuable Player
    edited April 2017
    Zoomie said:
    Sadly I doubt it.  The Vive seems to have a much more active add-on & hardware development community than the Rift - especially in China where a lot of these add-ons seem to originate. 
    So will Oculus Rift users be left needing highend computes to get good results? Surely, Oculus must have something in the pipe line because this is massive news for those of use who want to see high resolutions.
  • ZoomieZoomie Posts: 1,777 Valuable Player
    edited April 2017
    nalex66 said:
    We may not see it as an add-on for CV1, but certainly CV2 will have it.
    This. 
    I wasn't suggesting that eye tracking and Foveated rendering will be exclusive to Vive or omitted from the next Rift.  I guarantee Oculus/FB is looking into this tech for future generations.  I interpreted "is there any mention of something in the pipe line for Oculus eye tracking like we see here for the Vive?" to mean "will we see add-ons like this for the CV1?".  I think that answer is likely no.  

    And sure, you scrubs with your weak GPU can use eye tracking to let you run with the big boys.  :)
    I say that as someone who finally bought the flagship GPU - for the first time - with a 1080.
    Any sufficiently advanced technology is indistinguishable from magic.
    - Arthur C Clarke
  • RedRizlaRedRizla Posts: 7,010 Valuable Player
    We might just see 4k screens for CV2 after all.
  • ZenbaneZenbane Posts: 15,132 Valuable Player
    Zoomie said:
     I guarantee Oculus/FB is looking into this tech for future generations.
    It was sometime last year that this forum discussed the company Facebook bought out who specializes in eye-tracking. So yeah... it's definitely gonna be a thing.

    If it gets released on the Vive first, hopefully that will help create a market for VR content that can take advantage of it in a fun way. By the time the Rift releases their version... we can all have noteworthy titles to look forward to.
    Are you a fan of the Myst games? Check out my Mod at http://www.mystrock.com/
    Catch me on Twitter: twitter.com/zenbane
  • ZoomieZoomie Posts: 1,777 Valuable Player
    I hadn't even thought about the gaming applications of eye tracking.  This tech will improve VR just on the performance gains of foveated rendering alone.  Gaze control is already implemented in some applications - just moving your head to control a pointer.  My concern would be eye strain if we start using gaze as an input device.  I already take off my Rift with mild dry-eye or strain after prolonged use.
    Any sufficiently advanced technology is indistinguishable from magic.
    - Arthur C Clarke
  • ZenbaneZenbane Posts: 15,132 Valuable Player
    About that Gaze Control...
    The first eye tracking video game better involve Superman!

    Imagine being able to shoot them eye lasers by simply squinting irl? lol



    https://forums.oculus.com/community/discussion/comment/479301#Comment_479301

    Are you a fan of the Myst games? Check out my Mod at http://www.mystrock.com/
    Catch me on Twitter: twitter.com/zenbane
  • ogflatlanderogflatlander Posts: 45
    Brain Burst
    I agree falken76.  The way I see it is these 2 companies are really pushing VR forward.  To me, Oculus is giving us solid games and pushing the software side of VR.  Vive is really pushing the hardware advances for the VR community.  Hopefully this ends with two really strong companies pushing each other and other techs able to jump in without fear.
  • snowdogsnowdog Posts: 7,478 Valuable Player
    lovethis said:
    We might just see 4k screens for CV2 after all.
    I don't think so. Even with foveated rendering 2 x 4K screens at 90fps is still going to be out of reach for average graphics cards in the average gamer's PCs for the CV2s from HTC and Oculus to have them. The idea is to get the cost of VR adoption down so that the mainstream market can afford it. We'll certainly see an increase in resolution and FOV but not by that much.

    Oculus and Valve want these headsets to be within reach, spec-wise, of the average PC gamer. I posted on the Oculus Reddit thingummybob that the reason why so many gamers out there are anti-VR is because they can't afford the cost of a VR Ready PC, let alone the price of a headset, and are jealous of the people that can.
    "This you have to understand. There's only one way to hurt a man who's lost everything. Give him back something broken."

    Thomas Covenant, Unbeliever
  • MinituMinitu Posts: 79
    Hiro Protagonist
    I don't see it as practical thing for consumers at this point, but its interesting as a way to test use cases and test/improve FOV rendering.
    Soon upgrading to Ryzen
  • snowdogsnowdog Posts: 7,478 Valuable Player
    Minitu said:
    I don't see it as practical thing for consumers at this point, but its interesting as a way to test use cases and test/improve FOV rendering.
    Yup, they reckon it's going to be $200/£200+ so I won't be going anywhere near it when a Rift version appears.

    It also looks like something like this will also mean that Lens Lab prescription lenses wouldn't be able to fit into an HMD at the same time too.
    "This you have to understand. There's only one way to hurt a man who's lost everything. Give him back something broken."

    Thomas Covenant, Unbeliever
  • elbofforelboffor Posts: 2,572 Valuable Player
    edited April 2017
    Meanwhile on facebook...
    This is my forum signature.
    There are many others like it, but this is mine.
  • AQfumesAQfumes Posts: 133
    Art3mis
    It's pretty cool technology. No doubt Oculus has their r&d working on similar tech for the future. Vive is always allowing third-party companies to create these add-ons for their head set, but they hardly pay for any additional add-ons to be developed out of their own pockets. 

    It it makes me wonder if it's htc's strategy to spend little on their headset for now to grow profit. I mean they could be thinking why invest so much when other companies are already doing it for them and attracting customers for the Vive. 

    One downside I can see from that logic is quality control. 
  • snowdogsnowdog Posts: 7,478 Valuable Player
    HTC aren't investing a great deal of money on accessories because they don't have it to spare.
    "This you have to understand. There's only one way to hurt a man who's lost everything. Give him back something broken."

    Thomas Covenant, Unbeliever
  • ZoomieZoomie Posts: 1,777 Valuable Player
    I tend to agree AQfumes,

    Oculus has always maintained tight control of their end-product while Valve and HTC have embraced the home-brew and mod community.  Vive is on the ragged edge with innovative but unproven tech, and Oculus is the refined and polished experience.  The result is that HTC appears to be the one actually driving the tech forward by allowing a robust community of smaller companies to build for their device.  Facebook meanwhile has thrown money at the best people their money can buy.  It'll be interesting to see which strategy pays off in the long term.  To the average person, it will appear that smaller development firms are driving the tech, and the engineers at Oculus are just taking the end result and adding it to their device.  Behind the scenes, they have likely been working on it in parallel and will have a perfect version before it's released to the public.
    Any sufficiently advanced technology is indistinguishable from magic.
    - Arthur C Clarke
  • AQfumesAQfumes Posts: 133
    Art3mis
    @Zoomie I agree. You wrote it out exactly the way I'm looking at the situation as well. HTC and Valve are letting these third-party companies pretty much improve their product for them and they're saving a bundle and appearing to the average consumer to be leading the way in this technology. 

    Oculus on the other hand are investing their own money on the same tech but are waiting until they have it just right and ready for the consumer before they release their intentions with it.

    I understand that HTC and Vavle don't have the same deep pockets that Facebook backed Oculus has to invest in new tech, but I tend to think they should have some to spare by now; especially if they're supposedly out selling the Rift 2:1. 

    They seem to be holding out on investing much profits back into their own product unless they're forced to and before seeing how much investments through add-ons they can get off other companies first. 

    Which would also give cause for Valve pushing to get Oculus to open their exclusive games to be sold on Steam and letting the public smear Oculus for choosing not to do this. They don't want to invest any money on developing games but want to profit off said games to help sell their headset. 

    Oculus shouldn't feel ashamed at all for not letting Valve profit off the games they put their money into developing, when clearly Valve doesn't have intentions anytime soon in returning the favor. 

    Im glad that HTC and Valve are helping these small tech companies with allowing them to promote their technology on the Vive platform, but it seems in the end HTC/Valve will get the better part of the deal. 

  • KillCardKillCard Posts: 1,078
    Wintermute
    snowdog said:
    lovethis said:
    We might just see 4k screens for CV2 after all.
    I don't think so. Even with foveated rendering 2 x 4K screens at 90fps is still going to be out of reach for average graphics cards in the average gamer's PCs for the CV2s from HTC and Oculus to have them. The idea is to get the cost of VR adoption down so that the mainstream market can afford it. We'll certainly see an increase in resolution and FOV but not by that much.

    Oculus and Valve want these headsets to be within reach, spec-wise, of the average PC gamer. I posted on the Oculus Reddit thingummybob that the reason why so many gamers out there are anti-VR is because they can't afford the cost of a VR Ready PC, let alone the price of a headset, and are jealous of the people that can.
    I wouldnt be so sure about this, especially with Nvidia's VR Works and how they've already implemented a bunch of optimization on the software/driver side such as the single-pass stereo rendering which already cuts required processing by almost half. Everyone has 4K in mind as the goal to reach for VR, I don't think its a matter of "we wont get 4K in the CV2", I think its a matter of "we wont get the CV2 until we can get 4K".

    The CV1/Vive1 need a good 2-3 years to settle into an affordable price range for the average user, I think by the time the average PC can handle 4K in VR with single pass stereo and foveated rendering the CV2 will be released implementing all of these things.

    Plus.. tech evolves fast, like .. really fast. I wouldn't underestimate it. I think CV2 with 4K is a real possibility.
  • Paddy234Paddy234 Posts: 36
    Brain Burst
    This is brilliant news as it means we will get 4k and even 8k resolution much sooner due to this technology. The resolution on the vive and CV1 strains the eyes much too much and hinders the immersion  
  • ZoomieZoomie Posts: 1,777 Valuable Player
    edited April 2017
    I haven't been following the development of small screens, but does a manufacturer like Samsung have a sufficiently high PPD screen to support 4k if the GPU's could handle it?  If not, we're going to need development in many areas to fully realize higher resolutions.  

    @Atmos73 Yes, Facebook has a habit of buying out competing or innovative tech and rolling it into their own product line.  It works for them just like it has worked for Apple.  On the plus side, at least it doesn't stifle innovation because all these tech startups are hoping they get bought out by one of the giants.  Look at the payday guys like Palmer Luckey received.
    Any sufficiently advanced technology is indistinguishable from magic.
    - Arthur C Clarke
  • MervinRaeMervinRae Posts: 41
    Brain Burst
    edited April 2017
    Well, that that technology exists is known since many years. Nice someone builded a device-add on for the Vive. So we can be certain the Rift 2 will have that as well. And I am sure FB does have ways to implement this technology and planned it most probably since years for its 2. generation.

    The Nvidia optimizations are very cool as well. And if it is really so that the single pass stereo rendering will save almost half gpu power.. Really impressive if this should be so.

    Not to forget the Asyn.time-warp from Oculus itself to bypass at least some frame drops in busy scenes.

    On the other hand I do not know how fast the resolution will grow as indeed it is more important to increase the userbase and have every (media) notebook being able to run VR - seen commercially and for the all-around success of VR.

    It is also true that resolutuon is not all. You can see it in Robo-Recall, or the Bulletstorm Demo, or the Ghost in the Shell Demo - if games would all look like that we would not grump that much. A good Artstyle, good Textures (+ several maps), good lightning, good leveldesign, some filtering and we would have great looking games.

    People want 4k in the hope of better looking games while actual PCs can`t nearby render the actual games in 2k properly.
    People want great looking prof. game titles but that needs a massive userbase. A userbase where you do not even have to have a desktop pc but where a (slightly better) multimedia-notebook should do.

  • stargate88stargate88 Posts: 46
    Brain Burst
    Let's see what we have:
    • 4K displays: checked
    • Optimized graphics drivers: checked
    • Better GPU (1080 ti): checked
    • Eye tracking: checked
    • Wireless video (TPCast): checked
    • Inside out tracking (Acer, Lenovo): checked
    • More confortable HMD: checked
    • CV2: hum...

  • nalex66nalex66 Posts: 5,401 Volunteer Moderator
    edited April 2017
    Zoomie said:
    I haven't been following the development of small screens, but does a manufacturer like Samsung have a sufficiently high PPD screen to support 4k if the GPU's could handle it?  If not, we're going to need development in many areas to fully realize higher resolutions.  
    The screens are getting there. I see a lot of display industry reports through work, and everyone is heavily investing in development of small screen technology for AR and VR. Things have changed so much since the early days when Oculus had to take what they could get from the mobile phone left-overs. 

    Here's an example of a tiny 2048 x 2048 display with a 120 Hz refresh rate and a 1" diagonal size. 
    i7 5820K @ 4.25GHz | EVGA GTX 1080 SC | Gigabyte GA-X99-UD4 | Corsair DDR4 3000 32GB | Corsair HX 750W
    SSDs: Intel 660p M.2 2TB, Samsung 860 Evo 1TB, 850 Evo 1TB, 840 Evo 1TB | Startech 4 controller PCIe USB 3.0
Sign In or Register to comment.