Bump mapping — Oculus
Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Bump mapping

Beta84Beta84 Posts: 82
edited June 2014 in Game Design
I heared in a video from Michael Abrash that Bump mapping dosen't work anymore for VR. Is that right? (Can't tested by myself don't have DK1, I'm waiting for my DK2)

Comments

  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    Beta84 wrote:
    I heard in a video from Michael Abrash that Bump mapping doesn't work anymore for VR. Is that right? (Can't test by myself, don't have DK1, and I'm waiting for my DK2)
    It depends on how it is used. Extending the virtual screen to wrap around you should not affect how you generate the VR content any more than putting a larger monitor on your desktop would negate the usefulness of bump mapping (i.e. not at all).

    Please supply a link to where Michael Abrash said that -- I would like to see the context he was referring to if and when he made such a statement...

    EDIT: Okay, google found a reference:
    ...
    [–]nairol 7 points 3 months ago
    In your Alpha 91 video you say that you use normal mapping. Recently Michael Abrash (VR guru at Valve) said in a Steam Dev Days talk that normal mapping doesn't look good in VR. Here are his slides. (The normal mapping statement is on page 40)
    Just wanted to let you know. But I guess in this early stage of development it doesn't really matter that much.
    I don't yet understand how the gameplay would be like but visually it already looks amazing. You have my Greenlight. :)
    [–]ccsander 9 points 3 months ago
    I have to disagree on the normal mapping comment by Abrash, or at least agree with faith303 that there are exceptions. If you have a high-polygon model and use normal mapping to add even more details, it can look stunning in VR. You just can't get away with super low-poly models and expect normal mapping will make it look ok because it won't. It will look terrible. If you use normal maps for fine details like pores or wrinkles on skin, however, it is very effective because your eyes still perceive that as real geometry in VR at that scale.
    [–]subcide 3 points 3 months ago
    And as bad as normal mapping may look, no normal mapping isn't exactly better.
    permalinkparent
    [–]faith303 4 points 3 months ago
    Thanks a lot for sharing the slides. We are aware of these facts. There are some exceptions. It can look good under specific circumstances. ...
    This discussion makes it seem like you cannot substitute bump mapping for high resolution polygons, but you CAN use it to enhance the appearance of high resolution polygons. In the Rift, a stereoscopic view may make bump mapping appear to be painted onto flat polygon surfaces -- an effect that I noticed in some 3D animated movies which was rather annoying... Easier to get away with if you are not using a stereoscopic display device.

    Here is the Michael Abrash quote that seems to be having unintended meaning attached to it:
    Then there’s content. For example, no one knows yet which art styles work in VR. Detailed scenes
    that look great on a screen can look like cheesy stage sets in VR – and simple scenes can seem
    startlingly real. Normal maps don’t look good, and textures sometimes do and sometimes don’t. So
    we’re going to have to come up with a whole new visual vocabulary for VR too.
    All he is really implying is that you cannot trust methods that worked good on desktop displays to also look good in VR, so we really have to try things and figure out what works best, IMHO...
  • cyberealitycybereality Posts: 26,156 Oculus Staff
    Normal mapping looks fake in VR (or even just 3D). It's obvious you're looking at a flat quad.

    You can use parallax maps, which do adjust for the camera angle (but are somewhat expensive compared to normal maps).

    You can also look into tessellation using height maps, which is probably the best solution visually since it actually creates the geometric detail.
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    Normal mapping looks fake in VR (or even just 3D). It's obvious you're looking at a flat quad.
    ...
    If the quads are too large, as I mentioned. This is even a problem in some earlier 3D animated movies (which look fine in 2D). However, if you have sufficiently small polygons, they look much better with some bump mapping added on, even in 3D VR... The key is for the bump mapping to not occupy too much of your FoV.

    Of course, that is assuming that you do not get too close. Here are some examples:
    http://www.fabiensanglard.net/bumpMapping/index.php

    Good quality bump mapping can fool the eye even in VR, especially in the background where stereoscopic parallax has less effect. Up close, more polygons are better. But in typical VR environments, you can save on polys by using lost of bump mapping on distant objects.
  • cyberealitycybereality Posts: 26,156 Oculus Staff
    I don't think it has anything to do with how big or small the polygon is. The problem is that there is no stereo parallax between the eyes. So you see the proper lighting for a higher-res mesh, but none of the stereopsis. It destroys the illusion.
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    I don't think it has anything to do with how big or small the polygon is. The problem is that there is no stereo parallax between the eyes. So you see the proper lighting for a higher-res mesh, but none of the stereopsis. It destroys the illusion.
    That part in bold orange above treads dangerously close to speaking in absolutes, which is logically fraught with linguisic danger.

    Contrarily, the appropriate use of bump mapping has EVERYTHING to do with how small the polygon is and how far it is from the eyes (i.e. arc minutes):
    http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.211.7514&rep=rep1&type=pdf

    Stereo parallax is less critical for distant objects, where bump mapping can be used in place of extra polygon detail. It is normal to decrease polys for distant objects anyway, and you can improve the perceived detail and reduce polys even more with some appropriate bump mapping. I could keep trying to persuade you, but I am busy and I will let you discover these things for yourself if you are really interested.

    There is no need for a gazillion polys when bump mapping can reduce that burden for small details in small polys (or large polys very far away). Just be careful to keep bump mapping within the stereoacuity threshold of two to 10 arcseconds, where parallax has little perceptual effect (i.e. insufficient to stimulate stereopsis) while perception of bump mapped detail still adds to perceived realism.

    Vision Science III - Binocular Vision Module, Lecture 13 – Stereopsis:
    http://arapaho.nsuok.edu/~salmonto/vs3_materials/Lecture14.pdf

    Like I said, it depends on how you use it. Bump maps and textures are commonly used in 3D movies, and can look very good if used judiciously. I have seen movies that were startlingly overdone, though, and we need to avoid that problem in VR. It is all a matter of subjective perceptual perspective anyway, and different people will have different preferences. Just use what looks best within the hardware budget of your target audience.

    I suspect that Michael Abrash was just oversimplifying a bit for people who like to follow a simple unconditional set of rules. I sometimes do the same thing to minimize TL;DR posts. Bump mapping still has its uses in VR. Just be careful in when and where you apply it.

    I am not going to argue with you or with Michael Abrash (my hero from the DDJ and Zen book days). I am just supplying evidence to support my experience on how this stuff works. We are all free to believe whatever we want, until the law says otherwise. I just like to keep it real, and forbidding a useful tool under all conditions, by speaking in absolutes, is highly illogical.
  • cyberealitycybereality Posts: 26,156 Oculus Staff
    It seems you are misunderstanding what I am saying, and the point of this thread.

    Take, for example, a simple normal mapping shader (in HLSL):
    float4 BumpMapPixelShader(PixelInputType input) : SV_Target
    {
        float4 textureColor;
        float4 bumpMap;
        float3 bumpNormal;
        float3 lightDir;
        float lightIntensity;
        float4 color;
    
    
        // Sample the texture pixel at this location.
        textureColor = shaderTextures[0].Sample(SampleType, input.tex);
    	
        // Sample the pixel in the bump map.
        bumpMap = shaderTextures[1].Sample(SampleType, input.tex);
    
        // Expand the range of the normal value from (0, +1) to (-1, +1).
        bumpMap = (bumpMap * 2.0f) - 1.0f;
    
        // Calculate the normal from the data in the bump map.
        bumpNormal = (bumpMap.x * input.tangent) + (bumpMap.y * input.binormal) + (bumpMap.z * input.normal);
    	
        // Normalize the resulting bump normal.
        bumpNormal = normalize(bumpNormal);
    
        // Invert the light direction for calculations.
        lightDir = -lightDirection;
    
        // Calculate the amount of light on this pixel based on the bump map normal value.
        lightIntensity = saturate(dot(bumpNormal, lightDir));
    
        // Determine the final diffuse color based on the diffuse color and the amount of light intensity.
        color = saturate(diffuseColor * lightIntensity);
    
        // Combine the final bump light color with the texture color.
        color = color * textureColor;
    	
        return color;
    }
    
    Notice that the eye/camera position is not part of the calculation at all. This means as you change the camera position (or switch between the eyes) there is no change to the lighting effect. It will appear in stereo 3D as a painted-on, or tattooed, shadowing on a flat surface. Whether the polygon is big or small does not, in fact, have any bearing on this calculation. Though, you are correct that at large distances this illusion will remain convincing since there is not much (if any) stereopsis after a certain point.
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    It seems you are misunderstanding what I am saying, and the point of this thread. ... Though, you are correct that at large distances this illusion will remain convincing since there is not much (if any) stereopsis after a certain point.
    I take things quite literally, including speaking in absolutes. All I was saying is there are cases where bump mapping works fine in VR. It is really stereoacuity that matters -- really small polygons nearby, or large ones far away, where bump mapping will not look flat and painted on. Like I said, some 3D movies (like "Monster House 3D") did a horrible job at this, and if it looks bad on the giant screen, it will also look bad in VR.

    If I did not understand something, it must have contained implicit assumptions that I literally cannot see. That is why I like to define just about everything with lots of extra adjectives and such, leaving nothing to be guessed or assumed. But then others insert their own interpretation between the lines and claim I said stuff which I did not say. If not written, then not there, plan and literal and logical. I cannot interpret things other than their literal meaning, and sarcasm or satire often look like simple disrespect. My language skills were tested as "unprecedented", so that must account for some of our communications differences (or a failure in the classified military testing methodology).
  • tomftomf Posts: 88 Oculus Staff
    All of rendering is a hack. But normal maps work significantly worse in VR than on a monitor because the combination of stereo and fine positional motion mean it is very obvious when something is a flat plane with "odd" lighting, and not actual geometry. Yes, if it's distant geometry and/or tiny perturbations it will probably still work. But it's something like a factor of 10x between "looks good on a monitor" and "looks good in VR" - it really is quite a startling difference.

    On the same note, anything that relies too much on interpolated normals smoothing a polygonal mesh has the same problem. It becomes very obvious that the mesh is a bunch of discrete flat triangles and not a smooth surface, even when the shading is faking a smooth surface. In fact the disparity between the two can be quite jarring. This is most obvious when looking at specular highlights or environment-map reflections. One hack for these is to not use the vector to each actual eye in he lighting equations, but to instead use the vector to the "middle eye" for rendering both views. This at least avoids glaring stereo disparity, even if it doesn't look "correct". It doesn't solve the problem entirely though.

    This is a tricky subject, and no doubt we'll discover other accepted techniques that don't work that well in VR.
    Tom Forsyth - OculusVR coder
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    tomf wrote:
    All of rendering is a hack. But normal maps work significantly worse in VR than on a monitor because the combination of stereo and fine positional motion mean it is very obvious when something is a flat plane with "odd" lighting, and not actual geometry. ... One hack for these is to not use the vector to each actual eye in he lighting equations, but to instead use the vector to the "middle eye" for rendering both views. This at least avoids glaring stereo disparity, even if it doesn't look "correct". It doesn't solve the problem entirely though.

    This is a tricky subject, and no doubt we'll discover other accepted techniques that don't work that well in VR.
    No doubt ray tracing will replace all these perceptual hacks some day. Until then it is a delicate balance requiring subjective decisions best made by the end user. We each have different tastes, and even differences in how we perceive depth. The problem with normal mapping may bother me more in 3D movies because I like to sit front center, for full FoV, where my increased stereoacuity lets me see such defects much more than those sitting farther back in the movie theater. VR is much like sitting in the front row...

    I was mostly arguing the case for bump mapping as a viable option because of this thread making claims in near absolutes. There may be times when the hardware cannot supply the desired level of detail and bump mapping may come to the rescue to supply extra subtle detail. And it must be kept subtle to stay below the conscious perceptual annoyance level. It really depends on whether the game play or the eye candy is the most important element of the game, and which details are meant to draw the player's attention. Slight of hand and perceptual leading and pacing go a long way in making the player feel the extra details without looking too closely at them. When we have ray tracing engines, I am sure we will discover new perceptual "uncanny valley" effects to deal with.
  • raidho36raidho36 Posts: 1,312
    Let's say bump-mapping is a viable solution if you don't expect any stereoscopy whatsoever on the triangle in question. That is if it's very far away that there's no stereoscopic divergence, or the texture almost completely flat and bump-mapped light deviations are too slight to be picked as a relief where lack of expected stereoscopic cues would yield perception mismatch. Otherwise you would see a striking fakeness of bump-mapped lighting effect, in this case you should use either actual mesh or parallax map (those are work a charm in VR).
  • MrMonkeybatMrMonkeybat Posts: 640
    Brain Burst
    So does VR makes micro polygon tessellation or parallax occlusion mapping a must have then.
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    It is all about stereoacuity. I will concede that it is easier to ban a tool outright, than to enforce appropriate usage when that usage depends on the subjective bounds of perception. Some of us like to explore such bounds, which is why absolute rules and proclamations annoy us.

    Using such a tool wisely can be difficult. A case in point was the movie "Monster House 3D" (mentioned above) which looked pretty bad in 3D when sitting near the front row (some 3D objects looked painted onto flat surfaces), but probably looked fine to the majority of the audience sitting behind me. At least in VR, such problems will be mostly dependent on HMD display resolution rather than viewing distance, and software can adapt to the known resolutions of supported models.

    Or more expeditiously (from a political perspective), just ban all tools and methods that might be misused. :o

    EDIT: The alleged "Bump mapping doesn't work anymore for VR" quote was obviously a simplified generalization, rather than an absolute truth.
  • cyberealitycybereality Posts: 26,156 Oculus Staff
    I don't think anyone was saying you absolutely can't use normal mapping for VR. Of course developers will want to use the available tools, and a game may already be heavily reliant on these features if being ported from 2D. The point was just that the effect is more apparently "fake" in stereo 3D and not as effective as on a monitor. As I mentioned, there are other more modern techniques that will look good (i.e. tessellation and displacement maps).
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    I don't think anyone was saying you absolutely can't use normal mapping for VR. Of course developers will want to use the available tools, and a game may already be heavily reliant on these features if being ported from 2D. The point was just that the effect is more apparently "fake" in stereo 3D and not as effective as on a monitor. As I mentioned, there are other more modern techniques that will look good (i.e. tessellation and displacement maps).
    As you can see in the link I gave above to Fabien Sanglaard's site, bump mapping is often combined with parallax mapping (using a heightmap texture) to avoid this problem, perhaps more effectively than displacement mapping alone, and certainly side-stepping this whole stereoacuity issue that makes simplified poly meshes look flat in VR when viewed too closely or at too large a scale factor.

    Perhaps you guys are talking about 2D bump mapping without clearly stating that you are applying such a constraint to this problem domain. If so, why would you do that? That just makes things confusing and illogical.

    A quick google search found this bump map filter with parallax displacement for 3D use:
    ... The Bump map filter is a filter that alters an image by using another image called the bump map. The alteration is done by moving pixels in the image left or right based on the luminance of the corresponding pixel in the bump map. ... This means that you can warp images, change the depth of items, create stereograms or create 3D images from 2D images. There are some limitations, but the with some work, you can still get great results! ...
    Again, why would anybody use bump mapping in a stereoscopic application WITHOUT parallax mapping? Though admittedly, that is exactly what they did in the "Monster House 3D" movie, so maybe this is not as obvious to (some) others as it is to me (and others)...

    EDIT: It feels really strange to disagree with people I respect so much (especially Michael Abrash) when I know I am right (based on my own statistically interpreted contextual inferences). I suspect we are just disagreeing over conceptual technical definitions and poorly defined artificial constraints, or perhaps I missed some basic assumption you all managed to infer from hidden context. Explicitly literal logical linguistics are so much easier for me to assimilate than implicitly unspoken contextual assumptions...
  • MrMonkeybatMrMonkeybat Posts: 640
    Brain Burst
    Sound like you are not disagreeing with them just with there definition. Most of the time when people refer to bump mapping it does not have displacement when they have displacement or parallax the refer to it as displacement or parallax mapping.

    For over a decade now allot of game engines have saved performance by doing bump map shading without paralax When something like Unreal engine has parallax they tout that.

    The wikepedia article on bump mapping defines it as ontly affecting the shading not displacement and use this image to show the diference.
    800px-Bump_map_vs_isosurface2.png
    http://en.wikipedia.org/wiki/Bump_mapping
  • Beta84Beta84 Posts: 82
    So I could use parallax mapping or should I use displacement mapping / PN triangles tessellation?
  • geekmastergeekmaster Posts: 2,866
    Nexus 6
    Beta84 wrote:
    So I could use parallax mapping or should I use displacement mapping / PN triangles tessellation?
    I prefer 3D displacement mapping (which I learned as "bump mapping" back in the day, why there were terminology disagreements earlier in this thread). I see Googly sources show the meaning as visualized in the orange spheres in a recent post above. So, technically speaking in modern terms, it seems that traingle displacement mapping is not really considered as "bump mapping" these days. Before the internet, companies tended to develop their own custom technical terminology. Widepread distribution of terminology definitions via the Internet is helping standards form with more recognition.
  • djsainthubertdjsainthubert Posts: 4
    NerveGear
    please someone convert Perfect Dark Zero to VR for the lols... the bump mapping in the game is hilarious
Sign In or Register to comment.