Is there a way to turn on anti aliasing MSAA for GearVR in Unreal Engine 4.14? — Oculus
Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Is there a way to turn on anti aliasing MSAA for GearVR in Unreal Engine 4.14?

I'm working with Unreal 4.14 which supposedly even has a new forward rendering path for VR, though not sure if it is meant just for desktop vr or for mobile too.

I'm attempting to get antialiasing working on a Galaxy S6 with GearVR, but I've spent the past 2 days trying out all sorts of things and nothing makes a difference.

Basically, I start with a blank project set to mobile + scalable graphics quality. Then I've tried the following. Always packing to Android ETC2:

. default project settings (theoretically deferred path by default).
. With the new forward rendering option.
. Setting any combinations of the following console variables:
  . r.MSAACount
  . r.MSAACompositingSampleCount
  . r.DefaultFeature.Antialiasing
  . r.MobileOnChipMSAA
  . r.MobileMSAA
  . r.MobileContentScaleFactor
  . gearvr.EnableMSAA

Nothing seems to enable anti aliasing. I also posted this on SO, gamedev SE, Unreal AnwerHub and the Unreal forums. and no one has replied anything other than "MSAA should be on by default for GearVR."

Has anyone experienced this? Does anyone have any thoughts?

Best Answer

Answers

  • TuxerLulzTuxerLulz Posts: 2
    NerveGear
    edited December 2016


  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    Could it be that 4.14 is broken for Gear VR ?

    4.14.1 is out btw
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    rpalandri said:
    Hello!
    The forward rendering path for VR is really for PC only, the mobile side has always been forward only.

    By default, MSAA should be on when you start GearVR, but issues in the config files might have messed that up. How it works is, if gearvr.EnableMSAA is 1 when the plugin starts, then whatever the value of r.MobileOnChipMSAA is, it will be set as 1 (activated). r.MobileOnChipMSAA is the CVar that actually controls the rendertarget being set as an MSAA rendertarget, which activates the feature.

    How we can debug this:
    when gearvr.EnableMSAA is set, it will log :
    UE_LOG(LogHMD, Log, TEXT("Enabling r.MobileOnChipMSAA, previous value %d"), CVarMobileOnChipMSAA->GetInt());
    Can you see this "Enabling r.MobileOnChipMSAA, previous value..." log in your run? 

    If yes, and you still don't see MSAA, where the magic really happens is in OpenGLRenderTarget.cpp, line 201, where this is called:
    glFramebufferTexture2DMultisampleEXT
    instead of this
    FOpenGL::FramebufferTexture2D.
    Could you verify in which path your renderer goes?

    Thanks a lot

    Thanks a lot for the info. in the meantime I had to switch to another task (saving mp4 files even if on  separate thread freezes the UE game thread and the gearVR IMU stays on, allowing you to see a frozen quad...). But I will try out your test as soon as I can, which will be in a couple days probably. Ill report back when I do.
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    @rpalandri


    I just did a quick test. BTW, I am new to unreal. So I might need a bit of help here and there...

    So first, I assume you meant for me to check the output log via adb. right? If so, yes, when the app starts on the device I get:

    12-09 05:18:12.031  5369  5384 D UE4     : ------- EnterVRMode -------, tid = 5384
    12-09 05:18:12.031  5369  5384 D UE4     : [2016.12.09-10.18.12:048][-8607492]LogHMD: Enabling r.MobileOnChipMSAA, previous value 0
    12-09 05:18:12.031  5369  5384 D UE4     : [2016.12.09-10.18.12:048][-8607492]LogHMD: GearVR has started

    But the rendering is completely aliased. So MSAA seems to somehow be off anyway.

    Then I searched for the UE_LOG you mentioned. The one that prints the above. It is in GearVR.cpp (and does not seem to be conditionally compiled...).

    Last,  I also see the gl multisample fbo attachment at line 201 of OpenGLRenderTarget.cpp. But I don't know how I should check that this line is being hit, since it would run on the device and not the editor in my computer right? I tried sarting the editor from VS (developmentEditor configuration) and then hitting launch from the editor to upload to the device and run. But it didn't hit any breakpoints.

    What else should I do?

  • rpalandrirpalandri Posts: 63 Oculus Staff
    Hey!
    Sorry you're running into issues with UE4.

    So 1), we're seeing that  "[2016.12.09-10.18.12:048][-8607492]LogHMD: Enabling r.MobileOnChipMSAA, previous value 0" is being called, so good! the value should be 1.

    To check in which part of the code the engine goes in OpenGLRenderTarget.cpp, you could add a logging line, like :  UE_LOG(LogTemp, Log, TEXT("Going through MSAA codepath")) in the multisample FBO case, and UE_LOG(LogTemp, Log, TEXT("Going through non-MSAA codepath")) in the other,so that we're sure which path we're going on.

    One reason why it would switch off the MSAA codepath is if your driver/device doesn't support the MSAA extension (GLES doesn't support MSAA by default, it's a per-vendor extension that does). When VRAPI starts, it prints out all the supported extensions. Can you verify it contains GL_EXT_multisampled_render_to_texture? Can you also send us your exact model of device, and OS build version? A log of the execution of a UE4 program would also be helpful (containing the VRAPI start, vrmodes, everything :) )

    Can you also send us a screenshot of your aliased scene? Although on-device MSAA is definitively helpful, it doesn't remove all the edges.

    Thanks a lot
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    @rpalandri

    The device is:
    . Samsung Galaxy S6.
    . Software and Baseband version: G920PVPS4CPJ1
    . Hardware version: G920P.02
    . Model number: SM-G920P
    . Android version: 6.0.1
    . Kernel version: 3.10.61-8847946

    I'm also attaching logs (all, ue output only, vrapi output only).

    I see that the UE log is the one that seems to list all device gl extensions. And I do see:
    12-09 21:42:27.481  8399  8423 D UE4     : [2016.12.10-02.42.27:488][-8606092]LogRHI:   GL_EXT_multisampled_render_to_texture

    Interestingly too, I see that the gles version seems to be 3. I was expecting GLES2 since I do not have v3 checked in the project settings under platform android.

    About putting logs in OpenGLRenderTarget.cpp, I guess I need the source code version of the engine right? And recompiling the engine itself?

    I am currently running the 4.14.1 version from the Epic launcher.
    I tried modifying the file, adding:

    UE_LOG(LogTemp, Log, TEXT("------------------------ !!@# : MSAA | 1"));

    at line 200. And a similar log for the else branch:

    UE_LOG(LogTemp, Log, TEXT("------------------------ !!@# : NO MSAA | 1"));

    and hitting compile in the editor but it did not print anything when running on the device.
    Also, when I modified the cpp it warned me that it was under version control and had to be made read/write.

    About sending you an image of the aliasing I am wondering how could I do this, since the aliasing becomes obvious only when putting on the headset and staring thru the lenses.
  • rpalandrirpalandri Posts: 63 Oculus Staff
    This is gonna sound weird, but are you sure you're not running MSAA? Everything here seems to be pointing towards MSAA being on (which I also replicate on an S6 on 4.14.1). If you send us a screenshot (pressing the home and lock button at the same time), I'll put it in the headset to repro :)

    MSAA 2x is good, but far form magic, especially if the aliasing doesn't come from the geometry but from the textures (which something like FXAA/SMAA/TAA would solve, but not MSAA)
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    @rpalandri

    I'll try to get you a screenshot soon.

    And yeah, honestly it might just be on. The reason why I think it might not is cause in Unity, a similar scene with MSAA at 4 or 8 definitely makes a difference. My scene is actually made up so far exclusively of an UMG widget. A sort of flat 'card' with some text and an image.

    . I see a lot of shimmering in the image, which as you say, would be related to texture filtering and mipmaps. 

    . I also see the text shimmering, this I'm not sure how it could be addressed; not sure about Unreal settings related to text since I am new to it. The only thing I've seen so far is the UMG DPI setting but I'm not sure if it is just for metrics when designing the widget or if it has an effect on rendering.

    . Finally, the widget's (a square) borders are very jagged. This is what made me think MSAA might be off. I assume the widget is rendered over a quad. If so, the quad is geometry and I would expect MSAA to affect its edges.

    Makes sense?
  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    Afaik MSAA antialiases only edges of geometry. Since UMG widgets aren't geometry per se, the images and text on them won't be AA'ed. You should probably use stereo layers instead (render your UMG into stereo layers).
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    @motorsep


    I'm still in the process of making sure that multisample targets are set in the fbo (how @rpalandri pointed out), just to be safe. But if we are correct that this is not an msaa issue, shouldn't the UMG edges be antialiased anyway? While the content inside the widget might be rendered in a different way, I'd expect the quad itself to be a a quad in the usual vertex attribute way.


    About stereo layers, not sure what those are. Is it an Unreal thing?


    Thanks
  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    If you make solid color UMG widget, you'd see that edges of that widget (since it'd be a rectangular shape anyway) are AA'ed. Anything inside the widget won't be, since MSAA doesn't AA textures/text, afaik.

    https://docs.unrealengine.com/latest/INT/BlueprintAPI/Components/StereoLayer/index.html
    and the whole thread about how to use it: https://forums.oculus.com/developer/discussion/44130/gear-vr-rendering-umg-widget-in-stereolayer-component-how-bp-only
  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    @saldavonschwartz Are you using Multiview option by chance? It seems that AA isn't working when using Multiview (and performance is horrible too). I posted about that here https://forums.oculus.com/developer/discussion/46725/gear-vr-multiview-is-buggy-in-4-14-1
  • rpalandrirpalandri Posts: 63 Oculus Staff
    Hey,
    @motorsep is right on the money there. Although I feel UMG should probably be MSAAd, due to the fact that UMG renders after all the normal rendering is done (and therefore after the original MSAA resolve, due to how MSAA works on mobile chipsets), I can definitively see why there would be issues with it. As good practice, try to have alpha dithering on the sides of your quad content anyway to avoid that. 

    Layers are a VR compositor feature that enable high-quality quads, cylinders and cubemaps to be displayed on the headset. We are exposing this compositor feature to users via the UE4 StereoLayer blueprint component, whose links motorsep gave you :) For a quick explanation, if you're interested, how it works is that the quad isn't rendered in UE4 at all. We give the VR compositor the quad's texture and position, and the compositor doesn't distort the pre-rendered quad but raytraces the quad's geometry through the lenses. That enables per-display-pixel sampling (much higher than the 1024x1024 rendertarget that we use for game rendering), and avoids dual sampling the texture (once in the game, once in the compositor). 

    Outside of layers, which can only be used for a couple quads in the whole game, VR introduces a lot of issues in rendering high frequency textures like text. Devs have to be extremely conscious of the choices they make in textures, filtering, and mipmapping. John Carmack has excellent public talks on the subject :)




  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    @rpalandri
    @motorsep


    Hi again. Still haven't been able to make much progress on this.
    I'd like to do a recap of what I have so far, just in case.

    The situation is that I am using UMG widgets in world space and I see a lot of aliasing (more than I'd expect with MSAA x4) in images, text and the widget borders themselves.

    Here you can see a screenshot:



    When actually using the GearVR headset of course the artifacts are further amplified.
    While text and image aliasing might have fixes that are unrelated to MSAA, you can see the orange rect which is a bare minimum UMG widget displayed in world space. The edges are pretty jagged. And this is what makes me think MSAA could be off.

    BTW @motorsep I am not using multi view. At least not that I know of: it is not checked under project settings.

    Another interesting thing is that in Project Settings, I do not have GLES3 checked but according to the adb logs which I posted on a previous comment, the still seems to be running on GLES3 instead of 2. 

    Now on to the current hypotheses:

    1. Is MSAA on? Or why are the rect edges so jagged?

    I don't know. We did confirm that:

    a. The adb log spits out: "LogRHI:   GL_EXT_multisampled_render_to_texture"
    This seems to imply that multisampling is supported.

    b. The adb log spits out: "LogHMD: Enabling r.MobileOnChipMSAA, previous value 0"
    This seems to imply that MSAA is being turned on when running on GearVR mode. Though it does not tell what level of MSAA (2,4,8?).

    I was not able confirm that:

    glFramebufferTexture2DMultisampleEXT(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0 + RenderTargetIndex, RenderTarget->Target, RenderTarget->Resource, MipmapLevels[RenderTargetIndex], 2);

    is being called however. I tried downloading the source for UE, adding log statements around the above and compiling the engine. The engine compiles, the editor starts up and I am able to open my project. But when I try to package for Android it errs out saying there are a lot of missing files (related to all sorts of subsystems in the engine, like physics). So not sure what I'm doing wrong since I followed the instructions on Epics site to run from source.

    2. Does mipmapping and filtering make a difference for the images?

    Again, not sure... I'd expect it should but it is not clear to me if changing parameters from the Texture editor should be visible immediately even inside the general editor. For instance, if I change Mip Gen Settings, Texture Group or Filtering, I see no difference in the image. I also hit Save just in case (since I don't know if changing the settings takes effect before hitting save) and nothing seems different.

    3. Would Stereo Layers help?
    I'm actually gonna give this a try, since I tried the sample project in the thread @motorsep
    linked to and it definitely looks good. Also I was under the false impression that stereo layers required the surface tracking the headset but from the example project it is clearly not the case :).

    Again, not sure. The doc link @motorsep provided is of little use: like many areas in the UE docs, it appears to simply be the generated doc from the source code, which merely lists method names without any context on how to use.

    From the second link @motorsep sent, what I gathered so far is that stereo layers are meant to place HUD-like elements that track with the headset. But this would not serve my use case since the cards I show in the image are to stay put in their world positions. The player can stare around in the world and stare at the cards and activate them with their gaze, but the cards themselves should not follow the camera or headset.

    Last, another thing I notice when running on the device (the Galaxy S6), is that the black eye vignettes stutter a lot as I move the phone around. But I'm not sure this might have anything to do with the aliasing.
  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    MSAA x4 only available when enabled through Developer's Options on the phone, afaik. And as I recall turning that on disabled AA completely in my Gear VR app.

    If you set your StereoLayer tracking to "world", it will stay in one place and won't follow your camera.

    I am not sure what you are doing and what debug options you have on through phone's developer menu, but I am using 4.14.1, ES2 renderer, no HDR, ETC2 textures and no experimental features, and MSAA x2 works extremely well.
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    @motorsep


    I might upload a stripped down version of my project soon, in case you want to take a look. But I've been running into a lot of issues as you can see: UE from source does not package the app, MSAA seems off to me (as you can see in the image with the orange quad) even though I have no experimental features on... I don't know.

    Before I upload a sample project, I am trying to implement the stereo layer approach. I am looking at the sample @rpalandri posted in https://forums.oculus.com/developer/discussion/44130/gear-vr-rendering-umg-widget-in-stereolayer-component-how-bp-only but I don't see where the stereo layer is at all.

    Like, I tested it on my phone and works great. Then I started looking around and I see he has a widget same as me, then he does a RTT (render to texture) thru the static C++ function he exposes to BP. All that is fine. But where is any mention of stereo layers in the project?

  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    StereoLayer is a component you add to BP actor or to your player's actor.
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    Ok I found the stereo layer on the ground actor for some reason. I'll report back once I have a sample project.

    @motorsep
    But in the example from the thread you linked to there is no such component neither on the CameraActor nor DefaultPawn which are the 2 classes involved in the concept of "the player". As a matter of fact, the only 2 non standard entities in the project are:

    . the menu BP.
    . the cat UMG widget.
    . the C++ static function.
    . The level blueprint which just executs the same logic the menu executes on tick, but instead once on start.

    @rpalandri I believe you created the sample. Would yo mind pointing me to where the stereo layer is being created / set up? Also, does this work only for Oculus hardware or does it also work for Google Cardboard, Daydream and HTC Vive?

  • rpalandrirpalandri Posts: 63 Oculus Staff
    edited December 2016
    Hello!
    Lots of questions in there, that I'll try to explain. UE4 Mobile VR (and mobile VR in general) have a looot of hacks in it, which makes it not very straightforward to understand. Will try to help.

    b. The adb log spits out: "LogHMD: Enabling r.MobileOnChipMSAA, previous value 0" 
    That is where the MobileOnChipMSAA variable is set, which actually enables the 2x MSAA. Depending on device, you can probaby run this on 4x MSAA. We currently have MSAA set to 2x only everywhere, but what we should do (and are planning to do for 4.15 or 16) is activate MSAA 4x on the GPUs which have a cheap 4x path like the Mali T880. Right now, you can only change this in code, in OpenGLRenderTarget,

                            const uint32 SampleCount = 2; to 4.

    About the OpenGL version, you are correct that adb will tell you when running ES2 that the actual OpenGL context is ES3. Although the shaders generated in the .usf -> GLSL shadercompilation pass are GLES2 compliant (they do not use any ES3/3.1 features), the #version tag at the beginning of the shader is set on the device at runtime, and for every GearVR phone, that will be a #version 300 tag. The OpenGL context will also be set to 3, even if all the calls are compatible with ES2.

    About my stereolayer project, sorry for hiding the actor containing the layer :neutral: . Right now I know the quad stereo layers are also supported on vive, and I don't know about daydream. I'm fairly sure right now only GearVR (not even rift) support the cylinder layers though.

    About the aliasing you're seeing on the edges, can you try / did you try to put alpha borders on the textures you choose? On my stereolayer app when you're running it, do you also have AA issues?

  • motorsepmotorsep Posts: 1,367 Oculus Start Member
    Is MSAA x4 cheaper than MSAA x2 on Snapdragon devices too? If so, why not just make MSAA x4 default for Gear VR in 4.15 already? :) 

    P.S. I am still on S6 since I consider this a bottom line hardware for mobile VR, so I hope MSAA x4 is cheaper on it too ;)
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    @motorsep
    @rpalandri


    Ok I made a little project to test everything, maybe you want to take a look.

    The project is a simple scene with 4 cards. Starting from the right, these are my observations:

    1. A quad widget with a texture used for rounded edges. Notice how the aliasing on the edges is pretty bad. At this point I'm realizing that the issue is because of using the texture. But I'm not sure how would this be solved. Higher res texture? Some specific filtering or mipmapping setting in the texture editor?

    2. The card I was originally working on when I started this question. It uses the same texture for the rounded edges and you'll notice the aliasing once again. Also, this card has text and another image, both of which show some aliasing too. I'd assume the way to mitigate the image aliasing would be the same as for the black rounded border image (which as stated in 1, I'm not sure what it would be -- I'd assume mipmapping it). As for the text, no idea how it would be made to alias less.

    3. The stereo layer approach. This actually looks pretty good. But:

    a. Why does it not show the rounded border if it's rendering the middle card which has rounded borders? Maybe I need to tweak some resolution or total render area / viewport parameter?

    b. I actually need gaze input for this whole project. i.e.: gaze at a button in the card and stuff happens. It seems like it would be a pain to raycast onto the texture and then somehow extrapolate the hit point to the original card widget to activate whatever real control the original card would have.

    c. What about performance? Now I have all the machinery of the original widget plus the extra RTT + stereo layer overhead.

    4. Plain widget (orange colored). For some reason this time around it does not show aliasing on the edges like it did in the image I posted earlier. So I think this shows that MSAA is actually enabled. Though if I added elements to it I'd be back at 2.

    Let me know if you have any thoughts. Thanks again for the help.

  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    @rpalandri btw, when you say to try turning MSAA x4 on by changing:

    const uint32 SampleCount = 2; to 4.

    Would this require recompiling the engine source or can it be done with the Epic Launcher engine versions? I ask because I don't know if you read a previous comment where I explained that I downloaded the engine source and recompiled adding logs to see if multisampling was being set for the FBO, and while I was able to compile the engine and run the editor, attempting to package the sample project for Android would fail. It would say there were a lot of missing files which I have no idea why would be missing (since compiling the engine and running inside the editor worked fine).


  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    Ok I closed the question as answered because MSAA itself is enabled and this was answered. But I still have pending questions:

    @rpalandri

    About MSAA:

    1. Does changing the sample count from the default of 2 to 4 require a full engine source recompilation?

    About UMG:

    1. Is there a way to control antialiasing of text? That is, of the textures generated from the text? Maybe something like filtering options or maybe something like Unity's canvas scaler settings?


    2.What would the optimal settings be as far as mipmapping and whatnot for the UMG images in the widget?


    About stereo layers:

    3. Why do they not render the rounded borders of the original UMG widget?


    4. What should I do with the original widget? Rotating it 180deg so that it does not show seems not optimal.

    5. What about raycasting and interacting with controls in the original widget? Do I need to compute the diff between the widget pos / rot and the stereo layer's and mimic any interaction onto the original widget? 
  • rpalandrirpalandri Posts: 63 Oculus Staff
    Alrighty I'll try to answers the questions I know the answer to :)

    @motorsep 4x MSAA is not free on Snapdragon/Adreno (especially on older snapdragon chips), unlike on Mali, which is why we defaulted it to 2x. A better way to do this, coming normally around 4.16, would be to do 4x Mali, 2x Snapdragon by default.

    For the other questions:
    1) yeah changing it from 2 to 4 requires entire engine recompilation. Also, pressing the "Compile" button in the editor does NOT recompile the engine! (it'd have a hard time doing that, since the editor is running the engine ;) ). It recompiles your game's specific C++ classes, but not the underlying engine. 

    UMG.1) I'm not sure about UMG's rendering of text, I haven't seen antialiasing options but I'm far from a UMG expert. You can try rendering to a bigger UMG texture, then mipmapping it down to levels you want.

    UMG.2) Depends of how far you're gonna be from the widget. You want to have a mipmap whose size is approximatively equivalent to the number of pixels the layer will take on the screen. There are 2 or 3 carmack posts out there on layers and mipmapping for text, I'd try to read those.

    Layers.3) A layer only renders the texture you give it to. It's really straightforward. In your app (I opened it), you gave it a quad texture, so it rendered a quad. If you want rounded borders, you need to put a transparency-based rounded border on your texture. That is not, if you look, how the widget does its round borders though! The widget does it by creating a mesh whose borders are round, not by modifying the texture, therefore that will have no effect on the texture, and therefore no effect on the layer. I'm sure however that you can do your own UMG widget class that creates a rounded texture, which will work with stereo layers.

    Layers.4) In this https://forums.oculus.com/developer/discussion/44130/gear-vr-rendering-umg-widget-in-stereolayer-component-how-bp-only thread, I showed 2 ways of doing widget to texture render, one using blueprint (that I don't like at all), one using code. The blueprint one uses render to material, and then extracts the texture from the material (which therefore needs the material to be rendered to, which will only happen if the widget is in view). In that case, the best way I know of like you said, is to turn the widget 180 degrees. If you use the code version though (which uses a function library that can be trivially added to any blueprint project), you can ask the widget to render to a texture manually, which shouldn't necessitate the widget to be in view. 

    Layers.5) In order to use gazetracking, you want to use this: https://docs.unrealengine.com/latest/INT/Engine/UMG/HowTo/InWorldWidgetInteraction/ to basically emulate mouse input with gaze. It should be pretty straightforward, but to be honest I haven't used it yet.

    You also had a question on performance. StereoLayers, due to the number of copies it requires (one more than non-stereolayers), will be a bit more expensive than non-stereolayers. In the parallel universe where I have free time to work on this, I'd like to have a way to output the layer's swapchain directly into the UMG renderer to avoid that extra copy. I'll try to get on that some day, promise! However, since Stereolayers are used for UI, I don't think you should have performance issues since those are the moments where the game doesn't require too much extra perf from physics, stuff like that. 

    To finish, I'd like to reiterate that if you want to do quads without stereolayers, you 1000000% need to use alpha blending on the side of your textures. I REPEAT. use alpha blending. If you have to remember anything from those long posts, it's that. The only reason why I say "without stereolayers" here is that we already manually do alpha blending for you guys when you do stereolayers, so it's not necessary to do it twice.



  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    edited December 2016
    @rpalandri

    just an update, which unfortunately I did not get to do before you replied.

    About MSAA:

    I already figured out how to recompile the engine, etc. But the line you had suggested to change MSAA sample count (OpenGLRenderTarget.cpp: 152) does not actually change the sampleCount. That whole code path inside the #ifdef android does not execute when launching the game in the device. Further, there is a comment above line 152 which implies the sampleCount is hardcoded in other places. 

    And the FBO after that line implies that code path is only for multi view which I am not using since it is experimental and @motorsep said was kind of buggy.

    So I wasn't able to increase MSAA from x2 to x4. I'd need to know where else in the engine code this needs to be changed.

    About Stereo Layers:

    I'm using the C++ version of RTT btw (where you exposed the C++ static method as a BP function). But I'm not sure I follow how to get the stereo layer to render the rounded corners.

    From what I know about UMG so far, it provides no out of the box (vertex or texture based) way of doing rounded corners for its elements. So The 2 ways I was able to do rounded corners were:

    1. Creating an UMG border element with a "rounded rect texture" as the brush.
    2. Creating a copy of the material the whole widget uses to render in the world and:
      a. setting the blend mode to mask.
      b. adding a sampler to the opacity mask value, where I sample a "rounded rect" texture.

    The disadvantage of approach 1 is that even if I make the UMG border use the rounded rect texture, since I place an UMG image element as a child of it (on top of it), this image covers the top rounded corners of the border. So in this sense approach 2 works best because it literally masks the otherwise sharp square UMG widget.

    If I use approach 2 and then do a RTT I believe I lose the rounded corners because these are a masking step that takes place AFTER the UMG widget has rendered, so I think it is not captured by rendering the widget to another texture. At least that's my assumption on why I was not getting rounded corners when doing the stereo layer.

    If I use approach 1, I do get the rounded borders (only on the bottom of the card because of what I explained earlier), but the resulting render target that eventually becomes the texture of the stereo layer has a black background. So there is no transparency.

    Does this make sense? I mean that in UMG, I assign a texture that is a black rounded rect with transparency (for the corner areas visible since the inner black rounded rect has rounded corners) and while the UMG widget in world space respects the transparency in the borders, the resulting render target instead has black in the borders instead of transparency. 

    So I'm not sure how I would add a transparent mask to the texture after rendering and before adding it to the stereo layer, if I understood correctly your previous message.

    The thing that nags me the most about the stereo layer is having to still have the widget around... seems clunky and a performance waste. But maybe I'm wrong and it's not that big of a deal; I honestly just don't know enough about UE yet.

    And yes, I'll take your advice on Alpha blending.
  • rpalandrirpalandri Posts: 63 Oculus Staff
    I'm not sure how to achieve that, but on the sample I gave motorsep, I had transparency working in the UMG-rendered stereolayer (it was a cat picture with text underneath it with transparent background), so the same kinda technique should work there. Haven't played much with UMG. good luck.
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    @rpalandri Sorry but I'm not sure I understood what you meant by alpha blending in this context. Alpha blending to me just means using the alpha channel for transparency. From this, I'd understand that you mean, that I should make a square texture with rounded borders within the texture and the area difference on each edge of the square texture (square edge - rounded rect) be transparent. If this is what you mean, I was already doing that; otherwise I would not have gotten rounded borders. Or do you mean something else?
  • saldavonschwartzsaldavonschwartz Posts: 20
    Brain Burst
    @rpalandri
    Hey btw, I went back to your UILayer example and the stereo layer does not show transparency. In the original widget, you have the text "this is why they don't let me do UI anymore" over a transparent background (with the cat image ending just above this text). But the rendered target fills the transparent background with black. This is consistent with what I experienced: that there seems to be no transparency when rendering to texture.
Sign In or Register to comment.