Unity 5 Bug: Left Eye renders incorrectly on device — Oculus
Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Unity 5 Bug: Left Eye renders incorrectly on device

rjmig88rjmig88 Posts: 83
edited April 2015 in Oculus Go Development
Update:
Making the OVRCameraRig a top level object fixes this left eye rendering issue for me. It seems to occur if you make it a child of another game object and move that object around on device on android.

Original Issue:

System Info:
Unity 5.0.0p2
Mobile SDK 0.4.3.1
Exynos Gear VR Android 5.0.1 Lollipop
Android ETC2 (OpenGL ES 3.0) forced. Using default Mobile SDK Project Settings.

On device only the left eye is rendering incorrect light values/etc in complex scenes involving multiple shaders. I haven't quite figured out what is going on yet. Here is an example picture from the Top Down Assets (https://www.assetstore.unity3d.com/en/#!/content/5860) demo scene:
http://i.imgur.com/y3k7qd8.jpg

The left and right eye is rendering correctly both in editor(set to android platform) and on Windows build.

I'm not really sure where to begin to troubleshoot this issue. I've noticed on a simpler scene if I use the VRGUI class - viewtopic.php?f=37&t=4944 then it triggers the issue if I have mixed shaders (some objects using the bump diffuse shaders, others using the parallax diffuse shaders, etc.) It goes away if I use the same shader for everything. My best guess currently is anything relying on RenderTexture on device will trigger this issue.

Has anyone else ran across this issue? Has anyone else been able to resolve it?

Thanks!

Comments

  • rjmig88rjmig88 Posts: 83
    I saw Unity just released 5.0.0p3. I upgraded to it and the same issue still occurs.
  • stan4oculusstan4oculus Posts: 15
    NerveGear
    Hi,

    I do not know if this is related or not but we get left eye flickering a lot.
    It is left eye only and on phone only.
    We use 4.6.3 Unity and 4.4 android.
    Flicker happens only if we've got forced Gles3.
    Oculus camera is set to mono so exactly same picture is expected to be rendered but its not apparently.
    Muti-threaded rendering is on. if it is off then both eyes flicker horribly.
    With Gles 2 it seems to be ok.

    Stan
  • rjmig88rjmig88 Posts: 83
    Hi Stan,

    I've had left eye flickering as well. That went away with the Lollipop update so I'd highly encourage you to try that. The issue I'm reporting in this thread seems to be an issue with render textures, Unity 5, and the mobile SDK. I have a sample project I'm willing to share with Oculus and Unity if they're interested.

    Cheers!

    - Randy
  • rjmig88rjmig88 Posts: 83
    Just tried mobile SDK 0.5.0 on Unity 5.0.0p3. I also tried forcing OpenGL ES 2.0. I'm still having the same issues on device only. http://i.imgur.com/LbnjnJy.jpg

    If I run this script (attached to the OVR camera rig or attached to the right/left eye anchor) then the left eye renders incorrectly for mixed shader content. If I don't run the UI then both eyes are fine.
    using UnityEngine;
    using System.Collections;
    
    
    public class FpsCounter : VRGUI
    {
        // Attach this to a GUIText to make a frames/second indicator.
        //
        // It calculates frames/second over each updateInterval,
        // so the display does not keep changing wildly.
        //
        // It is also fairly accurate at very low FPS counts (<10).
        // We do this not by simply counting frames per interval, but
        // by accumulating FPS for each frame. This way we end up with
        // correct overall FPS even if the interval renders something like
        // 5.5 frames.
    
        public float updateInterval = 0.5F;
    
        private float accum = 0; // FPS accumulated over the interval
        private int frames = 0; // Frames drawn over the interval
        private float timeleft; // Left time for current interval
    
        private string fpsStr;
    
        public override void OnVRGUI()
        {
            GUILayout.BeginArea(new Rect(Screen.width / 2.25f, Screen.height / 2.25f, Screen.width, Screen.height));
            GUILayout.Label(fpsStr);
            GUILayout.EndArea();
        }
    
    
        void Start()
        {
            timeleft = updateInterval;
        }
    
        void Update()
        {
            timeleft -= Time.deltaTime;
            accum += Time.timeScale / Time.deltaTime;
            ++frames;
    
            // Interval ended - update GUI text and start new interval
            if (timeleft <= 0.0)
            {
                // display two fractional digits (f2 format)
                float fps = accum / frames;
                string format = System.String.Format("FPS: {0:F2}", fps);
                fpsStr = format;
    
                //	DebugConsole.Log(format,level);
                timeleft = updateInterval;
                accum = 0.0F;
                frames = 0;
            }
        }
    }
    

    And here is the code for VRGUI. It is really simple, it just creates a RenderTexture, draws the UI to that texture, then puts it in world space out in front of the camera.
    using UnityEngine;
    using System.Collections;
    /// <summary>
    /// Source code by boone188:
    /// https://forums.oculus.com/viewtopic.php?f=37&t=4944
    /// </summary>
    public abstract class VRGUI : MonoBehaviour 
    {
    	public Vector3 guiPosition      = new Vector3(0f, 0f, 1f);
    	public float   guiSize          = 1f;
    	public bool    useCurvedSurface = true;
    	public bool    acceptMouse      = true;
    	public bool    acceptKeyboard   = true;
    	public int     cursorSize       = 32;
    	public Texture customCursor     = null;
    	
    	private GameObject    guiRenderPlane    = null;
    	private RenderTexture guiRenderTexture  = null;
    	private Vector2       cursorPosition    = Vector2.zero;
    	private Texture       cursor            = null;
    	
    	private bool isInitialized = false;
    	
    	private void Initialize () 
    	{
    		// create the render plane
    		if (useCurvedSurface)
    		{
    			guiRenderPlane = Instantiate(Resources.Load("VRGUICurvedSurface")) as GameObject;
    		}
    		else
    		{
    			guiRenderPlane = Instantiate(Resources.Load("VRGUIFlatSurface")) as GameObject;
    		}
    		
    		// position the render plane
    		guiRenderPlane.transform.parent        = this.transform;
    		guiRenderPlane.transform.localPosition = guiPosition;
    		guiRenderPlane.transform.localRotation = Quaternion.Euler(0f, 180f, 0f);
    		guiRenderPlane.transform.localScale    = new Vector3(guiSize, guiSize, guiSize);
    		
    		// create the render texture
    		guiRenderTexture = new RenderTexture(Screen.width, Screen.height, 24);
    		
    		// assign the render texture to the render plane
    		guiRenderPlane.GetComponent<Renderer>().material.mainTexture = guiRenderTexture;
    		
    		if (acceptMouse)
    		{
    			// create the cursor
    			if (customCursor != null)
    			{
    				cursor = customCursor;
    			}
    			else
    			{
    				cursor = Resources.Load("SimpleCursor") as Texture;
    			}
    		}
    		
    		isInitialized = true;
    	}
    	
    	protected void OnEnable()
    	{
    		if (guiRenderPlane != null)
    		{
    			guiRenderPlane.SetActive(true);
    		}
    	}
    	
    	protected void OnDisable()
    	{
    		if (guiRenderPlane != null)
    		{
    			guiRenderPlane.SetActive(false);
    		}
    	}
    	
    	protected void OnGUI()
    	{
    		if (!isInitialized)
    		{
    			Initialize();
    		}
    		
    		// handle mouse events
    		if (Event.current.isMouse)
    		{
    			// return if not accepting mouse events
    			if (!acceptMouse)
    			{
    				return;
    			}
    		}
    		if (acceptMouse)
    		{
    			// save the mouse position
    			cursorPosition = new Vector2(Input.mousePosition.x, Screen.height - Input.mousePosition.y);
    		}
    		
    		// handle key events
    		if (Event.current.isKey)
    		{
    			// return if not accepting key events
    			if (!acceptKeyboard)
    			{
    				return;
    			}
    		}
    		
    		// save current render texture
    		RenderTexture tempRenderTexture = RenderTexture.active; 
    		
    		// set the render texture to render the GUI onto
    		if (Event.current.type == EventType.Repaint)
    		{			
    			RenderTexture.active = guiRenderTexture;
    			GL.Clear (false, true, new Color (0.0f, 0.0f, 0.0f, 0.0f));
    		}
    		
    		// draw the VRGUI
    		OnVRGUI();
    		
    		if (Event.current.type == EventType.Repaint)
    		{	
    			if (acceptMouse)
    			{
    				// draw the cursor
    				GUI.DrawTexture(new Rect(cursorPosition.x, cursorPosition.y, cursorSize, cursorSize), 
    					cursor, ScaleMode.StretchToFill);
    			}
    			
    			// restore the previous render texture
    			RenderTexture.active = tempRenderTexture;
    		}
    	}
    	
    	public abstract void OnVRGUI();
    }
    

    Has anyone else been having issues? This is a major blocker for me in general and for the VR jam.
  • cyberealitycybereality Posts: 26,156 Oculus Staff
    Can you provide me with a logcat from the device?
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • cyberealitycybereality Posts: 26,156 Oculus Staff
    Also, can you share a sample project that shows the issue? I can have someone take a look.
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • rjmig88rjmig88 Posts: 83
    Thank you cybereality! I'll send you a PM with a link to my project and with instructions on how to trigger the rendering bugs. Another bug I found today I added some trees to the scene and somehow the tree texture on device got baked into the house texture, even removing the trees from the scene and rebaking it in unity. That doesn't happen either in the editor or on windows. The tree texture goes away if I enable the fps counter (left eye still busted), but otherwise remains on the "normal" run. The project I'm including has the trees as well as everything else.

    Picture with tree texture incorrectly baked on the roof on device:
    http://i.imgur.com/zpIjlRQ.jpg

    I've uploaded the logs from both runs to this post.
  • IronManIronMan Posts: 139
    I'm pretty sure in the mobile docs they say "don't use OnGUI()". If they don't, they should.

    Besides being highly inefficient, time warp on android has a specific order in which all the rendering takes place.

    Take a look at how OVRScreenFade works and try to use OnCustomPostRender to get your GUI code working. You may have to modify OVRPostRender to run and get added on PC. I don't remember what the current released state of that code is.

    Generally, on mobile, anything you want to render on top of your scene (and work correctly in both eyes) you should use the OnCustomPostRender message.
  • rjmig88rjmig88 Posts: 83
    Thank you so much! I totally missed that part in the mobile documentation where it says Do not use Unity’s OnGUI() calls. Since it is labeled under CPU Optimizations my thoughts were that you just had to be careful since it may get called multiple times per frame. I didn't realize that it would result in graphical glitches due to rendering order.

    I'll give your method a shot and report back.
  • rjmig88rjmig88 Posts: 83
    I'm still working on it. So far I'm having absolutely zero success going off the OVRScreenFade example. One problem is that the GUI calls in Unity can only be made when the engine is calling OnGui(), otherwise this exception is being thrown - ArgumentException: You can only call GUI functions from inside OnGUI.

    Does stuff rendered in on OnPostRender() or OnCustomPostRender() still get distorted properly?

    I personally would think it is a reasonable request to have the mobile engine rendering path support OnGui() and have parity to the desktop DK2 for rendering. The VRGUI approach was a really nice technique to throw UI up quickly and natively with options to vision track or be in fixed world coordinates. Right now it seems like I have to resort to adding UI directly in the scene like Oculus's Menu_Sample.unity scene shows which can be problematic if the user gets too close to something and the UI clips another object.

    Also, that still doesn't solve the issue where the left eye is rendering on the demo scene incorrectly. That scene isn't using any OnGui() calls.
  • IronManIronMan Posts: 139
    You'd have to draw your render texture using the GL commands that OVRScreenFade uses.. not using GUI calls.
  • rjmig88rjmig88 Posts: 83
    I got that... Some reason the fade example doesn't work on device but works on the DK2. Either way it still doesn't solve this issue which isn't using the UI at all:
    http://i.imgur.com/y3k7qd8.jpg
  • IronManIronMan Posts: 139
    OK.. so I went back to your original post to see exactly what the problem was because all I was seeing was the white (corrupted) texture on the roof of the building and poor performance (judder). So I originally missed the comments about using parallax and bump shaders and all that. Those kind of shaders aren't going to work on mobile. You may get by using Mobile / Bumped Diffuse, but without per pixel lighting, that's not going to gain you anything. The other shaders that use render textures or grab pass or whatever will just kill your performance. Same with the cutout textures on the trees.. Any time you see "cutout" in the Unity shader names, that's using alpha test which performs really bad on mobile.

    You'll want to use baked lighting etc. Read up on it Unity integration guide.

    As for your GUI stuff.. OnGUI isn't going to work.. If you're just wanting to display some UI, I'd take a look at Unity's new UI instead.

    Hope that helps.
  • rjmig88rjmig88 Posts: 83
    I understand the performance implications of using various shaders. These are mobile optimized objects. I'm a professional game programmer with 5 years of industry experience working on a large project. I'll ensure my game has constant 60 fps. This thread is reporting rendering bugs in the left eye.

    Are you suggesting that the Gear VR is incapable of implementing advanced rendering techniques past 1999? That we should just stick to basic textures and static lights? That anything more won't render correctly in each eye?

    I've ripped out my OnGui() code and other than the roof issue its still having these other issues:
    Tree texture incorrectly being put on the house's roof, same issue as: viewtopic.php?f=67&t=21985
    Terrain messed up on the sample scene from the art pack in the left eye.
    Both of these render correctly with a normal camera and not the OVRCameraRig.
  • roswell108roswell108 Posts: 168
    Not sure if you're still having trouble, but the VRGUI works with Unity 5 and the mobile SDK. People tend to see 'OnGUI' and assume that is an issue without looking any further but it shouldn't be if you have added VRGUI to convert it.

    Typically, OnGUI is ignored when running with the mobile SDK. At least that's how it was. It's hard to say if that has been changed or broken by recent updates. It doesn't attempt to render to both eyes, though, so it should not cause only one to stop rendering. It bypasses the stereoscopic view, so it would usually break the full view.

    I am attaching my modified copy of the VRGUI.cs script code that will fix issues with it rendering incorrectly and causing block overlays when used with the mobile SDK
    using UnityEngine;
    using System.Collections;
    
    public abstract class VRGUI : MonoBehaviour 
    {
    	public Vector3 guiPosition      = new Vector3(0f, 0f, 1f);
    	public float   guiSize          = 1f;
    	public bool    useCurvedSurface = true;
    	public bool    acceptMouse      = true;
    	public bool    acceptKeyboard   = true;
    	public int     cursorSize       = 32;
    	public Texture customCursor     = null;
    	
    	private GameObject    guiRenderPlane    = null;
    	private RenderTexture guiRenderTexture  = null;
    	private Vector2       cursorPosition    = Vector2.zero;
    	private Texture       cursor            = null;
    	
    	private bool isInitialized = false;
    	
    	private void Initialize () 
    	{
    		// create the render plane
    		if (useCurvedSurface)
    		{
    			guiRenderPlane = Instantiate(Resources.Load("VRGUICurvedSurface")) as GameObject;
    		}
    		else
    		{
    			guiRenderPlane = Instantiate(Resources.Load("VRGUIFlatSurface")) as GameObject;
    		}
    		
    		// position the render plane
    		guiRenderPlane.transform.parent        = this.transform;
    		guiRenderPlane.transform.localPosition = guiPosition;
    		if (transform.rotation.y == 180f) {
    			guiRenderPlane.transform.localRotation = Quaternion.identity;
    		} else {
    			guiRenderPlane.transform.localRotation = Quaternion.Euler(0f, 180f, 0f);
    		}
    		guiRenderPlane.transform.localScale    = new Vector3(guiSize, guiSize, guiSize);
    		
    		// create the render texture
    		guiRenderTexture = new RenderTexture(Screen.width, Screen.height, 24);
    		
    		// assign the render texture to the render plane
    		guiRenderPlane.GetComponent<Renderer>().material.mainTexture = guiRenderTexture;
    		
    		if (acceptMouse)
    		{
    			loadCursorGraphic();
    		}
    		isInitialized = true;
    	}
    	
    	protected void OnEnable()
    	{
    		if (guiRenderPlane != null)
    		{
    			guiRenderPlane.SetActive(true);
    		}
    	}
    	
    	protected void OnDisable()
    	{
    		if (guiRenderPlane != null)
    		{
    			guiRenderTexture.DiscardContents();
    			guiRenderPlane.SetActive(false);
    		}
    	}
    
    	private void loadCursorGraphic() {
    		if (cursor == null) {
    			// create the cursor
    			if (customCursor != null)
    			{
    				cursor = customCursor;
    			}
    			else
    			{
    				cursor = Resources.Load("Textures/SimpleCursor") as Texture;
    			}
    		}
    	}
    	
    	protected void OnGUI()
    	{
    		OnEnable();
    		if (!isInitialized) {
    			Initialize();
    		}
    		
    		// handle mouse events
    		if (Event.current.isMouse)
    		{
    			// return if not accepting mouse events
    			if (!acceptMouse)
    			{
    				return;
    			}
    		}
    		if (acceptMouse)
    		{
    			loadCursorGraphic();
    			// save the mouse position
    			cursorPosition = new Vector2(Input.mousePosition.x, Screen.height - Input.mousePosition.y);
    		}
    		
    		// handle key events
    		if (Event.current.isKey)
    		{
    			// return if not accepting key events
    			if (!acceptKeyboard)
    			{
    				return;
    			}
    		}
    		
    		// save current render texture
    		RenderTexture tempRenderTexture = RenderTexture.active; 
    		
    		// set the render texture to render the GUI onto
    		if (Event.current.type == EventType.Repaint)
    		{
    			RenderTexture.active = guiRenderTexture;
    			GL.Clear (true, true, Color.clear, 1.0f);
    		}
    		
    		// draw the VRGUI
    		OnVRGUI();
    		
    		if (Event.current.type == EventType.Repaint)
    		{	
    			if (acceptMouse)
    			{
    				// draw the cursor
    				GUI.DrawTexture(new Rect(cursorPosition.x, cursorPosition.y, cursorSize, cursorSize), cursor, ScaleMode.StretchToFill);
    			}
    			
    			// restore the previous render texture
    			RenderTexture.active = tempRenderTexture;
    		}
    	}
    	
    	public abstract void OnVRGUI();
    }
    

    The issue is more often that the mobile SDK doesn't clear things properly. If you are having issues with a specific item, try adding a 'GL.Clear (true, true, Color.clear, 1.0f);' just before whatever you have displaying it for the first time.

    In my case, I have a menu with a floating RenderTexture and when I first initialize the RenderTexture in the scene is when I fire off a quick 'GL.Clear (true, true, Color.clear, 1.0f);'
  • rjmig88rjmig88 Posts: 83
    Thank you roswell108 for the insight. I tried your updated VRGUI code and it still triggers the roof to render incorrectly in the left eye in my sample. It looks like the mobile SDK sets the camera's clearFlags correctly for each eye. It does give me some new insights though poking around the mobile SDK source code.

    I have Unity's 4.6 UI system working for VR fine now. This just leaves the material leaks as discussed in the other thread and the original issue I reported with the terrain.
  • roswell108roswell108 Posts: 168
    rjmig88 wrote:
    Thank you roswell108 for the insight. I tried your updated VRGUI code and it still triggers the roof to render incorrectly in the left eye in my sample. It looks like the mobile SDK sets the camera's clearFlags correctly for each eye. It does give me some new insights though poking around the mobile SDK source code.

    I have Unity's 4.6 UI system working for VR fine now. This just leaves the material leaks as discussed in the other thread and the original issue I reported with the terrain.

    Hopefully you can get it all sorted out. The issue I had was with a skybox turning black. I had to set both cameras for the OVR rig to skybox, but also change the code for the center camera it adds in OVRManager to skybox and on top of all of that do the GL.Clear right when it started up the GUI.

    I do have an issue with the left eye dropping textures for some scenes still. If I come across anything that fixes it, I will definitely make sure to post results here. Speaking of... if you see any of the warnings (insert GPU tiling warning here), resolving them will help. A lot of Unity posts say they can be ignored, but they are usually linked to an issue with the rendering on the mobile SDK.
  • rjmig88rjmig88 Posts: 83
    Thanks! Just tried updating to Unity 5.0.1f1 and these issues remain.
  • roswell108roswell108 Posts: 168
    rjmig88 wrote:
    Thanks! Just tried updating to Unity 5.0.1f1 and these issues remain.

    The list of changes looked so promising, too.
  • CorvusVRCorvusVR Posts: 53
    Hiro Protagonist
    Some assets are not rendering in the left eye. Any suggestions/fixes?

    Unity 4.6.4f1 (free)
    Mobile SDK 5.0

    Forced OpenGL ES 2.0 in settings and AndroidManifest.xml

    Edit: also tried Unity 5.0.1p1
  • rjmig88rjmig88 Posts: 83
    Making the OVRCameraRig a top level object fixes this left eye rendering issue for me. It seems to occur if you make it a child of another game object and move that object around on device on android.
  • creat326creat326 Posts: 84
    This sucks. I'm spending about 80% of the time trying to work around unity 5 bugs instead of actually working on the game at this moment.
    I don't know how much we can do in 2 weeks if most stuff is not working properly :(
  • SvenVikingSvenViking Posts: 296
    Art3mis
    For me he problem seems to be solved by placing UI elements (even completely invisible) over the camera, and avoiding certain shaders.
    Scorched Battalion - VR Jam edition available for Gear VR and DK2 (Win/Mac/Linux)
    Jungle Juice
    Sven Co-op
  • sqsezssqsezs Posts: 1
    NerveGear
    rjmig88 said:
    I saw Unity just released 5.0.0p3. I upgraded to it and the same issue still occurs.

    from the player settings change from multipass to singlepass and the issue will dissapear
Sign In or Register to comment.