[SOLVED] Easy Movie Texture + 3DCeption (2 Big Ears) sound clicks — Oculus
Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

[SOLVED] Easy Movie Texture + 3DCeption (2 Big Ears) sound clicks

tomazvovktomazvovk Posts: 61
Hiro Protagonist
edited March 2017 in Unity Development
I am trying to sync a 360 video with tbe spatial sound using Easy Movie Texture for video playback, and TwoBigEars for 3d sound. I've managed to do this part. The problem is, that there are click-like sounds during video playback. I tried to play the sound separately and there are no problems. Is it possible that video takes too much CPU?

Comments

  • vrdavebvrdaveb Posts: 1,596 Oculus Staff
    If you use H.264 or 265, the video will be decoded in hardware. You can also attach Unity's profiler to see what's consuming CPU. Clicking sounds are sometimes due to audio buffer underruns - what kind of filtering are you doing with TBE? Does the issue repro with the Oculus spatializer?
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    edited September 2016
    Video is HEVC (H.265), stereo, 2048x2048. I checked with profiler and there are spikes happening on CPU around the times the sound gets glitchy. Most of CPU time is spent on VR.WaitForGPU during these glithces. This takes anywhere from 14ms to 24ms. If I understand that correctly, this is the time CPU waits for GPU to update?

    I made a clean project with the same video and audio and results are pretty much the same. 

    I'm using Unity 5.4.0p3, Oculus Utilities 1.8.0 and TB Audio Engine 0.9.9. with Easy Movie Texture.
    Video and Sound (tbe format) are loaded from SD card. I'm using Galaxy S6, Android 6.0.1.

    Is there a way to play tbe file format using Oculus spatializer?
  • vrdavebvrdaveb Posts: 1,596 Oculus Staff
    > If I understand that correctly, this is the time CPU waits for GPU to update? 

    Yes. Can you send the output of adb logcat? That will include the time spent on GPU.

    > Is there a way to play tbe file format using Oculus spatializer?

    Not directly. You would have to target Unity's built-in audio system and then set the spatializer plugin to Oculus in the project's audio settings.
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    Okay, here is the logcat output from clean project.
  • vrdavebvrdaveb Posts: 1,596 Oculus Staff
    > 09-29 18:55:16.071 15392 15420 D Unity   : tris>          min: 178802  max: 178802  avg: 178802

    This is almost double the recommended number of polygons, which can lead to stalls and dropped frames. Given the low numer of draw calls, my guess is you're drawing a finely-tessellated sphere. Any chance you can use a cubemap projection or lower-res sphere?
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    Yeah I used the default spheres from Easy Movie Texture and forgot to move each sphere to it's own layer, so they were both renderer twice per each eye. Sorry for that.

    I replaced the spheres with the one I'm using in my real project (1224 polygons) and result is the same. I disabled the video and sound is still glitchy. I have a scene where a guy bangs on gongs and this is where it gets the worst. 

    I am using default Unity audio settings, same for TBE.
  • vrdavebvrdaveb Posts: 1,596 Oculus Staff
    edited September 2016
    In Audio Settings, can you try setting DSP buffer size to "Default"? There is a known issue if you use "Best Latency".
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    Hi, I tried with all available options under Audio Settings > DPS buffer size. I tried to disable Default Unity audio and this results in much better sound, no clicking until it reaches the part with gongs. The sound in this part "vibrates" and echoes and it's quite loud.

    I am starting to suspect that something is wrong with the tbe file itself. Default one that comes with TBE plugin works great.


  • nothingtokeepnothingtokeep Posts: 3
    NerveGear
    Hey tomazvovk!

    The official support group for the Spatial Workstation and the SDK is here: https://www.facebook.com/groups/1812020965695437/

    Would you be able to follow through there? The issue you are facing is likely caused by distortion on the device. Turning the mix down before export or changing the volume of the TBSpatDecoder object should help.

    -Varun
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    I tried lowering the volume from 1 to 0.5 and then to 0.1. No change. I joined the Facebook group, waiting for confirmation.

    Thanks!
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    Forgot to mention that this has been resolved by upgrading to newer TBE plugin version.
  • whatthefangwhatthefang Posts: 2
    NerveGear
    Hi tomazvovk said:
    I am trying to sync a 360 video with tbe spatial sound using Easy Movie Texture for video playback, and TwoBigEars for 3d sound. I've managed to do this part. ~
    I'm new to unity, how did you sync the audio and video? I'm using the inbuilt 5.6 video player but I hope the process is the same, also was there a reason you used EasyMovieTexture rather than the new built in movie player?

    Many Thanks.
  • tomazvovktomazvovk Posts: 61
    Hiro Protagonist
    edited June 2017
    Hey, please check the included pdf documentation that comes with FB360 Rendering SDK. Basically what you need to do is to get current video time in miliseconds (from your video player) and pass it to the spatial decoder:
    TBE.TBSpatDecoder Decoder;
    ...
    void Awake() { 
       // Get the instance of TB Spat Decoder on the game object 
       Decoder = DecoderGameObject.GetComponent<TBE.TBSpatDecoder>();
    }
    ...
    void Update() {
       // Get the elapsed time/playback position from the 
       // video player/decoder you are using. 
       // Make sure the value is in milliseconds ! 
       float videoTime = VideoPlayer.getTimeMs() ;
    
       // Pass the video time to TB Spat Decoder
       // This will automatically force it to synchronise
       // with the video ( since the syncMode in Awake() was changed
       // to TB_SYNC_EXTERNAL) 
       Decoder.setExternalClockInMs(videoTime) ; 
    }
    
    I used EasyMovieTexture since native video player was not part of Unity yet.
  • whatthefangwhatthefang Posts: 2
    NerveGear
    Lol thanks, I did read the document, but hadn't a clue what to do with it, so i put that into a script and attach it to whatnow? yup total newb :)

    thanks again.
Sign In or Register to comment.