Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Uploading video WITH 360 AUDIO to Samsung Gear VR

ashcowanashcowan Posts: 11
edited March 2016 in Audio SDK Development
Hi there,

I have been uploading 360 videos to the 360 Videos folder on the Samsung Gear VR micro SD card for some time now. All these videos have been with stereo sound. I am now ready to upload videos with 360 binaural sound, which will respond the motion of the head and stay in sync with the changing video. How can I do this? I see that the Strangers Patrick Watson video has separate sound files. Is this the way to go?

Thanks

Ashley

Comments

  • BrianHookBrianHook Posts: 101 Oculus Staff
    We're actually unfamiliar with that format since it was developed externally. You might try the Gear VR forums in case someone knows there?
  • SamsungDTLSamsungDTL Posts: 11
    NerveGear
    MilkVR supports 5.1 spatial audio that can just be baked into the mp4 file. An example video with that spatial audio is here that can be played in MilkVR: https://milkvr.com/cdn/assets/CarBee51.mp4. More details on our spatial audio spec can be found here: https://milkvr.com/#/content/spec. We are usually able to convert the typical binaural separate wave + mp3 files into this 5.1 format. This also has the advantage that it is easier to stream.
  • xfghsfghxfghsfgh Posts: 9
    I have done this and it works!

    Created the 5.1 mix in logic, bake to mp4 and put on the card.

    Only annoying thing is MilkVR isn't available in the UK so I have to use Hola and then turn off Wifi.
  • calgapacalgapa Posts: 5
    NerveGear
    Hi xfghsfgh. Do you have the exact settings for the spatial sound encoding ? I have created several AAC sound files, baked them into an mp4 but none are played by MILK Vr. I have Logic Pro as well as Premiere.

    Please help.
  • Spheric-collectionSpheric-collection Posts: 4
    NerveGear
    BrianHook wrote:
    We're actually unfamiliar with that format since it was developed externally. You might try the Gear VR forums in case someone knows there?
    Brian,

    See your conference on youtube, i wish i was there but....

    My question is, How can i proceed without any Videogames software (unity, wwise ect) to provide sound file for oculus.
    I hope i can work with Third order ambisonic and then decode the sound scene in any axis i need. The best way would be to have a sound engine (API or i don't know what) that read ambisonic files and decode them in binaural considering the head movement tracking.
    I had to admit that i can't found information to make a file for Oculus.Can you give me a cooking lesson to implement an external mix (i mean created on a standard DAW)in an Oculus File.

    Thank you very much for considering my question.
  • BrianHookBrianHook Posts: 101 Oculus Staff
    I'm not quite sure what you're trying to do -- can you tell me what specific application you're trying to develop?
  • Spheric-collectionSpheric-collection Posts: 4
    NerveGear
    BrianHook wrote:
    I'm not quite sure what you're trying to do -- can you tell me what specific application you're trying to develop?
    Hi Brian,

    I get some answer since the last time. The thing is that every VR player don't tell much about audio specs.
    How many stream, which codec, sample rate etc... a lot of questions that remain unsolved.

    It seems that the best way is to deliver 4 binaural Files for each the quadrant (0°,90°,180°,270°). MilkVR choose this way, but i can't find a desktop player that is doing the same.

    Samsung VR is our final player, but i need to check my final mixing before sending them to my clients. Is their any bridge between OculusDK2 and samsung VR, A player that is doing the same thing (Mixing the 4 binaural files considering the head tracking position).

    Thanks
  • SassotaSassota Posts: 19
    NerveGear
    I think you did, but I will ask anyway,

    Did you simulate this inside DAW doing simply volume automations? The result was good?

    Good luck on your quest
  • Spheric-collectionSpheric-collection Posts: 4
    NerveGear
    I do!

    It was convincing. But i need something that do it following the headtracker.
  • SassotaSassota Posts: 19
    NerveGear
    Do you use FMOD? I don´t know if you can track oculus head position writing an specific code, but using parameters inside an event, you can change between four audio files more freely and test more possible scenarios.
  • ashcowan wrote:
    Hi there,

    I have been uploading 360 videos to the 360 Videos folder on the Samsung Gear VR micro SD card for some time now. All these videos have been with stereo sound. I am now ready to upload videos with 360 binaural sound, which will respond the motion of the head and stay in sync with the changing video. How can I do this? I see that the Strangers Patrick Watson video has separate sound files. Is this the way to go?

    Thanks

    Ashley

    I do have the same question Ashley. I came from Audio Post Production studio with Pro Tools as my main DAW. I would like to know how to encode 360 audio streams (whatever it is called :-), to a video, music video for example. I have no idea how to go about it. I have been trying to find information on the internet without and answers yet. Some of the questions I have are, 1. what is the audio format? 2. How many channels of audio needed? What software/plugin to be used to encode that audio file? Of course, the head tracking thing is the feature which makes exciting.

    This is the example of the video I am referring: https://youtu.be/SB8wg-icjfo

    I believe there are 4 channels of audio tracks.

    Wayne
  • SamsungDTL wrote:
    MilkVR supports 5.1 spatial audio that can just be baked into the mp4 file. An example video with that spatial audio is here that can be played in MilkVR: https://milkvr.com/cdn/assets/CarBee51.mp4. More details on our spatial audio spec can be found here: https://milkvr.com/#/content/spec. We are usually able to convert the typical binaural separate wave + mp3 files into this 5.1 format. This also has the advantage that it is easier to stream.

    Can't access those files anymore... any other link available?
    xfghsfgh wrote:
    I have done this and it works!

    Created the 5.1 mix in logic, bake to mp4 and put on the card.

    Only annoying thing is MilkVR isn't available in the UK so I have to use Hola and then turn off Wifi.

    xfghsfgh: did it spatialize properly? Could you confirm that the channel order is: left, right, center, lfe, left surround and right surround?

    Thanks
  • gp360gp360 Posts: 7
    NerveGear
    I've also just tested this and it works fine. Output a standard 5.1 mp4 from Premiere straight into Milk VR, and the tracking is spot on. Problem now is trying to work out how to get it to work with the Oculus VR app..
  • YggdrasilburyYggdrasilbury Posts: 7
    NerveGear
    Does anyone here have experience from baking mp4 files with ambisonic/spatial audio that works with the head tracker?
    And if so, would that work for Cardboard, too?
  • dahnovkadahnovka Posts: 1
    Does anyone here have experience from baking mp4 files with ambisonic/spatial audio that works with the head tracker?
    And if so, would that work for Cardboard, too?

    bump, looking for the same info
  • audiospillaudiospill Posts: 6
    NerveGear
    Does anyone here have experience from baking mp4 files with ambisonic/spatial audio that works with the head tracker?
    And if so, would that work for Cardboard, too?
    I'm also trying to figure out how to get spatial audio working on Samsung Gear VR through the Oculus player. I have come across the workflow mentioned on another thread which involves muxing a 4 channel .wav file with an .mkv video file, but when I try to play this on the player, it just plays the static stereo - no spatial audio.

    Weird thing is the same video works fine on YouTube with spatial audio. To put it up there, I had to inject spatial metadata (https://support.google.com/youtube/answer/6395969?hl=en-GB&ref_topic=2888648) but I'm guessing this isn't required for the Oculus player?

    Anyone who has managed to successfully upload video with spatial audio to Samsung Gear VR, please share how you did it.

    Thanks!
  • RonaldvanderSpekRonaldvanderSpek Posts: 5
    NerveGear
    edited June 2016
    It Works. :D thank you all... i missed the _360 et the end of the file. 
  • ThamosThamos Posts: 6
    Hi !
    I'm making a video for the oculus player on a Gear VR but i don't know if there is some metadata or a specific name like _amb or _ambix. Does anyone knows ?
    Thanks
  • PetrozaPetroza Posts: 147 Oculus Staff
    edited July 2016
    The other thread has more info. https://forums.oculus.com/community/discussion/comment/343286/#Comment_343286

    To answer specific questions here:
    MP4 is not supported because the Android AAC decoder downmixes everything to stereo which prevents us doing ambisonics, this may change in the future.

    The recommended format is MKV(H.264 video with Vorbis Audio)

    There is no metadata required, if it's a 360 video with 4 channel audio it assumes that is first order ambisonics in the AmbiX (ACN/SN3D) format.
  • Skaven252Skaven252 Posts: 45
    Brain Burst
    I got it to work too, with 4-channel spatial audio recorded with a Zoom H2n!

    However, I'm having an odd problem when I try to use 5.1 channel audio. It either works with spatial sound in theater mode, or without spatial sound in 360 mode. If the file is named video.mkv, it plays in theater mode and the soundscape is spatialized. If I simply rename the file to video_360.mkv, it plays in 360 but the audio does not spatialize.

    So... do 360 videos only support 4-channel spatialization?
  • RobTFRobTF Posts: 2
    NerveGear
    Has anyone created a good step by step process of adding in the audio into video?
Sign In or Register to comment.