How to Voip in Unity? — Oculus
Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

How to Voip in Unity?

rhillCampfirerhillCampfire Posts: 23 Oculus Start Member
I'm trying to set up Voip in Unity but can't hear anything.

I've set up a Voip.SetVoipStateChangeCallback() which confirms that msg.GetNetworkingPeer().ID is correct and msg.GetNetworkingPeer().State is Connected.

From there I pass .GetNetworkingPeer().ID to this function:
void SetupAudioSource(UInt64 otherPlayerID)
{
 if (voipAudio == null) { 
  voipAudio = gameObject.AddComponent<VoipAudioSourceHiLevel>();
 }
 voipAudio.senderID =otherPlayerID;
}

The audio source is only 0.2 metres from the player, so it's not too far to hear.


Any ideas as to what I might be overlooking?

( The other fun fact is the audio levels on all non-Voip audio get a lot quieter once the connection starts. Is there some sort of ducking built into this? )

Comments

  • rhillCampfirerhillCampfire Posts: 23 Oculus Start Member
    (I just tried it with the AddComponent<VoipAudioSourceHiLevel>() left out, and confirmed the "ducking" occurs even without it.)
  • rhillCampfirerhillCampfire Posts: 23 Oculus Start Member
    When I logcat this, I do get a lot of "Voip starved!!!, want 1024, but only have 480 available" if that's a sign of anything.
  • rhillCampfirerhillCampfire Posts: 23 Oculus Start Member
    If it helps, I get one
    Voip starved!!!, want 1024, but only have 0 available
    and then a bunch of 
    Voip starved!!!, want 1024, but only have 480 available

    So is that a sign it's getting that far and quitting?

    I logged some other stuff right after the int copied = parent.pcmSource.GetPCM(scratchBuffer, sourceBufferSize); but before the Voip starved!!! message

    data.Length is 2048
    scratchBuffer.Length is 7200
    channels is 2
    sizeToFetch is 1024

    So what does GetPCM do? It's passed scratchBuffer, which has a length of 7200, and sourceBufferSize which is 1024. So why does it return 480?
  • rhillCampfirerhillCampfire Posts: 23 Oculus Start Member
    Okay, so GetPCM is in turn calling this function:
    https://developer3.oculus.com/documentation/platform/latest/sdk-reference/function-voip-getpcmfloat/

    Which does say "The voip system will generate data at roughly the rate of 480 samples per 10ms." and these things are getting called about every 10ms. So I guess that's what's expected, but then why would I get Voip starved messages for getting the amount of data that's expected?


  • rhillCampfirerhillCampfire Posts: 23 Oculus Start Member
    So, in the VoipAudioSourceHiLevel.OnAudioFilterRead function, it returns if it's starved. If I comment that out so it still finishes if it's starved, I do get audio, except it sounds Dalek-like, presumably because it's missing samples. Which at least confirms for me that I am receiving data from the other Gear.
  • bpadgetbpadget Posts: 3
    NerveGear
    Hi!  Sorry you're having problems.  Some amount of audio starvation is normal, but I think you're encountering a bug we recently fixed.  Can you try using the 1.8 SDK unitypackage in your project instead?
  • johnluxfordjohnluxford Posts: 13 Oculus Start Member
    We seem to be having a similar problem. We have peers connecting and transmitting data, and voip is connecting in the same way but no sound is produced. We're on Unity 5.4.1p4 with Platform SDK 1.8, and we've tried with the VoipAudioSourceHiLevel component added manually to the head of the remote players as well as dynamically once the connection status says we're connected.

    Listening for data via Voip.SetMicrophoneFilterCallback() produces consistent PCM data of length 480, and both sides appear to be connected (one starts, the other accepts successfully and both see their voip status change to connected). We get some occasional voip starved notices, but mostly only if we have VoipAudioSourceHiLevel attached and enabled before the connection is established. Once it's established, notices all stop.

    I've added a Debug.Log() at the start of VoipAudioSourceHiLevel.OnAudioFilterRead() to see what's going on, and it seems that once the connection is established, VoipAudioSourceHiLevel.OnAudioFilterRead() stops being called. This is true both in-editor and in a desktop build.

    Below is the code that we're using to ensure the VoipAudioSourceHiLevel is attached, which is called whether we add it manually or not, but seems to make no difference. player.headParent is a reference to the remote head's GameObject, and ID contains the remote user ID from msg.GetNetworkingPeer().ID.

    Any ideas of things we can try to get this working?

    VoipAudioSourceHiLevel voip = player.headParent.GetComponent<VoipAudioSourceHiLevel> ();
    if (voip == null) {
    	voip = player.headParent.AddComponent<VoipAudioSourceHiLevel> ();
    	voip.audioSource.spatialize = true;
    }
    voip.senderID = ID;
    voip.enabled = true;
  • tnovaktnovak Posts: 4 Oculus Staff
    @johnluxford, if you periodically call CAPI.ovr_Voip_GetPCMFloat(), does it return non-zero values once the connection is established?

    VoipAudioSourceHiLevel works by creating an AudioSource and attaching a filter to it.  Unity is expected to call OnAudioFilterRead() whenever it needs more audio for playback; the filter then fills Unity's buffer with audio samples returned by GetPCMFloat().

    1. Are there any other filters attached to headParent's AudioSource?
    2. Does audioSource.Play() get called, and is audioSource.isPlaying true once the connection gets established?

  • cbdileocbdileo Posts: 6 Oculus Start Member
    Has anyone got VOIP working in Unity? For me, CAPI.ovr_Voip_GetPCMFloat() always return 0 and then eventually the OnAudioFilterRead(float[] data, int channels) on VoipAudioSourceHiLevel stops being called. I feel like i'm missing a step

  • tnovaktnovak Posts: 4 Oculus Staff
    @cbdileo, how are you setting up your VoIP connection?  What's the connection state you get in the callback installed via SetVoipStateChangeCallback()?
  • cbdileocbdileo Posts: 6 Oculus Start Member
    @tnovak I did this. And for the SetVoipStateChangeCallback() callback, when I set it up, I doesn't seem to get called.
    var audioSource = gameObject.AddComponent<VoipAudioSourceHiLevel>();
    audioSource.senderID = longValueIGenerated;
    audioSource.spatialize = true;
    Voip.Start(longValueIGenerated);
  • cbdileocbdileo Posts: 6 Oculus Start Member
    @tnovak Another note is I'm not using a Rift but a Vive. Not sure if that matters. 
  • johnluxfordjohnluxford Posts: 13 Oculus Start Member
    edited October 2016
    @tnovak we've upgraded Unity to 5.4.2p1 and the Audio SDK to 1.1.1 since my last post, just to include all potentially relevant info. Just ran some tests and here's what we've found:
    • CAPI.ovr_Voip_GetPCMFloat() returns data with a length of 480 every time (in a coroutine running every second).
    • Our Voip.SetMicrophoneFilterCallback() handler is being called consistently with pcmDataLength being 480 too. This stopped being called before after the connection was established, but that seems to have been fixed now. Still no voices though.
    • voip.audioSource.isPlaying was false, so in our setup method after setting voip.senderID and voip.enabled = true, I added voip.audioSource.Play() and confirmed in my microphone filter callback that isPlaying is true on the audio source.
    I see the Platform SDK 1.9 is out. Gonna try upgrading to that next. Might that help?
  • johnluxfordjohnluxford Posts: 13 Oculus Start Member
    edited October 2016
    Quick correction: GetPCMFloat returns more data when I increased my float[] size, but it's all zeroes.

    Edit: Getting lots of Voip starved messages now.
  • johnluxfordjohnluxford Posts: 13 Oculus Start Member
    Upgraded to Platform SDK 1.9, no change. Lots of Voip starved and manually calling GetPCMFloat is all zeroes.

    Oh, and there are no other audio filters on the component. I've tried with the ONSP Audio Source component attached and without it too.
  • tnovaktnovak Posts: 4 Oculus Staff
    @johnluxford: the fact that ovr_Voip_GetPCMFloat() is returning a non-zero size, but fills the buffer with silence, is concerning.  Does the buffer passed to the microphone filter callback contain all zeros as well?

    AFAICT one scenario in which this would happen is if the SDK failed to acquire the microphone.  Are you running this on Android by any chance?  Microphone access on Android is exclusive, so the game using Unity's Microphone class might prevent the platform SDK from recording audio.  A way to verify this would be grepping for "AudioRecord" in logcat (specifically, errors reported by "startRecording" or "initRecording").

    Re: "Voip starved" errors, calling audioSource.Play() too early might trigger those. VoipAudioSourceHiLevel by default waits until there's enough audio frames already buffered before it starts playing.

    Can you repro this with a smaller, self-contained Unity app?  If so, would you mind sharing the code?  I'd be happy to help debug this further.
  • johnluxfordjohnluxford Posts: 13 Oculus Start Member
    I don't have a simpler test case created yet, but I think we've made some progress. The data we're getting from Voip.SetMicrophoneFilterCallback() is non-zero and coming out like this:
    Length: 480, -12, -10, -10, -9, -8, -7, -6, -3, -3, -2, -1, 0, 0, 1, 2, 2, 5, 5, 7, 7, 9, 10, 11, 11, 11, 13, 13, 14, 15, 16, 16, 17, 17, 17, 18, 17, 17, 17, 17, 17, 17, 17, 17, 16, 15, 15, 14, 14, 14, 13, 13, 12, 12, 13, 11, 11, 11, 10, 8, 8, 7, 7, 7, 5, 4, 4, 2, 1, 0, 0, -2, -4, -5, -6, -6, -7, -7, -10, -9, -10, -10, -10, -12, -10, -10, -10, -9, -10, -9, -8, -8, -8, -8, -6, -7, -6, -5, -4, -4, -4, -3, -3, -2, -2, -2, -2, -1, -1, -1, 0, 0, 0, 0, 0, 1, 0, 1, -1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, -1, 0, -1, -2, -3, -3, -3, -5, -7, -7, -7, -9, -10, -11, -12, -13, -14, -15, -17, -17, -17, -18, -19, -18, -20, -21, -20, -20, -21, -20, -20, -19, -19, -17, -17, -15, -15, -15, -14, -13, -13, -12, -10, -10, -10, -9, -8, -7, -7, -7, -6, -4, -3, -4, -3, -3, -2, -1, 0, 1, 2, 4, 4, 5, 6, 8, 8, 9, 11, 11, 15, 16, 16, 16, 18, 19, 20, 20, 21, 22, 22, 22, 23, 23, 23, 24, 23, 23, 21, 21, 20, 19, 18, 17, 17, 17, 15, 14, 12, 12, 11, 11, 11, 9, 9, 9, 9, 8, 8, 7, 7, 7, 6, 6, 5, 5, 4, 4, 3, 3, 2, 1, 0, 0, -1, -1, -1, -2, -2, -3, -3, -4, -5, -5, -6, -6, -6, -7, -5, -5, -5, -5, -5, -4, -4, -4, -3, -3, -2, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, -1, -1, -2, -2, -3, -3, -6, -6, -6, -7, -8, -9, -9, -10, -10, -11, -12, -12, -12, -12, -13, -12, -13, -13, -13, -12, -12, -12, -12, -12, -13, -14, -14, -15, -15, -15, -16, -16, -16, -16, -17, -17, -17, -18, -19, -18, -19, -17, -17, -17, -17, -15, -14, -13, -12, -11, -9, -8, -6, -5, -4, -2, 0, 1, 3, 4, 6, 8, 9, 11, 13, 13, 13, 14, 15, 15, 16, 16, 16, 16, 16, 16, 17, 16, 16, 16, 16, 16, 17, 17, 16, 16, 16, 16, 15, 14, 15, 15, 15, 15, 14, 14, 15, 14, 13, 13, 11, 11, 10, 10, 8, 7, 7, 4, 3, 2, 0, -2, -3, -5, -6, -8, -9, -10, -11, -13, -12, -14, -13, -14, -16, -15, -15, -14, -14, -14, -13, -12, -11, -10, -9, -8, -6, -5, -4, -2, -2, -1, 0, 1, 0, 1, 2, 2, 3, 4, 5, 5, 5, 5, 6, 6, 6, 7, 8, 8, 8, 8, 7, 8, 9, 10, 10, 11, 11, 12, 12, 12, 13, 13, 12, 13, 13
    Still getting all zeroes from CAPI.ovr_Voip_GetPCMFloat(), including the returned length. I think I had that wrong in my last post.

    For reference, here's the current setup of our remote player's head anchor:



    Other things to note:

    - Checking AudioSource.isPlaying periodically is always false, unless I manually call voip.audioSource.Play() after voip.enabled=true.

    - We're only on the Rift for this app, no need to worry about Gear VR yet at least :)
  • johnluxfordjohnluxford Posts: 13 Oculus Start Member
    edited November 2016
    @tnovak any ideas on what else we can try to troubleshoot why we don't seem to be receiving voice data?

    I can confirm the connection is established, and we're setting voip.senderID and voip.enabled=true, but the audio source never starts playing unless I manually call voip.audioSource.Play(). We've had Peer connections working well for a while, so we can see each other move around but no sound. I do see data coming from Voip.SetMicrophoneFilterCallback(), but only zeroes in CAPI.ovr_Voip_GetPCMFloat().

    Here's a gist of the output of a recent test, which includes relevant code bits added in:

    https://gist.github.com/jbroadway/9fbdc76da5c510245385657a62b39fb3

    The weirdest part though is that I tried adding a Debug.Log() at the very start of the OnAudioFilterRead() in VoipAudioSourceHiLevel.cs and it outputs my log output before the Voip starved messages right up until my AddVoipComponent() method is called (its source is included in the gist).

    So it seems like something is causing Unity to stop calling OnAudioFilterRead() on the component, even though it's enabled and only one of 3 components on the object (seen in the screenshot in my previous reply). What can stop Unity from calling it I wonder?
  • ignacio.fsstudioignacio.fsstudio Posts: 1
    NerveGear
    @johnluxford

    I know this is about 3 years later but I'm finding the same error on the Oculus vrvoicechat sample for Unity using two oculus go.

    I encountered the error about starvation, but it seems I was able to resolve it by setting the OculusSpatializer as the Spatializer plugin in the project's audio settings. However, I'm still getting no audio output.

    It seems both are able to connect and the voip session is established but OnAudioFilterRead is never called.

    Any help at all would be appreciated!
  • justin.oheirjustin.oheir Posts: 16 Oculus Start Member
    Check if microphone permissions are on
Sign In or Register to comment.