Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Problem with Oculus Avatars replication in ue4 multiplayer

Hello Everyone,

I'm trying to make multiplayer in Unreal Engine 4.15 and to implement Oculus Avatars as an representation of players.
Problem I'm having is that the client doesn't see avatars at all and movement of all avatars is copied from avatar possessed by server (server sees all avatars).  
I have managed to get personalized avatars from oculus platform for server and client player characters but still movement is copied from server and clients can not see avatars at all.

Hierarchy looks like this:  
I have player character blueprint that manages player movement and all interactions.  
This Player Character has child actor added as component of localAvatar class which is copied from AvatarSample project shared by Oculus.

Here is how I have set replication for child actor component:


I tried to set different replication methods for event that initializes avatars but I couldn't get the desired effect.  
When I changed replication method of initializePlayerAvatar to replicated on client, client was able to see it's own and server's avatars but server could see client avatar.  

This event looks like this and it's called by pressing U key:


InitializeAvatar method:


    void ALocalAvatar::InitializeAvatar(FString OculusId)
    {
    IOnlineIdentityPtr IdentityInterface = Online::GetIdentityInterface("OculusSubsystemPlatform");
    if (IdentityInterface.IsValid())
    {
    OnLoginCompleteDelegateHandle = IdentityInterface->AddOnLoginCompleteDelegate_Handle(0, FOnLoginCompleteDelegate::CreateUObject(this, &ALocalAvatar::OnLoginComplete));
    IdentityInterface->AutoLogin(0);
    }
    uint64 OculusIdAsUInt64 = FCString::Strtoui64(*OculusId, NULL, 10);
    if (AvatarComponent)
    {
    AvatarComponent->RequestAvatar(OculusIdAsUInt64);
    }
    
    AvatarHands[ovrHand_Left] = nullptr;
    AvatarHands[ovrHand_Right] = nullptr;
    }

So my questions are:  
How to separate avatar movement from server so that movement is not copied between all avatars?  
How to properly replicate avatars so they could be seen both by server and clients?

Best regards,  
OrionPS

Comments

  • pieterdubpieterdub Posts: 11 Oculus Staff
    edited September 2017
    Hi OrionPS,
    I have been working on a multiplayer Avatar sample in UE.  In my sample, I am spawning the avatar pawns on the server with the following replication settings:
    bReplicates = true; 
    bAlwaysRelevant = true;
    bReplicateMovement = true;

    Client doesn't see Avatars at all:
    Where are you spawning the Avatar?  It it part of the scene?  Or are you spawning it in BP?
    Does the client see the locally controlled Avatar?

    I believe you need to spawn the Avatars on the server so replication copies them to all clients.
    In my GameMode (only runs on server), I use the OnPostLogin event to spawn each player's avatar and possess that pawn with the player controller that is a parameter of the OnPostLogin event:


    here is the spawn that is down the line from the OnPostLogin:


    So once the avatar's have been spawned and possessed with player controllers on the server, they are automatically replicated to all clients.

    Movement:
    When you say movement of the avatars is copied from the Server avatar, do you mean the movement of the hands/head etc or are you moving the avatar around with the sticks (moving the position of the avatar)?  I assume you mean the hand/head movement but let me know if not.

    In my sample, I am recording avatar movement packets for the local avatar, and sending them to the server who then replicates those packets out to all clients, and then the clients play them back.

    Did you make any other modifications to the LocalAvatar code or are you using it as it was provided in the sample?  I think the code in the sample is recording the movement data packets from the local avatar into a set of queues and that is playing back movement to all avatars (see void FOvrAvatarManager::QueueAvatarPacket(ovrAvatarPacket* packet) ).  Right now, that function is writing the packet into all the queues in AvatarPacketQueues and isn't keeping track of which avatar the packet belongs to.  So the server writes data into this queue, and then it is replicated to all clients which isn't correct.  It was setup this way because the sample is single player and allows the user to spawn multiple remote avatars that all mimic the movement of the local avatar.

    What you would need to do, is set the Key of the AvatarPacketQueue and use that to keep the queues separate.  You could used the player's UserID as the Key for each queue.  I believe you would also need to make an RPC call from each client to send the packet data to the server and then the server writes that into the correct queue in the AvatarManager.  

    I don't currently use the AvatarManager in my sample.  I send the movement packets from the clients to the server, and each Avatar Pawn maintains a queue.  Server puts the correct packets into each avatar's queue using a PacketKey, and then replication copies that to each client machine.  Then in the tick function for any nonlocal avatars, I playback movement from its queue.

    Let me know if this helps at all.  Hopefully I will have the sample ready to distribute within a couple weeks.
  • ginopelosoginopeloso Posts: 47
    Brain Burst
    edited January 2018
    pieterdub said:
    Hi OrionPS,
    I have been working on a multiplayer Avatar sample in UE.  In my sample, I am spawning the avatar pawns on the server with the following replication settings:
    bReplicates = true; 
    bAlwaysRelevant = true;
    bReplicateMovement = true;

    Client doesn't see Avatars at all:
    Where are you spawning the Avatar?  It it part of the scene?  Or are you spawning it in BP?
    Does the client see the locally controlled Avatar?

    I believe you need to spawn the Avatars on the server so replication copies them to all clients.
    In my GameMode (only runs on server), I use the OnPostLogin event to spawn each player's avatar and possess that pawn with the player controller that is a parameter of the OnPostLogin event:


    here is the spawn that is down the line from the OnPostLogin:


    So once the avatar's have been spawned and possessed with player controllers on the server, they are automatically replicated to all clients.

    Movement:
    When you say movement of the avatars is copied from the Server avatar, do you mean the movement of the hands/head etc or are you moving the avatar around with the sticks (moving the position of the avatar)?  I assume you mean the hand/head movement but let me know if not.

    In my sample, I am recording avatar movement packets for the local avatar, and sending them to the server who then replicates those packets out to all clients, and then the clients play them back.

    Did you make any other modifications to the LocalAvatar code or are you using it as it was provided in the sample?  I think the code in the sample is recording the movement data packets from the local avatar into a set of queues and that is playing back movement to all avatars (see void FOvrAvatarManager::QueueAvatarPacket(ovrAvatarPacket* packet) ).  Right now, that function is writing the packet into all the queues in AvatarPacketQueues and isn't keeping track of which avatar the packet belongs to.  So the server writes data into this queue, and then it is replicated to all clients which isn't correct.  It was setup this way because the sample is single player and allows the user to spawn multiple remote avatars that all mimic the movement of the local avatar.

    What you would need to do, is set the Key of the AvatarPacketQueue and use that to keep the queues separate.  You could used the player's UserID as the Key for each queue.  I believe you would also need to make an RPC call from each client to send the packet data to the server and then the server writes that into the correct queue in the AvatarManager.  

    I don't currently use the AvatarManager in my sample.  I send the movement packets from the clients to the server, and each Avatar Pawn maintains a queue.  Server puts the correct packets into each avatar's queue using a PacketKey, and then replication copies that to each client machine.  Then in the tick function for any nonlocal avatars, I playback movement from its queue.

    Let me know if this helps at all.  Hopefully I will have the sample ready to distribute within a couple weeks.

    How did you replicate ovrAvatarPacket? (since it is neither a UCLASS, nor a USTRUCT nor a UENUM)

    Is it possible to have this sample? It would be very useful for a lot of users. Thanks!
  • pieterdubpieterdub Posts: 11 Oculus Staff
    Sample is super close to being released!  Apologies for the delay!  

    WRT the data replication, you are correct that you can't just replicate the AvatarPacket.  I take the data in the packet and copy it into a USTRUCT to be sent over the wire.

    In my NetworkAvatar.h:

    class UOvrAvatar:

    USTRUCT()
    struct FAvatarPacket2
    {
      GENERATED_USTRUCT_BODY()

      UPROPERTY()
      TArray<uint8> AvatarPacketData;
      UPROPERTY()
      uint32 packetSequenceNumber;
    };
    UCLASS()
    class AVATARSAMPLES_API ANetworkAvatar : public APawn
    {
      GENERATED_BODY()
      private:
        ovrAvatarPacket* CurrentPacket = nullptr;
        FAvatarPacket2 CurrentPacketStruct;
    ...

    Then in NetworkAvatar.cpp in my UpdatePacketRecording function which is runs only on the locally controlled avatar instance:
    if (PacketSettings.AccumulatedTime >= PacketSettings.UpdateRate)
    {
      PacketSettings.AccumulatedTime = 0.f; //reset the recording timer
      ovrAvatarPacket* packet = AvatarComponent->EndPacketRecording();
      if (packet == nullptr)
      {
        //nothing to do, so start recording again
        UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - packet == nullptr"));
        AvatarComponent->StartPacketRecording();
        return;
      }
      UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - created a packet"));
      uint8_t* Buffer;
      uint32_t BufferSize;

      BufferSize = ovrAvatarPacket_GetSize(packet);
      UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - BufferSize =   ovrAvatarPacket_GetSize(): %d"), BufferSize);
      Buffer = new uint8_t[BufferSize];
      if (ovrAvatarPacket_Write(packet, BufferSize, Buffer)) 
      {
        UE_LOG_ONLINE(VeryVerbose, TEXT("ANetworkAvatar::UpdatePacketRecording - packing USTRUCT"), BufferSize);
        //pack data into our Struct array
        CurrentPacketStruct.AvatarPacketData.Append(Buffer, BufferSize);
        ServerHandleAvatarPacket(CurrentPacketStruct); //RPC Call to server with USTRUCT data.
    ...

    and then on the opposite side, when you are receiving a packet from the server and need to play it back:
    .h: 
    UPROPERTY(ReplicatedUsing = OnRep_ReceivedPacket)
      FAvatarPacket2 R_AvatarPacket;

    cpp:
    void ANetworkAvatar::OnRep_ReceivedPacket()
    {
      //Check if the incoming packet belongs to a remote avatar rather than the local (we don't want to move the local as that    is user controlled)
      if (!bAvatarIsLocal)
      {
        UE_LOG_ONLINE(Verbose, TEXT("ANetworkAvatar::OnRep_ReceivedPacket() for non-local avatar: %s with   SequenceNum: %d"), *PacketKey, R_AvatarPacket.packetSequenceNumber);

      uint32_t BufferSize;
      uint8_t* Buffer;
      BufferSize = R_AvatarPacket.AvatarPacketData.Num();
      Buffer = new uint8_t[BufferSize];
      for (uint32_t elementIdx = 0; elementIdx < BufferSize; elementIdx++)
      {
        Buffer[elementIdx] = R_AvatarPacket.AvatarPacketData[elementIdx];
      }
      FOvrAvatarManager::Get().QueueAvatarPacket(Buffer, BufferSize, PacketKey,   R_AvatarPacket.packetSequenceNumber);

      delete[] Buffer;
      }
    }
  • NeontopNeontop Posts: 219 Oculus Start Member
    edited June 2018
    Hello @pieterdub I'm having the same problem with the AvatatSamples from SDK 1.27.
    Is is possible to have access to those codes.
    Thank you



  • beaulima9933beaulima9933 Posts: 50 Oculus Start Member
    If anyone wants access to the code for UE4.19 - 4.20 multiplayer just PM me :smile:
  • NeontopNeontop Posts: 219 Oculus Start Member
    edited August 2018
    @beaulima9933 thanks for the link.
    I'm trying to integrate the platform SDK so it will be more easy to access Oculus Online Subsytem functionalities.

  • NeontopNeontop Posts: 219 Oculus Start Member
    Anyone tried to implement the code of @beaulima9933 here ?

Sign In or Register to comment.