Unreal 4.20/4.21 Multiplayer Avatars & VOIP template! — Oculus
Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Unreal 4.20/4.21 Multiplayer Avatars & VOIP template!

beaulima9933beaulima9933 Posts: 34 Oculus Start Member
Hi guys, x-post from Unreal forum there to point to my latest open-source template (v5.0): https://forums.unrealengine.com/development-discussion/vr-ar-development/106631-single-multiplayer-touch-vive-proteus-blueprint-only-template 
  • Uses Single/Multiplayer Avatars, Oculus SDK 1.32 & Avatars 1.31 (works also with in-house Avatar)
  • Works Go/Rift/Quest
  • Use latest Oculus implementations (splash screens, guardian limits, camera fade, etc).
  • Compatible Unreal 4.20 & 4.21
  • Minimum use of C++

Most important, yes @NinjaGaijin I'm pointing at you! And everyone interested in Oculus Avatars!

We are commited to MAKE AVATARS WORK IN UNREAL ONCE AND FOR ALL!
so, I call Oculus guys and Unreal/Oculus community, let's fix the last problems remaining:

1) Rift: No voice visualization with Avatars
2) Rift/Go/Quest: Stereo Layers cannot be destroyed
3) Go/Quest: Occlusion culling doesn't work
4) Go: Inputs no registered
5) Go: Only blank (default) Avatar is loaded

Let's hope we'll fix these problems within rapidly, we'll keep you posted on our side on possible fixes.
If you find any fixes on your side, share them here or on the Unreal forum!

Download it at https://github.com/ProteusVRpublic/ProteusTemplate

So say we all.

Thanks, Proteus.

Comments

  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    Small corrections:
    Oculus SDK 1.31 (Unreal 4.20) , Oculus SDK 1.32 (Unreal 4.21)
    4) Go: Inputs not registered while using Avatars
  • NeontopNeontop Posts: 209 Oculus Start Member
    Hi @Proteus this is a great news for the UE4 community. You are working hard to share your experience and thanks for the template.I will download the 4.21 and test your template with it.
    Thank you.

  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    @Neontop
    So for a heads-up:
    1) Download Unreal Engine 4.21, Oculus-branch, latest commit 5b6d633 (Dec 20, 2018), found at https://github.com/Oculus-VR/UnrealEngine, unzip/download dependencies
    2) In the engine, change \Engine\Source\Runtime\Engine\Classes\Components\PoseableMeshComponent.h

    Line 70 remove private

    private:                void MarkRefreshTransformDirty();                // this is marked if transform has to be updated                bool bNeedsRefreshTransform;

    This has been corrected in latest launcher 4.21.2 hotfix, so it should be unnecessary once Oculus-branch will be updated.

    If you develop for Go/Quest, also change  \Engine\Plugins\Online\OnlineSubsystemOculus\Source\Private\OculusIdentityCallbackProxy.cpp

    Line 26

    from:

    DelegateHandle = Online::GetIdentityInterface()->AddOnLoginCompleteDelegate_Handle(to:DelegateHandle = OculusIdentityInterface->AddOnLoginCompleteDelegate_Handle(

    to be able to retrieve Oculus ID and verify Entitlement.


    2)Compile with latest VS2017 version


    3) Finally, in the plugin, uncomment

    Line 569/////////////////////////////
    ONLY FOR UNREAL 4.21 depthMesh->MarkRefreshTransformDirty();Line 633/////////////////////////////
    ONLY FOR UNREAL 4.21 depthMesh->MarkRefreshTransformDirty();Line 809/////////////////////////////
    ONLY FOR UNREAL 4.21 mesh.MarkRefreshTransformDirty(); 


    That's it!

  • NeontopNeontop Posts: 209 Oculus Start Member
    Hi @Proteus ;
    I will wait for an update of the Oculus branch  to avoid multiple download..
    By the way what do you think about the 4.21 ?
    Thanks. 


  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    @Neontop
    For me 4.21 I saw mostly improvements in mobile (Android - Go/Quest):
    • is more stable with OpenGL ES3.1
    • Vulkan is not tagged as experimental anymore, however I've not been able to run it, maybe it's because of something else
    • Software occlusion culling is good, as regular occlusion culling doesn't work
    • Maybe Round Robin Occlusion Queries could also help
    • Finally it uses Oculus SDK 1.32
    For Rift not much difference, if you're not using other 4.21 new features

  • NeontopNeontop Posts: 209 Oculus Start Member
    @beaulima9933 , I tested your template for Oculus dist 4.20.2 and it works perfectly with the Oculus Avatar.
    Later I will test  in MP  and check your code.
    I visited your BP and effectively you have done a lot of work to clean  and simplify all of those BP.
    Thanks ...😉

  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    @Neontop
    I've made a small change for latest Oculus-branch 4.21.2, since they put
    UProteusOvrAvatar as friend class, I've made the same:

    change

    \Engine\Source\Runtime\Engine\Classes\Components\PoseableMeshComponent.h

    Line 78 add (line after friend class UOvrAvatar;)

    friend class UProteusOvrAvatar;

  • NeontopNeontop Posts: 209 Oculus Start Member
    Thanks @beaulima9933, I will check this point when I will be back in The Netherlands. 
    Any news about accessing bones of the Oculus Avatar?

  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    @Neontop it's a WIP; Avatar skeleton is built at runtime, by assembling a set of Avatar Assets (approx 30-35 assets). Hands and Projector are treated differently, and hands follow a standard skeleton.
    For the hands, custom pose is defined by (i.e. left hand):
    OVRN_EXPORT (void) ovrAvatar_SetLeftHandCustomGesture(ovrAvatar* avatar, uint32_t jointCount, const ovrAvatarTransform *customJointTransforms);
    which point to an opaque pointer, thus adding a level of complexity (everything points to opaque pointers with Oculus Avatars, very frustrating). So I just have to translate the in-game desired Avatar hands into the custom transform Matrix, as found at the beginning of ProteusLocalAvatar.cpp
  • NeontopNeontop Posts: 209 Oculus Start Member
    edited February 6
    Hi @beaulima9933 thanks for those infos. As a Start Member is it possible for you to receive more support from the Oculus dev team for your plugin, because it will for the interest of all Ue4 Start Member's here ....or you have to dig  by yourself ?

  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    @Neontop @NinjaGaijin
    Good question. For now I just reverse engineer things to figure out how it works. I remember back in november 2017 when I received Touch engineering samples, I spent few days figuring out all the possible Avatar hands poses (by recording videos of ToyBox) and found a way to reproduce this behaviour on custom hands (that's the way I animate my non-avatar hands). BTW I studied the way they animate hands in the HandSample project (much later when they made it available) and it did not convinced me to adopt their way to animate hands (and in a more limited way).

    I know Oculus people work hard and they have their own agenda, coupled with Epic's one. But I think it would be in the interest of everyone to have a working Avatars project before the release of Quest.

    As described in the doc, for now the remaining bugs are:

    Android (Quest, Go, GearVR): No occlusion culling; Only blank Avatar is loaded, Inputs not registered with Go/GearVR remote
    PC: No voice visualization with Avatars
    Android/PC: Stereo layers don't destroy

    I did not yet tried cross-platform Avatars so I don't know for these ones.




  • beaulima9933beaulima9933 Posts: 34 Oculus Start Member
    Hey guys, I've updated my template to include custom poses for Avatar hands (set in blueprints) for Unreal!

  • NeontopNeontop Posts: 209 Oculus Start Member
    @beaulima9933, thanks for that new functionality for the Oculus Avatar. 

Sign In or Register to comment.