Welcome to the Oculus Developer Forums!

Your participation on the forum is subject to the Oculus Code of Conduct.

In general, please be respectful and kind. If you violate the Oculus Code of Conduct, your access to the developer forums may be revoked at the discretion of Oculus staff.

Dynamic Room Modeling and Unreal ?

motorsepmotorsep Posts: 1,501 Oculus Start Member
Just saw this: https://developer.oculus.com/documentation/audiosdk/latest/concepts/release/ and wondering if support for dynamic room modeling is coming to Unreal 4.19 

Thanks beforehand

Comments

  • holosparkbizholosparkbiz Posts: 4
    NerveGear
    Came here to post the same question. We use the Oculus spatializer including its shoebox reflection system and this looks like a great advance -- we'd love to have it working in Unreal.
  • PetrozaPetroza Posts: 149 Oculus Staff
    Hi Folks,

    We have a working implementation for UE4 but it's a bit hacky and so we need to clean it up and then package it. No ETA on that, but relatively straight forward to set it up in native code for a project which is probably the best way to go.

    First grab the latest native package of the Audio SDK and replace the old DLLs and headers with the new ones in your unreal project. If you're using middleware then just make sure to grab the latest version of the plugin for that middleware version, and the header that comes with it.

    Now you just need to set up the raycast callback and call ovrAudio_UpdateRoomModel once per frame. There is some coordinate and unit conversions which make this a little tricky.

    The raycast function would look something like this (note I made a global getter for the World, this is a hack in your game you should be able to get the World in a more elegant way):
    static const float DISTANCE_SCALE = 0.01f;
    
    FORCEINLINE FVector ToUnrealVector(const ovrAudioVector3f& InVec)
    {
        return FVector(-InVec.z, InVec.x, InVec.y);
    }
    
    FORCEINLINE ovrAudioVector3f ToOVRVector(const FVector& InVec)
    {
        ovrAudioVector3f Out = { InVec.Y, InVec.Z, -InVec.X };
        return Out;
    }
    
    static UWorld* GetMainWorld()
    {
        TArray<APlayerController*> playerList;
        GEngine->GetAllLocalPlayerControllers(playerList);
    
        return (playerList.Num() > 0) ? playerList[0]->GetWorld() : nullptr;
    }
    
    static void OculusAudioRayCast(ovrAudioVector3f origin, ovrAudioVector3f direction, ovrAudioVector3f* hit, ovrAudioVector3f* normal)
    {
        auto World = GetMainWorld();
        
        if (World != nullptr)
        {
            FVector Start = ToUnrealVector(origin) / DISTANCE_SCALE;
            FVector End = Start + (ToUnrealVector(direction) * 100000.0f) / DISTANCE_SCALE;
    
            FHitResult Hit;
            World->LineTraceSingleByChannel(Hit, Start, End, ECC_Visibility);
    
            *hit = (ovrAudioVector3f)(ToOVRVector(Hit.ImpactPoint* DISTANCE_SCALE));
            *normal = (ovrAudioVector3f)(ToOVRVector(Hit.ImpactNormal* DISTANCE_SCALE));
        }
    }

    Then the update function that runs each frame needs to call ovrAudio_UpdateRoomModel, I've also added some debug drawing so you can visualize the room to make sure nothing went wrong. The most common issue is raycasts hitting things like player's own head or some UI elements which screws up the room modeling. If everything is working the virtual room should roughly line up with the actual room.

    
            ovrResult Result = ovrAudio_UpdateRoomModel(OvrAudioContext);
    
            auto World = GetMainWorld();
    
            if (World != nullptr)
            {
                // draw room box
                ovrAudioVector3f Position;
                float Dimensions[3], Coefficients[6];
                Result = ovrAudio_GetRoomDimensions(OvrAudioContext, Dimensions, Coefficients, &Position);
    
                FVector Center = ToUnrealVector(Position) / DISTANCE_SCALE;
                FVector Extent(Dimensions[2], Dimensions[0], Dimensions[1]);
                Extent /= 2.0f; // half dimensions
                Extent /= DISTANCE_SCALE;
                Extent *= 0.99f; // Make it a little smaller to prevent z-fighting
    
                DrawDebugBox(World, Center, Extent, FColor::Purple);
    
                const int HIT_COUNT_MAX = 2048;
                static ovrAudioVector3f Points[HIT_COUNT_MAX] = { 0 };
                static ovrAudioVector3f Normals[HIT_COUNT_MAX] = { 0 };
    
                // draw hit points
                Result = ovrAudio_GetRaycastHits(OvrAudioContext, Points, Normals, HIT_COUNT_MAX);
    
                for (int i = 0; i < HIT_COUNT_MAX; ++i)
                {
                    FVector Position = ToUnrealVector(Points[i]) / DISTANCE_SCALE;
                    DrawDebugPoint(World, Position, 10.0f, FColor::Green);
                    FVector LineEnd = Position + ToUnrealVector(Normals[i]) * 3000.0f;
                    DrawDebugLine(World, Position, LineEnd, FColor::Emerald);
                }
            }


  • FreetimeCoderFreetimeCoder Posts: 2 Oculus Start Member
    Hello,

    I've been trying to get this to work but I couldn't even get project to include the right headers. We are using the FMOD plugin for Unreal so I tried to download the latest spatializer plugin and the include that comes with it as suggested. Not sure where to put it I placed it in the FMOD plugin's source folder as the dll goes in the plugin folder as well.

    However the OculusSpatializerFMOD.h includes a OVR_Audio_DynamicRoom.h which I assume is from the Audio SDK but I have no idea where to download that as all downloads are just plugins or native integration for something else.

    Kind Regards


  • jumblijumbli Posts: 238 Oculus Start Member
    Thanks @FreetimeCoder for posting this. I got the code running, but it doesn't sound right to me. Have you had good results?

    I can see successful raycasts hitting appropriate walls, but the shoebox reverb model doesn't get close to walls when in long thin rooms. Instead they seem to use an average of the raycasts to define the shape.

    I seem to find that using the dynamic room model I get some nice reverb but it hardly changes when the shape of the room changes and there are no distinct early reflections.

    The video below shows a comparison between the dynamic mode and legacy mode. In the legacy mode I get strong early reflections from expected directions that highlight the shape of the room. 

    @PeterStirling , can you please let me know if the shoebox reverb model should closely reflect the room size or is the behaviour in the video to be expected. I'm using the code above with FMOD and UE4.18 and the Oculus Audio Spatializer Plugin for FMOD 1.24.0 - thanks.


    Developer of Dimensional, The Relentless, Breath Tech, Jigsaw 360
    View my dev blog at JumbliVR.com
Sign In or Register to comment.