cancel
Showing results for 
Search instead for 
Did you mean: 

Getting head and finger orientation

AlienCurv
Protege

Hey gang - glad to see a great support forum for the Oculus Quest 2 with Unity. I'm migrating an app from open XR to oculus to take advantage of the oculus hand tracking. I'm experiencing issues with the data (or lack of) and wondering if anyone can provide advice.

 

I'm using our existing client/server framework to send data to the server. The data is being sent and received correctly - but the data is strange.

 

The finger data never changes from the skeleton. During testing, I'm only sending the tip of the index finger on the right hand. The outcome is to get the amount of bend of each finger. 

 

The camera transform sends the correct data, and it's working great. We just can't figure out how to get the finger data to know the position of each finger.

 

 

void Update() {

        // send the index finger data
        OVRPlugin.Skeleton skeleton;
        if (OVRPlugin.GetSkeleton(OVRPlugin.SkeletonType.HandRight, out skeleton))
          foreach (var bone in skeleton.Bones)
            if (bone.Id == OVRPlugin.BoneId.Hand_Index3)
              _streamClient.SendPacket(
                StreamClient.CommandEnums.FingerRight0,
                MapToByte(bone.Pose.Orientation.x),
                MapToByte(bone.Pose.Orientation.y),
                MapToByte(bone.Pose.Orientation.z));

        // send head data (pitch, yaw, roll)
        _streamClient.SendPacket(
          StreamClient.CommandEnums.pitchYawRoll,
          MapToByte(Camera.current.transform.localRotation.x),
          MapToByte(Camera.current.transform.localRotation.y),
          MapToByte(Camera.current.transform.localRotation.z));
}

 

 

 

 

Any assistance is greatly appreciated - thanks in advance!

1 ACCEPTED SOLUTION

Accepted Solutions

Oh, I think I discovered the bone rotation index. I'll write this down here so anyone in the future can find it. There's a documented list for the skeleton, which partially applies to the bone rotation array. I'll paste it here because it's confusing when the bone rotation and skeleton data are a little different per this page: Set Up Hand Tracking | Oculus Developers

 

The bone rotation array contains the following...

 

Hand_WristRoot = 0 // root frame of the hand, where the wrist is located
Hand_ForearmStub = 1 // frame for user's forearm
Hand_Thumb0 = 2 // thumb trapezium bone
Hand_Thumb1 = 3 // thumb metacarpal bone
Hand_Thumb2 = 4 // thumb proximal phalange bone
Hand_Thumb3 = 5 // thumb distal phalange bone
Hand_Index1 = 6 // index proximal phalange bone
Hand_Index2 = 7 // index intermediate phalange bone
Hand_Index3 = 8 // index distal phalange bone
Hand_Middle1 = 9 // middle proximal phalange bone
Hand_Middle2 = 10 // middle intermediate phalange bone
Hand_Middle3 = 11 // middle distal phalange bone
Hand_Ring1 = 12 // ring proximal phalange bone
Hand_Ring2 = 13 // ring intermediate phalange bone
Hand_Ring3 = 14 // ring distal phalange bone
Hand_Pinky0 = 15 // pinky metacarpal bone
Hand_Pinky1 = 16 // pinky proximal phalange bone
Hand_Pinky2 = 17 // pinky intermediate phalange bone
Hand_Pinky3 = 18 // pinky distal phalange bone

 

View solution in original post

7 REPLIES 7

AlienCurv
Protege

After posting this, I found a different method that returns me the hand state. I can see movement values, but I'm not sure what index is in the array for each finger of BoneRotations[].  Looking at the OVR Plugin code, I see a lot of BoneRotations, but how can I know which index is for what bone? 

 

I "think" index 7 is for the index finger? Is there any additional information about the bones for the skeleton that I can reference?

 

        OVRPlugin.HandState handState = default(OVRPlugin.HandState);

        if (OVRPlugin.GetHandState(OVRPlugin.Step.Render, OVRPlugin.Hand.HandRight, ref handState)) {

          if (handState.BoneRotations.Length > 9)
            _streamClient.SendPacket(
              StreamClient.CommandEnums.FingerRight0,
              MapToByte(handState.BoneRotations[7].x),
              MapToByte(handState.BoneRotations[7].y),
              MapToByte(handState.BoneRotations[7].z));
        }

 

Oh, I think I discovered the bone rotation index. I'll write this down here so anyone in the future can find it. There's a documented list for the skeleton, which partially applies to the bone rotation array. I'll paste it here because it's confusing when the bone rotation and skeleton data are a little different per this page: Set Up Hand Tracking | Oculus Developers

 

The bone rotation array contains the following...

 

Hand_WristRoot = 0 // root frame of the hand, where the wrist is located
Hand_ForearmStub = 1 // frame for user's forearm
Hand_Thumb0 = 2 // thumb trapezium bone
Hand_Thumb1 = 3 // thumb metacarpal bone
Hand_Thumb2 = 4 // thumb proximal phalange bone
Hand_Thumb3 = 5 // thumb distal phalange bone
Hand_Index1 = 6 // index proximal phalange bone
Hand_Index2 = 7 // index intermediate phalange bone
Hand_Index3 = 8 // index distal phalange bone
Hand_Middle1 = 9 // middle proximal phalange bone
Hand_Middle2 = 10 // middle intermediate phalange bone
Hand_Middle3 = 11 // middle distal phalange bone
Hand_Ring1 = 12 // ring proximal phalange bone
Hand_Ring2 = 13 // ring intermediate phalange bone
Hand_Ring3 = 14 // ring distal phalange bone
Hand_Pinky0 = 15 // pinky metacarpal bone
Hand_Pinky1 = 16 // pinky proximal phalange bone
Hand_Pinky2 = 17 // pinky intermediate phalange bone
Hand_Pinky3 = 18 // pinky distal phalange bone

 

Hey, I'm trying to calculate the angle between the hand and the wrist but I can't find it. I have tried to calculate the angle between Hand_WristRoot and Hand_ForearmSt but it does not work for me. Thanks.

I believe the handState.RootPose.Position from the OVRPlugin.GetHandState() will return the hand position. I haven't successfully been able to get the wrist position. I'm using the handState.RootPose.Position to get the position of the hand.

The problem is that Hand_WristRoot and Hand_Forearm return the same vector, so I can't calculate the angle. This does not happen with the rest of the bones of the hand. Thanks AlienCurv.

Perhaps use the Hand_WristRoot and the middle finger proximal phalange bone? 

 

Oh, also can you check the OVRPlugin.GetHandState() and see if the handState has RootPose.Orientation? If so, you can get the hand angle from there.

Thank you AlienCurv . I get the RootPose.Orientation from OVRPlugin.GetHandState(), but I don't know how to get the flexion angle with the wrist. Sorry, but I'm a bit lost, I'm starting with Unity and Oculus.