When I use the Oculus Link feature and enter Oculus Home, I have a smooth framerate and no jutter in my hand movement. However, when I open any game (Oculus, SteamVR, OpenComposite) it starts dropping a large fraction of my frames in both the compositor and the application. Instead of locking to the refresh rate, the FPS stays around 10% lower than the refresh rate. In some games the FPS is steadily below the refresh rate while others are constantly moving between the refresh rate and that "10%-ish" range. My in-game hands/controllers lag a lot when I move them making games choppy or unplayable depending on the genre. Even after closing the game I'm playing the issue persists inside Oculus Home, not fixing itself unless I restart the software. I have plenty of hardware headroom and none of my hardware components are shown as maxing out in the task manager. I tried lowered the target resolution and refresh rate, which didn't affect my FPS at all. I set the OVR Server process priority to high. I've tried multiple USB 3.0/3.1/3.2 ports on my computer with nothing changing either. Disabling/enabling supersampling in SteamVR had no effect. I didn't have any issues playing the same games with my CV1 before I bought the Quest 2. I tried Airlink and it appeared to have the same issue (along with bad latency since the router is far from my PC, so could just be a bad connection there).
I am currently using a GTX 980 Ti, Ryzen 3900x, 32GB DDR4 3200MHz, 3.5GB/s NVMe, and a USB 3.1 Gen 2 cable for Oculus Link.
Downgrading my GPU driver to a version below 456 (452 is what I chose) improved the frame dropping issue a lot, dropping 2-3 frames each second instead of 20-30. Is there anything else I can do to get it completely smooth? While it's better, movement is still a bit choppy as my controllers freeze for every frame drop.