I reset my S8 before finding out that my usb-c to usb cable was the culprit. I changed it for a usb-c to micro-usb adaptor + micro-usb cable and the stutter dissapeared.
What is weird is the "faulty" cable came with the S8, and that it charges without issue my phone at high speeds.
If any Samsung engineer reads this: please add error messages instead of stuttering without explanation. Would have saved me having to reconfigure my phone, and I'm not even certain that there is anything wrong with the cable (I just checked and using it for data transfer works). Maybe the GearVR port is just a bit loose.