cancel
Showing results for 
Search instead for 
Did you mean: 

Split Post Processing

getnamo
Honored Guest
I am working on integrating a leap motion pass-through effectinto UE4 and I was wondering if anyone knows how to achieve a disparate view for post-processing so that I can feed left/right eye images from the leap before geometry is rendered (or even after), or if anyone has had success in putting two quads in front of the camera to pull this off accurately.
25 REPLIES 25

opamp
Protege
The trick is to realise that the CameraController/Camera remains in its own transform and the POV is updated directly by the rift Plugin relative to the camera location(The rift never updates the cameras transform).

so if you want the rifts location in world space you would add the two vectors together.
ie. cameramanager->camera->WorldLocation + HMDgetOrientationAndPosition->Location.

I beleive HMDOrientationandPosition->Rotation is in world space.

getnamo
Honored Guest
That makes sense, the question is now more along the lines of how to actually feed the two images as a render layer or a post process which is different for each eye.

According to epic this isn't currently possible, but I wonder if there isn't a way around this?

opamp
Protege
as youve got the location and rotation of the rift in world space you could always have 2 quads positioned in front of the rift
at the correct relative locations and the correct distances and update there transforms every tick.
A bit hacky but it would work.

I take it you've managed to figure out how to get these images updated to Utexture2d's without stalling the game thread?

Im not to clued up on the leap motion but wouldnt it be easier to just have a couple of your own skeletal meshes for the arms and update the bone transforms via an anim blueprint rarther than using some images generated by the leap SDK?

getnamo
Honored Guest
Thanks for the feedback opamp, though its not finger/arm location tracking I'm going for, I've got that working quite well in the plugin

just doing this


gives you this


In addition the plugin already supports real-time image forwarding to UE's Texture2d.


What I'm trying to achieve is a pass-through effect (see initial video). Essentially it gives the Rift AR capabilities by allowing you to see the world around you in IR. Its absolutely good enough for typing on a keyboard or drinking with the rift on. A very compelling experience, which is currently working in unity because they just render straight to the cameras and have easy render layer support, whereas the UE Rift implementation is much weirder (opaque).

Maybe 2 well placed 3d quads might work, if they can be somehow be applied to only the specific eye and if it can be passed through rendering before everything else (so you can overlay 3d objects on top of the real world). I worry that this way will be hard to calibrate for a proper 1:1 control even if you can do split-eye feeding in the vanilla UE engine and why rendering a full 2d texture directly in each eye before the scene would be ideal.

andrewtek
Expert Protege
Is there any way to flag an object so that it is only left/right eye rendered? If not, does the rendering pipeline have any flags to indicates which eye is being rendered?

opamp
Protege
"getnamo" wrote:
Thanks for the feedback opamp, though its not finger/arm location tracking I'm going for, I've got that working quite well in the plugin...


As Ive spent the evening bashing my head against a wall in regards to one of my own projects I thought I'd give myself a break and have a go at a per-eye postprocessing shader.

The below shader should get you started although it will only look correct in the rift.

http://i.imgur.com/AlGoWIo.png

opamp
Protege
"andrewtek" wrote:
Is there any way to flag an object so that it is only left/right eye rendered? If not, does the rendering pipeline have any flags to indicates which eye is being rendered?


I dont believe so,

There is'nt any sort of camera layer system with UE4.
The only one that exists is bOwnerNoSee. I really think they need to implement something like that.
I use to love the system they've got with Unity as all sorts of cool tricks where possible.

As for flags for eye rendering you'd have to look into the unreal plugin source code(something I really havnt had the time to do). But I doubt it.

andrewtek
Expert Protege
"opamp" wrote:

The below shader should get you started although it will only look correct in the rift.
...


Thanks for the blueprint. That is very cool! Something like this could definitely be used to do some interesting things. I tried this trick to create a material that would put a red texture in the right eye, and a gray-scale version of that texture in the left eye. The result was interesting. The brain combines the resulting colors for a grayish/red texture. I could definitely see some inversion puzzle mechanic uses for this.

andrewtek
Expert Protege
Getnamo, would you be willing to share the steps you took to integrate with Leap Motion? Thanks!