cancel
Showing results for 
Search instead for 
Did you mean: 

Data Streaming from Unity VR app to mobile (iOS/android/web) client

patmw
Explorer

Hi there,

 

I'm currently working on a VR app for a client, which will run on the Oculus Quest/Quest2. One of the base requirements is that the VR app can be interacted with from an external client application through two-way communication.

 

The mobile client should be able to trigger certain events within the Quest app - for example, pause/continue/next scene/trigger sound, etc. 

 

The mobile client should also be able to receive data from the Quest app - things such as the user's performance metrics in a certain part of the experience.

 

In my initial research, I've stumbled upon several options: UDP/threading/OSC/PUN/Mirror/Normcore/WebRTC, and I'm feeling slightly overwhelmed (I have no solid networking experience in the past).

 

If possible I would like to avoid some of the heavier networking frameworks, as I want to avoid having to refactor all my code to accommodate for these. There isn't necessarily a need for full synchronization of game-states like there would on a real-time multiplayer game, only small custom data packets.

 

Any guidance would be massively appreciated!

I'm way in over my head at the moment.

 

Thanks in advance.

Pat

23 REPLIES 23

brazzoni
Protege

I would really appreciate any guidance or referrals to documentation for this as well. I've looked into the peer-to-peer networking documentation. That looks easy enough, but my interpretation of it is that it is only to be used with other headsets.

 

In unity, there is https://docs-multiplayer.unity3d.com/. The documentation is great, so I'm considering it. However, I think the path of least resistance would be to create your mobile app as a 'unity game' as well. I've been searching for a few days, and it doesn't seem like Headset -> Mobile App is a well trodden path. It would be awesome if API used for the Oculus Mobile App was publicly released, or something like that!

 

Am I missing anything?

 

 

The peer-to-peer networking looked really promising, and it would be easy to pick up. I got lost trying to figure out what a peerID is, and whether or not I could create a peerID through a mobile app.

Hi brazzoni! 

 

Appreciate your input here. The way I got around it was, as you said, to build a mobile app within unity as well. The networking solution I went with was less graceful, I ended up implementing a custom UDP/TCP client-server system - the mobile app acting as a server and the VR app acting as a client - the implementation I went with was based off the following YouTube tutorial series:

 

https://youtube.com/playlist?list=PLXkn83W0QkfnqsK8I0RAz5AbUxfg3bOQ5

 

I recommend this approach as it worked for me, however you may find that Unity's built-in networking is an easier solution (it was released officially after I finished this project so I didn't get chance to try it myself) 

 

Hope this helps! 

Best, 

Patrick

I think I'll build a tcp server too. Unity's networking is really game dedicated. Fitting for a game dev platform!

 

The TCP server is going to be easier to learn atleast, and way more freedom at the end of the day. Thank you!

 

For anyone that might read this:

C#'s

TcpClient

And

TcpListener

Are the native functions.

 

 

 

 

 

 

MichaelReber
Honored Guest

Hey how's it going everyone. 

 

I am also attempting to do this as well. Any luck on getting this working using Unity's networking?

I've abandoned trying to do it with Unity's networking. I can make TCP connections between a react-native mobile and a unity build for the headset. I had to hack it together by logging the IP address of my headset and entering it into the mobile app. I'm struggling to find a good way to discover devices on a network. It would be awesome if I could use iOS's Bonjour or Androids NSD, but I'm kinda blocked on how I might run the Java code required for Androids NSD because I couldn't find any C# libraries for it.

 

I made an algorithm that discovers devices on a local network that are listening to a certain port. I got the IP address and Subnet from Node.js (os module). Then, used npm's NetMask to find the network address and I just iterated through every available address attempting to make connections. The next step would've been sending a handshake, so I can tell if the listener was a device that I programmed.

This is a bad system though because you're not guaranteed to be allowed to listen on the same port every time the app runs, and I worry about how flexible this system is on networks with different configurations.

Thanks for sharing! I am currently trying to set up a local connection were 360 videos playing on a HMD device are casted unto an app on an android device (table or phone).

 

This is probably less compilated than what you guys are doing but I'll update if I get this to work properly on my end 🙂

Please do keep us updated!

For my use case, the mobile phones/tablets will not be a customer's/users. They're devices that ship with our product, so for my use case I can change the hostname to something constant on the phone.

 

It didn't seem like you can change the host name on the Oculus without root access, but I just realized you can totally change the hostname on a packaged phone!

 

Might be useful