Adding input option to connect to IPhone FaceID Tracking via Unreal Engine's LiveLink App #1423
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…livelink-driver
sthalik
requested changes
Aug 25, 2023
| do | ||
| { | ||
| const qint64 sz = sock.pendingDatagramSize(); | ||
| datagram.resize(sz); |
There was a problem hiding this comment.
Don't use malloc in the main loop.
| // There is static data with length depending on IPhone name at the left of the datagram. | ||
| // Our three head rotation values are located 9th from the right of the datagram. | ||
| // Discard the rest of the face data | ||
| datagram = datagram.right(sizeof(float[9])).left(sizeof(float[3])); |
| // Discard the rest of the face data | ||
| datagram = datagram.right(sizeof(float[9])).left(sizeof(float[3])); | ||
| ok = true; | ||
| QDataStream dataStream(&datagram, QIODevice::ReadOnly); |
| } | ||
|
|
||
| if (ok) | ||
| { |
| datagram = datagram.right(sizeof(float[9])).left(sizeof(float[3])); | ||
| ok = true; | ||
| QDataStream dataStream(&datagram, QIODevice::ReadOnly); | ||
| dataStream.setByteOrder(QDataStream::BigEndian); |
There was a problem hiding this comment.
Floats are always little endian. You can read into a union just fine.
|
Any reason why this PR is stalled ? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The iPhone Face ID sensors can accurately capture all sorts of facial expression data, including head rotation data. Unreal Engine has an app called Live Link Face that sends that is meant to send that data in real-time over to Unreal Engine. This data is just sent over UDP so it is easy to read and extract the head rotation data from.
I just added a new input, mostly copying the existing UDP input, but made it extract and format the data sent by the Live Link app.
One note is that Live Link only sends head rotation data, nothing on position.
Another thing to note is that the Face ID sensors are present on the iPhone X/XR/XS and newer, as well as the iPad Pro 3rd gen and newer.