incorporating POSENET

[kavikode]

You have an excellent demo! This is what I have been looking for many years!

Can you please include the following to move the arms and limbs of the model in 3D with 2 web cameras?

https://youtu.be/7lMOdYc4F4c

Thank you!

[haruiz]

Hi @kavikode, I 'm currently working on that implementation, so I hope to have this done by next week, thanks for sharing that amazing demo, now is already in the roadmap: #2

[kavikode]

Thank you so much @haruiz ! I can't wait!

[Deamoner]

Yo, how is this looking? Do you have any architecture notes? I could jump in and help with this.

[haruiz]

Hey @Deamoner, I appreciate your interest, the idea is to use the posenet tensorflow.js model to animate the model, you can get more information on the roadmap of the project , I'm currently working on it, but you are welcome to contribute with that, let me know if you need anything, we can chat or discuss about it.

[kavikode]

@Deamoner are you interested in helping with a certain aspect? Can you please help to create the IK for each of limbs so that when you use one of the keypoints from PoseNet then the rest of the joints follow the selected IK of the limb?

Please see the following demo:
https://rawgit.com/takahirox/three.js/GLTFMMD/examples/webgl_loader_gltf_miku_ik.html

[Deamoner]

Let me review. Do you have a discord or chat?

…

Sent from my iPhone

On Jun 25, 2020, at 7:20 AM, kavikode ***@***.***> wrote:  @Deamoner are you interested in helping with a certain aspect? Can you please help to create the IK for each of limbs so that when you use one of the keypoints from PoseNet then the rest of the joints follow the selected IK of the limb? Please see the following demo: https://rawgit.com/takahirox/three.js/GLTFMMD/examples/webgl_loader_gltf_miku_ik.html — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kavikode]

@Deamoner thanks! I'm on discord Kavita Krishnaswamy#5836. my email kavi1@umbc.edu

[haruiz]

@Deamoner and @kavikode, I could create a Slack channel, so we can talk more in detail.

[kavikode]

@haruiz and @Deamoner that sounds like a wonderful plan! Please do email me the details. I feel so excited because I have been independently trying to solve this problem for more than 2 years. Looking forward to our discussion.

[Deamoner]

Feel free to just put the link here and I'll join. I'm just finishing up a posenet dance app.

[haruiz]

@Deamoner , @kavikode guys, this is the link to access to the slack channel: https://join.slack.com/t/riggingjs/shared_invite/zt-fcjr9p12-GqIj7zEjoNnH7j5AwWT7MQ

[kavikode]

@haruiz I'd like to port 3D Human Pose and Mesh Estimation to Tensorflow.js.

Please see the following: https://youtu.be/tDz2wTixcrI

Can you please help me port it to Tensorflow? This may solve some of the problems.

[haruiz]

@haruiz I'd like to port 3D Human Pose and Mesh Estimation to Tensorflow.js.

Please see the following: https://youtu.be/tDz2wTixcrI

Can you please help me port it to Tensorflow? This may solve some of the problems.

@kavikode let's do it, I will create a new repo for it, so we can start working on it!

[delebash]

Hi all @haruiz I modified your version and was able to stream the data for hand pose to my character in a 3D program and it works. The problem is the hand detection and joint accuracy. It is just not good enough for real use in animation of a 3D character. Take a look at this project https://github.com/vladmandic/human. He has made significant improvements on the default tfjs version. I have also discussed with him about making another version that would use better datasets and bounding box detection for greater stability and accuracy. My current project concept is to be able to stream the raw data where any application can pick it up via its own plugin and format the data for its animation system. This leaves the whole motion capture loosely coupled from any other software. But thank you for your software as I have used it to test if this can be done. Well done!!!!

[haruiz]

Thanks so much, @delebash for your feedback, I love that you found rigging.js useful, in the next few weeks I will put all my effort to make good progress, and try to implement the rest of the features that are in the roadmap, so I will follow up with your recommendations and I'll consider them for improving rigging.js and release a more complete version!!

[ubilrodriguez]

Excuse me, you can send me that avatar, I am doing an investment job, because the project does not find that avatar, there is only a sample avatar.