[Bugs] Some problems of kinematic models. (face direction and kinematic chain modeling)

[IsshikiHugh]

Thanks for your great work! When studying your wonderful motion representation, I found two small problem which might influence your effects. And they are all in common/skeleton.py.

1. Face Direction

This problem is already mentioned here: #112 #107. The direction of the across vector might be wrong.

HumanML3D/common/skeleton.py

Lines 58 to 62 in 6b72fcd

	l_hip, r_hip, sdr_r, sdr_l = face_joint_idx
	across1 = joints[:, r_hip] - joints[:, l_hip]
	across2 = joints[:, sdr_r] - joints[:, sdr_l]
	across = across1 + across2
	across = across / np.sqrt((across**2).sum(axis=-1))[:, np.newaxis]

Which is different from your code in motion_representation.ipynb.

HumanML3D/motion_representation.ipynb

Lines 85 to 89 in 6b72fcd

	" r_hip, l_hip, sdr_r, sdr_l = face_joint_indx\n",
	" across1 = root_pos_init[r_hip] - root_pos_init[l_hip]\n",
	" across2 = root_pos_init[sdr_r] - root_pos_init[sdr_l]\n",
	" across = across1 + across2\n",
	" across = across / np.sqrt((across ** 2).sum(axis=-1))[..., np.newaxis]\n",

This may lead to some errors, but thanks to the redundant 263-dimensional information, this can be recovered to a certain extent

2. Kinematic Chain

This problem may not be as serious as the above.

HumanML3D/common/skeleton.py

Lines 81 to 99 in 6b72fcd

	root_quat[0] = np.array([[1.0, 0.0, 0.0, 0.0]])
	quat_params[:, 0] = root_quat
	# quat_params[0, 0] = np.array([[1.0, 0.0, 0.0, 0.0]])
	for chain in self._kinematic_tree:
	R = root_quat
	for j in range(len(chain) - 1):
	# (batch, 3)
	u = self._raw_offset_np[chain[j+1]][np.newaxis,...].repeat(len(joints), axis=0)
	# print(u.shape)
	# (batch, 3)
	v = joints[:, chain[j+1]] - joints[:, chain[j]]
	v = v / np.sqrt((v**2).sum(axis=-1))[:, np.newaxis]
	# print(u.shape, v.shape)
	rot_u_v = qbetween_np(u, v)
	R_loc = qmul_np(qinv_np(R), rot_u_v)
	quat_params[:,chain[j + 1], :] = R_loc
	R = qmul_np(R, R_loc)

Here, each joints' position(except roots') will be converted to the rotation about it's parent along kinematic chain. However, not every chain start from root, which means line 85 is not very suitable especially for the arms' chains.

I thought it will be more suitable if it's like:

R = quat_params[:, chain[0]]

Again, thanks for your great work!

[EricGuo5513]

Hi, thanks for pointing this out. Now it would be too late to change the code, since many people have done experiments on the obtained dataset. However, I will highlight your issue, so people will see your comments.

[IsshikiHugh]

Hi, thanks for pointing this out. Now it would be too late to change the code, since many people have done experiments on the obtained dataset. However, I will highlight your issue, so people will see your comments.

Thanks!