ICON+SCANimate

[Bill-WangJiLong]

Hi, thank you very much for the great work, it's cool. I want to implement ICON+SCANimate now, but I have two questions I would like to ask you.

1. SMPL fit
   The smpl parameter output of ICON is a .npy file with a size of a few kb, while the SMPL fit file format of SCANimate is a .npz file with a size of several hundred kb. Please tell me in the infer.py
   np. save(
   f"{args.out_dir}/{cfg.name}/obj/{data['name']}_smpl.npy", smpl_info, allow_pickle=True
   )
   Is it enough to change the file format to npz in this sentence? Do I need to do anything else to generate the parameter npz file of SMPL?
2. Remove untextured meshes
   Can I modify the obj file when infer.py is saved, please give me some details if it is convenient, thank you!
   Thanks again for your great work!

[Bill-WangJiLong]

Further, I seem to have solved the problem of the npz file. The problem now is that the size of the pose array in the npy file saved by ICON is different from the size of the CAPE data set used by SCANimate. After flattening, the ICON is 138 (23x6) or I It can be modified to 207 (23x3x3), but the CAPE is 72(24x3), which leads to the following error, how should I solve it, thank you.

[Bill-WangJiLong]

Now I seem to have solved this problem, but the result with SCANimate is very poor, can't see that it is a human shape, but I haven't tried to remove all invisible faces. Is this related to the problem, or should I make some changes to SCANimate?

[YuliangXiu]

Hi, thank you very much for the great work, it's cool. I want to implement ICON+SCANimate now, but I have two questions I would like to ask you.

1. SMPL fit
   The smpl parameter output of ICON is a .npy file with a size of a few kb, while the SMPL fit file format of SCANimate is a .npz file with a size of several hundred kb. Please tell me in the infer.py
   np. save(
   f"{args.out_dir}/{cfg.name}/obj/{data['name']}_smpl.npy", smpl_info, allow_pickle=True
   )
   Is it enough to change the file format to npz in this sentence? Do I need to do anything else to generate the parameter npz file of SMPL?
2. Remove untextured meshes
   Can I modify the obj file when infer.py is saved, please give me some details if it is convenient, thank you!
   Thanks again for your great work!

1. The raw output from ICON cannot be directly used for SCANimate, but it should contain all the info used for SCANimate.
2. Just remove the invisible faces, you can find the related functions from query_color

[Bill-WangJiLong]

Hi, thank you very much for the great work, it's cool. I want to implement ICON+SCANimate now, but I have two questions I would like to ask you.

1. SMPL fit
   The smpl parameter output of ICON is a .npy file with a size of a few kb, while the SMPL fit file format of SCANimate is a .npz file with a size of several hundred kb. Please tell me in the infer.py
   np. save(
   f"{args.out_dir}/{cfg.name}/obj/{data['name']}_smpl.npy", smpl_info, allow_pickle=True
   )
   Is it enough to change the file format to npz in this sentence? Do I need to do anything else to generate the parameter npz file of SMPL?
2. Remove untextured meshes
   Can I modify the obj file when infer.py is saved, please give me some details if it is convenient, thank you!
   Thanks again for your great work!

1. The raw output from ICON cannot be directly used for SCANimate, but it should contain all the info used for SCANimate.
2. Just remove the invisible faces, you can find the related functions from query_color

Thank you for your reply. I have revised the first question by referring to other answers. Thank you for your reminder on the second question.
Another question I have is, do I need to change any parts of SCANimate besides changing the output of ICON? Such as training parameters or LOSS functions, except for training epochs.
Although I was able to run through the process, the results I got were very poor, but this may also be related to the fact that I used too little data during the experiment. I want to make sure that my method is correct.

[YuliangXiu]

Hi, thank you very much for the great work, it's cool. I want to implement ICON+SCANimate now, but I have two questions I would like to ask you.

1. SMPL fit
   The smpl parameter output of ICON is a .npy file with a size of a few kb, while the SMPL fit file format of SCANimate is a .npz file with a size of several hundred kb. Please tell me in the infer.py
   np. save(
   f"{args.out_dir}/{cfg.name}/obj/{data['name']}_smpl.npy", smpl_info, allow_pickle=True
   )
   Is it enough to change the file format to npz in this sentence? Do I need to do anything else to generate the parameter npz file of SMPL?
2. Remove untextured meshes
   Can I modify the obj file when infer.py is saved, please give me some details if it is convenient, thank you!
   Thanks again for your great work!

1. The raw output from ICON cannot be directly used for SCANimate, but it should contain all the info used for SCANimate.
2. Just remove the invisible faces, you can find the related functions from query_color

Thank you for your reply. I have revised the first question by referring to other answers. Thank you for your reminder on the second question. Another question I have is, do I need to change any parts of SCANimate besides changing the output of ICON? Such as training parameters or LOSS functions, except for training epochs. Although I was able to run through the process, the results I got were very poor, but this may also be related to the fact that I used too little data during the experiment. I want to make sure that my method is correct.

I just use the default setting for training, but firstly you should make sure the inputs of SCANimate are correct.

Convert the ICON's pose parameters into the right format, which are compatible with SCANimate's input. And only use the visible vertices to train all the networks (skinning, geometry, texture) of SCANimate.

[Bill-WangJiLong]

Hi, thank you very much for the great work, it's cool. I want to implement ICON+SCANimate now, but I have two questions I would like to ask you.

1. SMPL fit
   The smpl parameter output of ICON is a .npy file with a size of a few kb, while the SMPL fit file format of SCANimate is a .npz file with a size of several hundred kb. Please tell me in the infer.py
   np. save(
   f"{args.out_dir}/{cfg.name}/obj/{data['name']}_smpl.npy", smpl_info, allow_pickle=True
   )
   Is it enough to change the file format to npz in this sentence? Do I need to do anything else to generate the parameter npz file of SMPL?
2. Remove untextured meshes
   Can I modify the obj file when infer.py is saved, please give me some details if it is convenient, thank you!
   Thanks again for your great work!

1. The raw output from ICON cannot be directly used for SCANimate, but it should contain all the info used for SCANimate.
2. Just remove the invisible faces, you can find the related functions from query_color

Thank you for your reply. I have revised the first question by referring to other answers. Thank you for your reminder on the second question. Another question I have is, do I need to change any parts of SCANimate besides changing the output of ICON? Such as training parameters or LOSS functions, except for training epochs. Although I was able to run through the process, the results I got were very poor, but this may also be related to the fact that I used too little data during the experiment. I want to make sure that my method is correct.

I just use the default setting for training, but firstly you should make sure the inputs of SCANimate are correct.

Convert the ICON's pose parameters into the right format, which are compatible with SCANimate's input. And only use the visible vertices to train all the networks (skinning, geometry, texture) of SCANimate.

Thank you very much for the guide, I will give it a try

[Bill-WangJiLong]

Hi, thank you very much for the great work, it's cool. I want to implement ICON+SCANimate now, but I have two questions I would like to ask you.

1. SMPL fit
   The smpl parameter output of ICON is a .npy file with a size of a few kb, while the SMPL fit file format of SCANimate is a .npz file with a size of several hundred kb. Please tell me in the infer.py
   np. save(
   f"{args.out_dir}/{cfg.name}/obj/{data['name']}_smpl.npy", smpl_info, allow_pickle=True
   )
   Is it enough to change the file format to npz in this sentence? Do I need to do anything else to generate the parameter npz file of SMPL?
2. Remove untextured meshes
   Can I modify the obj file when infer.py is saved, please give me some details if it is convenient, thank you!
   Thanks again for your great work!

1. The raw output from ICON cannot be directly used for SCANimate, but it should contain all the info used for SCANimate.
2. Just remove the invisible faces, you can find the related functions from query_color

Thank you for your reply. I have revised the first question by referring to other answers. Thank you for your reminder on the second question. Another question I have is, do I need to change any parts of SCANimate besides changing the output of ICON? Such as training parameters or LOSS functions, except for training epochs. Although I was able to run through the process, the results I got were very poor, but this may also be related to the fact that I used too little data during the experiment. I want to make sure that my method is correct.

I just use the default setting for training, but firstly you should make sure the inputs of SCANimate are correct.

Convert the ICON's pose parameters into the right format, which are compatible with SCANimate's input. And only use the visible vertices to train all the networks (skinning, geometry, texture) of SCANimate.

Hello, sorry to bother you, I tried according to your method, including modifying the pose parameters and removing the invisible faces, but in SCANimate, neither the cano model nor the reposed model I trained had legs, and the final result still appeared Some redundant points, where is the problem? Should I modify the result generated by ICON or should I modify SCANimate somewhere?
cano

reposed

final output

[tao-he]

@Bill-WangJiLong did you get the correct avatar by SCANimate?

[Bill-WangJiLong]

@Bill-WangJiLong did you get the correct avatar by SCANimate?

The problem I mentioned above seems to be caused by inconsistent global scales, but I ultimately failed to achieve good results. I am not sure if my method is incorrect or the data is insufficient