Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow passing SMPL/HumanML3D initialization parameters #43

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

UuuNyaa
Copy link

@UuuNyaa UuuNyaa commented Nov 16, 2022

I have made it possible to initialize SMPL/HumanML3D by specifying the model path.
Now, you can load a model of any path with the following code:

class dotdict(dict):
    """dot.notation access to dictionary attributes"""
    __getattr__ = dict.get
    __setattr__ = dict.__setitem__
    __delattr__ = dict.__delitem__

SMPL_DATA_PATH = os.path.join(DATA_PATH, 'body_models', 'smpl')
SMPL_MODEL_PATH = os.path.join(SMPL_DATA_PATH, 'SMPL_NEUTRAL.pkl')
JOINT_REGRESSOR_TRAIN_EXTRA_PATH = os.path.join(SMPL_DATA_PATH, 'J_regressor_extra.npy')

data_loader = data.DataLoader(
    HumanML3D(mode='train', base_path=DATA_PATH, split='test', num_frames=max_frames),
    batch_size=batch_size, shuffle=True,
    num_workers=8, drop_last=True, collate_fn=t2m_collate
)

logging.info("Creating model and diffusion...")
model = model_util.MDM(
    smpl_model_path=SMPL_MODEL_PATH,
    joint_regressor_train_extra_path=JOINT_REGRESSOR_TRAIN_EXTRA_PATH,
    **model_util.get_model_args(dotdict({
        'dataset': dataset,
        'latent_dim': 512,
        'layers': 8,
        'cond_mask_prob': 0.1,
        'arch': 'trans_enc',
        'emb_trans_dec': False,
    }), data_loader)
)

diffusion = model_util.create_gaussian_diffusion(dotdict({
    'diffusion_steps': 1000,
    'diffusion_sampling_steps': 100,
    'noise_schedule': 'cosine',
    'sigma_small': True,
    'lambda_vel': 0.0,
    'lambda_rcxyz': 0.0,
    'lambda_fc': 0.0,
}))

@UuuNyaa UuuNyaa changed the title Allow passing SMPL initialization parameters Allow passing SMPL/HumanML3D initialization parameters Nov 17, 2022
@UuuNyaa
Copy link
Author

UuuNyaa commented Dec 6, 2022

Now we can specify the sampling timesteps for ddim.

@UuuNyaa
Copy link
Author

UuuNyaa commented Dec 20, 2022

I did motion editing (In-Betweening) on my CPU with ddim sampling.

$ time PYTHONPATH=mdm python3 motion_edit.py
TRANS_ENC init
EMBED TEXT
Loading CLIP...
100%|████████████| 100/100 [00:19<00:00,  5.19it/s]

real    0m23.509s
user    3m57.046s
sys     0m3.641s

CPU model and Settings

  • CPU: AMD Ryzen 9 3900XT 12-Core Processor
    seed = 1
    max_frames = 196
    guidance_param = 2.5
    device = 0
    text_condition = "a person jumps"
    num_samples = 1
    batch_size = 1
    dataset = 'humanml'
    prefix_end = 0.25
    suffix_start = 0.75
    diffusion_steps = 1000
    diffusion_sampling_steps = 100

    input_motion = np.load(os.path.join(DATA_PATH, 'dataset', 'joints', '000021.npy'))[:, :22]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants