Error when exporting model

[shihweichieh2023]

Hi there

I am training my model on Colab, I think the training all went okay but I have this after I ran the last cell. I think my RAVE is up to date? can anyone help with this? thank you so much.

I am using Colab template here:
https://github.com/devstermarts/Notebooks/blob/main/RAVE_Training_Template--Colab.ipynb?short_path=d168939

error message:
INFO:root:library loading
INFO:root:DEBUG
INFO:torch.distributed.nn.jit.instantiator:Created a temporary directory at /tmp/tmpagfj7kc8
INFO:torch.distributed.nn.jit.instantiator:Writing /tmp/tmpagfj7kc8/_remote_module_non_scriptable.py
I0122 10:47:15.689762 137650626262848 export.py:338] building rave
/content/miniconda/lib/python3.9/site-packages/torchaudio/transforms/_transforms.py:92: UserWarning: return_complex argument is now deprecated and is not effective.torchaudio.transforms.Spectrogram(power=None) always returns a tensor with complex dtype. Please remove the argument in the function call.
warnings.warn(
I0122 10:47:19.099579 137650626262848 export.py:372] warmup pass
I0122 10:47:19.344898 137650626262848 export.py:377] optimize model
I0122 10:47:19.616676 137650626262848 export.py:382] script model
Traceback (most recent call last):
File "/content/miniconda/bin/rave", line 8, in
sys.exit(main())
File "/content/miniconda/lib/python3.9/site-packages/scripts/main_cli.py", line 34, in main
app.run(export.main)
File "/content/miniconda/lib/python3.9/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/content/miniconda/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/content/miniconda/lib/python3.9/site-packages/scripts/export.py", line 384, in main
scripted_rave = script_class(
File "/content/miniconda/lib/python3.9/site-packages/scripts/export.py", line 85, in init
raise ValueError("Stereo mode not yet supported with AdaIN")
ValueError: Stereo mode not yet supported with AdaIN

[devstermarts]

Remove the --stereo flag when exporting the model. The V3 architecture doesn't support it.

[carloscastellanos]

I am having a similar issue. I think in my case it may be that I trained on stereo files. I set the architecture to v2 and the number of channels to 2 (and removed the --stereo flag).. Does rave not support stereo? I'm using the hexorcismos CoLab notebook: https://colab.research.google.com/drive/1ih-gv1iHEZNuGhHPvCHrleLNXvooQMvI?usp=sharing

INFO:root:library loading
INFO:root:DEBUG
I0202 17:25:31.399044 136738913838144 export.py:495] building rave
/content/miniconda/lib/python3.9/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
I0202 17:25:32.253728 136738913838144 export.py:505] model found : /content/drive/MyDrive/rave/checkpoints/runs/test_run_e5ef914ed9/version_4/checkpoints/epoch-epoch=0647.ckpt
Traceback (most recent call last):
File "/content/miniconda/bin/rave", line 8, in
sys.exit(main())
File "/content/miniconda/lib/python3.9/site-packages/scripts/main_cli.py", line 38, in main
app.run(export.main)
File "/content/miniconda/lib/python3.9/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/content/miniconda/lib/python3.9/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/content/miniconda/lib/python3.9/site-packages/scripts/export.py", line 513, in main
pretrained.load_state_dict(
File "/content/miniconda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2153, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for RAVE:
size mismatch for encoder.encoder.net.0.weight_v: copying a param with shape torch.Size([96, 32, 7]) from checkpoint, the shape in current model is torch.Size([96, 16, 7]).
size mismatch for decoder.net.21.weight_g: copying a param with shape torch.Size([64, 1, 1]) from checkpoint, the shape in current model is torch.Size([32, 1, 1]).
size mismatch for decoder.net.21.weight_v: copying a param with shape torch.Size([64, 96, 7]) from checkpoint, the shape in current model is torch.Size([32, 96, 7]).
size mismatch for discriminator.discriminators.0.layers.0.net.0.weight_v: copying a param with shape torch.Size([96, 2, 5, 1]) from checkpoint, the shape in current model is torch.Size([96, 1, 5, 1]).
size mismatch for discriminator.discriminators.0.layers.1.net.0.weight_v: copying a param with shape torch.Size([96, 2, 5, 1]) from checkpoint, the shape in current model is torch.Size([96, 1, 5, 1]).
size mismatch for discriminator.discriminators.0.layers.2.net.0.weight_v: copying a param with shape torch.Size([96, 2, 5, 1]) from checkpoint, the shape in current model is torch.Size([96, 1, 5, 1]).
size mismatch for discriminator.discriminators.0.layers.3.net.0.weight_v: copying a param with shape torch.Size([96, 2, 5, 1]) from checkpoint, the shape in current model is torch.Size([96, 1, 5, 1]).
size mismatch for discriminator.discriminators.0.layers.4.net.0.weight_v: copying a param with shape torch.Size([96, 2, 5, 1]) from checkpoint, the shape in current model is torch.Size([96, 1, 5, 1]).
size mismatch for discriminator.discriminators.1.layers.0.net.0.weight_v: copying a param with shape torch.Size([96, 2, 15]) from checkpoint, the shape in current model is torch.Size([96, 1, 15]).
size mismatch for discriminator.discriminators.1.layers.1.net.0.weight_v: copying a param with shape torch.Size([96, 2, 15]) from checkpoint, the shape in current model is torch.Size([96, 1, 15]).
size mismatch for discriminator.discriminators.1.layers.2.net.0.weight_v: copying a param with shape torch.Size([96, 2, 15]) from checkpoint, the shape in current model is torch.Size([96, 1, 15]).

[carloscastellanos]

It seems like the issue is partially explained here: #275

So yes, I guess v2 & v3 architectures do not support stereo

[FlexCouncil]

I got the same results, but also with v1/default. Did you find a way to export stereo models and generate with v1, or any other configuration?

[carloscastellanos]

Yes, I added "model.RAVE.n_channels = 2" to the config.gin (as they did on the issue I linked). I was able to export it and it works but I think it's just 2 mono channels, not true stereo

[FlexCouncil]

I added “model.RAVE.n_channels = 2” to the .gin and exported, too, but when I generated I got a hanging “0it [00:00, ?it/s]” progress meter. The same thing happened with a mono-trained model, although I didn’t have to alter the .gin file. The only way I could get the mono-trained model to generate was with some workaround code I found on a YouTube video (https://www.youtube.com/watch?v=IkuzquGec4Y&pp=ygUQcmF2ZSBhdXRvZW5jb2Rlcg%3D%3D):

—

import torch
import soundfile as sf
import librosa as li
from IPython.display import Audio, display

x, sr = li.load(“path-to-input-audio.wav", sr=44100)

model = torch.jit.load(“path-to-torchscript-model.ts").eval()
x = torch.from_numpy(x).reshape(1,1,-1)
z = model.encode(x)
x_hat = model.decode(z).detach().numpy().reshape(-1)

sf.write("model_output.wav", x_hat, sr)

—

But I couldn’t find any way to generate from a stereo-trained model on v2.3.1.—I can’t even get a dual mono generation. That’s why I’m thinking of migrating back to a previous version.