Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while loading checkpoint when training #41

Open
ManoniLo opened this issue Dec 8, 2021 · 0 comments
Open

Error while loading checkpoint when training #41

ManoniLo opened this issue Dec 8, 2021 · 0 comments

Comments

@ManoniLo
Copy link

ManoniLo commented Dec 8, 2021

I have to train ICNet on Camvid dataset.
I initialize the network with classification weights of the ResNet with a single stage training.

I set .config file and train_mem_saving.py input arguments as specified in https://github.com/oandrienko/fast-semantic-segmentation/blob/master/docs/icnet.md.

When initializing the model from the checkpoint "tmp/resnet_v1_50.ckpt", I get this error:

"Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key AuxOutput/biases not found in checkpoint
[[node save_1/RestoreV2 (defined at /content/libs/trainer.py:286) ]]".

It is strange since it sholdn't try to load variable 'AuxOutput/biases', since it does not belong to ResNet checkpoint.
I correctly set 'fine_tune_checkpoint_type' to 'classification' in .config file.

I am running the project on Google Colab which supports which supports tensorflow versions only greater than 1.13.1.
Has anyone successfully run the project with these versions of tf?

Thank you in advance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant