Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when using the finetuned model generated with example data #1

Open
ArmandMichaud opened this issue May 3, 2024 · 1 comment

Comments

@ArmandMichaud
Copy link

ArmandMichaud commented May 3, 2024

Hello,
I'm trying to use finetuning, with the example for now, but I get an error. Finetuning didn't create an error, but using the generated model did. I don't get any errors when using a pre-trained model or a newly trained model.

python3 AIONER_Run.py -i ../example/input/ -m bioformer-softmax-es-finetune.h5 -v ../vocab/AnEM_label.vocab -e ALL -o example/output/

`
...
Total params: 42,598,935
Trainable params: 42,598,935
Non-trainable params: 0


Traceback (most recent call last):
File "AIONER_Run.py", line 252, in
NER_main_path(args.inpath, para_set, args.outpath, args.model)
File "AIONER_Run.py", line 194, in NER_main_path
nn_model.load_model(modelfile)
File "/home/amichaud/Documents/main/AIONER/src/model_ner.py", line 128, in load_model
self.model.load_weights(model_file)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 2211, in load_weights
hdf5_format.load_weights_from_hdf5_group(f, self.layers)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 708, in load_weights_from_hdf5_group
K.batch_set_value(weight_value_tuples)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/keras/backend.py", line 3576, in batch_set_value
x.assign(np.asarray(value, dtype=dtype(x)))
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 858, in assign
self._shape.assert_is_compatible_with(value_tensor.shape)
File "/home/amichaud/.local/lib/python3.8/site-packages/tensorflow/python/framework/tensor_shape.py", line 1134, in assert_is_compatible_with
raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (128, 19) and (128, 23) are incompatible

@lingluodlut
Copy link
Collaborator

I think you used the unmatch lable.vocab files. The bioformer-softmax-es-finetune.h5 model should use ../vocab/AIO_label.vocab, as this model is trained to predict (Gene, Chemical, Disease, Variant, Species, CellLine, ALL) entity types, not AnEM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants