Key error 0 on evaluation set

[bollampa]

Hello,

I built a sample model with only 18 records.
The tokenization used was from spacy and the format is BILUO instead of BIO and I only have one class which is Org. I modified the get labels function and also verified the model config json file.

My labels looks like this,

["O", "B-ORG", "I-ORG", "L-ORG", "U-ORG", "[CLS]", "[SEP]"]

The training goes fine, but validation fails with the following error.

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
06/12/2020 14:07:59 - INFO - main - ***** Running evaluation *****
06/12/2020 14:07:59 - INFO - main - Num examples = 18
06/12/2020 14:07:59 - INFO - main - Batch size = 8
Evaluating: 0%| | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
File "run_ner.py", line 594, in
main()
File "run_ner.py", line 582, in main
temp_2.append(label_map[logits[i][j]])
KeyError: 0

I have attached the train and valid files.
Please point me in the right direction.

train.txt
valid.txt

[heraclex12]

I have the same issue. Have you resolved it?

I think you can treat 0's index as PAD tag/or O tag. And then compute scores exclude PAD/O tag.