Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sh run_crf_ner.sh run error #9

Open
hhdo opened this issue Oct 14, 2020 · 2 comments
Open

sh run_crf_ner.sh run error #9

hhdo opened this issue Oct 14, 2020 · 2 comments

Comments

@hhdo
Copy link

hhdo commented Oct 14, 2020

I don't know why occur this problem, thank you for reading

OSError: Model name '../pretrained_models/RoBERTa-zh-Large/' was not found in tokenizers model name list (roberta-base, roberta-large, roberta-large-mnli, distilroberta-base, roberta-base-openai-detector, roberta-large-openai-detector). We assumed '../pretrained_models/RoBERTa-zh-Large/' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url.

I have download all 3 file, including config.json\pytorch_model.bin\vocab.txt

Thank you in advance~

@liuyukid
Copy link
Owner

Maybe you're actually using a Bert model。

@hhdo
Copy link
Author

hhdo commented Oct 20, 2020

Maybe you're actually using a Bert model。

.sh file like this below.

#!/bin/bash

DATA_DIR='../datasets/mydataset/'
MODEL_TYPE='roberta'
MODEL_NAME_OR_PATH='../pretrained_models/RoBERTa-zh-Large/'
OUTPUT_DIR='../output/roberta/'
LABEL='../datasets/cluener/labels.txt'

CUDA_VISIBLE_DEVICES='1' python ../examples/run_crf_ner.py \
--data_dir $DATA_DIR \
--model_type $MODEL_TYPE \
--model_name_or_path $MODEL_NAME_OR_PATH \
--output_dir $OUTPUT_DIR \
--labels $LABEL \
--overwrite_output_dir \
--do_train \
--do_eval \
--evaluate_during_training \
--adv_training fgm \
--num_train_epochs 3 \
--max_seq_length 128 \
--logging_steps 0.2 \
--per_gpu_train_batch_size 16 \
--per_gpu_eval_batch_size 16 \
--learning_rate 5e-5 \
--bert_lr 5e-5 \
--classifier_lr  5e-5 \
--crf_lr  1e-3 \

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants