Skip to content

Latest commit

 

History

History

lm

Folders and files

NameName
Last commit message
Last commit date

parent directory

..

configs

configs

 
 

README.md

README.md

 
 

__init__.py

__init__.py

 
 

dataloader.py

dataloader.py

 
 

modeling.py

modeling.py

 
 

optimization_adafactor.py

optimization_adafactor.py

 
 

train.py

train.py

 
 

train_tpu_adafactor.sh

train_tpu_adafactor.sh

 
 

utils.py

utils.py

 
 

validate.py

validate.py

 
 

validate.sh

validate.sh

 
 

README.md

What everything does

  • validate.py gets perplexity. You can use the script validate.sh which contains some arguments I used.
  • train.py trains a grover model from scratch. The script train_tpu_adafactor.sh could also help. You probably don't want to do this unless you have a lot of money for TPUs. However, it might be a good idea to finetune Grover to a different domain.

Setting up tensorboard

During training, you can use tensorboard by running the following commands:

ssh -L 6006:localhost:6006 myservername
tensorboard --logdir="grover":"gs://MYOUTPUTPATH" --port=6006