Skip to content

Releases: kdexd/lang-emerge-parlai

Best Pretrained Checkpoint

10 Nov 04:33
Compare
Choose a tag to compare

Attached checkpoint was the best one when the following script was executed at this commit:

python3 train.py --use-gpu --memoryless-abot --num-epochs 99999

Evaluation of the checkpoint:

python3 evaluate.py --load-path world_best.pth 

Reported metrics:

Overall accuracy [train]: 96.47 (first: 97.76, second: 98.72, atleast_one: 100.00)
Overall accuracy [val]: 98.61 (first: 98.61, second: 100.00, atleast_one: 100.00)

Minimal snippet to reconstruct the world using this checkpoint:

import torch

from bots import Questioner, Answerer
from world import QAWorld

world_dict = torch.load('path/to/checkpoint.pth')
questioner = Questioner(world_dict['opt'])
answerer = Answerer(world_dict['opt'])
if world_dict['opt'].get('use_gpu'):
    questioner, answerer = questioner.cuda(), answerer.cuda()

questioner.load_state_dict(world_dict['qbot'])
answerer.load_state_dict(world_dict['abot'])
world = QAWorld(world_dict['opt'], questioner, answerer)