-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test and eval sets the same? #50
Comments
Is it not already separated with the train_test_split |
Normally, there are supposed to be three datasets: train, test and eval. In SimpleT5, there seems to be confusion on calling the non-training datasets Here you can see in the code there are 3 data loaders: def train_dataloader(self):
"""training dataloader"""
return DataLoader(
self.train_dataset,
batch_size=self.batch_size,
shuffle=True,
num_workers=self.num_workers,
)
def test_dataloader(self):
"""test dataloader"""
return DataLoader(
self.test_dataset,
batch_size=self.batch_size,
shuffle=False,
num_workers=self.num_workers,
)
def val_dataloader(self):
"""validation dataloader"""
return DataLoader(
self.test_dataset,
batch_size=self.batch_size,
shuffle=False,
num_workers=self.num_workers,
) See how |
I noticed using the same dataset for test and eval? Why do this and not separate out an eval set?
See self.test_dataset being used twice here.
The text was updated successfully, but these errors were encountered: