AttributeError: 'DatasetDict' object has no attribute 'train_test_split'

[david-waterworth]

The following code fails with "'DatasetDict' object has no attribute 'train_test_split'" - am I doing something wrong?

from datasets import load_dataset
dataset = load_dataset('csv', data_files='data.txt')
dataset = dataset.train_test_split(test_size=0.1)

AttributeError: 'DatasetDict' object has no attribute 'train_test_split'

[SBrandeis]

Hi @david-waterworth!

As indicated in the error message, load_dataset("csv") returns a DatasetDict object, which is mapping of str to Dataset objects. I believe in this case the behavior is to return a train split with all the data.
train_test_split is a method of the Dataset object, so you will need to do something like this:

dataset_dict = load_dataset(`'csv', data_files='data.txt')
dataset = dataset_dict['split name, eg train']
dataset.train_test_split(test_size=0.1)

Please let me know if this helps. 🙂

[david-waterworth]

Thanks, that's working - the same issue also tripped me up with training.

I also agree #767 would be a useful addition.

[SBrandeis]

Closing this now

[nehanawalkar]

dataset_dict = load_dataset(`'csv', data_files='data.txt')
dataset = dataset_dict['split name, eg train']
dataset.train_test_split(test_size=0.1)

I am getting error like
KeyError: 'split name, eg train'
Could you please tell me how to solve this?

[sadidul012]

dataset = load_dataset('csv', data_files=['files/datasets/dataset.csv'])
dataset = dataset['train']
dataset = dataset.train_test_split(test_size=0.1)

[dhikaputri]

!curl -L "https://app.roboflow.com/ds/YQYgzFyKns?key=f0IwaEetrr" > roboflow.zip; unzip roboflow.zip; rm roboflow.zip

from datasets import load_dataset

dataset = load_dataset("imagefolder", data_dir="/content/")
dataset["train"][0]

dataset["train"][-1]

train_ds = load_dataset("imagefolder", data_dir="/content/train/")
test_ds = load_dataset("imagefolder", data_dir="/content/test/")
val_ds = load_dataset("imagefolder", data_dir="/content/valid/")

train_ds.features

and i got error
AttributeError Traceback (most recent call last)
in <cell line: 1>()
----> 1 train_ds.features

AttributeError: 'DatasetDict' object has no attribute 'features'

[david-waterworth]

This has been closed, you should open a new issue describing what your problem is.