Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why do I get different a lower accuracy with EasySet? #146

Open
TeddyPorfiris opened this issue May 2, 2024 · 1 comment
Open

Why do I get different a lower accuracy with EasySet? #146

TeddyPorfiris opened this issue May 2, 2024 · 1 comment
Labels
question Further information is requested

Comments

@TeddyPorfiris
Copy link

TeddyPorfiris commented May 2, 2024

When I import the Omniglot dataset and define the train and test set as follows, I get an accuracy of 98%.

# training set of Omniglot
train_set = Omniglot(
    root="./data",
    background=True,
    transform=transforms.Compose(
        [
            transforms.Grayscale(num_output_channels=3),
            transforms.RandomResizedCrop(image_size),
            transforms.RandomHorizontalFlip(),
            transforms.ToTensor(),
        ]
    ),
    download=True,
)

# test set
test_set = Omniglot(
    root="./data",
    background=False,
    transform=transforms.Compose(
        [
            # Omniglot images have 1 channel, but our model will expect 3-channel images
            transforms.Grayscale(num_output_channels=3),
            transforms.Resize([int(image_size * 1.15), int(image_size * 1.15)]),
            transforms.CenterCrop(image_size),
            transforms.ToTensor(),
        ]
    ),
    download=True,
)

But when I downloaded the Omniglot dataset images from an external website and used EasySet (jsons attached) to have it work with easyfsl, I get an accuracy of 79%.

train_json = "train.json"
test_json = "test.json"

train_set = EasySet(train_json)
test_set = EasySet(test_json)

Why is this? The rest of the code is the same for both situations (I am following the my_first_few_shot_classifier tutorial). Thanks a lot!
test.json
train.json

@TeddyPorfiris TeddyPorfiris added the question Further information is requested label May 2, 2024
@ebennequin
Copy link
Collaborator

Without knowing the dataset you are using, there could be a number of reasons why the performances would drop. If you cannot trust your version of the Omniglot dataset, I strongly suggest you use the verified implementation from torchvision which we use in the notebook.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants