Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit test for different subsamples #290

Open
gmartinonQM opened this issue Mar 27, 2023 · 0 comments · May be fixed by #312
Open

Unit test for different subsamples #290

gmartinonQM opened this issue Mar 27, 2023 · 0 comments · May be fixed by #312
Labels
developers Proposed by developers. enhancement New feature or request good first issue Good for newcomers

Comments

@gmartinonQM
Copy link
Contributor

gmartinonQM commented Mar 27, 2023

I have noticed that samples generated by the Subsample class all share the same random state.

random_state=random_state,

This suggests that all bootstraps are drawned with the same seed. However, quick experiments show that this is not the case.

Would it be possible to add a unit test to ensure that all samples drawn are different ? The same is true for blockbootstrap by the way.

Here is a suggestion :

import numpy as np
import pytest

from mapie.subsample import Subsample

def test_split_SubSample_different_amples() -> None:
    """Test that outputs of subsamplings are all different."""
    X = np.array([0, 1, 2, 3])
    cv = Subsample(n_resamplings=2, random_state=1)
    trains = [x[0] for x in cv.split(X)]
    tests = [x[1] for x in cv.split(X)]
    with np.testing.assert_raises(AssertionError):
         np.testing.assert_equal(trains[0], trains[1])
    with np.testing.assert_raises(AssertionError):
         np.testing.assert_equal(tests[0], tests[1])
@gmartinonQM gmartinonQM added the enhancement New feature or request label Mar 27, 2023
@LacombeLouis LacombeLouis added this to Need triage in Developments via automation Apr 19, 2023
@thibaultcordier thibaultcordier moved this from Need triage to Bugs to be fixed (Low Priority) in Developments Apr 20, 2023
@thibaultcordier thibaultcordier added the good first issue Good for newcomers label Apr 20, 2023
@thibaultcordier thibaultcordier linked a pull request Jun 19, 2023 that will close this issue
9 tasks
@thibaultcordier thibaultcordier added the contributors Proposed by contributors. label Jul 4, 2023
@thibaultcordier thibaultcordier added developers Proposed by developers. and removed contributors Proposed by contributors. labels Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
developers Proposed by developers. enhancement New feature or request good first issue Good for newcomers
Projects
Developments
Bugs to be fixed (Low Priority)
Development

Successfully merging a pull request may close this issue.

2 participants