Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit random seed options in ANTs #1464

Open
cookpa opened this issue Dec 14, 2022 · 6 comments
Open

Revisit random seed options in ANTs #1464

cookpa opened this issue Dec 14, 2022 · 6 comments

Comments

@cookpa
Copy link
Member

cookpa commented Dec 14, 2022

Following discussion in #1462

I believe reproducibility can be achieved by setting the env var ANTS_RANDOM_SEED and using single-threaded computation to avoid numerical issues in the mutual information metric.

But the usage of some scripts might be misleading, eg in antsBrainExtraction.sh, "-u 0" fixes the Atropos random seed, but doesn't set a random seed to be used by all the other processes.

@cookpa
Copy link
Member Author

cookpa commented Dec 15, 2022

Hi @ntustison ,

Thinking about this some more, I think antsAI wasn't the cause of the issue you were seeing recently, because it defaults to a fixed seed:

ANTs/Examples/antsAI.cxx

Lines 1245 to 1265 in 74afcc5

int antsRandomSeed = 1234;
itk::ants::CommandLineParser::OptionType::Pointer randomSeedOption = parser->GetOption("random-seed");
if (randomSeedOption && randomSeedOption->GetNumberOfFunctions())
{
antsRandomSeed = parser->Convert<int>(randomSeedOption->GetFunction(0)->GetName());
}
else
{
char * envSeed = getenv("ANTS_RANDOM_SEED");
if (envSeed != nullptr)
{
antsRandomSeed = std::stoi(envSeed);
}
}
if (antsRandomSeed != 0)
{
randomizer->SetSeed(antsRandomSeed);
}

I think it was antsRegistration being variable. I'm thinking for now that we could set a fixed seed inside the script if "-u 0" is detected.

Longer term, it would be good to make all the programs consistent in how they handle random number generation. antsRegistration does, in decreasing order of priority,

  1. Command line --random-seed uses
  2. $ANTS_RANDOM_SEED
  3. System time

Having this be consistent for all tools would be nice because it removes the need to add a random option explicitly to every executable call. I guess one criticism is that you could run a very complex pipeline with the same random numbers over and over, but I don't know if anyone cares to use ANTS_RANDOM_SEED outside of regression testing anyway.

@cookpa
Copy link
Member Author

cookpa commented Dec 22, 2022

Solution in #1468 aims to maximize backwards compatibility and hopefully won't break anyone's regression tests

@cookpa cookpa added this to the Harmonize random sampling milestone Dec 22, 2022
@TheChymera
Copy link
Contributor

What's the situation with recent-ish ANTs (2.4.3)? is setting a global ANTS_RANDOM_SEED value analogous to using --random-seed for each antsRegistration call?

@cookpa
Copy link
Member Author

cookpa commented May 22, 2023

@TheChymera yes, ANTS_RANDOM_SEED should be used unless --random-seed is specified at run time.

@TheChymera
Copy link
Contributor

@cookpa thank you :)

@cookpa
Copy link
Member Author

cookpa commented May 20, 2024

The last part of my to-do list for this issue was antsLongitudinalCorticalThickness.sh, but because it uses ANTS, it doesn't give identical results even if ANTS_RANDOM_SEED is set. I tried fixing the seed here

unsigned int time_seed = (unsigned int)time(nullptr);
srand(time_seed);

but it was insufficient.

Since deterministic results aren't a documented feature of the longitudinal script, I'm inclined to leave it alone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants