Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fine-tuning option #142

Merged
merged 6 commits into from May 11, 2024
Merged

Add fine-tuning option #142

merged 6 commits into from May 11, 2024

Conversation

ElliottKasoar
Copy link
Member

Aims to resolve #38

  • Adds --fine-tune option to janus train

    • Given the current differences, this feels more natural than an entirely different command
    • Currently only checks that foundation_model is in the config for fine-tuning, and not present otherwise, and that the value of foundation_model is either a valid MACE-MP/MACE-OFF string or a path that exists
    • We could load the model (if given a path, or even if given a valid string), but we would be unable to pass this directly to MACE
  • Adds tests based on data and configs from @alinelena in add fine tuning #38

    • Modifications include:
      • File names and locations for train/test/validation data
      • max_num_epochs reduced from 10 to 1 for both configs
      • Uses mace_mp_small.model model for fine-tuning, as we use it elsewhere
      • Changes default_dtype to float64 to match model for fine-tuning
    • Skipped if unable to load run_train from MACE, so not currently tested in CI, but can be tested localled
    • The fine-tuning test is quite slow (~1 minute) - are there ways to reduce this without inhibiting the function of the test? It may be inevitably given the task

@ElliottKasoar ElliottKasoar added enhancement New feature or request janus labels May 9, 2024
@ElliottKasoar ElliottKasoar self-assigned this May 9, 2024
Copy link
Collaborator

@oerc0122 oerc0122 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine, just a couple of stylistic things.

tests/test_train_cli.py Outdated Show resolved Hide resolved
tests/test_train_cli.py Outdated Show resolved Hide resolved
oerc0122
oerc0122 previously approved these changes May 10, 2024
@alinelena alinelena merged commit adf3dbc into stfc:main May 11, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request janus
Projects
None yet
Development

Successfully merging this pull request may close these issues.

we may want fine tuning as a separate option since we need to check more stuff.. add fine tuning
3 participants