Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ablation pipeline fails for new-style models #451

Closed
cthoyt opened this issue May 18, 2021 · 0 comments · Fixed by #456 · May be fixed by #450
Closed

Ablation pipeline fails for new-style models #451

cthoyt opened this issue May 18, 2021 · 0 comments · Fixed by #456 · May be fixed by #450
Assignees
@cthoyt
Labels
bug Something isn't working
Milestone
PyKEEN v1.5.0

Comments

@cthoyt
Copy link
Member

cthoyt commented May 18, 2021

Describe the bug

The ablation pipeline does not support new-style models (e.g., subclasses of pykeen.models.ERModel). The following stacktrace is produced by the code below:

Traceback (most recent call last):
  File "/Users/cthoyt/.virtualenvs/pykeen/lib/python3.9/site-packages/optuna/_optimize.py", line 217, in _run_trial
    value_or_values = func(trial)
  File "/Users/cthoyt/dev/pykeen/src/pykeen/hpo/hpo.py", line 199, in __call__
    result = pipeline(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/pipeline/api.py", line 880, in pipeline
    model_instance = _build_model_helper(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/pipeline/api.py", line 619, in _build_model_helper
    return model_resolver.make(
  File "/Users/cthoyt/dev/class_resolver/src/class_resolver/api.py", line 159, in make
    return cls(**(pos_kwargs or {}), **kwargs)  # type: ignore
  File "/Users/cthoyt/dev/pykeen/src/pykeen/models/unimodal/quate.py", line 115, in __init__
    super().__init__(
TypeError: __init__() got an unexpected keyword argument 'regularizer'
Traceback (most recent call last):
  File "/Users/cthoyt/dev/pykeen/scratch/ablation_test.py", line 5, in <module>
    ablation_pipeline(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/ablation/ablation.py", line 188, in ablation_pipeline
    _run_ablation_experiments(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/ablation/ablation.py", line 211, in _run_ablation_experiments
    hpo_pipeline_result = hpo_pipeline_from_path(rv_config_path)
  File "/Users/cthoyt/dev/pykeen/src/pykeen/hpo/hpo.py", line 410, in hpo_pipeline_from_path
    return hpo_pipeline_from_config(config, **kwargs)
  File "/Users/cthoyt/dev/pykeen/src/pykeen/hpo/hpo.py", line 415, in hpo_pipeline_from_config
    return hpo_pipeline(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/hpo/hpo.py", line 723, in hpo_pipeline
    study.optimize(
  File "/Users/cthoyt/.virtualenvs/pykeen/lib/python3.9/site-packages/optuna/study.py", line 400, in optimize
    _optimize(
  File "/Users/cthoyt/.virtualenvs/pykeen/lib/python3.9/site-packages/optuna/_optimize.py", line 66, in _optimize
    _optimize_sequential(
  File "/Users/cthoyt/.virtualenvs/pykeen/lib/python3.9/site-packages/optuna/_optimize.py", line 163, in _optimize_sequential
    trial = _run_trial(study, func, catch)
  File "/Users/cthoyt/.virtualenvs/pykeen/lib/python3.9/site-packages/optuna/_optimize.py", line 268, in _run_trial
    raise func_err
  File "/Users/cthoyt/.virtualenvs/pykeen/lib/python3.9/site-packages/optuna/_optimize.py", line 217, in _run_trial
    value_or_values = func(trial)
  File "/Users/cthoyt/dev/pykeen/src/pykeen/hpo/hpo.py", line 199, in __call__
    result = pipeline(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/pipeline/api.py", line 880, in pipeline
    model_instance = _build_model_helper(
  File "/Users/cthoyt/dev/pykeen/src/pykeen/pipeline/api.py", line 619, in _build_model_helper
    return model_resolver.make(
  File "/Users/cthoyt/dev/class_resolver/src/class_resolver/api.py", line 159, in make
    return cls(**(pos_kwargs or {}), **kwargs)  # type: ignore
  File "/Users/cthoyt/dev/pykeen/src/pykeen/models/unimodal/quate.py", line 115, in __init__
    super().__init__(
TypeError: __init__() got an unexpected keyword argument 'regularizer'

Process finished with exit code 1

To Reproduce

from pykeen.ablation import ablation_pipeline
from pykeen.constants import PYKEEN_EXPERIMENTS

if __name__ == '__main__':
    ablation_pipeline(
        datasets='FB15k-237',
        models=[
            'QuatE',
            'MuRE',
        ],
        losses=['NSSA', 'BCE', 'MR'],
        directory=PYKEEN_EXPERIMENTS.joinpath('test'),
        optimizers='Adam',
        training_loops=['LCWA', 'sLCWA'],
    )

Expected behavior

Run the ablation pipeline without error

Environment:

Key Value
OS posix
Platform Darwin
Release 20.4.0
User cthoyt
Time Tue May 18 11:36:21 2021
Python 3.9.5
PyKEEN 1.4.1-dev-7d7f4ac8
PyTorch 1.8.1
CUDA Available? false
CUDA Version N/A
cuDNN Version N/A
@cthoyt cthoyt added the bug Something isn't working label May 18, 2021
@cthoyt cthoyt self-assigned this May 18, 2021
This was referenced May 18, 2021
Draft
Closed
@cthoyt cthoyt added this to the PyKEEN v1.5.0 milestone May 19, 2021
cthoyt added a commit that referenced this issue May 20, 2021
@cthoyt
Fix handling of regularizer in ablation study
7c75d07 
Closes #451
Trigger CI
@cthoyt cthoyt mentioned this issue May 20, 2021
Merged
cthoyt added a commit that referenced this issue May 22, 2021
@cthoyt @PyKEEN-bot @mali-git
🦆 🎪 Fix handling of regularizer in ablation study (#456)
193f2bf 
Closes #451

Co-authored-by: PyKEEN_bot <pykeen2019@gmail.com>
Co-authored-by: Mehdi Ali <33023925+mali-git@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cthoyt cthoyt

Labels
bug Something isn't working
Projects
None yet
Milestone
PyKEEN v1.5.0
1 participant
@cthoyt