Enable optional consolidation of multiple yaml files into one

[danrgll]

To streamline configuration management, we could consolidate multiple YAML configurations—such as pipeline_space, custom_searcher, and others—into a single run_args file. Instead of referencing these configurations, they would be defined directly under their respective keys within run_args. This approach would simplify setup and can enhance usability by allowing users to manage all settings from one centralized file.

Note: This consolidation could be optional, not removing the existing functionalities.

[Neeratyoy]

Can you give an illustration or example or snippet of how you are visualizing the final feature?

[danrgll]

Exemplary idea:

run_args:
  # run_pipeline provided via neps.run()
  # or
  run_pipeline:
    path: "<path_to_run_pipeline>"
    name: "<name_of_run_pipeline>"

  pipeline_space: "pipeline_space.yaml"
  # or
  pipeline_space:
    epochs:
      lower: 1
      upper: 6
      log: True
    optimizer:
      choices: [ "adam", "sgd", "adamw" ]
    constant:
      value: 3

  root_directory: "my_directory"
  max_evaluations_total: 20
  post_run_summary: True
  searcher: "bayesian_optimization.yaml"
  # or
  searcher:
    algorithm: bayesian_optimization
    initial_design_size: 7
    surrogate_model: gp
    acquisition: EI
    log_prior_weighted: false
    acquisition_sampler: random
    random_interleave_prob: 0.1
    disable_priors: false
    prior_confidence: high
    sample_default_first: false