Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support hot reloading of configuration #67

Open
1 of 6 tasks
JelteF opened this issue Nov 1, 2021 · 13 comments
Open
1 of 6 tasks

Support hot reloading of configuration #67

JelteF opened this issue Nov 1, 2021 · 13 comments
Assignees
Labels
enhancement New feature or request 🔥Hot Reload Tasks related to DAB's Hot Reload feature proposal
Milestone

Comments

@JelteF
Copy link
Contributor

JelteF commented Nov 1, 2021

Allow specifiying something like a --watch to make the runtime watch the provided configs and reload when they change.

Relevant comment from @jarupatj in #50:

You should look at using IOptionsMonitor
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-5.0#options-interfaces

#892

Tasks

  1. do-not-merge refactor
    ayush3797 seantleonard
  2. enhancement 🔥Hot Reload
    aaronburtle
  3. enhancement 🔥Hot Reload
    aaronburtle
  4. enhancement 🔥Hot Reload
    aaronburtle abhishekkumams
  5. refactor
    aaronburtle
  6. refactor 🔥Hot Reload
    aaronburtle
@seantleonard
Copy link
Contributor

can this be closed with #50 merged? @JelteF

@JelteF
Copy link
Contributor Author

JelteF commented Nov 29, 2021

I don't think this should be closed, because it is about the more user facing version of hot roloading configs. One that does not require dotnet watch but simply run the binary with a --watch flag.

However, I don't think this should be an M0 milestone. Work on dev tooling team can start just fine without this support. So I'm marking this as M1 instead now. @jarupatj please comment on this if you disagree with that for some reason.

@JelteF JelteF modified the milestones: M0, M1 Nov 29, 2021
@Aniruddh25 Aniruddh25 added this to To do in Data Gateway Runtime via automation Feb 10, 2022
@Aniruddh25
Copy link
Contributor

Solving this #189 will help here.

@Aniruddh25 Aniruddh25 moved this from To do to M1 in Data Gateway Runtime Feb 10, 2022
@Aniruddh25 Aniruddh25 modified the milestones: M1, M2 Apr 16, 2022
@Aniruddh25 Aniruddh25 moved this from M1 to To do in Data Gateway Runtime Apr 16, 2022
@Aniruddh25
Copy link
Contributor

Should we support hot reloading at all? Once runtime is bootstrapped, should we allow modifications to it?
In M1, we might expect customers to restart the docker image.
For M2, we will throw away the container and use a different one from the warm pool whenever any config changes.

Moving to M2 for now so we can discuss more.

@aaronburtle
Copy link
Contributor

Modified this issue to cove hot reloading of the configuration file. This will clear duplicate issues, which will all be tracked here.

@aaronburtle aaronburtle changed the title Support hot reloading Support hot reloading of configuration Apr 3, 2023
@aaronburtle
Copy link
Contributor

Would be great to have a switch on the CLI, something like --watch that will automatically reload the configuration file (and thus re-generate the in-memory schema) whenever the file is changed. Something like https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-watch

That would greatly improve development velocity.

@aaronburtle
Copy link
Contributor

In order to set the LogLevel based on the mode in the runtimeconfig we need to use a configuration provider that can reload the configuration after updating the Loglevel.

More information can be found here: https://learn.microsoft.com/en-us/dotnet/core/extensions/logging?tabs=command-line

@aaronburtle aaronburtle removed this from the Mar2023 milestone Apr 3, 2023
@aaronburtle aaronburtle added this to the Apr2023 milestone Apr 3, 2023
@aaronburtle
Copy link
Contributor

Moved to April, waiting on new configuration resolver.

@michaelstaib
Copy link
Collaborator

Form the GraphQL side you need to implement an ITypeModule ... this will manage the phase out of schemas...

Here you can see an example.
https://github.com/ChilliCream/graphql-platform/blob/13349fa4af03514507ed5e3aaf9a771c696b56a6/src/HotChocolate/Fusion/src/Core/DependencyInjection/FileWatcherTypeModule.cs

The TypeModule is registered with the schema.

@Aniruddh25 Aniruddh25 modified the milestones: 0.7, 0.8 May 8, 2023
@Aniruddh25 Aniruddh25 modified the milestones: 0.8, 0.9 Jun 27, 2023
@Aniruddh25 Aniruddh25 added the enhancement New feature or request label Jul 10, 2023
@aaronburtle aaronburtle modified the milestones: 0.9rc, 0.10rc Sep 25, 2023
@Aniruddh25 Aniruddh25 modified the milestones: 0.10rc, 0.11rc Nov 7, 2023
aaronburtle added a commit that referenced this issue Nov 11, 2023
## Why make this change?

Related to, covering the runtime section of the config only
#67

Because we are not able to make easy use of the `IOptionsMonitor` which
would have allowed for the updating of individual sections of the config
automatically (ie: just refreshing the `runtime options`), we monitor
and reload the entire file. However, we currently do not have support
for reloading the sections outside of the `Runtime` section of the
config. Because of this, we will not document and announce this feature
until the other sections can be safely reloaded.

Additionally, certain options are used to configure services at startup,
and despite refreshing the `RuntimeConfig`, those services will still be
configured with the values from initialization on startup. This means
adding a mechanism for refreshing those services to make use of the
updated options after a Hot Reload, and that work will come in a future
PR.



## What is this change?

Adds a config file watcher during startup which will monitor for any
file changes to the config and then trigger an update event in the case
that the file is changed.

We instantiate the config file watcher when the runtime provider is
first asked to get a config. We then register a method owned by the
runtime config provider that will be called back to in the case that we
wish to hot reload. However, we only do this for when we start the
service in local development. So that means when we are `not` late
configured, and we are in a Host Mode of Development on startup. In
other cases we do not watch the config file for changes, and do not make
this callback in the case that the config file does change.

To get the location of the file that we are going to watch we use the
config loader's file system and stored data regarding the config file.

## How was this tested?

Unit testing was added to verify correct hot reloading of the runtime
options that we intend to have changing.

## Sample Request(s)

Any valid request after the change ought to work.

---------

Co-authored-by: Sean Leonard <sean.leonard@microsoft.com>
aaronburtle added a commit that referenced this issue Nov 15, 2023
## Why make this change?

This is related to the following Hot Reload PR
#1726

and this issue #67


It adds a design doc to get feedback and iterate on how to implement the
Hot Reloading feature. Hot Reloading will be on going work as all of the
needed features to support it entirely are added.

## What is this change?

Adds the design doc and a basic diagram for the Hot Reload feature.

---------

Co-authored-by: Aniruddh Munde <anmunde@microsoft.com>
@seantleonard
Copy link
Contributor

hot-reload property in config?

@aaronburtle
Copy link
Contributor

I have created a number of sub-tasks to help track the progress on Hot Reload.

Currently, we are refreshing the RuntimeConfig to match any updates that we see in the associated config file when we are in the local scenario and start the engine in developer mode.

To facilitate extending Hot Reload in the future, a refactor of the RuntimeConfigProvider is taking place

#1966

as well as some changes to make hot reload work with the new multi config code

#1967

Additionally, one of the key features we want support for hot reloading is reloading of the log-level, and so this property is being added to the config and we have a task to track being able to hot reload this property in both dev and production mode

#1968

Additionally, there are some tasks for updating those settings which depend on the config only at startup, such as the OpenApiDoc and Auth settings

#1869

#1969

@seantleonard
Copy link
Contributor

Moving this + hot reload related tasks to post GA -> tracking via 1.2rc milestone.

@seantleonard seantleonard added the 🔥Hot Reload Tasks related to DAB's Hot Reload feature proposal label May 14, 2024
@seantleonard seantleonard modified the milestones: 1.2rc, Feature Backlog May 17, 2024
@seantleonard
Copy link
Contributor

The top-level task tracking hot reload can remain in backlog as we address sub-tasks. Ideally a sprint's tracked work items should be resolvable. This task has so many pre-reqs , we need to resolve those first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request 🔥Hot Reload Tasks related to DAB's Hot Reload feature proposal
Projects
No open projects
Development

No branches or pull requests

7 participants