Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker compose redis worker #455

Merged
merged 18 commits into from Jul 15, 2022
Merged

Docker compose redis worker #455

merged 18 commits into from Jul 15, 2022

Conversation

nhoening
Copy link
Contributor

@nhoening nhoening commented Jul 5, 2022

Closes #418

Also adds more documentation and lets us add schedules as a job in the CLI.

@nhoening
also allow warnable config settigs (mail, redis) to be set as env vars
e570d71 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
allow the CLI command for schedule creation to also simply add jobs t…
fffdd5c 
…o the queue (--as-jobs)

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
add redis and worker node to docker compose setup
d099cd6 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
update pre-commit tooling to Python 3.9
3517350 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Document worker queues in docker-compose; various improvements in oth…
9793cec 
…er parts, e.g. update and correct usage of CLI forecasting, redis as independent page under hosting

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening nhoening requested a review from Flix6x July 5, 2022 20:51
@nhoening nhoening marked this pull request as ready for review July 5, 2022 20:51
@nhoening
Merge branch 'main' into docker-compose-redis-worker
405661b 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
run black
2ba7944 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Merge branch 'docker-compose-redis-worker' of github.com:FlexMeasures…
f1b7c21 
…/flexmeasures into docker-compose-redis-worker
@coveralls
Copy link
Collaborator

coveralls commented Jul 5, 2022 •
edited

Pull Request Test Coverage Report for Build 2679470627

  • 11 of 51 (21.57%) changed or added relevant lines in 3 files are covered.
  • 252 unchanged lines in 13 files lost coverage.
  • Overall coverage decreased (-0.9%) to 67.485%
Changes Missing Coverage Covered Lines Changed/Added Lines %
flexmeasures/utils/config_utils.py 0 1 0.0%
flexmeasures/cli/data_add.py 2 14 14.29%
flexmeasures/cli/jobs.py 9 36 25.0%
Files with Coverage Reduction New Missed Lines %
flexmeasures/data/models/data_sources.py 1 56.16%
flexmeasures/data/schemas/utils.py 1 81.25%
flexmeasures/utils/config_utils.py 1 31.3%
flexmeasures/cli/jobs.py 2 27.66%
flexmeasures/api/v3_0/assets.py 3 96.15%
flexmeasures/data/services/sensors.py 7 26.09%
flexmeasures/api/v3_0/sensors.py 18 70.1%
flexmeasures/api/dev/sensors.py 20 61.64%
flexmeasures/data/models/charts/defaults.py 21 52.04%
flexmeasures/data/models/charts/belief_charts.py 27 13.33%
Totals Coverage Status
Change from base Build 2638960995: -0.9%
Covered Lines: 7150
Relevant Lines: 10025
💛 - Coveralls

@nhoening nhoening added this to the 0.11.0 milestone Jul 6, 2022
@nhoening
changelog entry
c6a51e2 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
changelog entry
c29b6de 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Flix6x
Flix6x reviewed Jul 9, 2022
View reviewed changes
Copy link
Contributor

@Flix6x Flix6x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to have the asynchronous stuff dockerized, too now. Here are my initial comments. I still need to test.

documentation/dev/docker-compose.rst Show resolved Hide resolved
documentation/dev/docker-compose.rst Outdated Show resolved Hide resolved
documentation/host/queues.rst Outdated Show resolved Hide resolved
documentation/tut/forecasting_scheduling.rst Outdated Show resolved Hide resolved
documentation/tut/forecasting_scheduling.rst Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
Flix6x
Flix6x requested changes Jul 11, 2022
View reviewed changes
documentation/tut/forecasting_scheduling.rst Show resolved Hide resolved
documentation/tut/forecasting_scheduling.rst Outdated Show resolved Hide resolved
documentation/tut/forecasting_scheduling.rst Outdated Show resolved Hide resolved
documentation/tut/forecasting_scheduling.rst Show resolved Hide resolved
documentation/tut/forecasting_scheduling.rst Show resolved Hide resolved
@nhoening
small doc corrections from review
4b8b4d9 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
fix parameter names in example
e0963f7 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
improve handling of new worker names - random if not given, add itera…
95bfd2b 
…ted suffix number if taken

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
allow to clear failed queue and add show-queues command
b605519 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
improve reporting on added jobs in CLI
0d8744a 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Merge branch 'main' into docker-compose-redis-worker
ebc248f 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening nhoening requested a review from Flix6x July 12, 2022 11:15
Flix6x
Flix6x requested changes Jul 14, 2022
View reviewed changes
Copy link
Contributor

@Flix6x Flix6x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only remaining comment is about showing and clearing the failed queue. If this is holding the PR back, feel free to revert the applicable commits and to open a separate PR for that.

flexmeasures/cli/jobs.py Outdated Show resolved Hide resolved
@nhoening
fix clearing the registry of failed jobs; improve a variable name and…
6b1bf4f 
… ordering of CLI args

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening nhoening requested a review from Flix6x July 15, 2022 08:22
Flix6x
Flix6x approved these changes Jul 15, 2022
View reviewed changes
flexmeasures/cli/jobs.py Outdated Show resolved Hide resolved
flexmeasures/cli/jobs.py Outdated Show resolved Hide resolved
@nhoening
small improvement to UX of jobqueue-handling CLI
1d1e436 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening nhoening merged commit d428472 into main Jul 15, 2022
@nhoening nhoening deleted the docker-compose-redis-worker branch July 15, 2022 21:55
@nhoening nhoening mentioned this pull request Jul 27, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Flix6x Flix6x Flix6x approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.11.0
Development

Successfully merging this pull request may close these issues.

Add Redis and a worker to Docker-Compose
3 participants
@nhoening @coveralls @Flix6x