Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Event Listener - Trigger #8

Open
Tracked by #12
franTarkenton opened this issue Mar 30, 2023 · 3 comments
Open
Tracked by #12

Implement Event Listener - Trigger #8

franTarkenton opened this issue Mar 30, 2023 · 3 comments
Assignees

Comments

@franTarkenton
Copy link
Member

franTarkenton commented Mar 30, 2023

In order to speed up the processing of the cmc grib data, this task will do the following:

  • determine how to create a event queue subscriber that can monitor for changes to data
  • Breakout subsequent tasks once problem is better understood

Focus of this work is to speed up the downloading / processing of the cmc grib data to enable hourly processing.

Task relates to: bcgov/nr-rfc-admin#21 and #7

@franTarkenton
Copy link
Member Author

franTarkenton commented Apr 5, 2023

working on this however is time consuming and difficult to debug due to infrequency that events are emitted for the data we are monitoring for.

Have resolves some timeout issues, but still haven't managed to observe an event from the stream we are attempting to monitor.

Branch with the work for this ticket: feat/event_listener

@franTarkenton
Copy link
Member Author

link to doc on how to setup the other side (github action) for remote trigger:
https://dev.to/rikurouvila/how-to-trigger-a-github-action-with-an-htt-request-545

@franTarkenton
Copy link
Member Author

franTarkenton commented Apr 12, 2023

Created the PR: #14 which contains code showing how to setup an event listener process.

Currently all the script does is listen for events and log them to the log that is emitted to STDOUT. Breaking this task up into smaller pieces described below

Next steps is to create tickets for the following tasks.

  1. containerize code - create dockerfile and github action that will build the dockerimage and cache either in dockerhub or github container registry.
  2. Create helm chart that will do the actual deployment to the openshift namespace
  3. Figure out best way to call github action remotely. See link in comment above.
  4. If time permitting create PR to metpx (https://github.com/MetPX/sarracenia) repo with documentation fixes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review / QA
Development

No branches or pull requests

1 participant