Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics should follow data-flow #31

Open
mariusae opened this issue Dec 5, 2019 · 0 comments
Open

Metrics should follow data-flow #31

mariusae opened this issue Dec 5, 2019 · 0 comments

Comments

@mariusae
Copy link
Collaborator

mariusae commented Dec 5, 2019

In the current implementation of metrics (#18), metrics are collected and aggregated from all tasks involved at the completion of a session.Run. This has a few drawbacks. First, all tasks may not be available at the end of session.Run (e.g., due to failed machines). Second, it ties stats collection to evaluation: we may in the future, persist task outputs to, e.g., s3, and clear task outputs from individual machines.

The solution to this is clear: we can flow Scopes along with task outputs; the executor can then aggregate them as the data flows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant