Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New engine logging #13450

Merged
merged 12 commits into from
May 18, 2024
Merged

New engine logging #13450

merged 12 commits into from
May 18, 2024

Conversation

desertaxle
Copy link
Member

@desertaxle desertaxle commented May 17, 2024

Adds logging for run creation and completion to the new engine so that it feels familiar and homey.

Also adds support for logging prints to the new engine.

Also fixes some issues with task run naming.

Example

For this flow:

from prefect import flow, task


@flow(retries=3, log_prints=True)
def my_flow(foo: bool):
    my_other_flow(42)
    print("Hello, world!")
    return slow_task.submit()

@flow
def my_other_flow(bar: int):
    pass

@task(timeout_seconds=10, task_run_name="bloop")
def slow_task():
    print("Sleeping...")
    import time
    time.sleep(1)

if __name__ == "__main__":
    my_flow(foo=True)

Running will produce this output:

22:14:31.434 | INFO    | prefect.engine - Created flow run 'didactic-prawn' for flow 'my-flow'
22:14:31.437 | INFO    | prefect.engine - View at http://127.0.0.1:4200/flow-runs/flow-run/785b8466-010a-4338-8f68-b0e1e10d5649
22:14:32.088 | INFO    | Flow run 'didactic-prawn' - Created subflow run 'burrowing-finch' for flow 'my-other-flow'
22:14:32.090 | INFO    | prefect.engine - View at http://127.0.0.1:4200/flow-runs/flow-run/ae098ba9-e966-43e1-bdf3-910f8ad50b7a
22:14:32.172 | INFO    | Flow run 'burrowing-finch' - Finished in state Completed()
22:14:32.185 | INFO    | Flow run 'didactic-prawn' - Hello, world!
22:14:32.191 | INFO    | Flow run 'didactic-prawn' - Submitting task slow_task to thread pool executor...
22:14:32.209 | INFO    | prefect.engine - Created task run 'slow_task-0' for task 'slow_task'
22:14:32.242 | INFO    | Task run 'bloop' - Sleeping...
22:14:33.267 | INFO    | Task run 'bloop' - Finished in state Completed()
22:14:33.294 | INFO    | Flow run 'didactic-prawn' - Finished in state Completed('All states completed.')

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • If this pull request adds new functionality, it includes unit tests that cover the changes
  • This pull request includes a label categorizing the change e.g. maintenance, fix, feature, enhancement, docs.

For documentation changes:

  • This pull request includes redirect settings in netlify.toml for files that are removed or renamed.

For new functions or classes in the Python SDK:

  • This pull request includes helpful docstrings.
  • If a new Python file was added, this pull request contains a stub page in the Python SDK docs and an entry in mkdocs.yml navigation.

@desertaxle desertaxle marked this pull request as ready for review May 18, 2024 03:27
@desertaxle desertaxle requested a review from a team as a code owner May 18, 2024 03:27
@desertaxle desertaxle requested a review from cicdw May 18, 2024 03:27
Copy link
Member

@cicdw cicdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this looks great!

@cicdw cicdw merged commit 0bad970 into main May 18, 2024
24 of 25 checks passed
@cicdw cicdw deleted the new-engine-logging branch May 18, 2024 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants