Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable rich markup if rich handler is not present #3682

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

sbrugman
Copy link
Contributor

@sbrugman sbrugman commented Mar 4, 2024

Description

Fix for #3626

Only used in Data Catalog (2x)

Done in Github UI to start the discussion.
Will add unit tests, and add release notes.

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • Read the contributing guidelines
  • Signed off each commit with a Developer Certificate of Origin (DCO)
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes
  • Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

@sbrugman sbrugman marked this pull request as ready for review March 4, 2024 18:43
@sbrugman sbrugman requested a review from merelcht as a code owner March 4, 2024 18:43
Comment on lines +63 to +66
def has_rich_handler(logger: logging.Logger) -> bool:
"""Returns true if the logger has a RichHandler attached."""
return any(isinstance(handler, RichHandler) for handler in logger.handlers)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to save this result to avoid calling this hundred times? It's probably have insignificant performance cost.

Copy link
Contributor Author

@sbrugman sbrugman Mar 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added functools.cache (if we assume the logger is not modified at runtime).
It's also possible to cache it on the DataCatalog in this case.

Side questions: has there been any profiling of Kedro? What are the most expensive parts of the framework?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sbrugman Sorry I missed the notification. I think it depends on what you are targeting.

#1476 is trying to optimise the CLI performance, those are insignificant time for a kedro run but significant for CLI UX.

For a kedro run, "it depends". The slowest one is always the node computation itself or the dataset I/O. Kedro itself does not have too much penalty, it can be even faster than a script if you can use some of the tricks to speed up (parallel runner, async I/O etc).

The main "penalty" I would say is saving & loading the same data, which could be reduced by using CacheDataset or something similar to avoid the unnecessary loading/blocking time.

@merelcht merelcht added the Community Issue/PR opened by the open-source community label Mar 27, 2024
@josephfinlayson
Copy link

I would like this to go in please

@noklam noklam enabled auto-merge (squash) April 22, 2024 21:37
@noklam noklam disabled auto-merge April 22, 2024 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community Issue/PR opened by the open-source community
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

None yet

4 participants