Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to use Kedro after uninstalling Rich. #3838

Merged
merged 57 commits into from May 21, 2024

Conversation

lrcouto
Copy link
Contributor

@lrcouto lrcouto commented Apr 24, 2024

Description

Allows Kedro to work regardless of the rich library being installed. At this moment, installing Kedro will still install rich, but the user will have the option to remove the library and still have everything work.

Because cookiecutter versions 2.3 and above depend on rich, it is also necessary to remove the version installed automatically by Kedro and install a compatible one to be able to run without rich.

Logging, prompts, and the kedro ipython command will work without rich given a compatible cookiecutter version is also being used. Our own unit tests will not, they will fail without rich.

Development notes

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • Read the contributing guidelines
  • Signed off each commit with a Developer Certificate of Origin (DCO)
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes
  • Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

lrcouto and others added 6 commits April 24, 2024 12:48
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
kedro/logging.py Outdated
@@ -30,6 +30,10 @@ class RichHandler(rich.logging.RichHandler):
"""

def __init__(self, *args: Any, **kwargs: Any):
if rich is None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the basic idea that users who don't install rich should default to this? So for users who already have rich installed and no longer want it they need to pip uninstall rich? If that is the case we should document the steps they need to take.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noklam do you have any thoughts on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought on this it shouldn't touch the log handler.

If the custom logger has Rich handler, it should raise error when it is not installed. To get rid of this the solution is just remove rich from the handler.

The thing that user cannot change now is all the traceback error / CLI that ties with rich. Essentially one cannot run Kedro without rich. @astrojuanlu

I am surprised the _ProjectLogging isn't changed here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SajidAlamQB yes, the way this is set up at the moment, if the user does not want rich to be used, it would need to be uninstalled.

@SajidAlamQB
Copy link
Contributor

Hey @lrcouto, this is an interesting approach, I think we will also need to modify kedro/framework/project/default_logging.yml, kedro/templates/project/{{ cookiecutter.repo_name }}/conf/logging.yml as they both also point to rich.

kedro/logging.py Outdated Show resolved Hide resolved
kedro/logging.py Outdated Show resolved Hide resolved
lrcouto and others added 12 commits April 30, 2024 11:43
Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
@astrojuanlu
Copy link
Member

The thing that user cannot change now is all the traceback error / CLI that ties with rich. Essentially one cannot run Kedro without rich.

Don't quite understand that part, could you elaborate @noklam ?

kedro/logging.py Outdated Show resolved Hide resolved
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
@noklam
Copy link
Contributor

noklam commented May 3, 2024

@astrojuanlu Alright I was confused because I see no changes on these:

rich.pretty.install()

and

kedro/kedro/logging.py

Lines 56 to 60 in 9da8b37

if self.rich_tracebacks and not _is_databricks():
# Rich traceback handling does not work on databricks. Hopefully this will be
# fixed on their side at some point, but until then we disable it.
# See https://github.com/Textualize/rich/issues/2455
rich.traceback.install(**traceback_install_kwargs) # type: ignore[arg-type]

Now I realise the logic is in the early return.

My question is, why do we want to be clever about the RichHandler? Alternatively, we can have another "default" setting, and if rich is not importable we simply use another default. One thing that doesn't work is if conf/logging.yml still have rich enabled.

In that case, I think it's reasonable to fail rather than being smart about it.

path = os.environ.get(
"KEDRO_LOGGING_CONFIG", Path(__file__).parent / "default_logging.yml"
)
logging_config = Path(path).read_text(encoding="utf-8")
self.configure(yaml.safe_load(logging_config))

@@ -40,4 +40,4 @@ loggers:
level: INFO

root:
handlers: [rich, info_file_handler]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need both handlers? would it duplicates the log?

@lrcouto
Copy link
Contributor Author

lrcouto commented May 3, 2024

@astrojuanlu Alright I was confused because I see no changes on these:

rich.pretty.install()

and

kedro/kedro/logging.py

Lines 56 to 60 in 9da8b37

if self.rich_tracebacks and not _is_databricks():
# Rich traceback handling does not work on databricks. Hopefully this will be
# fixed on their side at some point, but until then we disable it.
# See https://github.com/Textualize/rich/issues/2455
rich.traceback.install(**traceback_install_kwargs) # type: ignore[arg-type]

Now I realise the logic is in the early return.

My question is, why do we want to be clever about the RichHandler? Alternatively, we can have another "default" setting, and if rich is not importable we simply use another default. One thing that doesn't work is if conf/logging.yml still have rich enabled.

In that case, I think it's reasonable to fail rather than being smart about it.

path = os.environ.get(
"KEDRO_LOGGING_CONFIG", Path(__file__).parent / "default_logging.yml"
)
logging_config = Path(path).read_text(encoding="utf-8")
self.configure(yaml.safe_load(logging_config))

I was looking into this idea of adding a non-rich default yesterday, but that would mean having to try to import rich on init.py to check if it is installed, and I was trying to avoid adding more imports. I think it's a valid idea regardless. We could have a non-rich version that gets defaulted to if rich is not present, and that would allow us to not have to mess with the logging handler. What do you think?

Signed-off-by: lrcouto <laurarccouto@gmail.com>
@lrcouto lrcouto marked this pull request as ready for review May 10, 2024 05:00
@astrojuanlu
Copy link
Member

Ugh, thanks for the investigation @lrcouto. We'll give this some thought.

pyproject.toml Outdated
@@ -16,7 +16,7 @@ dependencies = [
"build>=0.7.0",
"cachetools>=4.1",
"click>=4.0",
"cookiecutter>=2.1.1,<3.0",
"cookiecutter>=2.1.1,<2.3", # 2.3 onwards depends on rich, which we want to avoid
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't pin cookiecutter so strictly, but instead clarify in docs that if users don't want to use rich they might have to downgrade their cookiecutter version.

@astrojuanlu
Copy link
Member

In line with #2928 (comment), maybe let's rename the PR to make it clear that we're not marking rich as an optional dependency yet (and therefore this doesn't yet fully address #2928)

@lrcouto lrcouto changed the title Make Rich an optional dependency Make it possible to use Kedro after uninstalling Rich. May 14, 2024
Signed-off-by: lrcouto <laurarccouto@gmail.com>
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, but nothing blocking.

Great work! 👍

@@ -241,3 +241,21 @@ Customise the configuration loader arguments in `settings.py` as follows if your
```python
CONFIG_LOADER_ARGS = {"default_run_env": "base"}
```

### How to use Kedro without the rich library
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section will fit better in the logging docs: https://docs.kedro.org/en/stable/logging/index.html

Copy link
Contributor Author

@lrcouto lrcouto May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about putting it in the logging page, as it already mentions rich a lot, but since it affects features other than logging I thought it'd fit better as more general configuration information. What do you think about adding it to both pages?

@@ -39,6 +44,8 @@

FunctionParameters = MappingProxyType

RICH = True if importlib.util.find_spec("rich") is not None else False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe name this RICH_INSTALLED?

RELEASE.md Outdated Show resolved Hide resolved
lrcouto and others added 6 commits May 15, 2024 15:08
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>
…o-org/kedro into remove-rich-as-mandatory-dependence
Signed-off-by: lrcouto <laurarccouto@gmail.com>
@lrcouto lrcouto linked an issue May 20, 2024 that may be closed by this pull request
4 tasks
Copy link
Contributor

@SajidAlamQB SajidAlamQB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @lrcouto for taking this on, tested this manually and this looks good to me. 🌟

@lrcouto lrcouto enabled auto-merge (squash) May 21, 2024 16:05
@lrcouto lrcouto merged commit a53500a into main May 21, 2024
41 checks passed
@lrcouto lrcouto deleted the remove-rich-as-mandatory-dependence branch May 21, 2024 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make rich optional / not a core dependency of kedro
6 participants