Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to configure logging in 2.8.11 using TOML. #2879

Closed
hirolau opened this issue Jan 7, 2020 · 4 comments · May be fixed by #3184
Closed

Unable to configure logging in 2.8.11 using TOML. #2879

hirolau opened this issue Jan 7, 2020 · 4 comments · May be fixed by #3184
Labels

Comments

@hirolau
Copy link
Contributor

hirolau commented Jan 7, 2020

In 2.8.11 the TOML parser freezes the config dict, this makes it no longer possible to load logging using that config.

Traceback (most recent call last):
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/logging/config.py", line 538, in configure
    formatters[name])
TypeError: 'FrozenOrderedDict' object does not support item assignment

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app-root/bin/luigid", line 11, in <module>
    load_entry_point('luigi==2.8.11', 'console_scripts', 'luigid')()
  File "/opt/app-root/lib/python3.6/site-packages/luigi/cmdline.py", line 31, in luigid
    DaemonLogging.setup(opts)
  File "/opt/app-root/lib/python3.6/site-packages/luigi/setup_logging.py", line 83, in setup
    configured = cls._section(opts)
  File "/opt/app-root/lib/python3.6/site-packages/luigi/setup_logging.py", line 48, in _section
    logging.config.dictConfig(logging_config)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/logging/config.py", line 795, in dictConfig
    dictConfigClass(config).configure()
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/logging/config.py", line 541, in configure
    'formatter %r: %s' % (name, e))
ValueError: Unable to configure formatter 'simple': 'FrozenOrderedDict' object does not support item assignment

I have not had time to look into possible solutions for now.

@hirolau
Copy link
Contributor Author

hirolau commented Jan 7, 2020

One way to fix this would be to not freeze the logging section in the parser:

In luigi/luigi/configuration/toml_parser.py instead of:

        # freeze dict params
        for section, content in self.data.items():
            for key, value in content.items():
                if isinstance(value, dict):
                    self.data[section][key] = recursively_freeze(value)

It would be possible to do:

        # freeze dict params
        for section, content in self.data.items():
            if key.lower() == 'logging':
                 continue
            for key, value in content.items():
                if isinstance(value, dict):
                    self.data[section][key] = recursively_freeze(value)

Another solution, that is probably better, is to let the config parser make a mutable copy. That would however require us to write a more complicated "recursively_unfreeze" function.

@stale
Copy link

stale bot commented May 6, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

@stale stale bot added the wontfix label May 6, 2020
@stale stale bot closed this as completed May 20, 2020
@DVlahovic
Copy link

DVlahovic commented Jul 31, 2020

Also experiencing this issue when attempting to configure logging via toml.

I tried editing my toml_parser.py as suggested by @hirolau and can confirm the solution works (I assume you meant section.lower())

# freeze dict params
        for section, content in self.data.items():
            if section.lower() == 'logging':
                 continue
            for key, value in content.items():
                if isinstance(value, dict):
                    self.data[section][key] = recursively_freeze(value)

@mzpqnxow
Copy link
Contributor

mzpqnxow commented Jul 11, 2022

I'm confused about why this is WONTFIX. I have a fork with the change (and one other change that was blocking toml config processing) that I will send, will they be accepted? I don't see a way to work around this in the toml file (maybe I'm missing something here?)

EDIT: Is the (non-code-change) "solution" to not include the logging section in the main TOML file? If so, the example toml configuration file in the repository should probably be updated- I had issues using that file verbatim

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants