Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3.0b0: variables passed to @deploy(data_defaults) not accessible via host.data in jinja2 templates #1068

Open
cmenke opened this issue Feb 14, 2024 · 0 comments
Assignees
Labels
Bug Label for all kind of bugs.
Milestone

Comments

@cmenke
Copy link

cmenke commented Feb 14, 2024

Describe the bug

Version: v3.0b0

When rendering jinja templates with the files.template()-operation, variables passed to @deploy(data_defaults) are not accessible via host.data.

This appears to be a regression from v2.9.

To Reproduce

Here's a simplified example:

# modules/test.py
from io import StringIO
from pyinfra.api import deploy
from pyinfra.operations import files

# foo is NOT defined in host/group data
DEFAULTS = {"foo": "bar"}

@deploy("Test", data_defaults=DEFAULTS)
def test_template():
    files.template(
        src=StringIO("foo={{ host.data.foo }}"),
        dest="/tmp/test"
    )
# task.py
import modules.test

modules.test.test_template()

Expected behavior

Running pyinfra inventory.py task.py works in v2.9 and writes "foo=bar" to /tmp/test.
However in v3.0b0, an error is thrown:

jinja2.exceptions.UndefinedError: 'pyinfra.api.host.HostData object' has no attribute 'foo'

The full output can be found below.

Meta

pyinfra inventory.py task.py
--> Loading config...
--> Loading inventory...
    inventory.py is neither a valid Python import or hostname.

--> Connecting to hosts...
    [192.168.6.11] Connected

--> Preparing operations...
--> Preparing Operations...
    Loading: task.py
    [192.168.6.11] Ready: task.py

--> Detected changes:
    Operation                                                                                             Hosts                  
    Test | Files/Template (src=StringIO(hash=dbe452ba519ba243e393e2a5e45eb31fedce21df), dest=/tmp/test)   1/1 (192.168.6.11)   

    Detected changes displayed above, skip this step with -y
                             
--> Beginning operation run...
--> Starting operation: Test | Files/Template (src=StringIO(hash=dbe452ba519ba243e393e2a5e45eb31fedce21df), dest=/tmp/test)
Traceback (most recent call last):
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 991, in template
    output = get_template(src).render(data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'pyinfra.api.host.HostData object' has no attribute 'foo'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operations.py", line 185, in _run_host_op_with_context
    return run_host_op(state, host, op_hash)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operations.py", line 53, in run_host_op
    return _run_host_op(state, host, op_hash)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operations.py", line 82, in _run_host_op
    for command in op_data.command_generator():
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/api/operation.py", line 238, in command_generator
    for command in func(*args, **kwargs):
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 1002, in template
    with open(src, "r") as f:
         ^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not StringIO
2024-02-14T11:06:16Z <Greenlet at 0x10c475800: _run_host_op_with_context(<pyinfra.api.state.State object at 0x10c2e77a0>, Host(192.168.6.11), 'ba63029b861efac1dc89396092183678302576e4')> failed with TypeError

--> Disconnecting from hosts...
--> An internal exception occurred:

  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 1002, in template
    with open(src, "r") as f:
    ^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/pyinfra/operations/files.py", line 991, in template
    output = get_template(src).render(data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/lib/python3.12/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'pyinfra.api.host.HostData object' has no attribute 'foo'

During handling of the above exception, another exception occurred:

TypeError: expected str, bytes or os.PathLike object, not StringIO

--> The full traceback has been written to pyinfra-debug.log
--> If this is unexpected please consider submitting a bug report on GitHub, for more information run `pyinfra --support`.
pyinfra --support
    If you are having issues with pyinfra or wish to make feature requests, please
    check out the GitHub issues at https://github.com/Fizzadar/pyinfra/issues .
    When adding an issue, be sure to include the following:

    System: Darwin
      Platform: macOS-14.2.1-x86_64-i386-64bit
      Release: 23.2.0
      Machine: x86_64
    pyinfra: v3.0b0
    Executable: /Users/cmenke/Library/Caches/pypoetry/virtualenvs/cm-infra-uVj_SpmL-py3.12/bin/pyinfra
    Python: 3.12.1 (CPython, Clang 15.0.0 (clang-1500.1.0.2.5))

Please let me know if you need any additional information or details.

@cmenke cmenke added the Bug Label for all kind of bugs. label Feb 14, 2024
@Fizzadar Fizzadar added this to the v3 milestone Mar 13, 2024
@Fizzadar Fizzadar self-assigned this Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Label for all kind of bugs.
Projects
None yet
Development

No branches or pull requests

2 participants