Failed to render 'TemplateTransformerTemplate' when using store_write

[winstonmyers]

I have had issues with the store_write function. It seems I am unable to write to the store with the following yaml config file.

Using pg_datanymizer 0.2.0

schema: <SCHEMA_NAME>
tables:
  - name: entity_master
    rules:
      entity_master_id:
        template:
          format: "{{ _0 }}"
      federal_ident:
        template:
          format: "{{ _1 }}{{ store_write(key='entity_master.federal_ident.' ~ prev.entity_master_id, value=_1) }}"
          rules:
            - random_num:
                min: 100000000
                max: 999999999
                uniq: true
    rule_order:
      - federal_ident
      - <ALL_OTHER_COLUMNS>
table_order:
  - "entity_master"
  - <ALL_OTHER_TABLES>

I consistently get the following output:

[ec2-user@<HOST_NAME>]$RUST_BACKTRACE=full pg_datanymizer -f  './anon_dump.sql' -c "./anon_config.yaml" "${DB_NAME}" -h "${DB_HOST}" -U "${DB_USER}" -W "${DB_PW}"
Prepare data scheme...
Fetch tables metadata...
[1 / 187] Prepare to dump table: <SCHEMA_NAME>.<TABLE_NOT_ENTITY_MASTER>
[Dumping: <SCHEMA_NAME>.<TABLE_NOT_ENTITY_MASTER>] [|##################################################|] 1
24 of 124 rows [100%] (0s)
[Dumping: <SCHEMA_NAME>.<TABLE_NOT_ENTITY_MASTER>] Finished in 0 seconds
[2 / 187] Prepare to dump table: <SCHEMA_NAME>.entity_master
Error: Failed transform Failed to render 'TemplateTransformerTemplate'

When I remove {{ store_write(key='entity_master.federal_ident.' ~ prev.entity_master_id, value=_1) }} it works fine, but I am unable to run a successful dump using store_write as well as store_force_write. Any idea what might be causing the issue?

[evgeniy-r]

Hello! I will take a look.

[evgeniy-r]

It seems it is working for current version (0.5.1).
Could you please test this with the current version?

[krysopath]

@evgeniy-r , using latest and v0.5.1: I can reproduce the bug, given this Config

---
tables:
  - name: authentication_request
    query:
      dump_condition: "requested_at >= '2022-06-01'::date"
      limit: 100000
    rules:
      login_session_id:
        template:
          format: "{{ _1 }}{{ store_write(key='session.' ~ prev.login_session_id, value=_1) }}"
          rules:
            - hex_token: {len: 128}

  - name: authentication_session
    query:
      dump_condition: "authenticated_at >= '2022-06-01'::date"
      limit: 100000
    rules:
      id:
        template:
          format: "{{ store_read(key='session.' ~ prev.id) }}"

      subject:
        hex_token: {len: 128}

I reach this output:

Prepare data scheme...
Fetch tables metadata...
[1 / 18] ...
[16 / 18] Prepare to dump table: public.authentication_request
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0/100000
Error: Failed transform Failed to render 'TemplateTransformerTemplate'

Since the session ids are uuid4, I tried with latest master branch first, then attempted v0.5.1 (that did not have uuid transformer) but no change.

[evgeniy-r]

Hello! Thank you for reproducing the bug, I will check this tomorrow.

[evgeniy-r]

@krysopath, run your dump with this branch: #183, please.

[krysopath]

@evgeniy-r on this branch I can use store_write and store_read successfully <3

[evgeniy-r]

It's very interesting, because I just add more verbose message for this case...
Could you give me the DB scheme? Or maybe just authentication_request and authentication_session table schemes?