Issue with v0.1.11

[viniciuschiele]

I've upgraded greenmask from v0.1.10 to v0.1.11 and I started getting this error:

2024/04/29 14:52:35 {"level":"fatal","error":"yaml: unmarshal errors:\n  line 12: cannot unmarshal !!str `my_schema` into []string","time":"2024-04-29T14:52:35Z","message":"error parsing transformer parameters"}

Sample of my config.yaml

common:
  tmp_dir: /tmp
  pg_bin_path: /usr/lib/postgresql/15/bin

storage:
  directory:
    path: /tmp

dump:
  pg_dump_options:
    dbname: my_database
    schema: my_schema
    jobs: 10

  transformation:
    - schema: my_schema
      name: users
      transformers:
        - name: Hash
          params:
            column: first_name

restore:
  pg_restore_options:
    dbname: my_database
    no-owner: true
    no-privileges: true
    jobs: 10
  scripts:
    pre-data:
      - name: Drop objects
        when: before
        query_file: ./drop_objects.sql

greenmask --config ./config.yaml dump

Has the configuration file format changed?

[wwoytenko]

No, but the last changes affected the logic of transformer parameters parsing. I will check and get back to you

[wwoytenko]

I've just checked. This problem was caused by fixes in #76 . I implemented a workaround there because mapstructure and viper libraries force parsing the map key with lowercase. To overcome this problem we started to use two parsers - viper for the explicit parameters and built-in go yaml/json for transformer params. With the case you mentioned, it appears that if there is a parameter that parses as a string into a slice of string it causes the error. So I will refactor the workaround I've made to avoid this problem.

As for now, I suggest downgrading to v0.1.10 if there aren't important changes for you. Or edit your config as below

dump:
  pg_dump_options:
    dbname: my_database
    schema: 
      - my_schema
    jobs: 10

[wwoytenko]

I've just refactored the workaround that was published in v0.1.11 #86. I will publish it in the next release

[viniciuschiele]

I tried using:

dump:
  pg_dump_options:
    dbname: my_database
    schema: 
      - my_schema
    jobs: 10

I got another error that may or not be related

2024/04/29 15:00:23 2024-04-29T15:00:23Z INF restoring dump dumpId=1714402820003
2024/04/29 15:00:23 2024-04-29T15:00:23Z INF executing script script="Drop objects" section=pre-data when=before
2024/04/29 15:00:23 2024-04-29T15:00:23Z FTL fatal error="pre-data stage restoration error: cannot aply script \"Drop objects\" before pre-data section: nothing to execute"
Error: /29 15:00:57 [ERROR] FAILED. container: application, exit code: 1

[wwoytenko]

I will check it

[wwoytenko]

I've just tested it. It is a related issue. #86 must fix it

[wwoytenko]

I've just published the v0.1.12 release with proper fixes. I hope this resolves all issues.

Let me know if that is okay or not

[viniciuschiele]

It did work, thank you!

[wwoytenko]

Thank you for reporting