Feature: conditional transform

[gregwebs]

A conditional transform states a SQL condition used to decide whether or not to transform a row. In datanymizer a where clause is given as a string. This API seems to work. Below groups is a table.

  - name: groups
    query:
      transform_condition: "id NOT IN (select group_id FROM employee_groups)"

Datanymizer implemented this by adding NOT to the given query. I fixed an issue that adding NOT also needs proper NULL-checking behavior: datanymizer/datanymizer@24e2521

[wwoytenko]

I wonder, does that condition apply to the whole record or a specific transformer?

So here is the definition of the transformation now:

    - schema: "humanresources"
      name: "employee"
      transformers:
        - name: "Hash"
          params:
            column: "jobtitle"
            function: "sha1"
            max_length: 10

In my point of view, there might be a condition for the whole record and for specific transformer.

The prototype might be:

    - schema: "humanresources"
      where: "{{ gt .GetRawColumnValue("id") 100 }}"
      name: "employee"
      transformers:
        - name: "Hash"
          where: '{{ not eq .GetRawColumnValue("jobtitle") "Developer" }}'
          params:
            column: "jobtitle"
            function: "sha1"
            max_length: 10

If we are talking about tuple filtering then it is connected with other features (database scale down). As for now, it is better to use the query property until the automatic scale-downing feature is released.

[gregwebs]

This applies to the whole record.

It is very useful to also selectively not anonymize certain column values. We have been able to achieve that with an if statement in template logic. However it would be useful to have a declarative way of doing it as you show.

[wwoytenko]

Just want to inform you that this feature is going to be published right after the next major v0.2.0 release (15 May). Probably v0.2.1 or v0.2.2 I will keep you posted.