Support a hierarchy in metadata attributes

[tuner]

Problem

How to handle a large number of attributes?

Metadata in the app is based on a csv/tsv/json file. The data types, scales, and domains are automatically inferred from the data. These can be further customized in the spec:

        "attributes": {
          "publicationCode": {
            "type": "nominal",
            "visible": false
          },
          "sampleTime": {
            "type": "ordinal",
            "scale": {
              "domain": ["primary", "interval", "relapse"],
              "range": ["#facf5a", "#f95959", "#455d7a"]
            }
          },
          ...

However, when the number of attributes grows large, certain problems arise:

1. The view-visibility menu becomes super long
2. Finding a particular attribute from the list is difficult
3. Customizing each attribute separately in the spec is burdensome
4. Toggling attributes' visibility is slow (plenty of clicks)

Often, if the number of attributes is large, they have some inherent structure – they could be split into different categories.

Solution

1. Allow the attributes to have an arbitrary number of group levels.
2. In the spec, one could specify a separator character, e.g., ., _, or /, that separates the groups
3. In the spec, attributes could be configured using the full attribute name or any prefix that determines a group at any level.

The metadata.tsv file might have the following columns:

1. sample
2. clinical.PFI
3. clinical.OS
4. signature.HRD
5. signature.APOBEC

The above attributes would comprise two groups: clinical and signature.

The attributes and groups could be configured as follows:

     "samples": {
        "data": { "url": "data/samples.tsv" },
        "attributeGroupSeparator": ".",
        "attributes": {
          "clinical.PFI": {
            "type": "quantitative",
            "scale": {
              "type": "threshold",
              "domain": [90, 182, 365, 740],
              "scheme": "yellowgreenblue"
            }
          },
          "clinical.OS": {
            "type": "quantitative",
            "scale": {
              "type": "threshold",
              "domain": [365, 730, 1095, 1460, 1825],
              "scheme": "lighttealblue"
            }
          },
         "signature": {
            "type": "quantitative",
            "scale": { "scheme": "yelloworangered" }
          },
          ...

The attributeGroupSeparator specifies which character separates the attribute names into "paths". In the above example, signature attribute targets all attributes that belong to the signature group: HRD and APOBEC. The configuration should be inheritable, i.e., one could provide defaults for a group and adjust individual attributes if necessary.

The grouping should form a view hierachy, which would be shown in the view visibility menu. The groups should be container views (hconcat), end their visibility should be toggleable.

The above example would be shown in the menu as follows:

• Sample name
• clinical
  • PFI
  • OS
• signature
  • HRD
  • APOBEC

Configuring groups

Groups are automatically extracted from the column names if the separator character has been specified. But what if they need some configuration – what if some of them should be hidden by default? Two options:

Separate config for attributes and groups

    "samples": {
       "data": { "url": "data/samples.tsv" },
       "attributeGroupSeparator": ".",
       "attributes": {
         "clinical.PFI": { ... },
         "signature": { ... },
       ",
       "attributeGroups": {
         "signature": {
           "visible": false,
           "title": "Mutational signatures"
         }
       }

Pros: it would be clear (based on the JSON schema) which properties can be used where. Cons: stuff has to be specified at multiple places.

Attributes and groups in the same object

    "samples": {
       "data": { "url": "data/samples.tsv" },
       "attributeGroupSeparator": ".",
       "attributes": {
         "clinical.PFI": { ... },
         "signature": {
           "visible": false,
           "title": "Mutational signatures"
         },
       }

Pros: more succinct. Cons: Not clear if, for example, the visible property should be applied to the group or to all attributes within the group, i.e., is the group invisible or are all the attributes separately invisible.