UVA adaptive round 18

[hokinus]

UVA adaptive round 18

[github-actions]

Run validation on files: 2024-04-28-UVA-adaptive.gz.parquet

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

❌ Error 202: At least 1 of the 'scenario_id' do(es) not correspond: 'A-2024-04-28, B-2024-04-28, C-2024-04-28, D-2024-04-28, E-2024-04-28, F-2024-04-28'. The scenarios ids for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. The scenarios ids for the target(s): inc hosp (sample) and for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. The scenarios ids for the target(s): inc death (sample) and for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. The scenarios ids for the target(s): inc hosp (quantile, mean) and for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. The scenarios ids for the target(s): inc death (quantile, mean) and for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

🟡 Warning 5043: All values associated with output type 'sample' should have a maximum of 1 decimal place

Target Columns:

🟡 Warning 606: The projection contains more projected week than expected. The additional weeks might not be included in the Ensembles and/or visualizations

Locations:

No errors or warnings found on Location

Sample:

❌ Error 904: Samples are expected in the submission for the target(s): inc death, inc hosp. please verify.

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

[github-actions]

Run validation on files: 2024-04-28-UVA-adaptive.gz.parquet

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

No errors or warnings found on scenario name and scenario id columns

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

🟡 Warning 606: The projection contains more projected week than expected. The additional weeks might not be included in the Ensembles and/or visualizations

Locations:

No errors or warnings found on Location

Sample:

No errors or warnings found on Sample

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

[LucieContamin]

Hi @hokinus ,

Thank you for the submission, I have some questions:

• It seems that the "inc death" and "inc hosp" don't have the same number of horizon projected. inc hosp is projected for 53 weeks and inc death for 55 weeks. 52 weeks is the expected horizon, would it be possible to verify it?
• Also, for the "sample" output type:
  • it seems that the column output_type_id is not set to NA (some rows have -1), would it be possible to verify that, please?
  • And for the sample ID information in the run grouping and stochastic run, if I understand correctly:
    • 100 trajectories was provided
    • And for the grouping information: there is no stochasticity, every models runs has a different run_grouping that can be group by age_group, location, horizon, scenario_id, target or by another level for example group by age group, horizon, scenario. Did I understand correctly?

Please let me know if any questions or issues
Best, Lucie

[github-actions]

Run validation on files: 2024-04-28-UVA-adaptive.gz.parquet
End of validation check: all the validation checks were successful

[hokinus]

Lucie,
I have updated the file, trimmed the projection horizon and fixed the output_type_id.
Different run_groupings correspond to single set of parameters of our model, but for each location the calibration step is done independently and the selection of trajectories for submission is done separately (so for example set of run_groupings for VA may be different than set for MD). The age groups are disaggregated from a single run and deaths are result of hospitalization postprocessing. So single run_grouping (as in independent model run) corresponds to location, horizon, scenario_id.

[LucieContamin]

Thanks for update and the additional information.

I am still a bit confused by the "grouping" (also called "pairing") information. When I look at the content of your file, I see different level of "grouping" for some trajectories which I am not sure I understand. It is expected to have the same level for all trajectories.
For example:

• the sample "4101" is "grouped" by scenario_id, target, age_group and horizon as it contains all the value for all the age groups, scenarios, both targets, all the horizons and one specific location (42).
• the sample "5013" is "grouped" by scenario_id, target, age_group, horizon and a sub-group of location, as it contains all the value for all the age groups, scenarios, both targets, all the horizons and multiple locations (01, 05, etc.)

Also, the minimal "grouping" level accepted in this round is by horizon and age group, which seem to match your submission file but not your additional information.

If I understand correctly, a single run_grouping (independent model run) generates data for all age group, both targets, all horizon and all scenario? In this case, the grouping information should be scenario_id, target, age_group and horizon and each location should be with a different sample ID (like sample 4101).
I apologize if I misunderstood something, but does my question makes sense?

[hokinus]

I might got confused by grouping by/pairing usage here, but your understanding is correct.
The trajectories with the same run_grouping are using the same parameter set. The selection of the trajectories is done independently for each location and I do not enforce the selection of the trajectories with the same parameter set across locations, that is why some of the run_grouping ids are shared between some of the locations and some are not.
If that is a problem, then I can:

1. reassign run_groupings so they are unique between locations
2. or enforce selection of the trajectories to be the same across locations, so then trajectories will be "paired" across locations as well, that may slightly reduce the spread of the trajectories selected for single locations though

Let me know what is preferred.

[hokinus]

Updated to have unique run_groupings per location.

[github-actions]

Run validation on files: 2024-04-28-UVA-adaptive.gz.parquet

Columns:

No errors or warnings found on the column names and numbers

Scenarios:

❌ Error 202: At least 1 of the 'scenario_id' do(es) not correspond: 'A-2024-04-28, B-2024-04-28, C-2024-04-28, D-2024-04-28, E-2024-04-28, F-2024-04-28'. The scenarios ids for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. The scenarios ids for the target(s): inc hosp (quantile, mean) and for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.
❌ Error 204: At least 1 of the required 'scenario_id' is missing: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. The scenarios ids for the target(s): inc death (quantile, mean) and for this round are: 'A-2024-03-01, B-2024-03-01, C-2024-03-01, D-2024-03-01, E-2024-03-01, F-2024-03-01'. Please verify.

Origin Date Column:

No errors or warnings found on the column 'origin_date'

Value and Type Columns:

No errors or warnings found on Value and Type columns

Target Columns:

No errors or warnings found in target and associated columns

Locations:

No errors or warnings found on Location

Sample:

No errors or warnings found on Sample

Quantiles:

No errors or warnings found on quantiles values and format

Age Group:

No errors or warnings found on Age_group

[github-actions]

Run validation on files: 2024-04-28-UVA-adaptive.gz.parquet
End of validation check: all the validation checks were successful

[LucieContamin]

Hi @hokinus ,

Thank you for the update and the additional information, it seems to be all good now!

Best, Lucie