You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our current code was written back when case reporting in many places was done daily, and was relatively reliable. This is not true anymore, and as a result our code is not as robust as it should be.
Looking for the most extreme symptom of this, I instrumented update_prevalence.py to see how many people are affected by (presumably false) reports of zero cases in at least one of two weeks we look at. Here are the new log messages when this was run on 2022-12-08:
WARNING: 271,562,497 people affected by No cases noted for a week - State level
WARNING: 909,848,811 people affected by No cases noted for a week - Country level
WARNING: 48,101,095 people affected by No cases noted for a week - County level
I looked at Nigeria, the largest country affected based on the logs:
No cases noted for a week - Country level (206,139,587 people): No cases reported in at least one week in Nigeria for period
Indeed, we report ~0 microCOVIDs for this scenario as of the latest data - what update_prevalence.py calls "effective_date" - 2022-12-07:
I looked at the JHU-aggregated data we are pulling over the last couple of months using the alasql CLI:
alasql "SELECT Confirmed FROM csv('https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/10-09-2022.csv') where Country_Region = 'Nigeria'"
Pulling that over multiple days showed these transition points in the reporting numbers:
2022-10-08: 265816 (e - 60)
2022-10-15: 265937 (e - 53) - 7 days later - 121 new cases/week
2022-10-22: 266043 (e - 46) - 7 days later - 106 new cases/week
2022-10-29: 266138 (e - 39) - 7 days later - 95 new cases/week
2022-11-05: 266192 (e - 32) - 7 days later - 54 new cases/week
2022-11-19: 266283 (e - 18) - 14 days later - 45.5 new cases/week
2022-12-07: 266283 (e = effective_date)
I dropped the repeating entries above - the numbers between the items below stay identical until the next item.
This causes the false zeros:
Place#cases_last_week grabs e-7 to e and calculates 0 cases that week.
Place#cases_week_before grabs e-14 to e-7 and calculates zero cases the week before too.
Per microCOVID's basic method of person risk we create a now-cast based on the change between the weekly numbers, so presumably a false zero on either value is going to cause trouble.
The text was updated successfully, but these errors were encountered:
Our current code was written back when case reporting in many places was done daily, and was relatively reliable. This is not true anymore, and as a result our code is not as robust as it should be.
Looking for the most extreme symptom of this, I instrumented
update_prevalence.py
to see how many people are affected by (presumably false) reports of zero cases in at least one of two weeks we look at. Here are the new log messages when this was run on 2022-12-08:I looked at Nigeria, the largest country affected based on the logs:
Indeed, we report ~0 microCOVIDs for this scenario as of the latest data - what update_prevalence.py calls "effective_date" - 2022-12-07:
I looked at the JHU-aggregated data we are pulling over the last couple of months using the alasql CLI:
alasql "SELECT Confirmed FROM csv('https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/10-09-2022.csv') where Country_Region = 'Nigeria'"
Pulling that over multiple days showed these transition points in the reporting numbers:
I dropped the repeating entries above - the numbers between the items below stay identical until the next item.
This causes the false zeros:
Place#cases_last_week
grabs e-7 to e and calculates 0 cases that week.Place#cases_week_before
grabs e-14 to e-7 and calculates zero cases the week before too.Per microCOVID's basic method of person risk we create a now-cast based on the change between the weekly numbers, so presumably a false zero on either value is going to cause trouble.
The text was updated successfully, but these errors were encountered: