Have SirCAL use the "Active" flag from the API metadata #1961
Labels
data quality
Missing data, weird data, broken data
devops
Engineering
Used to filter issues when synching with Asana
future-solution
Solutions to problems we don't have yet but still dread
good first issue
Sir Complains-a-Lot lets us know about potential data pipeline problems by noticing when our indicator signals havent been properly updated for a while. Sometimes a signal may be discontinued for any number of reasons, and it will stop getting updated with new data. To prevent SirCAL from alerting on such signals, it consults a list of "retired" signals. Unfortunately, maintaining this list requires enumerating them all by hand (such as here and here) and the list will only grow larger over time.
Instead of relying [solely?] on the "retired" signal list, SirCAL can make use of the
Active
status in the metadata available through the Delphi Epidata API to determine if a signal is worth alerting on -- if a signal is not marked as being "Active
", it is no longer expected to receive updates and it can thus be ignored.A caveat to this is that SirCAL currently uses the
covidcast
client package to access metadata. That client uses the/covidcast_meta
endpoint, but the "Active
" status is only available from an alternate endpoint,/covidcast/meta
.The text was updated successfully, but these errors were encountered: