-
Notifications
You must be signed in to change notification settings - Fork 4
/
background.Rmd
31 lines (22 loc) · 5.33 KB
/
background.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
---
title: "Background"
output:
html_document:
toc: false
toc_float: false
---
The European Covid-19 Forecast Hub is run in collaboration between the [Epiforecasts team](http://epiforecasts.io/), which is part of the [Centre for Mathematical Modelling of Infectious Diseases](http://cmmid.github.io/topics/covid19) at the [London School of Hygiene & Tropical Medicine](http://www.lshtm.ac.uk), and the [European Centre for Disease Control and Prevention (ECDC)](http://ecdc.europa.eu).
The Hub collates and combines short-term forecasts of Covid-19 across Europe (EU and EFTA countries and the UK) generated by different independent modelling teams using a wide range of approaches. The underlying approach was pioneered by the [Reich lab](https://reichlab.io/) and follows similar projects in the [USA](https://covid19forecasthub.org/) and in [Germany and Poland](https://kitmetricslab.github.io/forecasthub/forecast). Other related projects are the [US scenario hub](https://covid19scenariomodelinghub.org/) and the [German hospitalization nowcast hub](https://covid19nowcasthub.de/).
For more information on technical details and contributors, visit the [project pages on github](https://github.com/epiforecasts/covid19-forecast-hub-europe). Teams from anywhere in the world are invited to submit forecasts once a week for one or more of the countries. Take a look at the [submission instructions](https://github.com/epiforecasts/covid19-forecast-hub-europe/wiki) and [get in touch](contact.html) with any questions. Please note that the participating teams do not represent ECDC or the London School of Hygiene & Tropcial Medicine.
# Aims
The main aim of the European Covid-19 Forecast Hub is to provide decision makers and the general public with reliable information about the near-term future trajectory of the pandemic. This is achieved by collating forecasts from different models into an ensemble, an approach that has in the past proven to provide consistently better performance than any individual modelling approach.
Secondary aims are to gain insight into the predictive performance of different modelling approaches, to assess the quality of forecasts with respect to different measures of disease severity (e.g., cases or deaths), and to maintain a community of infectious disease modellers underpinned by an open-science ethos.
# Limitations
The European Covid-19 Forecast Hub aims at providing forecasts in real time and evaluating systematically how reliable these forecasts are. Please keep the following in mind when interpreting forecasts:
- Real-time forecasts of epidemics, and particularly of Covid-19, have proven challenging in the past. Case forecasts in particular can be difficult in the face of changing policies, population behaviour and testing practices. Deaths are a more lagged measure and thus can be easier to predict. Forecasts should always be interpreted in conjunction with the associated prediction intervals. While these may sometimes seem wide, they contain detailed information on how likely different possible future trajectories are considered to be.
- Experience from previous projects has shown that following changes in non-pharmaceutical interventions or the emergence of new variants of SARS-CoV-2, many models do not adapt immediately to the changed circumstances. This also affects the ensemble, which may then extrapolate current trends too far into the future. More generally, predicting changes in trends of case numbers has proven very challenging.
- Reported data and short-term trends can be subject to uncertainty in particular around holiday periods, when testing behaviour can be very difficult to predict across countries. Models may not reflect how this affects observed data and could misinterpret a transient aberration in the data as a genuine trend.
- Model outputs are collected up to four weeks into the future and generally decrease in accuracy over this time period. Confirmed cases up to 1 to 2 weeks ahead and deaths up to 3 weeks ahead are assumed to be largely unaffected by future changes in control measures. Forecasts beyond these time horizons are subject to increased uncertainty as future interventions, behaviour changes or other unforeseeable factors may affect outcomes.
- In the period until November 2021 our [evaluation results](reports.html) show that the ensemble forecast intervals for deaths contained the observed value roughly as often as intended even four weeks ahead. For cases, coverage levels of the ensemble intervals were close to the intended values at the one week horizon, but considerably decreased from two weeks onwards. Especially for cases we therefore recommend to focus on forecasts at short time horizons, as displayed by default in the visualization.
- Individual models that constitute the ensemble are shown for transparency but should not be taken on their own without fully acknowledging the limitations and engaging with the listed authors. Information on authorship and the underlying methodology is shown when hovering the mouse over the model names in the legend.
We constantly monitor the performance of forecasts. In the [evaluation section](reports.html) we provide, among other measures, the empirical coverage proportions of forecast intervals for different targets and forecast horizons. This makes it possible to assess whether past prediction intervals of the ensemble have been reliable for a given target.