Skip to content

Analysis of RT-PCR sensitivity by day since exposure or symptom onset

Notifications You must be signed in to change notification settings

HopkinsIDD/covidRTPCR

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DOI

This code complements the submission “Variation in False Negative Rate of RT-PCR Based SARS-CoV-2 Tests by Time Since Exposure

Methods

Zhao et al. (2020), Liu et al. (2020), Guo et al. (2020), Wölfel et al. (2020), Danis et al. (2020), Kujawski et al. (2020), Kim et al. (2020), and Young et al. (2020) each looked at the sensitivity of the RT-PCR by time since symptom onset.

The sensitivity rises to a peak 4 days after symptom onset then declines for the next couple of weeks.

If we know the risk of an individual, we can find the negative predictive value – the probability that someone who tests negative is actually negative. From Bi et al., we know that about 15% (77/517) household contacts later tested positive for covid-19.

We use logistic regression for the sensitivity of the RT-PCR with a cubic polynomial for the log of time since exposure and use that, along with the probability of infection given exposure, to estimate the negative predictive value of the RT-PCR. We use estimates of the incubation period from Lauer, Grantz, et al. (2020). From this, we can find the probability of having a covid-19 infection despite being RT-PCR negative.

We use Stan for this analysis.

Results

day False negative rate (95% CI) False omission rate (95% CI) Rel. reduction in prob. pos. Abs. reduction in prob. pos.
1 100.0% (100.0-100.0%) 11.3% (9.0-13.8%) 0.0% (-0.0-0.0%) 0.0% (-0.0-0.0%)
2 100.0% (96.0-100.0%) 11.2% (8.9-13.8%) 0.0% (0.0-3.6%) 0.0% (0.0-0.4%)
3 97.7% (58.8-99.9%) 10.7% (6.7-13.5%) 2.0% (0.1-38.3%) 0.2% (0.0-4.2%)
4 71.0% (29.6-94.1%) 8.1% (3.5-11.8%) 26.5% (5.3-67.7%) 3.0% (0.6-7.8%)
5 38.7% (18.4-64.6%) 4.7% (2.2-7.9%) 58.5% (32.7-79.7%) 6.5% (3.5-9.6%)
6 24.8% (14.0-39.9%) 3.0% (1.7-5.1%) 72.9% (57.3-84.4%) 8.2% (6.0-10.5%)
7 20.1% (12.5-31.0%) 2.5% (1.5-4.0%) 77.9% (66.3-86.2%) 8.7% (6.7-11.0%)
8 19.1% (12.0-29.1%) 2.4% (1.4-3.8%) 78.9% (68.3-86.6%) 8.9% (6.9-11.2%)
9 20.0% (12.8-30.2%) 2.5% (1.5-4.0%) 78.0% (67.4-85.8%) 8.8% (6.8-11.0%)
10 22.1% (14.5-32.7%) 2.7% (1.7-4.3%) 75.8% (64.7-84.0%) 8.5% (6.6-10.7%)
11 25.0% (16.7-36.0%) 3.1% (1.9-4.7%) 72.7% (61.1-81.5%) 8.2% (6.3-10.3%)
12 28.6% (19.5-40.1%) 3.5% (2.3-5.2%) 69.0% (57.0-78.6%) 7.7% (5.9-9.9%)
13 32.5% (22.6-44.4%) 3.9% (2.6-5.8%) 64.8% (52.6-75.1%) 7.3% (5.5-9.3%)
14 36.8% (26.2-49.0%) 4.4% (3.0-6.4%) 60.4% (47.9-71.5%) 6.8% (5.1-8.8%)
15 41.2% (30.0-53.6%) 4.9% (3.4-7.0%) 55.8% (43.5-67.5%) 6.3% (4.6-8.2%)
16 45.5% (33.8-58.0%) 5.4% (3.8-7.6%) 51.5% (39.1-63.5%) 5.8% (4.1-7.7%)
17 49.6% (37.6-62.1%) 5.9% (4.2-8.1%) 47.4% (35.2-59.6%) 5.3% (3.7-7.2%)
18 53.5% (41.2-65.7%) 6.3% (4.6-8.6%) 43.5% (31.8-55.9%) 4.9% (3.4-6.7%)
19 57.0% (44.6-68.9%) 6.7% (4.9-9.0%) 40.1% (28.7-52.3%) 4.5% (3.1-6.2%)
20 60.2% (47.8-71.6%) 7.1% (5.2-9.4%) 37.0% (25.9-49.1%) 4.2% (2.8-5.8%)
21 63.0% (50.8-74.2%) 7.4% (5.5-9.7%) 34.3% (23.6-46.2%) 3.9% (2.6-5.5%)

With little data on RT-PCR to time prior to symptom onset, the estimates of sensitivity at or below day five are low with large credible intervals. Due to the decline in sensitivity over time, the RT-PCR test is best deployed about a week after exposure. A day or two after exposure (3 or 4 days prior to symptoms), the test may have no utility at all, and thus the probability of having been infected is would be the same with or without an RT-PCR. Seven to nine days after exposure (roughly 2 to 4 days after symptom onset), the negative predictive value is around 97.5%, meaning there is about a 2.5% chance of actually being covid-19 positive despite testing negative (given a 15% attack rate).

Sensitivity analyses

Specificity

What if the specificity of the test is less than 100%? To test this we fit the same model except with a specificity of 90% and compared the results to the original (the sensitivity remained the same).

Day False omission rate (90% spec.) False omission rate (100% spec.) Rel. increase in FOR Abs. increase in FOR
1 12.4% 11.3% 9.9% 1.1%
2 12.4% 11.2% 10.0% 1.1%
3 11.8% 10.7% 9.6% 1.0%
4 8.8% 8.1% 7.6% 0.6%
5 5.0% 4.7% 7.2% 0.3%
6 3.3% 3.0% 7.5% 0.2%
7 2.7% 2.5% 7.5% 0.2%
8 2.6% 2.4% 8.1% 0.2%
9 2.7% 2.5% 8.8% 0.2%
10 3.0% 2.7% 8.8% 0.2%
11 3.3% 3.1% 9.1% 0.3%
12 3.8% 3.5% 9.1% 0.3%
13 4.3% 3.9% 9.3% 0.4%
14 4.9% 4.4% 9.5% 0.4%
15 5.4% 4.9% 9.4% 0.5%
16 5.9% 5.4% 9.5% 0.5%
17 6.5% 5.9% 9.5% 0.6%
18 6.9% 6.3% 9.5% 0.6%
19 7.4% 6.7% 9.6% 0.6%
20 7.7% 7.1% 9.7% 0.7%
21 8.1% 7.4% 9.5% 0.7%

The shape of the curve for 90% specificity is similar to that of 100%, though slightly elevated. The best time to test is still 2-4 days post-symptom onset.

Different pre-test probabilities

We tested pre-test probabilities of half, twice, and four times that of Bi et al. (2020).

day attack rate FOR FOR LB FOR UB
1 half 5.6 4.1 7.4
2 half 5.6 4.0 7.4
3 half 5.3 3.1 7.2
4 half 3.9 1.6 6.1
5 half 2.2 1.0 4.0
6 half 1.4 0.7 2.5
7 half 1.1 0.6 2.0
8 half 1.1 0.6 1.9
9 half 1.2 0.7 1.9
1 2x 22.5 19.4 25.8
2 2x 22.4 19.2 25.7
3 2x 21.6 14.3 25.2
4 2x 16.8 7.8 22.6
5 2x 9.9 4.9 16.1
6 2x 6.6 3.8 10.6
7 2x 5.4 3.3 8.4
8 2x 5.2 3.2 7.9
9 2x 5.4 3.4 8.1
1 4x 44.9 41.3 48.6
2 4x 44.8 40.8 48.6
3 4x 43.7 31.0 47.9
4 4x 36.0 18.6 44.5
5 4x 23.5 12.6 34.9
6 4x 16.6 9.9 24.4
7 4x 13.9 8.8 20.0
8 4x 13.3 8.6 18.7
9 4x 13.9 9.1 19.2

Different incubation period lengths

We originally assumed a 5-day incubation period, what if that was 3 or 7 days instead?

day inc period FOR FOR LB FOR UB
1 3d 11.3 8.9 13.7
2 3d 10.0 4.9 12.9
3 3d 5.3 2.4 8.9
4 3d 2.9 1.6 5.0
5 3d 2.3 1.4 3.8
6 3d 2.2 1.3 3.6
7 3d 2.4 1.5 3.8
8 3d 2.7 1.7 4.1
9 3d 3.1 2.0 4.6
10 3d 3.5 2.3 5.2
11 3d 4.0 2.7 5.7
12 3d 4.5 3.1 6.3
13 3d 5.0 3.5 6.9
1 7d 11.3 9.1 13.8
2 7d 11.3 9.1 13.8
3 7d 11.3 9.1 13.8
4 7d 11.1 8.2 13.6
5 7d 9.8 5.1 12.8
6 7d 6.8 3.1 10.7
7 7d 4.3 2.2 7.3
8 7d 3.1 1.7 5.0
9 7d 2.6 1.5 4.1
10 7d 2.4 1.5 3.8
11 7d 2.5 1.5 3.9
12 7d 2.8 1.7 4.1
13 7d 3.1 1.9 4.6

Changing the length of the incubation period changes the progression of the false omission rate (post-test probability given test negative). Since the sensitivity is calibrated with respect to the time of symptom onset, an earlier onset time leads to a quicker drop in false omission rate and a later onset time leads to a slower drop.

Shift ambiguous days one day earlier

The timing of the days since symptom onset are ambiguous in Guo et al. and Kim et al., where day 1 may mean one day since symptom onset or the first day of symptoms.

Shifting the timing of ambiguous days one day earlier improves the median sensitivity of the early tests, such that the best days for testing would shift from 2-4 days post-symptom onset to 1-3 days post-symptom onset. The log-likelihoods for each model are nearly identical with the shifted days being minutely better. The overlap in credible intervals indicates that the estimates are roughly equivalent.

Inconclusive tests as negatives or positives

Kujawski et al. and Danis et al. reported 21 inconclusive tests combined in their data. In our main analysis, we omit these swabs, but they could count as negative or positive tests instead.

The results are barely changed by including inconclusive tests as negatives or positives instead of being omitted.

Leave one study out

To see whether any study has undue influence over our results, we ran a leave-one-study out analysis to see if the results changed.

Incubation period as a random variable

## Running /Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB foo.c
## clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Rcpp/include/"  -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/"  -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/unsupported"  -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/BH/include" -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/src/"  -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/"  -I"/Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include" -DEIGEN_NO_DEBUG  -D_REENTRANT  -DBOOST_DISABLE_ASSERTS -DBOOST_PENDING_INTEGER_LOG2_HPP -include stan/math/prim/mat/fun/Eigen.hpp   -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include  -fPIC  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c foo.c -o foo.o
## In file included from <built-in>:1:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Dense:1:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Core:88:
## /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/src/Core/util/Macros.h:613:1: error: unknown type name 'namespace'
## namespace Eigen {
## ^
## /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/src/Core/util/Macros.h:613:16: error: expected ';' after top level declarator
## namespace Eigen {
##                ^
##                ;
## In file included from <built-in>:1:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
## In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Dense:1:
## /Library/Frameworks/R.framework/Versions/3.6/Resources/library/RcppEigen/include/Eigen/Core:96:10: fatal error: 'complex' file not found
## #include <complex>
##          ^~~~~~~~~
## 3 errors generated.
## make: *** [foo.o] Error 1

days_since_exposure fnr_med fnr_lb fnr_ub for_med for_lb for_ub rr_med rr_lb rr_ub abs_med abs_lb abs_ub
1 100.0 100.0 100.0 11.3 8.9 13.9 0.0 0.0 0.0 0.0 0.0 0.0
2 100.0 86.8 100.0 11.2 8.4 13.9 0.0 0.0 11.7 0.0 0.0 1.4
3 99.1 44.8 100.0 10.9 5.1 13.6 0.8 0.0 52.4 0.1 0.0 5.8
4 80.2 24.8 99.0 9.0 3.0 12.6 17.9 0.9 72.8 2.0 0.1 8.4
5 45.7 17.5 83.4 5.5 2.1 10.1 51.2 14.9 80.8 5.7 1.7 9.7
6 27.9 14.0 52.3 3.4 1.7 6.5 69.6 44.7 84.5 7.8 4.8 10.4
7 21.3 12.3 35.1 2.6 1.4 4.5 76.6 62.2 86.4 8.6 6.4 10.9
8 19.3 11.8 29.9 2.4 1.4 3.9 78.7 67.7 87.0 8.9 6.8 11.1
9 19.5 12.3 29.7 2.4 1.4 3.8 78.5 67.8 86.4 8.8 6.8 11.0
10 21.0 13.3 31.6 2.6 1.5 4.1 76.9 65.8 85.2 8.6 6.7 10.8
11 23.5 15.1 34.9 2.9 1.8 4.5 74.3 62.4 83.3 8.3 6.4 10.5
12 26.8 17.5 38.7 3.3 2.1 5.0 70.9 58.5 80.7 8.0 6.0 10.1
13 30.4 20.4 42.9 3.7 2.4 5.5 67.0 54.1 77.6 7.5 5.6 9.7
14 34.5 23.7 47.4 4.2 2.7 6.1 62.7 49.6 74.1 7.1 5.2 9.2
15 38.7 27.3 51.9 4.7 3.1 6.7 58.3 45.1 70.3 6.6 4.7 8.7
16 43.1 31.0 56.2 5.2 3.5 7.2 53.9 40.8 66.3 6.1 4.3 8.1
17 47.4 34.9 60.5 5.7 3.9 7.8 49.6 36.8 62.4 5.6 3.9 7.6
18 51.4 38.7 64.2 6.1 4.3 8.3 45.6 33.2 58.4 5.1 3.5 7.1
19 55.1 42.3 67.5 6.5 4.7 8.7 42.0 30.0 54.7 4.7 3.2 6.6
20 58.4 45.6 70.4 6.9 5.0 9.1 38.7 27.2 51.3 4.3 2.9 6.1
21 61.4 48.9 72.9 7.2 5.3 9.5 35.8 24.8 48.0 4.0 2.6 5.7

When the incubation period is treated as a random variable instead of a fixed value, the confidence intervals for the false negative rate are wider, especially in the first week of exposure.