How To Get COVID-19 Testing Right

Over the past two weeks, Gov. Andrew Cuomo completed COVID-19 antibody testing of 15,000 volunteers at grocery stores and community centers. Despite finding that 19.9 percent of those tested in New York City were positive, there is still so much about COVID-19 we do not know. We desperately need properly conducted antibody tests of the general population to not only manage healthcare resources but also plan how and when to safely reopen.

To gain the most reliable information, we should prioritize testing in the epicenter of America’s pandemic. Because of its presumably high infection rate, careful serum testing in New York City -- even on a relatively small number of people -- will provide crucial data for smart, credible, and evidence-based exit strategies for state governments across the country.

Population density, global travel, and lack of early preparedness made New York City an acute hotspot, with more than 170,000 confirmed cases and over 14,000 deaths. Despite these high numbers, we are still largely in the dark about the City’s true infection rate. Knowing the precise number is central to knowing where, when, and how to battle the virus and bring our country back from the brink.

Gov. Cuomo is requiring hospitals maintain 30 percent available ICU capacity before reopening the city. And Mayor Bill de Blasio is stockpiling 90 days worth of personal protective equipment (PPE) in order to be prepared for a second wave. But it is impossible to know if these criteria are reasonable and sufficient without accurate estimates of the true prevalence in New York. This is essential to establish accurate estimates for the likelihood and severity of a second wave, including predictions for the number of patients who will be hospitalized and/or require ICU care.

New York City’s high infection rate promises higher levels of accuracy with a smaller testing group than anywhere else in the world. In fact, a serum study of just 300 people in New York would be far more revealing than the heavily criticized 3,300-person study conducted in early April in Santa Clara County, California. The California study’s low observed antibody rate—just 1.5 percent—could have been caused entirely by testing errors. With many faulty or unreliable, such errors are amplified when the population being tested has a low rate of infection.

Because of the margin of error, testing in areas with relatively low rates of infection won’t allow us to confidently say whether the true population infection rate is, say, 2 percent or 0.2 percent. That may seem small in absolute terms, but it means we might not even know the real rate within a factor of 10. And that level of doubt about the true infection rate means we can’t know what resources we need and how to forecast and respond to a second wave. Even the smallest inaccuracies compound into serious misunderstandings and unpreparedness to combat the virus.

Bottom line: we won’t be able to prepare for what lies ahead without reliable test results. Using statistics and a result called the Central Limit Theorem, our analysis demonstrates that serum testing results are more robust to errors when done in an area with higher infection rates.

To understand why, imagine the testing equipment is a scale that can be off by up to 2 pounds. When using it to determine the weight (infection rate) of a newborn baby (a city), each measurement can be highly inaccurate, making it difficult to access the baby’s true weight. But when using it to determine the weight of a 150 pound adult, those errors become negligible. When the baseline weight (infection rate) is high, errors become less important, and you can be more confident in the accuracy of your measurements. New York City’s high rate of infection means studies done here will be far more reliable than the California study.

Even in New York City though, successful studies will require truly randomized and representative sampling. Recent efforts to test volunteers in New York stores were helpful, but biases inherent in this method likely skewed the results (e.g., people who feel sick are less likely to go to stores). By comparison, random and representative studies are designed to limit bias by testing people regardless of whether they’re symptomatic. The U.S. Census already provides the data we need to design a truly random and representative at-home testing study. This approach would require expertise and resources, but with our economic crisis worsening every day, the benefits far outweigh the costs.

We have to get testing right. Rapid antigen tests will make testing for infection more widespread so that swift action can be taken to reduce community transmission. However, this will not be sufficient if we do not even know how many infections require hospitalization or ventilation support. We cannot wait to test all 8.5 million New Yorkers before reopening, as LA is aiming to do. Mayor de Blasio just announced testing of 70,000 New Yorkers over the next two weeks. This time and money would be better spent on a careful, scientifically rigorous study of just 300 people which would be far more informative.

The end of COVID-19’s grip on our lives may be a long way off. But we have the road map to get there, and it starts with small-scale serum testing in hot spots—first and foremost, New York City. This data will then help us map out other cities and countries where the virus hasn’t peaked—and places where it has peaked and receded, but may return again—to keep us a step ahead. The way we successfully fight this virus isn’t with uncertainty and guesswork, but by efficiently using the sharpest research tools in the best places in order to save lives, rescue our economies, and return to normal.

About the Researchers

Carri W. Chan

Associate Professor of Business, Decision, Risk, and Operations Division

Assaf Zeevi

Kravis Professor of Business, Decision, Risk, and Operations Division

Daniel J Russo

Assistant Professor, Decision, Risk, and Operations Division