Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
| Title: | Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. |
|---|---|
| Authors: | Cramer EY; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Ray EL; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Lopez VK; COVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.; Bracher J; Chair of Econometrics and Statistics, Karlsruhe Institute of Technology, 76185 Karlsruhe, Germany.; Computational Statistics Group, Heidelberg Institute for Theoretical Studies, 69118 Heidelberg, Germany.; Brennen A; IQT Labs, In-Q-Tel, Waltham, MA 02451.; Castro Rivadeneira AJ; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Gerding A; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Gneiting T; Computational Statistics Group, Heidelberg Institute for Theoretical Studies, 69118 Heidelberg, Germany.; Institute of Stochastics, Karlsruhe Institute of Technology, 69118 Karlsruhe, Germany.; House KH; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Huang Y; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Jayawardena D; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Kanji AH; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Khandelwal A; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Le K; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Mühlemann A; Institute of Mathematical Statistics and Actuarial Science, University of Bern, CH-3012 Bern, Switzerland.; Niemi J; Department of Statistics, Iowa State University, Ames, IA 50011.; Shah A; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Stark A; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Wang Y; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Wattanachit N; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Zorn MW; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003.; Gu Y; Unaffiliated, New York, NY 10016.; Jain S; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Bannur N; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Deva A; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Kulkarni M; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Merugu S; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Raval A; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Shingi S; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Tiwari A; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; White J; Wadhwani Institute of Artificial Intelligence, Andheri East, Mumbai, 400093, India.; Abernethy NF; University of Washington, Seattle, WA 98109.; Woody S; Department of Integrative Biology, University of Texas at Austin, Austin, TX 78712.; Dahan M; Texas Advanced Computing Center, Austin, TX 78758.; Fox S; Department of Integrative Biology, University of Texas at Austin, Austin, TX 78712.; Gaither K; Texas Advanced Computing Center, Austin, TX 78758.; Lachmann M; Santa Fe Institute, Santa Fe, NM 87501.; Meyers LA; Department of Integrative Biology, University of Texas at Austin, Austin, TX 78712.; Scott JG; Department of Information, Risk, and Operations Management, University of Texas at Austin, Austin, TX 78712.; Tec M; Department of Statistics and Data Sciences, University of Texas at Austin, Austin, TX 78712.; Srivastava A; Ming Hsieh Department of Computer and Electrical Engineering, University of Southern California, Los Angeles, CA 90089.; George GE; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Cegan JC; US Army Engineer Research and Development Center, Concord, MA 01742.; Dettwiller ID; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; England WP; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Farthing MW; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Hunter RH; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Lafferty B; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Linkov I; US Army Engineer Research and Development Center, Concord, MA 01742.; Mayo ML; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Parno MD; US Army Engineer Research and Development Center, Hanover, NH 03755.; Rowland MA; US Army Engineer Research and Development Center, Vicksburg, MS 39180.; Trump BD; US Army Engineer Research and Development Center, Concord, MA 01742.; Zhang-James Y; Department of Psychiatry and Behavioral Sciences, State University of New York Upstate Medical University, Syracuse, NY 13210.; Chen S; School of Medicine, State University of New York Upstate Medical University, Syracuse, NY 13210.; Faraone SV; Department of Psychiatry and Behavioral Sciences, State University of New York Upstate Medical University, Syracuse, NY 13210.; Hess J; Department of Psychiatry and Behavioral Sciences, State University of New York Upstate Medical University, Syracuse, NY 13210.; Morley CP; Department of Public Health & Preventive Medicine, State University of New York Upstate Medical University, Syracuse, NY 13210.; Salekin A; Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13207.; Wang D; Department of Public Health & Preventive Medicine, State University of New York Upstate Medical University, Syracuse, NY 13210.; Corsetti SM; Department of Physics, University of Michigan, Ann Arbor, MI, 48109.; Baer TM; Department of Physics, Trinity University, San Antonio, TX 78212.; Eisenberg MC; Department of Complex Systems, University of Michigan, Ann Arbor, MI 48109.; Department of Mathematics, University of Michigan, Ann Arbor, MI 48109.; School of Public Health, Department of Epidemiology, University of Michigan, Ann Arbor, MI 48109.; Falb K; Department of Physics, University of Michigan, Ann Arbor, MI, 48109.; Huang Y; Department of Physics, University of Michigan, Ann Arbor, MI, 48109.; Martin ET; School of Public Health, Department of Epidemiology, University of Michigan, Ann Arbor, MI 48109.; McCauley E; Department of Physics, University of Michigan, Ann Arbor, MI, 48109.; Myers RL; Department of Physics, University of Michigan, Ann Arbor, MI, 48109.; Schwarz T; Department of Physics, University of Michigan, Ann Arbor, MI, 48109.; Sheldon D; College of Information and Computer Sciences, University of Massachusetts, Amherst, MA 01003.; Gibson GC; School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA 01003.; Yu R; Department of Computer Science and Engineering, University of California, San Diego, CA 92093.; Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115.; Gao L; Department of Statistics, University of Washington, Seattle, WA 98185.; Ma Y; Halıcıoğlu Data Science Institute, University of California, San Diego, CA 92093.; Wu D; Department of Computer Science and Engineering, University of California, San Diego, CA 92093.; Yan X; Department of Computer Science, University of California, Santa Barbara, CA 93106.; Jin X; Department of Computer Science, University of California, Santa Barbara, CA 93106.; Wang YX; Department of Computer Science, University of California, Santa Barbara, CA 93106.; Chen Y; Mechatronics, Embedded Systems and Automation Lab, Department of Mechanical Engineering, University of California, Merced, CA 95301.; Guo L; Jilin University, Changchun City, Jilin Province, 130012, People's Republic of China.; Zhao Y; University of Science and Technology of China, Heifei, Anhui, 230027, People's Republic of China.; Gu Q; Department of Computer Science, University of California, Los Angeles, CA 90095.; Chen J; Department of Computer Science, University of California, Los Angeles, CA 90095.; Wang L; Department of Computer Science, University of California, Los Angeles, CA 90095.; Xu P; Department of Computer Science, University of California, Los Angeles, CA 90095.; Zhang W; Department of Computer Science, University of California, Los Angeles, CA 90095.; Zou D; Department of Computer Science, University of California, Los Angeles, CA 90095.; Biegel H; Department of Mathematics, University of Arizona, Tucson, AZ 85721.; Lega J; Department of Mathematics, University of Arizona, Tucson, AZ 85721.; McConnell S; Construx, Bellevue, WA 98004.; Nagraj VP; Quality Assurance and Data Science, Signature Science, LLC, Charlottesville, VA 22911.; Guertin SL; Quality Assurance and Data Science, Signature Science, LLC, Charlottesville, VA 22911.; Hulme-Lowe C; Quality Assurance and Data Science, Signature Science, LLC, Austin, TX 78759.; Turner SD; Quality Assurance and Data Science, Signature Science, LLC, Charlottesville, VA 22911.; Shi Y; Department of Materials Science and Engineering, Rensselaer Polytechnic Institute, Troy, NY 12309.; Ban X; Department of Civil and Environmental Engineering, University of Washington, Seattle, WA 98195.; Walraven R; Unaffiliated, Davis, CA 95616.; Hong QJ; School for Engineering of Matter, Transport and Energy, Arizona State University, Tempe, AZ 85287.; School of Engineering, Brown University, Providence, RI 02912.; Kong S; Manhasset Secondary School, Manhasset, NY 11030.; van de Walle A; School of Engineering, Brown University, Providence, RI 02912.; Turtle JA; Infectious Disease Group, Predictive Science, Inc, San Diego, CA 92121.; Ben-Nun M; Infectious Disease Group, Predictive Science, Inc, San Diego, CA 92121.; Riley S; Medical Research Council Centre for Global Infectious Disease Analysis, Department of Infectious Disease Epidemiology, School of Public Health, Imperial College, W2 1PG London, United Kingdom.; Riley P; Infectious Disease Group, Predictive Science, Inc, San Diego, CA 92121.; Koyluoglu U; Financial Services, Oliver Wyman, New York, NY 10036.; DesRoches D; Oliver Wyman Digital, Oliver Wyman, Boston, MA 02110.; Forli P; Oliver Wyman Digital, Oliver Wyman, Sao Paolo, Brazil 04711-904.; Hamory B; Health & Life Sciences, Oliver Wyman, Boston, MA 02110.; Kyriakides C; Oliver Wyman Digital, Oliver Wyman, New York, NY 10036.; Leis H; Health & Life Sciences, Oliver Wyman, New York, NY 10036.; Milliken J; Financial Services, Oliver Wyman, New York, NY 10036.; Moloney M; Financial Services, Oliver Wyman, New York, NY 10036.; Morgan J; Financial Services, Oliver Wyman, New York, NY 10036.; Nirgudkar N; Core Consultant Group, Oliver Wyman, New York, NY 10036.; Ozcan G; Financial Services, Oliver Wyman, New York, NY 10036.; Piwonka N; Health & Life Sciences, Oliver Wyman, New York, NY 10036.; Ravi M; Core Consultant Group, Oliver Wyman, New York, NY 10036.; Schrader C; Health & Life Sciences, Oliver Wyman, New York, NY 10036.; Shakhnovich E; Health & Life Sciences, Oliver Wyman, New York, NY 10036.; Siegel D; Financial Services, Oliver Wyman, New York, NY 10036.; Spatz R; Core Consultant Group, Oliver Wyman, New York, NY 10036.; Stiefeling C; Financial Services, Oliver Wyman Digital, Toronto, ON, Canada M5J 0A1.; Wilkinson B; Financial Services, Oliver Wyman, London, UK W1U 8EW.; Wong A; Oliver Wyman Digital, Oliver Wyman, New York, NY 10036.; Cavany S; Department of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.; España G; Department of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.; Moore S; Department of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.; Oidtman R; Department of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.; Department of Ecology and Evolution, University of Chicago, Chicago, IL 60637.; Perkins A; Department of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556.; Kraus D; Department of Mathematics and Statistics, Masaryk University, 61137 Brno, Czech Republic.; Kraus A; Department of Mathematics and Statistics, Masaryk University, 61137 Brno, Czech Republic.; Gao Z; Microsoft, Redmond, WA 98029.; Bian J; Microsoft, Redmond, WA 98029.; Cao W; Microsoft, Redmond, WA 98029.; Lavista Ferres J; Microsoft, Redmond, WA 98029.; Li C; Microsoft, Redmond, WA 98029.; Liu TY; Microsoft, Redmond, WA 98029.; Xie X; Microsoft, Redmond, WA 98029.; Zhang S; Microsoft, Redmond, WA 98029.; Zheng S; Microsoft, Redmond, WA 98029.; Vespignani A; Institute for Scientific Interchange Foundation, Turin, 10133, Italy.; Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.; Chinazzi M; Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.; Davis JT; Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.; Mu K; Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.; Pastore Y Piontti A; Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.; Xiong X; Laboratory for the Modeling of Biological and Socio-technical Systems, Northeastern University, Boston, MA 02115.; Zheng A; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Baek J; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Farias V; Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142.; Georgescu A; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Levi R; Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142.; Sinha D; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Wilde J; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Perakis G; Massachusetts Institute of Technology, Cambridge, MA 02142.; Bennouna MA; Massachusetts Institute of Technology, Cambridge, MA 02142.; Nze-Ndong D; Massachusetts Institute of Technology, Cambridge, MA 02142.; Singhvi D; Technology, Operations and Statistics (TOPS) group, Stern School of Business, New York University, New York, NY 10012.; Spantidakis I; Massachusetts Institute of Technology, Cambridge, MA 02142.; Thayaparan L; Massachusetts Institute of Technology, Cambridge, MA 02142.; Tsiourvas A; Massachusetts Institute of Technology, Cambridge, MA 02142.; Sarker A; Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139.; Jadbabaie A; Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139.; Shah D; Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139.; Della Penna N; Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA 02139.; Celi LA; Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA 02139.; Sundar S; River Hill High School, Clarksville, MD 21029.; Wolfinger R; JMP Life Sciences, SAS Institute Inc, Cary, NC 27513.; Osthus D; Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM 87545.; Castro L; Information Systems and Modeling Group, Los Alamos National Laboratory, Los Alamos, NM 87545.; Fairchild G; Information Systems and Modeling Group, Los Alamos National Laboratory, Los Alamos, NM 87545.; Michaud I; Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos, NM 87545.; Karlen D; Department of Physics and Astronomy, University of Victoria, Victoria, BC, V8W 2Y2, Canada.; Physical Sciences Division, TRIUMF, Vancouver, BC, V8W 2Y2, Canada.; Kinsey M; Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.; Mullany LC; Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.; Rainwater-Lovett K; Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.; Shin L; Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.; Tallaksen K; Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.; Wilson S; Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723.; Lee EC; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Dent J; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Grantz KH; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Hill AL; Institute for Computational Medicine, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21218.; Kaminsky J; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Kaminsky K; Unaffiliated, Baltimore, MD 21205.; Keegan LT; Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT 84108.; Lauer SA; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Lemaitre JC; Laboratory of Ecohydrology, School of Architecture, Civil and Environmental Engineering, École Polytechnique Fédérale de Lausanne, 1015 Lausanne, Switzerland.; Lessler J; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Meredith HR; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Perez-Saez J; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Shah S; Unaffiliated, San Francisco, CA 94122.; Smith CP; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; Truelove SA; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21215.; International Vaccine Access Center, Johns Hopkins University, Baltimore, MD 21231.; Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21231.; Wills J; Unaffiliated, San Francisco, CA 94122.; Marshall M; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD 21218.; Gardner L; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD 21218.; Nixon K; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD 21218.; Burant JC; Unaffiliated, Amsterdam, The Netherlands.; Wang L; Department of Statistics, Iowa State University, Ames, IA 50011.; Gao L; Department of Finance, Iowa State University, Ames, IA 50011.; Gu Z; Department of Statistics, Iowa State University, Ames, IA 50011.; Kim M; Department of Statistics, Iowa State University, Ames, IA 50011.; Li X; School of Mathematical and Statistical Sciences, Clemson University, Clemson, SC 29634.; Wang G; Department of Mathematics, College of William & Mary, Williamsburg, VA 23187.; Wang Y; Department of Statistics, Iowa State University, Ames, IA 50011.; Yu S; Department of Statistics, University of Virginia, Charlottesville, VA 22904.; Reiner RC; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Barber R; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Gakidou E; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Hay SI; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Lim S; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Murray C; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Pigott D; Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA 98195.; Gurung HL; Emerging Technologies, IEM, Inc., Bel Air, MD 21015.; Baccam P; Emerging Technologies, IEM, Inc., Bel Air, MD 21015.; Stage SA; Emerging Technologies, IEM, Inc., Baton Rouge, LA 70809.; Suchoski BT; Emerging Technologies, IEM, Inc., Bel Air, MD 21015.; Prakash BA; College of Computing, Georgia Institute of Technology, Atlanta, GA 30308.; Adhikari B; Department of Computer Science, University of Iowa, Iowa City, IA 52242.; Cui J; College of Computing, Georgia Institute of Technology, Atlanta, GA 30308.; Rodríguez A; College of Computing, Georgia Institute of Technology, Atlanta, GA 30308.; Tabassum A; Department of Computer Science, Virginia Tech, Falls Church, VA 22043.; Xie J; College of Computing, Georgia Institute of Technology, Atlanta, GA 30308.; Keskinocak P; H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.; Asplund J; Advanced Data Analytics, Metron, Inc., Reston, VA 20190.; Baxter A; H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.; Oruc BE; H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.; Serban N; H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.; Arik SO; Google Cloud, Sunnyvale, CA 94089.; Dusenberry M; Google Cloud, Sunnyvale, CA 94089.; Epshteyn A; Google Cloud, Sunnyvale, CA 94089.; Kanal E; Google Cloud, Sunnyvale, CA 94089.; Le LT; Google Cloud, Sunnyvale, CA 94089.; Li CL; Google Cloud, Sunnyvale, CA 94089.; Pfister T; Google Cloud, Sunnyvale, CA 94089.; Sava D; Google Cloud, Sunnyvale, CA 94089.; Sinha R; Google Cloud, Sunnyvale, CA 94089.; Tsai T; Department of Health Policy and Management, Harvard University, Cambridge, MA 02138.; Yoder N; Google Cloud, Sunnyvale, CA 94089.; Yoon J; Google Cloud, Sunnyvale, CA 94089.; Zhang L; Google Cloud, Sunnyvale, CA 94089.; Abbott S; Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.; Bosse NI; Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.; Funk S; Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.; Hellewell J; Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.; Meakin SR; Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.; Sherratt K; Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, WC1E 7HT London, United Kingdom.; Zhou M; McCombs School of Business, The University of Texas at Austin, Austin, TX 78712.; Kalantari R; Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX 78712.; Yamana TK; Department of Environmental Health Sciences, Columbia University, New York, NY 10032.; Pei S; Department of Environmental Health Sciences, Columbia University, New York, NY 10032.; Shaman J; Department of Environmental Health Sciences, Columbia University, New York, NY 10032.; Li ML; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Bertsimas D; Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142.; Skali Lami O; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Soni S; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Tazi Bouardi H; Operations Research Center, Massachusetts Institute of Technology; Cambridge, MA 02139.; Ayer T; H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.; Winship Cancer Institute, Emory University Medical School, Atlanta, GA 30322.; Adee M; Radiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.; Chhatwal J; Radiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.; Dalgic OO; Health Economic Modeling, Value Analytics Labs, 34776 İstanbul, Turkey.; Ladd MA; Radiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.; Linas BP; Department of Medicine, Section of Infectious Diseases, Boston University School of Medicine, Boston, MA 02118.; Mueller P; Radiology-Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA 02114.; Xiao J; H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.; Wang Y; Department of Biostatistics, Columbia University, New York, NY 10032.; Department of Psychiatry, Columbia University, New York, NY 10032.; Wang Q; Department of Biostatistics, Columbia University, New York, NY 10032.; Xie S; Department of Biostatistics, Columbia University, New York, NY 10032.; Zeng D; Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599.; Green A; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; Bien J; Marshall School of Business, Department of Data Sciences and Operations (DSO), University of Southern California, Los Angeles, CA 90089.; Brooks L; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; Hu AJ; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; Jahja M; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; McDonald D; Department of Statistics, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada.; Narasimhan B; Department of Biomedical Data Sciences, Stanford University, Stanford, CA 94305.; Department of Statistics, Stanford University, Stanford, CA 94305.; Politsch C; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213.; Rajanala S; Department of Statistics, Stanford University, Stanford, CA 94305.; Rumack A; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213.; Simon N; Department of Biostatistics, University of Washington, Seattle, WA 98195.; Tibshirani RJ; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; Tibshirani R; Department of Statistics, Stanford University, Stanford, CA 94305.; Ventura V; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; Wasserman L; Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15213.; O'Dea EB; Odum School of Ecology, University of Georgia, Athens, GA 30602.; Drake JM; Odum School of Ecology, University of Georgia, Athens, GA 30602.; Pagano R; Unaffiliated, Tucson, AZ 85704.; Tran QT; Catalog Data Science, Walmart Inc., Sunnyvale, CA 94085.; Ho LST; Department of Mathematics and Statistics, Dalhousie University, Halifax, NS, B3H 4R2, Canada.; Huynh H; Virtual Power System Inc, Milpitas, CA 95035.; Walker JW; COVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.; Slayton RB; COVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.; Johansson MA; COVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.; Biggerstaff M; COVID-19 Response, Centers for Disease Control and Prevention; Atlanta, GA 30333.; Reich NG; Department of Biostatistics and Epidemiology, University of Massachusetts, Amherst, MA 01003. |
| Source: | Proceedings of the National Academy of Sciences of the United States of America [Proc Natl Acad Sci U S A] 2022 Apr 12; Vol. 119 (15), pp. e2113561119. Date of Electronic Publication: 2022 Apr 08. |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: National Academy of Sciences Country of Publication: United States NLM ID: 7505876 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1091-6490 (Electronic) Linking ISSN: 00278424 NLM ISO Abbreviation: Proc Natl Acad Sci U S A Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: Washington, DC : National Academy of Sciences |
| MeSH Terms: | COVID-19*/mortality; Public Health/trends ; United States/epidemiology ; Data Accuracy ; Forecasting ; Humans ; Pandemics ; Probability |
| Abstract: | Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. |
| Comments: | Erratum in: Proc Natl Acad Sci U S A. 2023 Apr 11;120(15):e2304076120. doi: 10.1073/pnas.2304076120.. (PMID: 37014868) |
| References: | PLoS Comput Biol. 2019 Nov 22;15(11):e1007486. (PMID: 31756193); Proc Natl Acad Sci U S A. 2019 Feb 19;116(8):3146-3154. (PMID: 30647115); Proc Natl Acad Sci U S A. 2019 Nov 26;116(48):24268-24274. (PMID: 31712420); Science. 2005 Oct 14;310(5746):248-9. (PMID: 16224011); J Infect Dis. 2016 Dec 1;214(suppl_4):S404-S408. (PMID: 28830111); PLoS Med. 2021 Oct 19;18(10):e1003793. (PMID: 34665805); Science. 1999 Sep 3;285(5433):1548-1550. (PMID: 10477515); BMC Public Health. 2019 Dec 10;19(1):1659. (PMID: 31823751); Philos Trans R Soc Lond B Biol Sci. 2019 Jul 8;374(1776):20180276. (PMID: 31104603); PLoS Comput Biol. 2021 Feb 12;17(2):e1008618. (PMID: 33577550); Sci Rep. 2019 Jan 24;9(1):683. (PMID: 30679458); Sci Data. 2021 Feb 11;8(1):59. (PMID: 33574342); Nat Commun. 2021 Aug 27;12(1):5173. (PMID: 34453047); Epidemics. 2018 Mar;22:13-21. (PMID: 28958414); Proc Natl Acad Sci U S A. 2021 Dec 21;118(51):. (PMID: 34903655); Lancet Infect Dis. 2020 May;20(5):533-534. (PMID: 32087114) |
| Grant Information: | MC_PC_19012 United Kingdom MRC_ Medical Research Council; MR/J008761/1 United Kingdom MRC_ Medical Research Council; U01 CK000531 United States CK NCEZID CDC HHS; U01 IP001137 United States IP NCIRD CDC HHS; U01 IP001122 United States IP NCIRD CDC HHS; R35 GM119582 United States GM NIGMS NIH HHS; 210758/Z/18/Z United Kingdom WT_ Wellcome Trust; R01 GM124104 United States GM NIGMS NIH HHS; 200861/Z/16/Z United Kingdom WT_ Wellcome Trust; MR/R015600/1 United Kingdom MRC_ Medical Research Council; U01 CK000538 United States CK NCEZID CDC HHS |
| Contributed Indexing: | Keywords: COVID-19; ensemble forecast; forecasting; model evaluation |
| Entry Date(s): | Date Created: 20220408 Date Completed: 20220415 Latest Revision: 20260417 |
| Update Code: | 20260417 |
| PubMed Central ID: | PMC9169655 |
| DOI: | 10.1073/pnas.2113561119 |
| PMID: | 35394862 |
| Database: | MEDLINE |
Journal Article