| Title: |
Computational Reproducibility and Robustness of Empirical Economics and Political Science Research Between 2022 and 2023 |
| Authors: |
Brodeur, Abel; Mikola, Derek; Cook, Nikolai; Fiala, Lenka; Brailey, Thomas; Briggs, Ryan; de Gendre, Alexandra; Dupraz, Yannick; Gabani, Jacopo; Gauriot, Romain; Haddad, Joanne; Duncan, Jamie; Duong, Kiet Tuan; Duprey, Thibaut; Dworschak, Christoph; Ellingsrud, Sigmund; Elminejad, Ali; Eissa, Yasmine; Erhart, Andrea; Etingin-Frati, Giulian; Fatemipour, Elaheh; Lima, Goncalo; Federice, Alexa; Feld, Jan; Fenig, Guidon; Firouzjaeiangalougah, Mojtaba; Fleisje, Erlend; Fortier-Chouinard, Alexandre; Engel, Julia Francesca; Fréchet, Nadjim; Fortier, Reid; Fries, Tilman; Ankel-Peters, Jörg; Frith, Michael James; Galipeau, Thomas; Gallegos, Sebastián; Gangji, Areez; Gao, Xiaoying; Garnache, Cloé; Gáspár, Attila; Gavrilova, Evelina; Ghosh, Arijit; Gibney, Garreth; Dreber, Anna; Gibson, Grant; Godager, Geir; Goff, Leonard; Gong, Da; González, Javier; Gretton, Jeremy D.; Griffa, Cristina; Grigoryeva, Idaliya; Grøtting, Maja; Guntermann, Eric; Campbell, Douglas; Guo, Jiaqi; Gugushvili, Alexi; Habibnia, Hooman; Häffner, Sonja; Hall, Jonathan D.; Hammar, Olle; Kordt, Amund Hanson; Hashimoto, Barry; Hartley, Jonathan S.; Hausladen, Carina I.; Kattan, Lamis; Havránek, Tomáš; He, Harry; Hepplewhite, Matthew; Herrera-Rodriguez, Mario; Heuer, Felix; Heyes, Anthony; Ho, Anson T. Y.; Holmes, Jonathan; Holzknecht, Armando; Hsu, Yu-Hsiang Dexter; Marino Fages, Diego; Hu, Shiang-Hung; Huang, Yu-Shiuan; Huebener, Mathias; Huber, Christoph; Huynh, Kim P.; Irsova, Zuzana; Isler, Ozan; Jakobsson, Niklas; Jananji, Raphaël; Jayalath, Tharaka A.; Mierisch, Fabian; Jetter, Michael; John, Jenny; Forshaw, Rachel Joy; Juan, Felipe; Kadriu, Valon; Karim, Sunny; Kelly, Edmund; Dang, Duy Khanh Hoang; Khushboo, Tazia; Kim, Jin; Sun, Pu; Kjellsson, Gustav; Kjelsrud, Anders; Kotsadam, Andreas; Korpershoek, Jori; Krashinsky, Lewis; Kundu, Suranjana; Kustov, Alexander; Lalayev, Nurlan; Langlois, Audrée; Laufer, Jill; Wright, Taylor; Lee-Whiting, Blake; Leibing, Andreas; Lenz, Gabriel; Levin, Joel; Li, Peng; Li, Tongzhe; Lin, Yuchen; Listo, Ariel; Liu, Dan; Lu, Xuewen; Connolly, Marie; Lukmanova, Elvina; Luscombe, Alex; Lusher, Lester R.; Lyu, Ke; Ma, Hai; Mäder, Nicolas; Makate, Clifton; Malmberg, Alice; Maitra, Adit; Mandas, Marco; Hoces de la Guardia, Fernando; Marcus, Jan; Margaryan, Shushanik; Márk, Lili; Martignano, Andres; Marsh, Abigail; Masetto, Isabella; McCanny, Anthony; McManus, Emma; McWay, Ryan; Metson, Lennard; Johannesson, Magnus; Kinge, Jonas Minet; Mishra, Sumit; Mohnen, Myra; Möller, Jakob; Montambeault, Rosalie; Montpetit, Sébastien; Morin, Louis-Philippe; Morris, Todd; Moser, Scott; Motoki, Fabio; Miguel, Edward; Muehlenbachs, Lucija; Musulan, Andreea; Musumeci, Marco; Nabin, Munirul; Nchare, Karim; Neubauer, Florian; Nguyen, Quan M. P.; Nguyen, Tuan; Nguyen-Tien, Viet; Niazi, Ali; Vilhuber, Lars; Nikolaishvili, Giorgi; Nordstrom, Ardyn; Nüß, Patrick; Odermatt, Angela; Olson, Matt; Øien, Henning; Ölkers, Tim; Oliver i Vert, Miquel; Oral, Emre; Oswald, Christian; Abarca, Alejandro; Ousman, Ali; Özak, Ömer; Pandey, Shubham; Pavlov, Alexandre; Pelli, Martino; Penheiro, Romeo; Park, RyuGyung; Pérez Martel, Eva; Petrovičová, Tereza; Phan, Linh; Acharya, Mahesh; Prettyman, Alexa; Procházka, Jakub; Putri, Aqila; Quandt, Julian; Qiu, Kangyu; Nguyen, Loan Quynh Thi; Rahman, Andaleeb; Rea, Carson H.; Reiremo, Adam; Renée, Laëtitia; Adjisse, Sossou Simplice; Richardson, Joseph; Rivers, Nicholas; Rodrigues, Bruno; Roelofs, William; Roemer, Tobias; Rogeberg, Ole; Rose, Julian; Roskos-Ewoldsen, Andrew; Rosmer, Paul; Sabada, Barbara; Akhtar, Ahwaz; Saberian, Soodeh; Salamanca, Nicolas; Sator, Georg; Scates, Daniel; Schlüter, Elmar; Sells, Cameron; Sen, Sharmi; Sethi, Ritika; Shcherbiak, Anna; Sogaolu, Moyosore; Ramirez Lizardi, Eduardo Alberto; Soosalu, Matt; Sørensen, Erik Ø.; Sovani, Manali; Spencer, Noah; Staubli, Stefan; Stans, Renske; Stewart, Anya; Stips, Felix; Stockley, Kieran; Strobel, Stephenson; Albrecht, Sabina; Struby, Ethan; Tang, John; Tanrisever, Idil; Yang, Thomas Tao; Tastan, Ipek; Tatić, Dejan; Tatlow, Benjamin; Tchuisseu Seuyong, Féraud; Thériault, Rémi; Thivierge, Vincent; Andersen, Synøve Nygaard; Tian, Wenjie; Toma, Filip-Mihai; Totarelli, Maddalena; Tran, Van; Truong, Hung; Tsoy, Nikita; Tuzcuoglu, Kerem; Ubfal, Diego; Villalobos, Laura; Walterskirchen, Julian; Andlib, Zubaria; Wang, Joseph Tao-yi; Wattal, Vasudha; Webb, Matthew D.; Weber, Bryan; Weisser, Reinhard; Weng, Wei-Chien; Westheide, Christian; White, Kimberly; Winter, Jacob; Wochner, Timo; Arrora, Falak; Woerman, Matt; Wong, Jared; Woodard, Ritchie; Wroński, Marcin; Yazbeck, Myra; Yang, Chung; Yap, Luther; Yassin, Kareman; Ye, Hao; Yoon, Jin Young; Ash, Thomas; Yurris, Chris; Zahra, Tahreen; Zaneva, Mirela; Zayat, Aline; Zhang, Jonathan; Zhao, Ziwei; Zhong, Yaolang; Bacher, Etienne; Bachler, Sebastian; Bacon, Félix; Bagues, Manuel; Balogh, Timea; Batmanov, Alisher; Barschkett, Mara; Basdil, B. Kaan; Baxa, Jaromír; Becker, Sascha; Beeder, Monica; Beland, Louis-Philippe; Bello, Abdel-Hamid; Markovits, Daniel Benenson; Benjamin, Grant; Bergeron, Thomas; Blimpo, Moussa; Binetti, Marco; Bonander, Carl; Bonneau, Joseph; Borbáth, Endre; Topstad Borgen, Nicolai; Topstad Borgen, Solveig; Borowsky, Jonathan; Brini, Elisa; Brown, Myriam; Brun, Martin; Bruns, Stephan; Buliskeria, Nino; Calef, Andrea; Cameron, Alistair; Campa, Pamela; Campos-Rodríguez, Santiago; Cantone, Giulio Giacomo; Carpena, Fenella; Carter, Perry; Castañeda Dower, Paul; Castek, Ondrej; Caviglia-Harris, Jill; Chauca Strand, Gabriella; Chen, Shi; Chzhen, Sya In; Chung, Jong; Collins, Jason; Coppock, Alexander; Cordeau, Hugo; Couillard, Ben; Crechet, Jonathan; Crippa, Lorenzo; Cui, Jeanne; Czymara, Christian; Daarstad, Haley; Dao, Danh Chi; Dao, Daniel; Schmandt, Marco David; de Linde, Astrid; De Melo, Lucas; Deer, Lachlan; De Vera, Micole; Dimitrova, Velichka; Dollbaum, Jan Fabian; Dollbaum, Jan Matti; Donnelly, Michael; Huynh, Luu Duc Toan; Dumbalska, Tsvetomira |
| Publisher Information: |
Institute for Replication (I4R) |
| Publication Year: |
2026 |
| Collection: |
EconStor (German National Library of Economics, ZBW) |
| Subject Terms: |
ddc:330; Reproduction; Robustness; Research Transparency; Open Science; Economics; Political Science |
| Description: |
This systematic and large-scale reproduction effort tests the reproducibility and robustness of economics and political science. We reproduced original analyses and conducted robustness checks of 110 articles recently published in leading economics and political science journals (all of which have mandatory data and code sharing policies). We found that over 85% of published claims were computationally reproducible. In robustness checks, our re-analyses led to 72% of statistically significant estimates to remain significant and in the same direction, and the median reproduced effect size is (nearly) the same as the originally published effect size (that is, 99% of the published effect size). Additionally, six independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness correlated with neither author characteristics nor data availability. ; Published in Nature 652, 151-156 (2026): https://doi.org/10.1038/s41586-026-10251-x |
| Document Type: |
report |
| Language: |
English |
| Relation: |
Series: I4R Discussion Paper Series; No. 287; https://hdl.handle.net/10419/338965; RePEc:zbw:i4rdps:287 |
| Availability: |
https://hdl.handle.net/10419/338965 |
| Rights: |
https://www.econstor.eu/dspace/Nutzungsbedingungen |
| Accession Number: |
edsbas.5BBAF901 |
| Database: |
BASE |