Dieses Ergebnis aus BASE kann Gästen nicht angezeigt werden. Login für vollen Zugriff.

The reliability of replications: a study in computational reproductions

Title:	The reliability of replications: a study in computational reproductions
Authors:	Breznau, Nate; Rinke, Eike Mark; Wuttke, Alexander; Adem, Muna; Adriaans, Jule; Akdeniz, Esra; Álvarez-Benjumea, Amalia; Andersen, Henrik K; Auer, Daniel; Azevedo, Flavio; Bahnsen, Oke; Bai, Ling; Balzer, Dave; Bauer, Paul C; Bauer, Gerrit; Baumann, Markus; Baute, Sharon; Benoit, Verena; Bernauer, Julian; Berning, Carl; Berthold, Anna; Bethke, Felix S.; Biegert, Thomas; Blinzler, Katharina; Blumenberg, Johannes N.; Bobzien, Licia; Bohman, Andrea; Bol, Thijs; Bostic, Amie; Brzozowska, Zuzanna; Burgdorf, Katharina; Burger, Kaspar; Busch, Kathrin; Castillo, Juan-Carlos; Chan, Nathan; Christmann, Pablo; Connelly, Roxanne; Czymara, Christian S.; Damian, Elena; Rooij, Eline A. de; Ecker, Alejandro; Edelmann, Achim; Eder, Christina; Eger, Maureen A.; Ellerbrock, Simon; Forke, Anna; Forster, Andrea; Freire, Danilo; Gaasendam, Chris; Gavras, Konstantin; Gayle, Vernon; Gessler, Theresa; Gnambs, Timo; Godefroidt, Amélie; Grömping, Max; Groß, Martin; Gruber, Stefan; Gummer, Tobias; Hadjar, Andreas; Halbherr, Verena; Heisig, Jan Paul; Hellmeier, Sebastian; Heyne, Stefanie; Hirsch, Magdalena; Hjerm, Mikael; Hochman, Oshrat; Höffler, Jan H.; Hövermann, Andreas; Hunger, Sophia; Hunkler, Christian; Huth-Stöckle, Nora; Ignácz, Zsófia S.; Israel, Sabine; Jacobs, Laura; Jacobsen, Jannes; Jaeger, Bastian; Jungkunz, Sebastian; Jungmann, Nils; Kanjana, Jennifer; Kauff, Mathias; Khan, Salman; Khatua, Sayak; Kleinert, Manuel; Klinger, Julia; Kolb, Jan-Philipp; Kołczyńska, Marta; Kuk, John; Kunißen, Katharina; Kurti Sinatra, Dafina; Langenkamp, Alexander; Lee, Robin C.; Lersch, Philipp M.; Liu, David; Löbel, Lea-Maria; Lutscher, Philipp; Mader, Matthias; Madia, Joan E.; Malancu, Natalia; Maldonado, Luis; Marahrens, Helge; Martin, Nicole; Martinez, Paul; Mayerl, Jochen; Mayorga, Oscar J.; McDonnell, Robert; McManus, Patricia; McWagner, Kyle; Meeusen, Cecil; Meierrieks, Daniel; Mellon, Jonathan; Merhout, Friedolin; Merk, Samuel; Meyer, Daniel; Micheli, Leticia; Mijs, Jonathan; Moya, Cristóbal; Neunhoeffer, Marcel; Nüst, Daniel; Nygård, Olav; Ochsenfeld, Fabian; Otte, Gunnar; Pechenkina, Anna; Pickup, Mark; Prosser, Christopher; Raes, Louis; Ralston, Kevin; Ramos, Miguel; Reichert, Frank; Roets, Arne; Rogers, Jonathan; Ropers, Guido; Samuel, Robin; Sand, Gregor; Sanhueza Petrarca, Constanza; Schachter, Ariela; Schaeffer, Merlin; Schieferdecker, David; Schlueter, Elmar; Schmidt, Katja; Schmidt, Regine; Schmidt-Catran, Alexander; Schmiedeberg, Claudia; Schneider, Jurgen; Schoonvelde, Martijn; Schulte-Cloos, Julia; Schumann, Sandy; Schunck, Reinhard; Seuring, Julian; Silber, Henning; Sleegers, Willem; Sonntag, Nico; Staudt, Alexander; Steiber, Nadia; Steiner, Nils D.; Sternberg, Sebastian; Stiers, Dieter; Stojmenovska, Dragana; Storz, Nora; Striessnig, Erich; Stroppe, Anne-Kathrin; Suchow, Jordan W.; Teltemann, Janna; Tibajev, Andrey; Tung, Brian; Vagni, Giacomo; Assche, Jasper van; Linden, Meta van der; Noll, Jolanda van der; Hootegem, Arno van; Vogtenhuber, Stefan; Voicu, Bogdan; Wagemans, Fieke; Wehl, Nadja; Werner, Hannah; Wiernik, Brenton M.; Winter, Fabian; Wolf, Christof; Wu, Cary; Yamada, Yuki; Zakula, Björn; Zhang, Nan; Ziller, Conrad; Zins, Stefan; Żółtak, Tomasz; Nguyen, Hung H.V.
Contributors:	German Research Foundation; Agencia Nacional de Investigación y Desarrollo (Chile); Breznau, Nate; Rinke, Eike Mark; Wuttke, Alexander; Adem, Muna; Adriaans, Jule; Akdeniz, Esra; Álvarez-Benjumea, Amalia; Andersen, Henrik K.; Auer, Daniel; Azevedo, Flavio; Bahnsen, Oke; Balzer, Dave; Bauer, Paul C.; Bauer, Gerrit; Baumann, Markus; Baute, Sharon; Benoit, Verena; Bernauer, Julian; Berthold, Anna; Bethke, Felix S.; Biegert, Thomas; Blumenberg, Johannes N.; Bobzien, Licia; Bohman, Andrea; Bol, Thijs; Bostic, Amie; Brzozowska, Zuzanna; Burgdorf, Katharina; Burger, Kaspar; Busch, Kathrin; Castillo, Juan-Carlos; Chan, Nathan; Christmann, Pablo; Connelly, Roxanne; Czymara, Christian S.; Damian, Elena; Rooij, Eline A. de; Ecker, Alejandro; Edelmann, Achim; Eder, Christina; Eger, Maureen A.; Ellerbrock, Simon; Forster, Andrea; Freire, Danilo; Gaasendam, Chris; Gavras, Konstantin; Gayle, Vernon; Gessler, Theresa; Gnambs, Timo; Godefroidt, Amélie; Grömping, Max; Groß, Martin; Gruber, Stefan; Gummer, Tobias; Hadjar, Andreas; Halbherr, Verena; Heisig, Jan Paul; Hellmeier, Sebastian; Heyne, Stefanie; Hirsch, Magdalena; Hjerm, Mikael; Hochman, Oshrat; Höffler, Jan H.; Hövermann, Andreas; Hunger, Sophia; Hunkler, Christian; Huth-Stöckle, Nora; Ignácz, Zsófia S.; Jacobs, Laura; Jacobsen, Jannes; Jaeger, Bastian; Jungkunz, Sebastian; Jungmann, Nils; Kauff, Mathias; Khatua, Sayak; Klinger, Julia; Kolb, Jan-Philipp; Kołczyńska, Marta; Kuk, John; Kunißen, Katharina; Kurti Sinatra, Dafina; Langenkamp, Alexander; Lersch, Philipp M.; Löbel, Lea-Maria; Lutscher, Philipp; Mader, Matthias; Madia, Joan E.; Malancu, Natalia; Maldonado, Luis; Marahrens, Helge; Martin, Nicole; Martinez, Paul; Mayerl, Jochen; Mayorga, Oscar J.; McDonnell, Robert; McManus, Patricia; McWagner, Kyle; Meeusen, Cecil
Publisher Information:	Royal Society (Great Britain)
Publication Year:	2025
Collection:	Digital.CSIC (Consejo Superior de Investigaciones Científicas / Spanish National Research Council)
Subject Terms:	Computational reproduction; Reliability; Replications; Social and behavioural sciences; Behavioural sciences
Description:	This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a 'transparent group' receiving original study and code or 'opaque group' receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and p-value cutoff), while the opaque group had less success (89.3%). Second-decimal place exact numerical reproductions were less common (76.9 and 48.1%). Qualitative investigation of the workflows revealed many causes of error, including mistakes and procedural variations. When curating mistakes, we still find that only the transparent group was reliably successful. Our findings imply a need for transparency, but also more. Institutional checks and less subjective difficulty for researchers 'doing reproduction' would help, implying a need for better training. We also urge increased awareness of complexity in the research process and in 'push button' replications. ; This research was funded by a grant from the German Science Foundation – Deutsche Forschungsgemeinschaft (DFG) – Project Number 464546557, and from the Chilean National Research and Development Agency – Agencia Nacional de Investigación y Desarrollo – ANID/FONDAP/1523A0005 ; Peer reviewed
Document Type:	article in journal/newspaper
File Description:	application/pdf
Language:	English
Relation:	Royal Society open science; https://doi.org/10.1098/rsos.241038; Sí; Royal Society Open Science 12(3): 241038 (2025); https://hdl.handle.net/10261/395835; http://dx.doi.org/10.13039/501100001659; https://api.elsevier.com/content/abstract/scopus_id/105001207223
DOI:	10.1098/rsos.241038
DOI:	10.13039/501100001659
Availability:	https://hdl.handle.net/10261/395835; https://doi.org/10.1098/rsos.241038; https://doi.org/10.13039/501100001659; https://api.elsevier.com/content/abstract/scopus_id/105001207223
Rights:	info:eu-repo/semantics/openAccess
Accession Number:	edsbas.3A76AAC7
Database:	BASE