Sources
Every primary source behind this anthology, the code behind its numbers, and the search terms the kickers keep out on purpose.
The Forking Paths
The exhibit's own Debrief names who caught this and states one honest caveat. Here is the full record behind both.
-
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366.
doi.org/10.1177/0956797611417632 -
Gelman, A., & Loken, E. (2013). The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No "Fishing Expedition" or "p-Hacking" and the Research Hypothesis Was Posited Ahead of Time. Unpublished manuscript, Department of Statistics, Columbia University. No journal, volume, or DOI: it has never been formally published.
sites.stat.columbia.edu/gelman/research/unpublished/p_hacking.pdf -
Gelman, A., & Loken, E. (2014). The Statistical Crisis in Science. American Scientist, 102(6), 460. The peer-reviewed version of the same argument, in plainer language.
americanscientist.org/article/the-statistical-crisis-in-science -
Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. A modelling result under stated assumptions, not an audit of the literature. Goodman & Greenland's peer-reviewed critique (PLoS Medicine, 2007, 4(4), e168) argues the model cannot prove most published claims are false; Ioannidis replied the same year.
doi.org/10.1371/journal.pmed.0020124 -
Data Colada. The research-credibility blog run by the same three authors, publishing since 2013.
datacolada.org
The kicker above says manufactured significance, not p-hacking. The term is common, but its own originators are wary of it. Simmons, Nelson and Simonsohn call it researcher degrees of freedom and never write p-hacking themselves. Gelman and Loken, who named the garden of forking paths, later wrote that they regret how fast fishing and p-hacking spread, because both words imply someone did this on purpose. Multiple comparisons is the older, broader statistical term for the same family of problem. The concept belongs to the researchers above; none of them coined the word.
The mechanic, choosing an analysis by hand and watching the p-value move, adapts FiveThirtyEight's Hack Your Way to Scientific Glory (Christie Aschwanden & Ritchie King, 2015). Their working example was partisan; this exhibit keeps the interaction and swaps in Simmons, Nelson and Simonsohn's own study instead.
The planes that came back
The hub's closing panel separates the Wald story from the Wald record. Its sources:
-
Wald, A. (1980). A Reprint of "A Method of Estimating Plane Vulnerability Based on Damage of Survivors." CRC 432. Center for Naval Analyses, reprinting the 1943 memoranda. No diagram anywhere in it: it is pure mathematics and numeric tables.
cna.org/analyses/1980/a-method-of-estimating-plane-vulnerability -
Mangel, M., & Samaniego, F. J. (1984). Abraham Wald's Work on Aircraft Survivability, with comment by J. O. Berger and rejoinder. Journal of the American Statistical Association, 79(386), 259–271. The worked example on the hub reaches us through this rejoinder.
doi.org/10.1080/01621459.1984.10478038 -
Casselman, B. (2016). The Legend of Abraham Wald. AMS Feature Column, June 2016. American Mathematical Society. The source for what does and does not survive in the record, including the "no diagram" finding above.
ams.org/publicoutreach/feature-column/fc-2016-06
Illusions 2 to 4
The Reversal, The Positive and The Missing Planes are prose in the essay for now. Their sources land here the day each one ships.
The code behind these numbers
No dataset and no server sit behind any figure on this site, including the ones above. Every number is a few dozen constants and about a hundred and thirty lines of statistics, run in your browser while you read, and all of it is public.