Bernoulli's Fallacy: Statistical Illogic and the Crisis of Modern Science

by Aubrey Clayton

There is a logical flaw in the statistical methods used across experimental science. This fault is not a minor academic quibble: it underlies a reproducibility crisis now threatening entire disciplines. In an increasingly statistics-reliant society, this same deeply rooted error shapes decisions in medicine, law, and public policy with profound consequences. The foundation of the problem is a misunderstanding of probability and its role in making inferences from observations.

Aubrey Clayton traces the history of how statistics went astray, beginning with the groundbreaking work of the seventeenth-century mathematician Jacob Bernoulli and winding through gambling, astronomy, and genetics. Clayton recounts the feuds among rival schools of statistics, exploring the surprisingly human problems that gave rise to the discipline and the all-too-human shortcomings that derailed it. He highlights how influential nineteenth- and twentieth-century figures developed a statistical methodology they claimed was purely objective in order to silence critics of their political agendas, including eugenics.

Clayton provides a clear account of the mathematics and logic of probability, conveying complex concepts accessibly for readers interested in the statistical methods that frame our understanding of the world. He contends that we need to take a Bayesian approach—that is, to incorporate prior knowledge when reasoning with incomplete information—in order to resolve the crisis. Ranging across math, philosophy, and culture, Bernoulli’s Fallacy explains why something has gone wrong with how we use data—and how to fix it.

• ISBN10: 0231199945
• ISBN13: 9780231199940
• Publish Date: 3 August 2021
• Publish Status: Active
• Publish Country: US
• Imprint: Columbia University Press
• Format: Hardcover
• Pages: 368
• Language: English

Reviews

[annieb123]

Originally posted on my blog: Nonstop Reader.

Bernoulli's Fallacy is an expository academic comparison of the statistical methods and accepted methodologies used by modern empirical scientists, analyzed and presented by Dr. Aubrey Clayton. Released 3rd Aug 2021 by Columbia University Press, it's 368 pages and is available in hardcover, audio, and ebook formats.

This is an esoteric book with urgent, potentially catastrophic, foundational implications for science (and society). The way we interpret, group, and present data has fundamental connections to what we see as "objective truth" and "facts". This is especially frightening when considered in the light of recent crises such as systemic racism, alleged election/voting fraud, and pandemic/public health methodology and data.

This is a deep dive into the subject material and will require a solid background in mathematics and statistical methodology at the very least. I have a couple degrees in engineering sciences (and a real love for bioinformatics), and it was significantly above my pay grade. I could understand much, but by no means all, of the author's exposition and there were tantalizing glimpses of deeper information which I simply couldn't grasp. Readers should expect to expend some effort here to even make an informed decision on the veracity of the author's claims.

It's an academic book, the author is an academic, and it reads very much like an academic treatise. The language isn't *quite* as impenetrable as many academic volumes. The text is well annotated throughout and the annotations will make for many hours of background reading enjoyment. I get the distinct impression that the author has made a herculean effort to use accessible language to make it more easily understood, but there is a basic level of understanding which will render it inaccessible to many readers. That being said, the author writes with style and humor and tries to make the read minimally pedantic. I can well imagine that he's a talented and popular lecturer.

At the end of the day, Disraeli wasn't wrong when he decried "lies, damned lies, and statistics". I am not strong enough in this particular field of study to say where on the above spectrum Dr. Clayton's exposition falls.

Five stars (readers should keep in mind that the subject will require significant effort). I would enthusiastically recommend that people in education and policy expend the necessary effort. It would be a good selection for public/university library acquisition, as well as for more academic settings in philosophy of mathematics and science and allied fields of study.

Disclosure: I received an ARC at no cost from the author/publisher for review purposes.

[Jeff Sexton]

Lies, Damn Lies, and Statistics. On the one hand, if this text is true, the words often attributed to Mark Twain have likely never been more true. If this text is true, you can effectively toss out any and all probaballistic claims you've ever heard. Which means virtually everything about any social science (psychology, sociology, etc). The vast bulk of climate science. Indeed, most anything that cannot be repeatedly accurately measured in verifiable ways is pretty much *gone*. On the other, the claims herein could be seen as constituting yet another battle in yet another Ivory Tower world with little real-world implications at all. Indeed, one section in particular - where the author imagines a super computer trained in the ways of the opposing camp and an unknowing statistics student - could be argued as being little more than a straight up straw man attack. And it is these very points - regarding the possibility of this being little more than an Ivory Tower battle and the seeming straw man - that form part of the reasoning for the star deduction. The other two points are these: 1) Lack of bibliography. As the text repeatedly and painfully makes the point of astounding claims requiring astounding proof, the fact that this bibliography is only about 10% of this (advance reader copy, so potentially fixable before publication) copy is quite remarkable. Particularly when considering that other science books this reader has read within the last few weeks have made far less astounding claims and yet had much lengthier bibliographies. 2) There isn't a way around this one: This is one *dense* book. I fully cop to not being able to follow *all* of the math, but the explanations seem reasonable themselves. This is simply an extremely dense book that someone that hasn't had at least Statistics 1 in college likely won't be able to follow at all, even as it not only proposes new systems of statistics but also follows the historical development of statistics and statistical thinking. And it is based, largely, on a paper that came out roughly when this reader was indeed *in* said Statistics 1 class in college - 2003. As to the actual mathematical arguments presented here and their validity, this reader will simply note that he has but a Bachelor of Science in Computer Science - and thus at least *some* knowledge of the field, but isn't anywhere near being able to confirm or refute someone possessing a PhD in some Statistics-adjacent field. But as someone who reads many books across many genres and disciplines, the overall points made in this one... well, go back to the beginning of the review. If true, they are indeed earth quaking if not shattering. But one could easily see them to just as likely be just another academic war. In the end, this is a book that is indeed recommended, though one may wish to assess their own mathematical and statistical knowledge before attempting to read this polemic.