I. A PostScript on Medical Diagnosticism and Math-Mal.1

My favorite story that encapsulates what’s wrong with modern medical diagnosticism comes from a friend’s childhood, circa the mid-1960s. Evidently the family dog was getting on in years and had something wrong with him. No one in the family was quite certain exactly what ailed the dog, so my friend - being like most young boys who love their dogs - finally prevailed upon his dad to incur the certain costs and take the dog to the veterinarian. After the vet’s examination, he didn’t really have any answers beyond ruling out some obvious things. The vet began to go through a list of diagnostic tests he could run to look for this or that disease… and my friend’s clearly skeptical father waited for the doctor to finish and then asked (and I’m paraphrasing badly): “How about tests for conditions that you have a treatment for?” There was a long pause and then the doctor said something to the effect, “I think we’re done here.”2

In other words, a big chunk of the vet’s practice was charging people to do tests that might tell you what was wrong with your dog - or, as we learned in the last chapter, might not, or might give you a false positive or false negative - but in any case, the test wouldn’t change a thing for the poor doggoe because the vet had no treatment for the many diseases for which he had tests.

I humbly submit that this constitutes a big chunk of modern medical practice for both humans and pets. Go back and look at the list of 110 medical diagnostic tests from Harvard health’s website: the claim is not about cure, but about saving one from the “anxiety” of “uncertainty.” Western medicine has diagnostics for all manner of things that it can’t even treat, while knowing full well that (a) no diagnostic test has 100% sensitivity or specificity and will therefore produce some number of false positives and false negatives, and (b) 3 out of 4 Haahvaahd doctors can’t even do the math necessary to tell you what the probability is that a positive test means you actually have the disease.

None of this addresses the fact that roughly 80% of the medical problems AND costs currently crushing the US and western health systems is a result of all of the downstream harms that result from chronic disease, specifically from people’s overconsumption of refined carbohydrates - particularly the liquid ones - and a dearth of vigorous exercise. Diet and exercise is what needs to be fixed, but that don’t sell no pharmakoeia, nor make the stock price go up, nor pay back medical school debt, so… here we are.3 We have unsustainable rates of: obesity in children, Type 2 diabetes in children, autism in children, and (not coincidentally) a “study” system of “science” based upon frequentist statistics that hides and obfuscates all of it. As CrossFit, Inc.’s second CEO, Jeff Cain, once quipped: “We don’t have a healthcare system, we have a disease economy… and an outbreak of fitness and wellness could collapse the whole thing.”

II. Sally Clark Meets “What-Are-the-Odds-ism” in Law.

Sally Clark (née Locker) was the only daughter of a distinguished police officer in the Wiltshire Constabulary. She married solicitor Steve Clark in 1990.4 After marriage to Steve, Sally took up the profession herself, completing law school and training at Macfarlanes, a firm in London. By 1994, the couple worked at a firm in Manchester and purchased a house in Wilmslow. They had a baby, Christopher, born on September 22, 1996.

Tragedy struck ~12 weeks later on December 13. Sally Clark noticed something wasn’t right after she put the baby to bed; Steve was at an office party at the time. The infant lost consciousness, prompting an hysterical Sally to call emergency services. The baby was rushed to the hospital via ambulance, but pronounced dead not long after arrival. A post-mortem examination deemed Christopher’s to be the result of a lower respiratory tract infection and likely “SIDS” (to Americans) - “cot death” to British.

Like many women and/or couples who either miscarry or lose an infant, the Clarks decided that “the best bereavement therapy was another baby.”5 Baby Harry was born several weeks premature. Eight weeks after his birth, he collapsed in his high chair and died.

In light of this second death, law enforcement and their associated medical experts conducted a “re-review” of their previous work on Christopher, decided that Harry’s death was the result of “shaken baby” syndrome, reinterpreted some of their prior findings, and then began criminal processes against the Clarks, focusing on Sally.

• At her original trial in 1999, she was convicted, 10-2, and then the Clarks filed her unsuccessful (first) appeal in 2000.

• The public outcry by many, including the Royal Statistical Society, eventually led to a Crown review of her conviction;

• her subsequent successful (second) appeal in 2003 quashed her conviction and released her from prison.

• Sally Clark died a few short years later from alcohol poisoning, undoubtedly the byproduct of this whole saga.

Some prefatory remarks regarding the case of Sally Clark and her two infant sons are in order.

III. If You Won’t Offer It to Your Enemy, It’s Not a Right.

What do I mean? Until you have been on the wrong end of a serious civil or criminal accusation and had to stand against all of the machinery that the State can bring to bear down upon you, any opinion on how guilt or innocence gets doled out is woefully under-informed. Notwithstanding how wonderfully DAs and cops are portrayed by Dick Wolfe in “Law and Order” spinoffs or other police procedural TV-shows, the criminal justice system is not, I am sorry to say, staffed by saintly public servants who care only for Justice, meted out free of any personal biases or political concerns. As I noted in a previous piece, government forensic labs alone are a never-ending source of later-overturned convictions because of all manner of scientific misconduct and fuckery,6 without even accounting for testilying by police or undercover informants, or simple human error in the acquisition, storage, processing, and analysis of evidence.

When I was a more senior, supervising attorney within the Marine Corps’ defense bar structure, and my daughters were old enough to ask pointed questions, the “how do you defend guilty people?” question got dropped while driving somewhere. The answer has not changed in 25 years: Defending guilty people is easy. You give them the best possible defense you can within the bounds of the law and, if at the end of the trial they’re convicted, you can sleep well knowing (a) that you’ve done your best for them, and (b) that the system worked as intended… i.e. a guilty person is going to jail. Defending innocent people, however, is really hard. What if it turns out you’re not even a good enough attorney to get an innocent man acquitted? And how do you sleep comfortably - or look yourself in the mirror each morning - knowing that an innocent person is now sitting in prison?

No society that wants to call itself Just can survive for long with a justice system that routinely and/or randomly convicts innocent people. If you can be snatched out of your bed on a whim and incarcerated with no opportunity to defend yourself then there is no security and we’re not far from “public safety committees” of masked vigilantes. The Anglo-American legal system is predicated upon the plausible reasoning mechanism baked into all of us, and the system sets the boundaries on some types evidence and what may be inferred from the evidence upon those same considerations (by other names). In the modern world, this includes scientific evidence, which brings us back around to Sally Clark’s case and how the doctors and lawyers walked right into the Prosecutor’s Fallacy.

IV. The Prosecutor’s Fallacy…?

In PR 8, we spent time going over conditional probability as it relates to medical diagnostic tests and what the results actually mean, using Bayes’ Theorem to ensure that our doctor isn’t telling us we’re going to die when, in fact, we have a very low probability of having the disease, even if we tested positive for it, because of the low prevalence of the disease in the population. At the heart of the Prosecutor’s Fallacy is the simple, yet utterly-seductive-because-it-is-sometimes-right-in-specific-classes serpent of errors known as the transposed conditional: it is confusing the Pr (A | B) with the Pr (B | A) and thinking they’re the same. This seems silly when written this way, but it is as simple as mistaking the values of these two statements: “The probability that it’s raining if there are clouds in the sky” for “The probability that there are clouds in the sky if it’s raining.” In the first statement, you get many cloudy days where it doesn’t actually rain, however, in the second statement statement, it’s a very different result: on days where it’s raining, you have clouds virtually every time (allowing for the odd sun shower where you can’t see the cloud that produces the moisture).7 The point here is that depending upon the relationship of the specific A and B, it can seem quite reasonable to treat one probability, Pr (A | B), like the other, Pr (B | A), or to confuse what each of these probabilities means - and how their probabilities are not absolute values, but always relative to other evidence present in the case.

V. Fallacies All the Way Down

A (seemingly) Bayesian analysis of the problems with the Crown’s statistics has already been done in many places on the internet, but I’m going to take the time to pick on one here from Cornell, not for any particular flaw, but because I think I think this kind of analysis, while seemingly well-intended, misses the point - and indeed even helps perpetuate a deeper error. The problem is that equations seem to exert a kind of magical thrall over the human mind and very quickly transform from being useful rubrics for understanding conditional probability into sacred oracles declaring innocence or guilt by percentage points.

Let me be clear up front: there is no such thing in the world as “the probability of [anyone’s] innocence.” That is a mental construct, a model, and it (perhaps unintentionally) buries reality by overgeneralizing what actually happened in the Sally Clark case, and actually happens in any case. Anyone thinking or purporting to be able to frame the complexities and interplay between the various pieces of evidence in a trial in an equation is engaged in criminal oversimplification or a bad case of reification - of thinking that probability is ontological - that it inheress in dice or coins or nature.

Let me explain: during any trial, some evidence is not disputed by either side. Events occurred in the past and are fixed - two infant boys roughly 10-12 weeks old died - and in many ways those agreed-upon evidentiary propositions set a baseline frame for all of the other evidence that comes in.8 This is the landscape upon which the competing sides will paint their additional propositions, we can call them A', B', C', D', etc., which will be contested by the other side and countered by their evidence and witnesses, A", B", C", etc. All of this evidence - these propositions - will have various levels of credibility assigned to them by the jury, and there will be contingent relationships among these propositions. For example, A, A', and A" may all have plausibility relationships to one another, as well as conditional relevance to other evidence, such as G", K', and M, and on and on in a complex web that is held together by all of the narrative rules that govern our experiences and evidentiary rules provided by Her Honor during instructions. Some pieces of evidence/propositions can only be used for certain purposes - impeachment - but not for their underlying ‘truth’. Other pieces include “expert opinion” or “scientific evidence,” which is usually regarding the state of other evidence, itself circumstantial to proving actual “guilt”.

All of THAT can absolutely NOT be reduced to a single-variable equation.

Watch how the Cornell paper’s author - in attempting to use Bayes’ Theorem to “debunk” the Crown’s statistician - becomes increasingly consumed by the equation, and stuffing numbers into it, convinced he’s calculating Sally Clark’s “chances of innocence.”

According to the Confidential Enquiry for Stillbirths and Deaths in Infancy (CESDI, an authoritative and detailed study of deaths of babies in five regions of England between 1993 and 1996), there is about 1 in 1303 [chance of a] baby dying a cot death. The chance is reduced to 1/8500 if the baby lives in a relatively wealthy, nonsmoking family and with a mother over 26, which fits the Clark’s case. The “expert” at the court against Clark assumed there is no link between cot death of siblings by squaring 1/8500. However, the siblings of children died cot death is 10 and 22 times more likely to die in similar way than average kids. The chances of a second cot death in the same family could be between 1 in 60 and 1 in 130.

Therefore, how to calculate the chance that Clark was innocent, which in other words the chance her children died of cot death. The equation is written as the following:

H refers to cot death, and D means the baby death.

As we mentioned, the chance two babied died of cot death could be high as 1/130, and the number 1/73 million the testimony given was impossible. In this case we lower 1/130 to 1/100,000, still a very rare chance that essentially no much different from the impossible 1/73 million. P (D|H), chance of baby died given cot death is 1. A, referring to the alternate hypothesis that the children did not die of cot death (all other possibilities: for example someone else murdered both children, or Sally Clark murdered one of them etc., is equal to . The Home Office statistic gives that fewer than 30 children are murdered by their mother each year in England and Wales where 650,000 are born each year. 30/650,000=0.000046. Since the chance two siblings are murdered is much more rare than single murder, we should use number much smaller than 0.000046, but in this case, we would just overestimate, using 0.0000046—number 10 times smaller. So, the chance Clark is innocent is that

I’m not trying to pick on the author, but he seems to seriously believe that he can calculate the “odds” are that she’s innocent with a formula - this one, specifically.

I’ll say it again for those in the back: there is no “probability of innocence.” This is something that exists entirely in the head of whoever wrote it.

The mistake in reasoning and understanding of the debunker is that notwithstanding his use of Bayes’ equation, he’s still a frequentist philosophically, even while using Bayes’ Theorem. He begins his analysis by citation to an official “enquiry” into cot deaths - “an authoritative and detailed study” - and then asserts that “there is about 1 in 1303 [chance of a] baby dying a cot death. The chance is reduced to 1/8500 if the baby lives in a relatively wealthy, nonsmoking family and with a mother over 26, which fits the Clark’s case.”

WHOOP! WHOOP! WHOOP! Transposed conditional alert!

It is not “chance” that is causing 1 in 1303 babies to die. The “1 in 1303 babies dies of cot death” is an incidence rate, a frequency is what Jaynes would call it. It is not a causal relationship. Likewise, the fact that the frequency is lower for “relatively wealthy, nonsmoking family with a mother over 26” (the 1/8500 number) is also not a causal relationship. “Chance” isn’t magically “reduced” when women have their 26th birthday, nor when a family’s income reaches a certain median income. In the same way that die don’t acquire 1-in-6-ness by virtue of a statistician using a limit equation to show that the die will likely asymptote to 1-in-6 over an infinite number of roles under certain assumed conditions.

What the incidence rate suggests scientifically - by inverse correlation - is that there is something about that “> 26 rich family mommy” cohort that is different than (presumably) general populace and that different thing - whatever it is - causes the >26 cohort to have a much lower incidence (or frequency, or prevalence)… but the numbers alone prove nothing at all. You can draw no causal connection, which makes the entire calculation - using prevalence rates to determine Pr (Innocence | Evidence) or Pr (Guilt | Evidence) worse than useless: because the person using these numbers actually believes they’re proving something about the guilt or innocence of the person being considered.

To make this more stark: if being over 26 and wealthy leads to lower frequency of cot death, let us suppose that the same “authoritative study” data also showed the mirror image: being a young (m < 21), poor mother, of certain dark-skinned minority was correlated with much higher incidences of “cot death” - say 1/400, 3 times more likely than the average. Under the Cornell author’s logic and analysis this would necessarily make those mothers… ahem… less mathematically innocent. But contrariwise, if left-handed, blue-eyed mothers had a much, much lower incidence of cot death - 1 in 30,000, well, I mean, practically a lock to be innocent, right?

Do you see how seductive frequentists statistics can be, and why frequentist statistics were so closely associated with, and used by, eugenicists like Fisher?

These are all from the “Spurious Correlation Department” or the “Correlation ≠ Causation Department” and that is the sub silentio siren’s song of frequentist statistics and “significance” testing. Sally Clark’s guilt or innocence had - and has - nothing at all to do with the incidence rates of cot deaths among the British public between 1993-1996, nor does her guilt or innocence depend upon her being > 26 and relatively well off. She doesn’t magically become more or less likely to have committed the crime because of patterns that we can see in a dataset, of which she happens to possess some of the same characteristics of the people in that prior “study.”

I’ve been raving throughout these pieces about Polya’s work in helping to provide such a thorough qualitative analysis of various forms of human pattern recognition, but here is where it runs into problems: our pattern recognition systems are so good, but so obscured from our conscious thought processes, that we find correlations even where there are none. We find signal even in noise. Not me, you say?

And that’s why DNA evidence about “1 in many millions chance of the blood type matching some random person” kinds of evidence can be useful, but has not been the boon for prosecutor’s that was envisioned when it first began appearing. DNA testing is much better at ruling out suspects than it is at conclusively proving someone committed a particular crime… but sometimes it does.

Evidence isn’t more or less plausible or persuasive because it comes from an expert or has numbers attached to it; it’s more or less plausible only in the context of all of the other evidence surrounding it. I previously used the example of having “a dead body in your living room” and how the propositions that follow give it context that makes the difference between a grieving family and a murder scene. DNA evidence isn’t a special or magical detection technology and using Bayes’ Theorem doesn’t cure all logical fallacies.

All of which is another way of saying something that Professor Briggs suggested I emphasize and here is a great place for it: all probability is conditional on the evidence used to form it. All propositions and probabilities, A, have some given X, upon which they are contingent and based. Pr (A | X) - always. There is simply no Pr (A | ) with nothing after the | - something is always there.

Final related note: “cot death” or SIDS is the name we give to our ignorance about the cause of some number of otherwise (seemingly) healthy babies being reported tragically dead by their parents soon after death (typically). In Sally Clark’s case, and in others like it, one might wonder if we should consider all possible causes. Sally Clark’s father has the following on the front of the website for his daughter:

The only common factor, which may or may not be relevant, is that both deaths followed shortly after vaccination. [There is research into the possibility that these vaccinations can cause death within a few days if infants have certain genetic defects.]

Huh. Interesting that vaccines are known to have adverse effects on some portion of recipients. Indeed, the entire vaccine injury compensation program in the United States is premised on the manufacturers needing to be afforded special liability protection because of the known adverse effects that will occur as a result of mass inoculation. And yet none of that has ever been mentioned in the context of “Cot death” or SIDS. But of course, the most expertists of experts assure us that SIDS couldn’t possibly be caused by vaccines because… frequentist statistics say so.