But what I can do — rather than offer a range of uncertain forecasts — is assess the range of plausible outcomes to this conflict and consider how each will affect countries and people in different ways. There’s no scientifically accurate way of eliminating uncertainty; so let’s embrace it. This is not a trivial exercise: I can recall only a few times in my own life when the range of possible geopolitical and economic outcomes, good or bad, has been as wild as it is today.

With his “known-unknowns” and “unknown-unknowns”, Donald Rumsfeld has actually provided us with a useful starting place to assess outcomes. The unknown-unknowns are often more important, but they are beyond our reach. “Whereof one cannot speak,” said Wittgenstein, “thereof one must be silent”. These are what economist Frank Knight was referring to when he coined the notion of “unquantifiable uncertainty”, or true uncertainty, as distinct from outcomes that carry a small probability.

So while little of value can be said about the unknown-unknowns, known-unknowns are generally worthy of deeper reflection. We all know about the truly bad scenarios stemming from the war. Airlines may run out of kerosene in six weeks; the world could run out of fertilizer and basic chemicals if the blockade continues for much longer. We know that Asia would be hit the hardest, followed by Europe, with the US affected least.

There are also potentially good outcomes. Europe has been offered a reminder that its investments in energy independence are worthwhile — and this includes both nuclear and renewables. The European Greens were mad to oppose nuclear power, but they were right about renewables, if only from the perspective of energy independence.

Beyond the war, which should end eventually, there are other positive economic scenarios coming down the track. The most interesting part of the latest World Economic Outlook by the International Monetary Fund was not the central forecast, the risk of global recession: but, rather, the wide range of possible scenarios it also weighed up. We truly live in a world of Knightian uncertainty.

More from this author

Trump’s gamble with the West’s future

By Wolfgang Munchau

I’m talking here about plausible scenarios, those that have a reasonable chance of occurring: the known-unknowns. Of particular interest are those scenarios at the extreme ends of the spectrum. Some middle-of-the-road scenarios are plausible, of course: economic forecasters always use them for their forecasts. But they have no higher probability of occurring than the good or bad ones. What you really want is information about the spread of outcomes, not the average.

If you want to understand uncertainty, after all, the mean is your enemy. Politicians should care about the voter with the median income; the voter who, when income is ranked from poorest to richest, is sitting in the middle. He would be a different person from the man with the average income, because the super-rich distort the picture. It’s the “Bill Gates walks into the pub” problem, where a single billionaire raises the average income of a group.

There’s an old joke about the statistician who died crossing a river that was three foot deep on average. Real statisticians, of course, know about this stuff. The people most at risk of such misjudgments are those with a Statistics 101 background, who have been taught some powerful techniques they then unleash on an unsuspecting public: economists, journalists, doctors, epidemiologists and unfortunately also climate scientists.

One of the most catastrophic uses of averages is represented by the communication strategy of climate scientists, when, 10 years ago this week, they set a target to keep the increase in average global temperature to below 2°C. But what kills us is not the average temperature — the 2°C extra — but the extremes: floods, heat, forest fires.

These are the metrics the climate community should have communicated. When you’re told, after all, that the temperature is going to rise by a few degrees, you get the deckchairs out. Indeed, I heard one well-known political commentator claiming that it might be nice for the weather to be a little warmer in Northern Europe. When, on the other hand, you receive a flood warning, you get the sandbags out.

In short, how we communicate risk determines how we act, and misusing statistics inevitably leads to misjudgments.

So rather than making sweeping average claims about the future, whether about the climate or war with Iran, a better way of thinking about our planet in the medium term is by reflecting on how different plausible scenarios could affect regions in different ways.

The most important positive scenario we might consider is the AI-led productivity boom. It has not happened yet — but there are data points suggesting that it may be starting to happen. If it does, it would benefit the US a lot more than the rest of the world. After all, America invented the technology; Silicon Valley’s foundational AI models are ahead of everyone else’s; and the US’s AI infrastructure is far and away the global leader. China, for its part, is next in line. The People’s Republic may not have invented AI: but it was easily the most successful second mover.

Unsurprisingly, Europe will gain least from the AI revolution. The EU is still in data-protection mode, as much a function of its cultural stance as individual government policies. From LLMs to driverless vehicles, indeed, the Europeans have decoupled from most of our century’s cutting-edge technologies. Pharmaceuticals is the one important exception, but here too China is catching up fast. China also is ahead on renewable investments, but here, at least, the Europeans do better than the Americans.

When it comes to bad outcomes, then, Europe is at greater risk than its rivals. And I don’t just mean technologically. There is a plausible scenario that the war in Ukraine could spread, for instance to the Baltic states, and the US would almost certainly not come to Europe’s rescue. Donald Trump’s line about how the Europeans “weren’t there for us” when the US attacked Iran will keep ringing in our ears as the scenario of a European war grows — in my opinion, a non-trivial tail risk.

Similarly, the Iran ceasefire could break down and the violence spread across the Middle East. Yet while there’s a lot of uncertainty about all these scenarios, we can say with some confidence that there is no realistic scenario of war hitting America’s shores.

More from this author

Europe will lose the AI century

By Wolfgang Munchau

Absent any definitive forecasts, from this brief analysis, I can make some sharp observations. First: the US is clearly in a better position than everyone else. Second: the US is more likely to benefit from any upside scenario, and less likely to be hit by a downside one. I’d therefore warn against those heralding the end of US global dominance, let alone the end of Trump’s presidency. This is mostly wishful thinking. The President has certainly made some foolish decisions, but the US is still at the head of the pack.

Things don’t look so rosy for Europe: it will not benefit from a global productivity boom and will be disproportionately exposed to the bad scenarios. And in those areas where it is strong, it is under attack by China. East and South East Asia sit somewhere in between the two polar opposites — beneficiaries of 21st-century technologies, but not as much as the US. But even though Asia is at greater risk of disruption from the blockade of the Strait of Hormuz, it is still more resilient to shocks than Europe.

Whatever happens, one single forecast, or even a focus on two or three scenarios, can’t hope to capture the deep asymmetry of the uncertainty we’re currently facing.

But people who embrace uncertainty don’t think in terms of baselines. They don’t look for polls or projections. Nor do they seek to control that uncertainty. They seek to learn from it. Perhaps we could all learn from them. Forecasts are for losers, and once you’re comfortable admitting that you just don’t know — you might end up seeing things more clearly.

Though rarely expressed so honestly, these are sentiments that leaders around the world could sympathize with. Most governments claim to be answerable to their people, yet if they rely on debt to fund their activities, they will find their sovereignty constrained by analysts sitting in distant financial districts. James Carville, the outspoken former adviser to Bill Clinton, famously quipped that he would like to be reincarnated as the bond market, since “you can intimidate everybody”.

When Labour entered power last year, Chancellor Rachel Reeves faced the same problem that bedeviled Erdoğan and Carville: how to reassure the bond markets that they could keep lending at reasonable rates. Her answer was to commit to a set of “fiscal rules”, pledging that government debt will be falling by the end of the parliament. This was not an original strategy: Reeves was merely adapting what has become a convention for British chancellors since George Osborne created the framework for austerity in 2010. But during Reeves’ hapless tenure, it has become clear that the mantra of fiscal responsibility conceals another tendency: government by metrics.

Bereft of imagination in a rapidly changing world, our political class clings ever more tightly to numerical targets and indicators, hoping to convey an impression of competence and stability. Instead, they have only reinforced the sense that we are ruled by management consultants rather than leaders. Nor have metrics provided the bulwark against chaos that they promised; for the numbers, it turns out, do not always show what people think they show.

An over-reliance on statistical models and measurements comes with two great dangers. The first is that the seemingly objective quality of numbers can seduce us into believing that they present an infallible picture of reality. In fact, data are gathered and modeled by institutions with all the shortcomings and flaws that one would expect of a human undertaking. The second problem is that our behavior is increasingly determined by what we can measure — or think we can measure — rather than what is actually important or desirable.

Both problems are abundantly illustrated by British politics today. When it comes to her fiscal rules, Reeves is essentially relying on economic forecasts from the Office for Budget Responsibility, an independent body, to tell her how much money she has to spend. The government ties its own hands, inviting the OBR’s economists to rate its finances before the markets get a chance to. The problem is that these forecasts are imprecise, and regularly revised as the various metrics change. Consequently, Reeves’ chancellorship has been an endless drama of budgetary “black holes” and vanishing “fiscal headroom”, as she scrambles to tailor her policies to the shifting estimates. Measuring has displaced thinking, to the point where the government’s decisions are being dictated by a numerical target rather than an actual plan for how to match the public finances to the nation’s needs.

In July, for instance, Reeves tried to tackle the problem of runaway welfare spending. Instead of setting out a strategy for reforming a fiendishly complex and thoroughly broken system of benefits — a problem whose roots go deep into British society ­— she set about salami-slicing various budgets to satisfy the fiscal rules. This so outraged Labour backbenchers that Reeves and Keir Starmer had to abandon their cuts in any case.

That same month, as if to emphasize the futility of the government’s approach, the OBR suggested it would be reconsidering some of its forecasts, which were too optimistic. The adjustment will likely force Reeves to raise billions of pounds in tax at her upcoming Budget. As the blogger M.F. Robbins has aptly put it, “you cannot micromanage policy based on statistical noise. The more you try to do this, the more you’ll find that, in fact, the noise is managing you.” The Chancellor is now trapped between a bond market waiting to see if she will keep her promises, and a party that wants her to abandon them. Last week, former transport secretary Louise Haigh published a frontal assault on the Chancellor’s strategy, demanding a less prominent role for the OBR in government. A group of 40-odd Labour MPs in Red Wall seats has previously expressed similar frustrations.

Labour’s love of metrics does not end with fiscal rules. Hoping to show a firm, technocratic grip on the nation’s problems, Starmer’s government has issued a blizzard of “missions” and “milestones” based on quantifiable targets. It wants 75% of five-year-olds to reach “a good level of development”. It aims to recruit 13,000 neighborhood police officers and 6,500 new teachers; to build 1.5 million homes and to achieve 95% clean electricity. The list goes on. At one point, the government even proposed a “digital dashboard”, so that the public could track its progress towards these goals on their smartphones.

All of this assumes that we want to treat politics like a stock market, assessing the fundamentals of a government before choosing to invest. We measure and rate everything else, so why not our politicians? One reason is that voters have an intuitive understanding of “Campbell’s Law”, which observes that a measure which becomes a target will likely come under “corruption pressures” — that is, it will be fudged or misrepresented to appear more impressive. More importantly, politics, like human experience, is fundamentally qualitative. The number of new houses does not tell us whether they are decent homes in neighborhoods where people actually want to live, just as the number of new police officers and teachers does not tell us whether the criminal justice and education systems are working as we would like. These are ratings we make with our eyes and ears.

It’s tempting to think that numbers can build trust by helping the public to understand the work of government. But if statistical and subjective reality clash, it has the opposite effect. People do not like being told a graph proves that their perceptions are wrong. In 2024, supporters of then president Joe Biden foolishly tried to convince struggling voters that they were doing great, actually; had they not seen the economic metrics? Something similar is happening with law and order issues in Britain today, as data on falling crime rates fail to assuage anger at low-level lawlessness and police impotence. The Covid pandemic was perhaps the most ambitious attempt in history at doing politics via statistical models, and it was ultimately a disaster for public trust in institutions. People were initially willing to give up their freedoms to “bend the curve” of infections, but as time wore on, the experts who were meant to save the politicians from resentment were themselves contaminated by it.

It is true that our impressions diverge from the numbers in part because we are biased. Political partisanship has long influenced perceptions of the economy in the United States, and now the same appears to be happening in Britain too. Then again, sometimes the official statistics are just spectacularly wrong, as has consistently been the case with immigration. In 2004, Tony Blair’s government estimated that welcoming migrants from Eastern European countries to work in Britain would result in a maximum of 13,000 people arriving annually. Just four years later, the Home Office revealed that 750,000 had come already. Another discrepancy emerged when EU citizens were invited to apply for settled status after Brexit. In 2019, the government thought there were 3.7 million EU nationals in the country, yet closer to 6 million have asked to remain. Similar misjudgments led former prime minister Boris Johnson to relax visa restrictions in 2021, resulting in a surge of new arrivals.

More from this author

The NHS can’t fix your lifestyle

By Wessie du Toit

Such errors stemmed in part from a woefully inadequate method for collecting data. The International Passenger Survey, designed in the Sixties for studying tourism, involves little more than questioning people at random at airports and other border crossings. It uses a tiny sample of migrants, and misses out many patterns of movement altogether. But until other data sources were added in the last few years — and despite immigration becoming a persistent political issue — this was the British state’s sole way of measuring it.

At one point, it could be argued that migration was an outlier in a generally solid field of British statistics. Since the Covid pandemic, though, the country’s chief number-crunching body, the Office for National Statistics, has been mired in crisis. Reports have been delayed, revised, and retracted, and it has admitted that some of its most important metrics are no longer reliable. It is a portrait in miniature of British misgovernance. In 2007, the ONS lost 90% of its London-based staff — a devastating flight of expertise — when a misguided attempt at devolution saw the agency moved to Newport in southern Wales. Governments have subsequently cut its funding, which as a cost-saving exercise is a bit like trying to lose weight by gouging out an eye. But the most pressing problem has been that fewer and fewer people are taking ONS surveys, which seems partly to reflect an unwillingness to cooperate with the state, or for that matter to answer phone calls and open the door to strangers.

This erosion of statistical competence has far-reaching implications, given the range of public and private actors who use ONS data. Their uncertainty has knock-on effects for the affordability of mortgages, the value of British businesses and assets, and the quality of government policy. Political discourse will, meanwhile, become increasingly farcical, as politicians stake their authority on numbers that provide only a tenuous picture of reality, and we will criticize them using the same illusory figures. We are already familiar with the seismic importance given to every tenth of a percentage point of GDP growth, even though such changes are far too small to measure accurately, and would tell us little of value to our lives in any case. So we are unlikely to change tack just because the measurement has been declared unreliable as well.

A grasp of metrics is essential for governing a modern society; we are right to expect that the state will make the numbers add up. The fate of Liz Truss, a prime minister effectively toppled by the bond markets when her plans were found wanting, is a reminder of the ruthless will to measurement at the heart of capitalism today. Donald Trump can bully government agencies to confect more favorable economic data, as he appeared to do last month by firing the head of the Bureau of Labor Statistics, but he cannot force businesses and investors to believe that data. Still, politics requires an insight that goes beyond our fixation with the quantifiable. The numbers do not lie, but nor do they tell us everything. It was once widely believed that demographic change in the United States would hand the future to the Democrats, and that the size of the EU’s GDP and population meant it could easily propel Ukraine to victory against Russia. Those claims appear less certain now.

Likewise, Britain’s current regime shows that an obsession with metrics is ultimately a profound weakness, mistaking the means of statecraft with its ends. Starmer and Reeves’ project appears to be falling apart before it is even clear what that project was, teaching us that a government led by managerial bean counters can be as fragile and incoherent as one led by clowns. Leaders who confuse meeting targets with governing well will achieve neither.

Today, we live in a curious age of skepticism, not merely towards the authority of expertise, but also towards promises of social change. Unreasonable attachment to our own ideas often sits alongside an unwillingness to consider the arguments of others, even as we’re often cynical about their motives. In this context, Adam Kucharski asks an awkward question. What, he wonders, in his new book Proof: the Uncertain Science of Uncertainty, does it mean to be convinced of something? More to the point, what does it take to convince you — what kind of evidence, and what kind of reasoning?

Not everything can be proved with mathematics and logic. Well done, Euclid, for using proof by contradiction to show that, if two angles in a triangle are equal, the sides opposite the angles will be equal too. But does that help me decide if a defendant is guilty, whether to get vaccinated, or how likely a social media post is to be true? Unlikely.

More from this author

The trouble with online safetyism

By Timandra Harkness

Proof is not the argument-ending rubber stamp that internet pedants lust for. Standards of proof, in science, medicine, or the law, did not fall, ready-made, from heaven. Ideas like “beyond reasonable doubt” or “statistical significance” were developed by humans to answer important questions within the constraints of what they had to hand: evidence, people, technical resources and time.

Today, we take for granted that any new medical treatment will have been through a standard array of tests, including randomized controlled trials (RCTs), analyzed with statistical methods now mostly performed by computers. All these tools were invented by researchers and statisticians to answer specific questions. Do babies thrive better on cow’s milk or mother’s milk? Does streptomycin cure tuberculosis? Which type of barley makes the best Guinness?

Nature, like a guilty defendant, does not generally crack under interrogation and deliver a full confession. Measuring the effectiveness of a treatment, or identifying the factors that influence outcomes, is equally a job for detectives: piecing together evidence, eliminating misleading or irrelevant clues, and working out ways to test a hypothesis. Even then, researchers may conclude that one thing seems to cause another, but they don’t really know why.

Does this matter? After all, as Kucharski points out, we use very effective medical techniques, such as defibrillation, every day without understanding why they work. We board airplanes without worrying that mathematicians still can’t explain exactly how the flow of air over the wings keeps them aloft. At some point, we just accept the evidence of results — without understanding causes. That’s especially true when familiar statistical methods are turbocharged by AI and fed by industrial-scale data, meaning social as well as medical interventions are often based on this kind of correlation, rather than a clear understanding of causation.

Even leaving aside the real problem of unintended consequences, however, there’s a drawback to this approach. Observing correlation lets us predict what happens if we do more of the same. But understanding causality allows us to conjure something new. Kucharski cites the philosopher Judea Pearl’s “ladder of causality” — rising from association, through intervention, to counterfactuals — to imagine an alternative version of reality. What would happen if we did X instead of Y? And, perhaps more controversially, what would have happened, had we not done Y?

As a medical mathematician, involved in advising the UK government during Covid as part of SAGE, Kucharski was at the sharp end of this kind of “what if?” modeling. He has more practical experience than most would want in using incomplete data to build scenarios of what might happen in the future — or, rather, in different futures. For Kucharski, after all, variables included not only natural factors, like how contagious different Covid strains would turn out to be, but also how people would behave, and how different government policies might affect that behavior.

Decisions made on the basis of these models, as Kucharski and his colleagues were well aware, would affect millions of lives. There was no laboratory in which to test alternative policies, only mathematical models with many assumptions baked in. The stakes were high, time and information were limited, and public debate was highly politicized.

Kucharski is careful to distinguish factual disputes from mere policy disagreements. As he puts it, “deciding what is true and false is not the same as deciding what is socially right and wrong.” He rightly criticizes the disingenuous claim that governments were just “following the science” approvingly quoting Austin Bradford Hill, best known for discovering the link between smoking and lung cancer. “‘It was no part of our job to tell the public how to behave with regard to smoking…To become propagandists would ruin us as scientists and make us “biased” presenters of further material’.”

Still, given the profound disagreements that emerged about the nature of Covid, and how to prevent and cure it, you might expect Kucharski’s book, like Brecht’s poem, to attack those who “don’t believe” in the facts. You’d be wrong. After all, if we are aware of what it takes to be convinced that something is true, should we not also think about what it takes to convince others? We ourselves are not machines, absorbing data and outputting hypotheses. We weigh evidence and argument, testing new theories about the world against all our existing knowledge and assumptions. It’s reasonable to assume that others do the same.

Kucharski’s personal experience of vaccine-hesitant friends and family was that they had varying reasons for doubt. Some felt they didn’t know enough. Others were deeply skeptical of pharmaceutical companies in general. The author found he could only convince them that vaccination was a good idea by “taking the time to understand why they doubted, then finding ways to address these specific doubts.”

This should be obvious to everyone, in a pluralist, democratic society. But we live in a world where “proof by intimidation (‘the evidence is clear’)” is too often invoked by those who regard the public as beneath rational argument. Ironically, research shows this kind of approach is not effective — and indeed tends to backfire. “Public trust,” warns Kucharski, “is eroded as problems are brushed away with appeals to authority.”

This points to a deeper problem with the widespread attitude that evidence, or proof, is merely ammunition for propagandists of every stripe. Those on the receiving end lose faith, not just in specific pieces of evidence, but in the very methods and institutions they should be using to weigh and test the arguments.

Kucharski quotes Jules Henri Poincaré, the French mathematician and philosopher, writing in 1908: “To doubt everything or to believe everything are two equally convenient solutions; both dispense with the necessity of reflection.” To rephrase, healthy skepticism gives way to fatalistic cynicism, what Brecht calls “the doubt which is a form of despair”.

Despite its title, “In Praise of Doubt” also targets those who doubt “not in order to come to a decision but to avoid a decision”. Writing in 1939, exiled from Nazi Germany and disillusioned by Stalinist Russia, he points the finger not only at those who fail to ask questions, or who suppress dissent — but also at those who don’t want to face the reality of their situation, or take responsibility for action.

More from this author

The cynical hysteria around AI

By Timandra Harkness

Fair enough. Public discourse today too often falls into lazy sloganizing, personal character attacks, and sloppy invocation of highly selected or dubious facts. We should all expect better standards of reasoning and evidence, and in turn deploy better arguments when trying to convince others. But political argument is generally not susceptible to the kinds of proof in Kucharski’s book. Types of evidence and reasoning appropriate to a laboratory, a court of law, or a philosopher’s musings, can only go so far.

Abraham Lincoln, fascinated by Euclid’s methods of mathematical proof, used “proof by contradiction” to show that one person could not legitimately enslave another. His capacity for logical argument against his opponents probably helped his election as President, but it did not end the practice of slavery. In the end, it took more than evidence and proof to change that social reality. It took both doubt that the existing state of affairs was the best (or only) possible world — and positively imagining an alternative reality. Yet it also meant convincing others that change was possible. And that, as Kucharski says, “is not just about data or research; it is a matter of psychology, politics and prior beliefs.”

In the end, nobody can prove in advance that it’s possible to convince enough people to change their minds. At some point, you just have to act on the strength of your conviction.

A paediatrician, Roy Meadows, called as an expert witness for the prosecution, told the court that the probability of the two deaths happening by chance was one in 73 million: that is, 8,500 times 8,500.

As it happens, that’s not true. This calculation assumes that the deaths are entirely uncorrelated, but we know that SIDS can run in families and be affected by environmental conditions. If you have one case of SIDS in your household, while incredibly rare, you are more likely to have a second; the 73 million figure is orders of magnitude too high. But that wasn’t Meadows’s big mistake.

His big mistake was the following: he assumed that if the probability of the two deaths happening by chance was one in 73 million, then the probability that Sally Clark was innocent was one in 73 million as well.

But this is wrong. Crucially, catastrophically wrong. As wrong as assuming that because only one human in eight billion is the President of the United States, there’s only a one-in-eight-billion chance that the President of the United States is human.

Nonetheless, Meadows’s testimony helped convict Clark in 1999. She spent three years in jail before her conviction was overturned on appeal. Her life was, obviously, ruined. It will not surprise you to learn that she drank herself to death four years later, alone. It’s a haunting story.

The mistake made in Clark’s case is a subcategory of a far wider failure of reasoning, a failure I discuss in my new book, Everything is Predictable: How Bayes’ Remarkable Theorem Explains the World. But in legal circles it comes up again and again — often enough to have its own name: the “prosecutor’s fallacy”.

There was a recent article in the New Yorker about the nurse Lucy Letby, convicted of murdering seven babies in a neonatal ward. The online version is blocked in the UK, because of contempt-of-court laws, despite her appeal against the convictions being denied: she still faces a retrial on one count of attempted murder.

This piece is not about Letby. I do not know the facts of the Letby case and would not be allowed to write about them if I did; whether that is a strength or a weakness of British law, I leave to others to discuss. But I do know that courts, in the UK, US and elsewhere, have a bad history when it comes to the use of statistical evidence. To understand why, we need to go back to the work of an 18th-century nonconformist minister.

The Reverend Thomas Bayes’s eponymous theorem was published in a paper called “An Essay towards solving a Problem in the Doctrine of Chances” in 1763, five years after his death. Previous work in probability theory had answered the question: how likely am I to see some event, given a hypothesis? For instance, if we assume that my dice are fair, and I roll three of them, I can expect to see three sixes one time in 216. That’s called sampling probability.

Suggested reading

Bayes should be the face of the new £50 note

By Graeme Archer

But most of the time, with statistics, we want to answer the opposite question: how likely is my hypothesis to be true, given some new event? That’s called inferential probability, and it’s a completely different thing.

Say I go to the doctor’s and I get a cancer test. It’s quite an accurate test: if I have cancer, it will correctly say so 99 times out 100; if I don’t have cancer, it will correctly say so 99 times out of 100. If I get a positive result, then, what’s the chance that I have cancer? Is it 99%?

No. The answer is you don’t know. At least, not with the information I’ve given you.

Imagine that this particular cancer is rare: only one person in 1,000 has it. You test 100,000 people at random. Of that 100,000, about 100 will have the cancer, and your test will pick up 99 of them. Of the remaining 99,900, your test will correctly say that 98,901 are cancer-free.

But that means that it will incorrectly say that 999 people do have cancer when they don’t. So of your 100,000 tests, 1,098 will come back positive, and only 99 of them are true positives. If you are one of them, then there is just a 9% chance you have cancer.

You can’t answer the question “How likely am I to have cancer, given this test?” without first answering the question “How likely did I think I was to have cancer in the first place?” That was Bayes’s big insight. You need what is called a prior probability. If the cancer was less rare, then your positive test would be more worrying: if one person in every 100 had it, then a positive result would mean about a 50% chance you have the disease.

It’s counterintuitive and weird. What do you mean, this 99% accurate test result is almost certainly wrong? But it is mathematically unavoidable.

You can probably see the bearing this has on statistical evidence used in court. Take DNA tests, for instance: you might do a DNA test on a sample from a crime scene. It matches a result on your database. There’s only a one in 3 million chance that someone’s DNA would match the sample by chance. Does that mean there’s only a one-in-3-million chance that your suspect is innocent? As you’ll realise by now, no it does not.

It depends on your prior probability. If your database is a random sample of the British population, then the prior probability that any given person is the culprit is one in 65 million. If you tested the whole population, you’d get about 20 matches just by chance.

But if you are a modern-day Hercule Poirot, and you’re only testing 10 people trapped by a snowstorm in a country mansion, then your prior probability is one in 10, and the chance it’s a false positive is about one in 300,000.

Real cases have turned on these details: a man called Andrew Deen was convicted of rape in 1990 on the basis of DNA evidence that an expert witness said there was only a one in 3 million probability of a chance match. His conviction was overturned — although he was found guilty again in a retrial — because a statistician pointed out that “How likely is it that a person’s DNA would match the sample, if they are innocent?” and “How likely is it that someone is innocent, given that their DNA matches the sample?” are very different questions.

In Sally Clark’s case, the problem was not testing, but clustering: two rare events happening simultaneously. But, again, the probability of seeing those events by chance is not the same as the probability that she was guilty.

Hers is far from the only case in which the use of statistics has raised suspicion. In 2022, the Royal Statistical Society wrote a report on statistical evidence in criminal trials, and noted that one of the most common reasons that medical professionals are accused of murders is because “an apparently unusual number of deaths occurs among their patients”.

But these cases are doubly difficult to evaluate, the report noted, because “they involve at least two levels of uncertainty”. As well as the normal uncertainty over whether an individual committed a murder, there is uncertainty over whether any murders occurred at all.

The Dutch paediatric nurse Lucia de Berk was convicted, in two trials in 2003 and 2004, of seven murders and three attempted murders of children under her care. A criminologist told her trial that “the probability of so many deaths occurring while de Berk was on duty was only one in 342 million”.

But even if that were the case — and again, it wasn’t; the RSS estimated that if you took into account all the relevant factors, the chance of seeing a cluster like that could be as high as one in 25 — that’s not the same as the chance that de Berk was innocent. In order to establish that, you would need to take into account the prior probability that someone would be a multiple murderer — a mercifully tiny chance. De Berk’s conviction was overturned in 2010, thanks in part to the work of Bayes-savvy statisticians.

Bayesian reasoning doesn’t only reveal the wrongly convicted — in some instances it could have led to the guilty being detected. During OJ Simpson’s trial for the murder of his wife and her friend, for instance, the prosecution showed that Simpson had been physically abusive. The defence, though, argued that “an infinitesimal percentage — certainly fewer than 1 in 2,500 — of men who slap or beat their wives go on to murder them” in a given year, so it wasn’t relevant to the case.

But this is simply the opposite mistake to the prosecutor’s fallacy. The probability that a man who beats his wife will go on to murder her in a given year might “only” be one in 2,500. But that’s not what we’re asking. What we want to know is, if a man beats his wife, and given that the wife is then murdered, what is the chance that he did it?

The scholar of risk Gerd Gigerenzer had a go at answering that. The base rate for murders among American women is about five in 100,000. Assuming the one-in-2,500 probability is correct, then of 100,000 women with abusive husbands, about 99,955 will not be murdered. But of the remaining 45, 40 will be murdered by their husbands. The correct probability that the husband did it should be nearly 90%. Bayesian thinking might have helped convict Simpson.

And although he does not use full Bayesian reasoning but a slimmed-down version, the insight that we should update our existing probability estimates with new information — cumulative monitoring, rather than one-off testing — could, argued the statistician Professor Sir David Spiegelhalter, have spotted both the catastrophe at the Bristol Royal Infirmary and the murders of Britain’s worst serial killer, Harold Shipman, earlier, and saved many lives.

The Lucy Letby case does not turn solely on statistical evidence, and I make no arguments about whether she is guilty or innocent here. But people, including juries, prosecutors and judges, have misunderstood probability in the past; Sally Clark and Lucia de Berk had their lives ruined by that misunderstanding. Thinking like a Bayesian might have helped prevent that.

Professor John Edmunds of the London School of Hygiene and Tropical Medicine appeared before the Science and Technology Committee of the House of Commons and said as much. The R value was now between 0.6 and 1, he thought, but “if you’d asked me two weeks ago I’d have given lower numbers, about 0.6 or 0.7, maybe up to 0.8.” It got quite a lot of attention. 

But I want to suggest (as in fact Edmunds himself did) that this isn’t the bad news people seem to think; in fact it’s a product of our success, rather than of our failure. Here’s why.

There’s an interesting statistical anomaly called Simpson’s paradox. It is that you can find a trend going one way in lots of individual datasets, but that when you combine those datasets, it can make the trend look like it’s going the other way.

That sounds quite dry, so let me use a famous example. In the autumn of 1973, 8,442 men and 4,321 women applied to graduate school at UC Berkeley. Of those, 44% of the men were admitted, compared with just 35% of the women.

More from this author

Is the lockdown doing more harm than good?

By Tom Chivers

But this is not the clear-cut example of sex bias that it seems. In most of the departments of the university, female applicants were more likely than male applicants to be admitted. In the most popular, 82% of women were admitted compared to just 62% of men; in the second most popular, 68% of women compared to 65% of men. Overall, there was a “small but statistically significant bias in favour of women”.

What’s going on? Well: men and women were applying for different departments. For instance, of the 933 applicants for the most popular department — let’s call it A — just 108 were women, but of the 714 applicants for the sixth most popular department (call it B), 341 were women.

Let’s take a look at just those two departments. Women were more likely to be admitted to both. But, crucially, the two departments had hugely different rates of acceptance: in Department A, 82% of female applicants and 62% of male ones were accepted; in Department B, 7% and 6%.

So of the 108 women applying to Department A, 89 of them were admitted (82%), while of the 825 men applying, 511 got in (62%). Meanwhile, of the 341 who applied to Department B, just 24 (7%) were admitted; for men, it was 22 out of 373 (6%).

You see what’s going on here? In both departments, women were more likely to be accepted. But added together, it’s a different story. Of 449 women who applied to the two departments, just 111 were accepted: 25%. Whereas of the 1,199 men who applied to the two, 533 got in: 44%.

suggested reading

Why the English sacrificed liberty for lockdown

By James Kirkup

So, to repeat: even though any individual woman applying to either department had a higher chance of being admitted, on average fewer women were admitted because they tended to apply to more competitive departments.

This doesn’t mean that we’ve once-and-forever ruled out any form of sex bias — it might be, for instance, that there is a lack of investment in the popular but female-dominated classes — but it does show that the simple, top-line, aggregated numbers can mislead. When divided up into smaller groups, they can tell a very different story.

Simpson’s paradox is a specific case of a wider class of problem known as the “ecological fallacy”, which says that you can’t always draw conclusions about individuals by looking at group data. A topical example: local authorities with above-average numbers of over-65s actually have a lower rate of death from Covid-19 than those with below-average numbers. But we know that older people are individually at greater risk. What’s going on seems to be that younger areas tend also to be denser, poorer, and more ethnically diverse, all of which drive risk up.

The apparent rise of the R value seems to be something like that. According to Edmunds, the rise is not because lockdown isn’t working (“it’s not that people are going about and mixing more”), but that it is working. There are, he says, separate epidemics in the community, and in care homes and hospitals.

“We had a very wide-scale community epidemic,” he told the committee, “and when we measured the R it was primarily the community epidemic. But that’s been brought down: the lockdown has worked, breaking chains of transmission in the community … now if you measure the R it’s being dominated by care homes and hospitals.”

suggested reading

I don't want life to go back to 'normal'

By Graeme archer

Let’s imagine that we had two epidemics, of equal size, one in the community and one in care homes. Say 1,000 people are infected in each, and in the community each person on average infects two people, while in the care homes on average each person infects three. The total R is 2.5^[1].

But now imagine you lock down and reduce both the R and the number of people infected, but by more in the community than in the care homes. Say that now there are 100 people infected in the community, and they each pass it on to an average of one person; and there are 900 people infected in the care homes, and they pass it on to an average of 2.8 people.

Now your average R is 2.62^[2]; it’s gone up! But — just as with the Berkeley graduate students above — when you divide up the data into its constituent parts, it’s actually gone down in each category.

I don’t know the numbers, but according to Edmunds something like this has gone on in the real world. The collapse in the number of people with and passing on the disease in the community means that now the epidemic in care homes is a much greater share of the average. And that means, even though the R in care homes hasn’t gone up, the average R in total has, because the average in care homes was higher to start with.

To be clear: this doesn’t mean that everything is fine, or that we’ve won, or anything. “The thing that worries me is that it might be the overall R that matters,” says Kevin McConway,  an emeritus professor of statistics at the Open University, who helped me understand these numbers. It’s not that the epidemics in the community and in care homes and hospitals are truly separate, islands cut off from each other — they’re interlinked, so if the disease spreads in care homes it can reinfect those of us outside it.

Edmunds said as much to the Science and Technology Committee: “Strictly speaking you have one R: there’s one epidemic and linked sub-epidemics; the epidemic in hospitals is not completely separate from the one in the community.” But to understand how it works, you need to look in this more granular fashion: the overall R is not much use on its own.

More from this author

Test article

By Tom Chivers

And while it doesn’t mean that we’ve won, it certainly shouldn’t be taken to mean that the British population has been lax in its approach to lockdown. Compliance has been very high, much higher than modellers anticipated.

But it does show that simple numbers can hide more complex stories. They feed, for instance, into modelling. One simple model is the SIR (susceptible, infected, recovered) model, where you assume everyone just interacts at random, mixing uniformly like molecules in a gas; but if the epidemics in care homes and the community behave very differently, then those models might give out very misleading numbers.

That’s why models such as the Imperial one try to simulate human behaviour to some degree; the extent to which it got that right is far from clear, but it was at least trying. Some more simple models that went around the internet did not. McConway, a statistician not a modeller, is profoundly wary of those: “I know enough [about modelling] to say I wouldn’t touch it because I’m not an expert; I see people getting it wrong in ways that I can recognise, whereas I’d get it wrong in ways I don’t recognise.” These subtle misunderstandings can drive major errors.

We’ve seen examples of this throughout the crisis. Early on, people (Donald Trump, notably) paid an awful lot of attention to the case numbers; but the case numbers didn’t really tell us how many people had the disease, just how many of them had been tested. And people have tried to place countries in a league table of death rates, but that’s not particularly informative either (although that’s not to say comparisons are entirely useless). Trying to boil down this messy, complicated situation to single numbers and saying whether they’re good or bad is rarely a good idea.

The really key thing, which I keep coming back to, is just how much uncertainty there is. “Modelling is bloody hard,” says McConway. “Prediction is bloody hard. The map is not the territory. We’ll know what’s happened when it’s happened.” Even something as apparently simple as the R value has to be treated with immense caution.

^[1] ((1,000*2)+(1,000*3))/2000=2.5

^[2] ((100*1)+(900*2.8))/1000=2.62

That’s because any statistic that is interesting enough to have become a news story, he claims, has probably passed through “so many filters that encourage distortion and selection”, that “the very fact that I am hearing a claim based on statistics is reason to disbelieve it”.

I have a similar rule of thumb, which I’ve said here before: if a statistic is interesting, it’s probably not true.

The trouble is that we can’t do without them. There are simply too many people and too many things in the world to understand what’s going on without simplifying them and aggregating them into statistics; we can’t think about every individual NHS patient, we need to think in terms of average survival rates or waiting times. We have to treat people like numbers if we want to do the best for those people, so we need to learn to be better at interpreting those numbers.

Suggested reading

Why flawed science should concern us all

By Tom Chivers

But statistics can be misleading — in some fascinating ways. In his book, Spiegelhalter, a Cambridge statistician and former president of the Royal Statistical Society, points some out; Simpson’s paradox, for instance. In 1996, the admissions data for five STEM subjects at Cambridge showed that men were more likely to be admitted: 24% of men’s applications were accepted, compared to 23% of women’s.

Is this evidence of bias? Well: possibly. But not, interestingly, of discrimination against women. In four of the five subjects – economics, engineering, computer science, medicine, and veterinary medicine – women were actually more likely to be accepted (in one, medicine, they were equal). Sometimes, that difference was quite stark – 16% of women compared to 12% of men were accepted onto the veterinary course; 32% of women against 26% of men into engineering.

How is this possible? Because medicine and veterinary medicine were much more popular, so your chances of being accepted were much lower; and because women were much more likely to apply for medical or veterinary degrees. By choosing the harder-to-access courses, women were less likely to be accepted overall, even though, individually, they were more likely to be accepted than a man applying for the same course. Similar findings have been seen elsewhere.

Another example is the “prosecutor’s fallacy”. Imagine a suspect is in court. If they’re innocent, there is only a one in a billion chance that their DNA would match the DNA found at the crime scene, and yet it does. Does this mean that there is only a one in a billion chance that they’re innocent? No; far from it. Nonetheless this has led to real women being wrongly convicted of child murder, after several of their children died of sudden infant death syndrome.

Suggested reading

Don't fight dirty with statistics

By Tom Chivers

An analogous problem is cancer testing. Say your new cancer test is 95% accurate; it will answer the question “do you have cancer” correctly 95% of the time. If you test positive, do you have a 95% chance of having cancer?

Not at all! Imagine that 1% of people have cancer. You run your test on 10,000 people. Of the 100 people who do have cancer, it rightly tells 95 of them that they do have cancer. Of the 9,900 people who don’t have cancer, it rightly tells 9,405 that they don’t have cancer. But this means that it has wrongly told 495 people that they do have cancer. If you test positive for cancer, your chances of having cancer are not 95%, but 95 in 590, or about 16%. The DNA-test situation is exactly the same: your test is “accurate” 999,999,999 times out of a billion, but will still regularly be wrong, because the thing it is looking for (the specific guilty person) is very rare.

These are just a couple of the reasons why even straightforward-sounding statistics can be misleading. They’re not even the big reasons – they’re just a couple that I thought you might not have heard of.

Things like statistical bias, small sample sizes, or multiple analyses of the same data will have more of an impact, as well as the Groucho Principle that the most dramatic findings are both more likely to be well-publicised and more likely to be false. But they’re illustrative of the fact that you need to be very careful of almost any statistical story you read.

Suggested reading

If you don't 'get' statistics it's just as bad as not 'getting' Shakespeare

By Graeme Archer

At this point, I should admit that I have a dirty little secret, which is that I am not very good at maths. I mean, I’m fine. I’m OK. I got an A at GCSE, in 1997. But I’m not good. This is a problem, because I write an awful lot about statistics. Spiegelhalter tries to make the case that you don’t need to be very good at maths to be good at statistics, and he sort of succeeds – quite a lot of it you can grasp via the concepts, or through visualisations. But still, as an indifferent mathematician, I often found it hard to follow him.

As a non-mathematician, I have a few shortcuts for working out whether a statistic is worth believing, which seem to have done all right for me so far. One, which Spiegelhalter stresses, is that often the best statistical analysis you can do is simply visualising the data. There was a bit of a recent kerfuffle about suicides among girls and young women going up 83% since 2012; but simply looking at the ONS chart showed that the numbers were small, the data was noisy, and the only way you got the 83% figure was by choosing the lowest year on record. (It’s an old trick.)

Another useful hack is to ask “is this a big number?” Lots of statistics are presented as bald numbers: “X people have died”. But out of how many? If 25 people have died of cancer out of a population of 50, that’s awful; if 25 people have died of cancer out of a population of 50 million, that’s a statistical miracle. That’s one reason why I’m always wary of stories that say, for instance, “10 or more black transgender women have been killed this year” without telling me how many black transgender women there actually are, and whether that is more or less than we should expect.

Similarly, if a story tells you that something has “gone up by 25%”, or that something “increases your risk by 90%”, without telling you what it’s increased it from and to – if it gives you the “relative risk” but not the “absolute risk”, in technical terms – that’s an indicator that it may be worth ignoring. If something doubles your risk from one in 100 million to one in 50 million, you probably don’t care.

A third is asking “can I be sure what’s causing this?” Spiegelhalter dedicates a chapter to how you establish causality. It is amazingly hard to do this in the absence of large-scale randomised controlled trials. Austin Bradford Hill, one of the scientists – with Richard Doll – who first established that smoking causes lung cancer, suggested ways of doing so; looking for plausible mechanisms, making sure that the supposed effect follows the supposed cause, seeing if there’s a “dose response” – that is, the more of the cause there is, the more of the effect you see – and, importantly, making sure that the effect is sufficiently large that it couldn’t be a coincidence or some statistical confounding.

If you’re looking to see if something (say, a rise in teen suicides) is caused by something else (say, social media) then it’s worth looking to see if any rise in teen suicides follows the growth in social media, and whether each doubling in social media use leads to a consistent increase in suicide rate.

Suggested reading

Do we really have a 'suicidal generation'?

By Tom Chivers

But also, it’s important to see whether the suicide rate tends to jump around a lot anyway. If it does, then perhaps there’s nothing to explain; it just happened to be low one year and high another. (Spiegelhalter has a useful trick for seeing whether you’d expect to see the variation you see, such as using the mean averages to perform a “Poisson distribution”, but just looking at a graph and seeing whether it bounces around a lot is a good start.)

The statistician George Box said: “All models are wrong, but some are useful.” When you’re reducing millions of people and events to a few numbers, you’re building a model of the world, and that model must be simpler than the world, so it will be wrong in some way. You’re always making assumptions and drawing arbitrary lines.

For instance, we talk about the “murder rate” in Britain; statisticians treat it like an arbitrary event that happens at random to people, like decaying subatomic particles, with a 0.0008% or so chance of it happening to any given person per year. But that’s not how it works. Each murder is the result of real people making real decisions, not some lightning bolt out of a clear blue sky; it just happens that in a large enough population, non-random events can be treated as though they were random.

We can’t do it any other way. We can’t look at every person, and you can’t possibly predict every individual event, whether it’s the outcomes of heart operations or teen suicides or murders. You have to aggregate them, round off the corners, make simplifying assumptions; we have to treat people like numbers. You’ll get it wrong a lot of the time, and you have to be immensely cautious about any conclusions you draw.

There is one simple step, though, which will help a lot of the time. It’s a bit nihilistic, perhaps, but it’s powerful. And that is the Groucho principle that we discussed at the beginning: if the statistic makes a headline, it’s probably wrong.

It is therefore no surprise that when the folks at Our World in Data compared the media’s coverage (specifically, in the New York Times and The Guardian) of various causes of death with mortality statistics for the general population they found a mismatch. They also looked at causes of death through a third lens – online searches by the public.

Further reading

Don’t sweat the small stuff, or why the ‘New Optimists’ are only half right

By Peter Franklin

The findings are summarised on the site in an article by Hannah Ritchie:

“…around one-third of the considered causes of deaths resulted from heart disease, yet this cause of death receives only 2-3 percent of Google searches and media coverage… just under one-third of the deaths came from cancer; we actually google cancer a lot (37 percent of searches)… but it receives only 13-14 percent of media coverage…”

This is pretty much as one might expect. Specific incidences of death caused by a common disease do not generally make the news – except in the case of famous individuals.

As for online searches, one would expect the public to turn to ‘Dr. Internet’ if they have concerns about their own health or that of people they know – and, given the wide range of possible symptoms and warning signs associated with cancer, it’s not surprising that it prompts a disproportionate share of searches.

Then we come to the most dramatic findings:

“…the largest discrepancies concern violent forms of death: suicide, homicide and terrorism. All three receive much more relative attention in Google searches and media coverage than their relative share of deaths. When it comes to the media coverage on causes of death, violent deaths account for more than two-thirds of coverage in the New York Times and The Guardian but account for less than 3 percent of the total deaths in the US.”

Ritchie says that compared to what actually happens in real life, homicide is over represented in the newspapers by a factor of 31 and terrorism a factor of 4,000.

Ritchie does acknowledge what she calls the “very nature of news”. As the word makes clear, news is all about novelty and therefore the unusual, the sudden and the spectacular. This, she says, leaves most of us “with a skewed perception of the world; we think the world is much worse than it is.”

Further reading

I'm Lionel, I'm a news-aholic

By Lionel Shriver

Commenting on the Our World in Data study, Bill Gates was moved to tweet the following:

“I’m always amazed by the disconnect between what we see in the news and the reality of the world around us… we must fight the fear instinct that distorts our perspective.”

But is it right to pathologise what Ritchie calls a “strong bias for outlier events”?

Weighing in on the debate, Nassim Taleb slammed the “naive empiricists” who equate the future significance of a danger with how common it happens to be over a particular period of time.

The most egregious example of this fallacy is a cheap trick that’s often pulled by faux sophisticates in the commentariat. This is how it works: firstly, look up the stats for some rare, but crushingly banal, way to die – like falling off a ladder or accidentally drowning in a bathtub. Then, tell your readers that they’re still more likely to die that way than be knifed by a mugger or blown-up by terrorist. Then reassure everyone that the country is not, in fact, going-to-hell-in-a-handcart and that all is well in the liberal paradise. Finish with a condescending quip about Brexit voters – and you’re done. File that column, sit back and bask in the respect of your peers.

Taleb exposes the flaws in this ‘thinking’. He does so by distinguishing between two domains of probability, which he calls ‘Extremistan’ and ‘Mediocristan’. Terrorism is an example of a risk that belongs to Extremistan. It may, at present, be small – but it has the potential to become massively and unpredictably more significant.

A single act of terror can pitch a nation into a series of foreign policy disasters, as with 9/11 and the US. An escalating campaign of terror (and counter-terror) can bring fear and division to an entire society as in Northern Ireland during the Troubles; or, yet worse, develop into a full-scale, bloody civil war – as it did in Syria. Crime, too, poses potentially systemic risks. If it gets out of control, the harm is not just to the immediate victims, but everyday life and long-term prosperity across an entire neighbourhood, city or country.

Further reading

If you want to get ahead, take a risk

By Timandra Harkness

The same cannot be said of bathtubs or ladders. We may have the luck to live in a time and place where they kill more people than terrorists do, but the risks associated with their use belong to Mediocristan. This means that the most extreme scenarios really aren’t that extreme at all – society will not be rocked to its foundations by an uncontrollable epidemic of bathtub drownings, for instance.

Could it be that the general public has an instinctive sense of the risks that have the greatest potential to turn into catastrophe? If so, in pressing our buttons, the media is not whipping up scare stories – but acting as a means by which legitimate public concerns can be identified, amplified and pushed up the political agenda.

But they were cheating. They picked 1998 deliberately since it was an outlier – an El Niño year much hotter than the years around it. If you were, on the other hand, to measure from 1997 or 1999, then there were lots of much hotter years on record; and the clear trend was that later years, on average, were hotter than earlier ones. It was a wobbly, noisy line, with some outliers, but the average temperature really was going up, and the only way you could hide that trend was by cherry-picking statistics.

I was thinking about this as I read the Sunday Times splash this week, which (using as-yet unavailable data from the Office for National Statistics) claimed that the “suicide rate among teenagers has nearly doubled in eight years”. It expressed concerns that we are raising “a suicidal generation”.

They said that the suicide rate among 15- to 19-year-olds in 2010 was just over three per 100,000. The ONS figures, due out in September, will (apparently) show that it is now over five per 100,000. Inevitably enough, the piece links the purported rise to the growth of social media since 2010.

Suggested reading

Why you can't rely on the news media to understand... science

By Tom Chivers

But this is – and I don’t want to get too technical here, but bear with me – absolute bollocks from top to bottom. It’s a masterclass in what scientists call “hypothesising after results are known”, or HARKing. If you have the data in front of you, then you can make it say almost anything you like.

First, it’s worth noting that very few teenagers kill themselves. The total number of suicide deaths among 15- to 19-year-olds in 2017 in England and Wales was 177, out of about 3.25 million. That means that small changes can look like big percentage swings. More important, though, the Sunday Times story did exactly what the climate deniers did. The year 2010 had the lowest rate of teen suicides of any year since at least 1981, when the ONS records begin. You could compare it with literally any other year and you’d see a rise.

Added to which, picking social media as your reason for is completely arbitrary. Social media did not start in 2010. The BBC TV series Sherlock, starring Benedict Cumberbatch and Martin Freeman, did, though. Maybe we should blame that.

You could, if you wanted to, use the same trick to tell the exact opposite story. Facebook was first released in 2004, when the suicide rate among 15- to 19-year-olds in England and Wales was 4.7. But after six years of social media being available, it had dropped to 3.1! It’s a life-saver, no?

Suggested reading

Why it's time to panic about kids and smartphones

By Peter Franklin

Or, you could use the trick to tell other, more political stories. For instance, one of the highest rates of suicide on record was 1998, shortly after the election of Tony Blair’s New Labour, when 6.1 out of every 100,000 teenagers killed themselves. But by the time they left office in 2010, it was – as we’ve seen! – an amazing 3.1; New Labour almost halved teen suicide! And with the election of three Tory-led governments in a row, it’s crept back up almost to where it was.

I have no idea if that’s true – and I should admit, I’ve done some shameless cherry-picking of my own, there, choosing 1998 instead of the year Blair actually swept to power, 1997, which was a less spectacular 5.5 per 100,000. You could tell a plausible-sounding story about cuts to mental health provision and social care and housing support and so on. But if you did so, you’d also have to explain why suicide in general, including among young people outside this very specific bracket, is on the decline.

The trouble is, for people who want to say that social media is causing an epidemic of suicide, is that there’s just no good evidence for it. For a start, as I just mentioned, there isn’t really an epidemic of suicide anyway. If you look at 10- to 29-year-olds, it’s gone from a consistent plateau of about 15 per 100,000 from the late 1980s to the early 2000s, down to about nine per 100,000 by about 2004, and that’s where it’s stayed, pretty much. There’s a similar story among the population as a whole.

Suggested reading

Why we should fear technophobia

By Tom Chivers

What’s more, when you look at the links between social media and mental wellbeing, they are deeply ambiguous and extremely weak – and the data doesn’t show causation, anyway. An enormous analysis of three major datasets by Oxford University scientists, published in Nature Human Behaviour last month, found a tiny, uncertain correlation between the two. But that correlation about the same as the correlation between well-being and regularly eating potatoes, or well-being and wearing glasses. We do not talk of potatoes as having “destroyed a generation”, or worry about bespectacled teens and suicide. The Human Behaviour study was also interesting because it showed how you could find scarier results – by doing exactly what the Sunday Times story did, in essence: by HARKing, getting your data first and then choosing the bits to display to make it look scary.

The sadness of it all is that this comes on the back of some extremely sensible advice being issued by the Royal College of Paediatrics and Child Health, and a surprisingly even-handed report by the House of Commons Science and Technology Committee. Both acknowledged the shortage of good evidence, and made common-sense suggestions for parents – in particular, rather than imposing hard limits, the RCPCH guidelines simply said that if you find that social media or screen time gets in the way of family time or things you want to do, then you should reduce it. It felt like a little turning point in the endless barrage of scaremongering. Hopefully it really was.

The climate deniers, by the way, appear to have largely stopped using the 1998 trick, after 2014, 2015 and 2016 were all hotter than 1998 and, in fact, each hotter than the last. The trick, in essence, stopped working. We don’t need extra reasons to hope that teen suicides go back down once more, but making it harder to scaremonger about social media would certainly be a bonus.

Consider this very modern problem: I have given a test drug to 20 patients, and 15 of them responded positively. That’s a 75% ‘response rate’, in the jargon, in this sample of patients.

But common sense will tell you that 75% isn’t the drug’s true response rate (would you expect exactly 15 patients to respond out of every further 20 who were exposed to the treatment? Or sometimes 14, sometimes 18, and so on?) So: the observed response rate isn’t the truth. But – and again, applying merely to common sense, and with the supposition that I didn’t rig the trial in some way – surely a response rate of 75% is more probably the true response rate than, say, 25%?

More from the Christmas Reading collection

Why the Unabomber's Manifesto is essential reading

By Jamie Bartlett

To peer at the truth about a theory concerning the world (in this case, the true response rate of the drug) usually requires an act of inductive inference: from experimental observation (“We observed a 75% response rate in this study”), to statement about the truth (“The true response rate is more likely to lie between 50% and 90% than it is to lie below 10%”). You’ll never know the truth (shades of Jack Nicholson) – you can’t expose every possible patient throughout the Earth’s history to the drug, and count how many of that infinite set would respond – yet you can still make valid claims about it. Roll that thought about your head a moment, and then tell me Statistics isn’t sexy.

That there exists an inductive logic that more or less works is thanks in part to the Reverend Thomas Bayes, whose simple inquiry in the 18^th century provides the answer to what I called a “very modern problem”.

Here’s his version, posthumously published by his friend Richard Price, in 1763:

Other than f-for-s, this is essentially the “what is the probability that the true response rate for this drug lies between 50% and 90%, given that we’ve observed a 75% response in this trial?” question with which we opened.

Bayes answers like this: we have some hypothesis of interest; in this case, perhaps it’s “The true response rate for this drug is higher than 50%”. Call that hypothesis H. Before the trial, I must have some positive belief that H is true: call that Pr(H). Pr(H) must lie between zero (H is impossible), and 1 (H is certainly true) (and if Pr(H) didn’t lie between zero and one, there could be no justification for the experiment).

After the trial, the evidence – that 15 out of 20 patients responded – must alter that belief to an extent warranted by the observed results (else why do any experimental science at all?). “All” Bayes’ theorem makes plain is how you should modulate your pre-trial belief, Pr(H), into your post-trial belief, Pr(H, given that 15 out of 20 patients responded). Mathematically, it’s almost trivial. Inferentially and scientifically, it’s a revolution.

Almost every bit of tech you use pivots on his theorem – every Google auto-complete, not to mention the search algorithms themselves; Turing’s work at Bletchley; astronomical investigations of far-flung galaxies; medical imaging; genetic markers for disease predictions; nearly everything with the modern label “Artificial Intelligence/Machine Learning”: all of these and thousands more exercises in human ingenuity make use, explicitly or otherwise, of the theorem which carries his name.

In one of those dead-ends to which civilisation can sometimes be prone, however, early 20th-century Statistics turned its face against Bayes: theories are either true, or false, and so to discuss them in terms of probability – worse, in terms of “your” probability – was an unwarranted intrusion of the subjective – of the self – into science, which (as every fule kno) is supposed to be entirely objective.

Further reading

Don't fight dirty with statistics

By Tom Chivers

Ronald Fisher, geneticist and statistician – a genius, a giant, but occasionally totally wrong – wrote Bayes out of the statistical universe, by inventing most of the methods that are taught to undergraduate scientists to this day, all of which pivot on “falsificationist” notions. As Popper held a theory to be scientific only to the extent to which an experiment could prove it false, so Fisher attempted an inductive logic by which data could “reject” a hypothesis.

Statisticians were aware of the limits of Fisher’s pseudo-objective approach (it often doesn’t work, it’s incoherent and, ironically, it is also unavoidably, enormously subjective), but continued to teach it – in four undergraduate years at the University of Glasgow, I received nine hours of Bayesian lectures, against heavens know how many hundreds of hours on ideas that flowed from Fisher.

Further reading

How gamification makes losers of us all

By Peter Franklin

I don’t have space to deal with Fisher’s issues here; suffice to say that Bayesian subjectivity isn’t an excuse for the post-modern relativism which is destroying the social sciences and nor does it imply that “there’s no such thing as the truth”. The biggest roadblock to Bayes wasn’t philosophical, but mechanical: for all but the simplest problems, the maths required to make the Bayesian machine work in practice is too “hard”.

And then the second Bayesian revolution happened. I started my PhD research just as that revolution picked up pace, in the early 1990s, with the “Gibbs sampler” publication of Gelfand and Smith. I’m blessed now to count among my direct colleagues (and friends) Professor Nicky Best (this year honoured by the Royal Statistical Society) who, along with coworkers David Lunn, Christopher Jackson, Andrew Thomas and the brilliant David Spiegelhalter invented BUGS: Bayesian inference using the Gibbs Sampler.

Gibbs sampling allows the statistician, no matter how complex his or her mathematical model – his or her representation of the theory under investigation – to draw simulated observations from that model’s idea of the world and to use those simulations for valid inference.

More from the Christmas Reading collection

Why Antifragile is the perfect read for today's turmoil

By Peter Franklin

“Given this noisy, blurred MRI scan of your brain: what is the representation of your actual brain that has highest probability? Which areas are most likely affected by stroke?” – my PhD questions were identical, mutatis mutandis, to Thomas’s 1763 “PROBLEM”, and in theory could be answered by application of his theorem. Prior to Gibbs, however, this was practically impossible: afterwards, it was (almost) trivial. Suddenly, every scientific problem that had a perfect Bayesian solution in theory almost always had nearly perfect Bayesian approximate solutions in practice.

Statistical science was rescued from its obscurity not by Fisher’s dead-end significance tests, but from a mechanical solution that made Bayes’ 1763 theorem relevant and applicable in almost any setting. Without doubt, the portrait for the new 50-pound note should be of Thomas Bayes, a truly Great Briton.

Two highly readable books by philosopher Ian Hacking also shaped my understanding of how, in his words, the “quiet statisticians have changed our world”: The Taming of Chance explains how the concept of statistical “laws” came to rival those of the physical world in terms of their impact on human affairs. And An Introduction to Probability and Inductive Logic is a great exposition of probability and the ideas of both Fisher and Bayes (and others). Probability, the language of uncertainty, is the thing that everyone (including undergraduate statisticians) thinks they “get”, until they think about it carefully and realise they don’t.

If you’d like to read more, Howson and Urbach’s Scientific Reasoning is magisterial. In the same year I wasn’t being taught Bayesian methods by Glasgow University’s statisticians, Peter Urbach’s partner, who headed the Drama department, gave me a copy of this book, and it changed my life. I recommend it to everyone; Howson and Urbach were philosophers of science, not statisticians, and it’s very readable. For completely un-mathematical friends I’ve been recommending Thinking Statistically by Uri Bram.

Predictions are hard, especially about the future, as Niels Bohr may have said. But that doesn’t stop people making them. May’s deal will get through, say some. She doesn’t have the votes, say others. Of course, if we do crash out, then the crystal ball gets even murkier. We might have to stockpile medicines. Energy prices might skyrocket. We might run out of Mars Bars.

Niels Bohr was right. Predictions are hard, and people are spectacularly bad at making them. But there are, also, some simple things you can do to get better at making them, in order to mitigate the consequences of whatever the future holds. I went on a workshop last week, with a bunch of senior and important people from various major banks and intergovernmental agencies, to learn how to do it better.

It was run by an organisation called the Good Judgment Project. The GJP was set up by Philip Tetlock who, in 1984, had noticed something. The young psychologist had joined an expert committee set up by the National Academy of Sciences to help prevent nuclear war. The experts were divided. Conservatives wanted to maintain a tough line against the Soviet Union; liberals thought that line was strengthening Kremlin hardliners.

Suggested reading

How Corbynomics is winning over Britain

By Matthew Goodwin

When, a few months later, Mikhail Gorbachev took office and started opening the USSR up, Tetlock was surprised to see that everyone thought this showed they’d been right all along. The conservatives thought the tough line had pushed the Kremlin into action; the liberals thought it would have happened anyway, and that the hard line had just slowed things down. People all thought they were right, whatever had actually happened.

So Tetlock set up an experiment to see how good experts were at predicting the future. First, he had to tie them to falsifiable predictions. Pundits had, and have, a tendency to vague answers that don’t really pin them to anything: “Food shortages could be likely”, and so on. Tetlock gave them specific questions with clear dates: “Will the yen be higher than it is now against the deutschmark in one month’s time?” He then asked the pundits to give percentage values for how likely that was: there is a 75% chance that this will happen, a 60% chance, etc.

He had nearly 300 pundits make an average of 100 predictions each, and then saw how well they did over the coming months. If your 60% predictions came in 60% of the time, and your 35% predictions came in 35% of the time, you were “well calibrated”. You were also given points for being confident and right – 90% guesses got a higher score than 50% ones – but punished for being confident and wrong.

What he found was that pundits’ calibration, on average, was no better than random guessing – or, as Tetlock described it, “than a dart-throwing chimpanzee”. But some were significantly better. What predicted who was better was not whether they were conservative or liberal, or even, particularly, their expertise in the field, but how they approached the problem. People who assumed the world was simple and had simple solutions did badly; people who thought it was complex, who realised they could be wrong, and who learnt from mistakes did better. It also helped if you were good with numbers, and good at spotting patterns, as in IQ tests.

Suggested reading

So you think you're open-minded...

By Tom Chivers

Tetlock would later set up the GJP, which runs tests to find the best forecasters and enters them in forecasting competitions against teams from intelligence agencies and universities. Teams of these “superforecasters” outperformed the best of the rest by a huge margin, even CIA analysts and academic experts.

The workshop that I attended couldn’t make everyone superforecasters. There is some inherent talent involved in being the best. But the one of the supers who ran it pointed out that not everyone can ever play at Carnegie Hall, but almost everyone can learn to play the violin. Practising the right skills can make anyone better.

This is a useful thing to do. Being a better forecaster, if you’re a major financial institution, lets you make better bets with your money, and so turn it into more money; if you’re a national security agency or an intergovernmental organisation, it lets you direct resources towards things that are likely to happen, and keep the world safer.

Some of the things you could do were obvious: invoking the “wisdom of the crowd”, for instance. The “ask the audience” option on Who Wants to Be a Millionaire comfortably outperforms the Phone A Friend option; more generally, if you ask 1,000 people to guess something, any one of them will probably be wrong, but on average they’ll all be wrong in different directions, so the average guess is usually close. And if you’re in a big group, you should take the median average, and then readjust your predictions, and take the average again. Doing so gives you a real, measurable improvement in forecasting.

Suggested reading

If you don't 'get' statistics it's just as bad as not 'getting' Shakespeare

By Graeme Archer

Another is to use the sort of precise, numerical terms we talked about above. The superforecasters were highly critical of things like Sadiq Khan’s claim that no-deal Brexit is “more likely than ever” or Sir Vince Cable’s that the same was “highly improbable”. These don’t really tie you to anything: more likely than ever? How likely was it before? How likely is “highly improbable”? Is it a 5% chance? 10? Instead, write down a real percentage, and then revisit it afterwards, and see how it did. Otherwise you’ll find a way to convince yourself you were right all along, and you’ll never improve.

A third is to use the “outside view” as well as the “inside view”. For instance, say you’re at a wedding, and someone asks you how likely it is that the couple’s marriage will last until one of them dies. You look at them, they seem very happy, and you know them to be well-matched, so you say “90%”. That’s the “inside view”: looking at the specifics of the situation in front of you.

But taking the “outside view” involves looking at the whole class of similar events. So you might first think “About 40% of British marriages end in divorce,” and use the inside view to adjust your estimate from there, by taking into account other details that might change it from the base rate – social class, educational level, how long they stared into each other’s eyes when making their vows.

Fourth, you might break down the problem into smaller parts. How likely is it that we’ll have a Labour-led government before June next year? Well, the Government feels like it’s teetering on the brink at the moment, so you might think it’s pretty likely, so you might say 60%.

Suggested reading

Lies, damned lies and energy statistics

By Peter Franklin

But if you break it down, Parliament would have to vote for a new election, which would probably mean at least some Tories voting for an election they could well lose, and then it would mean Labour forming the largest party. If you think how likely each of those things are individually – I’d say an election before June next year is at best 50% likely, and that Labour’s odds of winning aren’t much better than 50% either – and then multiply them out, you get 25%. That’s called a “Fermi estimate”, and again, using this has been shown to improve accuracy.

There are a few other things that help – I won’t list them all, but one is, if you have a group, to try to make sure that it’s got diverse viewpoints, so it’s not just teachers or financial analysts (or just liberals or just conservatives), but a mix of many types. You also ought to make sure that the loudest or highest-status person in the group doesn’t dominate, so that the “wisdom of the crowd” doesn’t get drowned out.

We weren’t a bunch of superforecasters. We were just ordinary people, albeit some of us (not me) from some quite important financial, academic and governmental organisations. But we had a stab at it, and put best-guess estimates on some things, and then shared them and readjusted them as we were supposed to. And, after doing all that, in answer to the question “Will a Brexit withdrawal agreement obtain the consent of the European Parliament by 1 April 2019?” – a specific and falsifiable version of the question “Will Britain agree a Brexit deal?” – we figured that there was a 60% chance.

Suggested reading

Don't fight dirty with statistics

By Tom Chivers

That seems scary enough to me – that there’s a 40% chance of either a no-deal collapse or another referendum. And when we did it, our guess roughly matched that of the superforecasters. They wouldn’t tell us exactly what their numbers were, because it’s all privileged information, but I’m told we weren’t far off their guess.

That was over a week ago, though. Since then, the deal has been unveiled, Cabinet ministers have resigned, the ERG has unsuccessfully launched a leadership challenge, and the Democratic Unionist Party have all but ended the deal with the Tories that gives May her majority. Again, I don’t know exactly what the superforecasters’ numbers are, but that 60%-ish figure that we came up with now looks deeply optimistic.

The three main options – leaving with a deal, leaving without a deal, and not leaving at all, by 1 April 2019 – are all, I understand, roughly equal in probability. The Good Judgment Project is, verifiably, the best forecasting outfit in the world, but even for them it is, pretty much, a complete tossup.

So, yes. Predictions are hard, especially about the future. You can find ways of making it easier. But it looks like when it comes to Brexit, we really are all in the dark.

However, as always, economics is a slippery business. For a start, there’s the practical challenge of measuring production. For various reasons, the most reliable way of doing so is at the point of purchase. Using this expenditure method, GDP is equal to personal consumption plus business investment plus government expenditure plus net exports. That, however, leaves the question of what to include under each of these headings.

Further Reading

Don't fight dirty with statistics

By Tom Chivers

In an article for the Global Poverty and Inequality Dynamic Research Network, Jacob Assa and Ingrid Harold Kvangraven point out that the standard methodology has changed in recent decades – and that the effect of those changes has been to make the western economies look richer (in comparison to non-western economies) than they really are:

“…changes made to GDP measurement over the past two decades have a bias towards countries traditionally in ‘the West’. As this has had a substantial impact on the assessment of comparative growth among countries in those two groups, we deem it as a form of kicking away the statistical ladder “

What changes are we talking about?

“Examples include the reclassification of financial intermediation services, Research and Development  activities (R&D) and owner-occupied dwellings as productive activities. This means finance has been ‘made productive’ (as Brett Christophers puts it), R&D is no longer considered an intermediate input and home ownership is now included in GDP in the form of the imaginary rent that homeowners would have paid to a landlord had they owned their home…”

The authors propose a new measure of “core GDP” that would exclude the funny money and focus “only what can be directly measured”:

“Such directly measured industries include agriculture, utilities, manufacturing, retail trade, transportation, and communication.”

In a fascinating paper they compare the official GDP figures to their “core GDP” measure and find that the choice of method makes a big difference to the relative sizes of the western and non- western economies. For instance, according to the official figures the non-West only overtook the West in the middle of 2000, but according to the core GDP figures the cross-over took place much earlier – at some point before 1990 (when the comparison starts).

Further Reading

Lies, damned lies and energy statistics

By Peter Franklin

Choice of statistical method doesn’t change how rich or poor people actually are, but the authors observe that the bias to the West does impact on “our perception of growth in the world” and also on the weight of western influence in global institutions like the World Bank and the IMF.

As well as this apparent unfairness to the non-West, I’d argue that the official measures of GDP are doing damage in the western nations too.

A key issue here is that of ‘imputed rent’ – which takes a bit of explaining. Official GDP figures include the rents that tenants pay to their landlords. They also include an estimate of what owner occupiers would pay if they too rented their homes instead of owning them – these are the imputed rents. It is as if owner occupiers were paying rent to themselves. This might seem fanciful, but the justification is that though owner occupation doesn’t generate an actual transaction, it still has a measurable financial value.

Assa and Kvangraven argue that including imputed rents in the measurement of GDP has “the peculiar effect of making countries with housing bubbles appear to grow more rapidly”. Well, fair comment, but on that basis one shouldn’t include any kind of rent in the GDP figures – imputed or actual.

Further Reading

Home Truths – Part III: The nine drivers of a broken market

By Liam Halligan

In fact, taking rent out of GDP would remove some deeply perverse incentives for policy-makers. Ever wondered why government does such a bad job of solving the housing crisis? Well, if it did a better job, and rental values came down, GDP would take a corresponding hit.

In the longer term, I think the real productive economy would benefit hugely from lower, more stable rents and land values – but that’s not the sort of thing factored into the models of bean-counting finance ministries.

A statistic about, say, the economy – GDP growth, employment levels – is a neutral statement of fact. The economy has grown by X%, unemployment has gone down by Y%. But it’s also it’s a stick to beat the enemy with. Growth is slow under Labour? Vote Conservative! Jobs are up under the Democrats – vote for us! The same is true of how much the world has warmed, the number of black people the police shoot, how many women go into STEM careers.

You can use statistics in one of two ways: you can use them to try to be right, to deploy them in the way that best describes reality; or you can use them to win the war against your ideological opponents. I would rather use them to be right. That’s why I have recently spent a surprising amount of time correcting a widespread misunderstanding.

Further reading

One dead animal is an online sensation, a million is a statistic

By Peter Franklin

Last week, as pointed out by Peter Franklin, The Living Planet Report was released. It studied at the numbers of vertebrate animals from 3,000 species in 14,000 different local populations, and found that, on average, the populations had declined by 60% since 1970.

That sounds, and is, catastrophic. But what it doesn’t mean, despite how many media outlets reported it, is that there are 60% fewer animals than there were in 1970.

Imagine if, instead of examining 14,000 populations, they’d only looked at two: rats, of which there were 1,000 in 1970, and white rhinos, of which there were 10. And let’s imagine that between 1970 and 2017, both populations had lost five members, so there are now 995 rats and five white rhinos. The average population has declined by more than 25%. But the total number of actual animals has declined by less than 1%.

This is a silly example, but it demonstrates that the “decline in average population” figure and the “decline in actual number of animals” figure don’t have to be related at all. I’m not saying that everything is OK and we don’t need to worry. There really is a human-caused mass extinction going on, and besides, there are good reasons to care more about the death of five white rhinos than of five rats. But it does mean the headlines were wrong.

Further reading

Don't be so quick to laugh at the 'grievance study' hoax

By Tom Chivers

Still, that didn’t stop several people from telling me off for my pedantry. Sure, the headlines might not be exactly true, they said. But they grab public attention and get the statistic reported. If we point out that they’re false, then people will think there’s no problem. If we – people who care about the environment and wildlife – let the stat stand, it will convince people to take action. We can win.

I know this attitude. I’ve bumped into it before. A statistic went around in 2014 that 10,000 people had died within six weeks of having their disability benefit stopped by ATOS, the company doing the DWP’s disability assessments. It sounded pretty awful; as though ATOS was declaring people fit for work while they were on their deathbed. But it turned out that the stat actually included people who had died up to six weeks before their benefit stopped.

That is, people died, because people sometimes die, and then the company successfully stopped paying benefits to them, because they were dead. It wasn’t that they were dying just after being told that they didn’t need their benefits any more.

Again, this doesn’t mean that what the DWP and ATOS were doing was OK. They were hiding statistics that could have shown whether people really were dying early; the coroner blamed their assessments for at least one person’s death, and by ATOS’s own admission the arrangement wasn’t working. But 10,000 people did not die after being declared fit to work.

Further reading

If you don't 'get' statistics it's just as bad as not 'getting' Shakespeare

By Graeme Archer

Again, I was told that this is quibbling with statistics. Who cares about the exact number. The point is, people are dying. Nitpicking the figure just gives the enemy ammunition.

This is a pretty standard position. A few years ago, a claim went around that men are 32 times more likely to be struck by lightning, and 11 times more likely to be killed by a meteorite, than they are to be falsely accused of rape. The claim was spectacularly false, of course. (I mean, come on. Literally no one has ever been confirmed killed by a meteorite. It doesn’t have to be very likely to be more likely than that.)

While it is true that false rape accusations are uncommon, those figures were off by a factor of several thousand, because the author had compared the risk of a man being accused of rape each time he has sex with the risk of dying in a meteorite strike in your entire life. But it went enormously viral, because it fit a popular narrative. And when its falseness was pointed out, its fans defended it as the sort of weapon – “war and fire”, they called it – that we, the good guys, need to use to win in the war against the bad guys.

It’s not always that people are consciously thinking “yes, that is false, but I will say it anyway”. Part of what’s going on here is simple motivated reasoning: you don’t bother to fact-check stats that support your side. When we see very obviously false stats like the ones above, a little alarm in our head should go off, but if it’s a ‘helpful’ stat sometimes we ignore it.

Further reading

Why you can't rely on the news media to understand... science

By Tom Chivers

There is, also, a mindset that – even once the error has been pointed out – wants to cling to it, because it is a useful weapon. That is counterproductive. We’re living in a world of increasing political polarisation. One of the problems behind it is that political opponents increasingly fail to share not only opinions, but facts: we don’t agree whether the world is warming, or whether immigrants are hurting wages, or whatever. That happens even when we don’t want it to. If we start knowingly making stuff up because it’s convenient, it will only make that process worse.

Aside from driving yet another wedge into the ever-widening gap in our society, it undermines our ability to trust things in general. If I start to think I can’t trust stats and evidence because most of them have been made up or gamed by partisans, then I won’t be able to make decisions off the back of them. Public policy suffers if the electorate can’t get access to good information.

In any case, if something is really bad, then you shouldn’t need to exaggerate your statistics to make your point. The holocaust of vertebrate species (and invertebrate species) is truly awful. Species have been devastated; some by 60%, some by less, many by more. We are living in a world with far fewer incredible creatures in it than the world our ancestors lived in. That is terrifying and tragic enough without making stuff up. False rape accusations really are rare; you don’t have to say they’re 30 times rarer than lightning strikes.

In fact, if you do make stuff up, you give your enemy ammunition. The DWP could point to obviously false claims about 10,000 deaths and say that it was all stupid hype. Anti-feminists can say, look, the feminists lie about false rape accusations: why should we ever believe them?

But this is neither a game, nor a war. The information around us is a common resource. We all take from it, like drawing water from a well. But we contribute to it as well, filling it up with facts and stats that we find. We can try to replenish it with useful, relevant, true things, based as best we can in reality.

Or we can poison it with falsehoods, as Donald Trump does, in the hope that it kills our enemies before it kills us. Poisoning the well can feel good – it can even feel like we’re doing the right thing – but in the end we all lose. We need to be right, because there isn’t anything to win.

But how relevant is it? According to the International Energy Agency IEA), hydroelectric, solar and wind power will together supply barely 4 per cent of global energy demand by 2040.

That makes renewable power look pretty unimportant.

Further reading

The enemies of renewable energy have lost – they should be delighted

By Peter Franklin

But, watch out – energy statistics, like most statistics, must be interpreted with caution. Narratives composed of numbers may appear to be more objective than those composed of words, but that’s all the more reason for vested interests to exploit their ambiguous qualities (or should that be quantities?).

For a start, official energy prognosticators have a pretty consistent record of underestimating the rate of renewable energy expansion. As with most new technologies, forecasters tend to err on the side of caution. Fair enough.

Then there’s the distinction to be made between energy and electricity. You’d be amazed just how often politicians and journalists confuse the two. Energy includes not just electricity but also fuels used for heating and hot water, industrial processes and transport. The key renewable technologies – i.e. wind turbines and solar photovoltaic panels – generate electricity, and the debate about how much we should use them relates to the electricity sector only. Therefore, to try to prove their irrelevance by using statistics relating to overall energy demand is pulling a fast one. It’s like saying that the Team GB cycling team won a disappointing number of swimming medals.

But there’s an even the trickier issue in energy statistics: the distinction between primary and secondary energy. A crude definition (pun intended) is that primary energy is energy in its original form as extracted from nature e.g. coal, oil, natural gas, biomass etc. Secondary energy is what you get when primary energy is transformed to produce a carrier of energy – such as electricity. (There are various complications to these definitions, but to get on with it, I’m simplifying).

The crucial point is that a lot of energy is lost in the process of converting one form of energy into another. When fossil fuels are combusted, say in a power station or a vehicle’s engine, they produce heat as well as power and most of that heat is wasted (disappearing up the cooling tower or out the radiator, exhaust pipe etc). In fact, there’s generally more heat produced than power, meaning that we need two or more units of fossil fuel energy to produce each unit of electrical/mechanical energy.

The relevance to renewables and energy statistics is this: renewables aren’t hot. They don’t produce large quantities of waste heat. Instead of consuming a fossil (or nuclear) fuel, they produce electricity from the wind and the sun – inputs that aren’t counted as primary energy because they are ambient, inexhaustible and no one owns them. Thus renewables are only measured in terms of the electricity they produce.

Further reading

You’ll never guess which is the most densely populated country in Europe

By Peter Franklin

In an eye-opening piece for Bloomberg, Liam Denning explains why this confuses the energy statistics. He does so by comparing two sets of energy projections – the IEA ones mentioned above and equivalent figures from BP. The key difference is that compared to the IEA projection, BP’s shows renewables taking a much higher share of primary energy:

“The estimates for thermal energy from fossil fuels and nuclear power are very similar…

“The really interesting difference concerns hydro, solar and wind power. BP’s higher figure isn’t because it is more bullish on these. Rather, in order to make the renewables figures comparable with the ones for fossil fuels and nuclear power, BP grosses them up as if they also produced waste energy. The IEA doesn’t do this, so its figure represents just the energy derived from a solar panel, wind turbine, or hydro plant. The IEA figure is 36 percent of the BP one, similar to the 38 percent conversion factor BP uses to adjust the data.

“There are pros and cons to both approaches. The IEA’s reflects the fundamentally different nature of renewable energy, but at the cost of making its share of the market look very low: Solar and wind are 11 percent of BP’s mix in 2040 but less than 4 percent of the IEA’s.”

In other words, showing renewable production as a share of all primary energy production (i.e. measured energy before conversion losses) flatters non-renewables because the fossil fuel and nuclear shares contain a massive amount of waste.

Or, as Denning points out, renewables don’t just displace non-renewable power, but also the non-renewable heat that gets vented in the process – and, of course, the fossil fuels used to produce both.

So be careful with energy statistics, they’re every bit as treacherous as the financial kind.

Further reading

In the age of Big Data, politicians should not believe their eyes

By Peter Franklin

The answer, is that we make prior assumptions about what (and who) is trustworthy and relevant. Furthermore, we tend to do so collectively – subscribing to the shared assumptions of the groups we want to be part of.

Fair enough. We need those filters, otherwise we’d be overwhelmed. The problem though is when the filter become so restrictive that it only allows through the information and ideas that confirm existing assumptions. From that point, it ceases to be a defence and becomes a cage – a bubble within which different perspectives have no place and new understanding becomes impossible.

Further reading

Don’t believe what those idiots are telling you about echo chambers

By Peter Franklin

Such bubbles are typically constructed from linguistic components – facts, theories, jargon, slogans, arguments, talking points, memes, cultural references. However, the thickest bubbles are made out of numbers not words.

While words are open to interpretation, numbers are an expression of objective truth: 2 plus 2 equals 4 and nothing else. We may fear the dark arts of statistical manipulation, but what could be more objective than a direct, unfaked observation?

Unfortunately, there are pitfalls there too. The point is made in an eye-opening piece by Aaron E Carroll for the New York Times. It concerns that most wholesome of corporate activities – the workplace ‘wellness’ programme.

By now, we all know that sedentary lifestyles aren’t good for us – and especially not in combination with mental stress. This, in turn, is bad for productivity, employee retention and the cost of providing health insurance. Hence the rationale for workplace-based schemes to promote healthier lifestyles.

But do they work?

“These… can offer screening for a variety of reversible conditions; access to weight-loss programs or gyms; encouragement and support; and sometimes even chronic disease management. Many of the analyses of these programs have shown positive results.

“Almost all of those analyses are observational, though. They look at programs in a company and compare people who participate with those who don’t”

Surely, that’s a perfectly fair test – a direct, unmanipulated measurement of the difference that participation makes.

There’s a catch though:

“The most common concern with such studies is that those who participate are different from those who don’t in ways unrelated to the program itself. Maybe those people participating were already healthier. Maybe they were richer, or didn’t drink too much, or were younger. All of these things could bias the study in some way.”

Of course, one can control for factors like age, sex, income etc. This requires a statistical trick or two, but as long as the effect is to compare like-with-like (e.g. slightly overweight middle-aged men with other slightly overweight middle-aged men) who could object?

Further reading

If you don't 'get' statistics it's just as bad as not 'getting' Shakespeare

By Graeme Archer

Only, there’s another catch:

“…we can never be sure that there aren’t unmeasured factors, known as confounders, that are changing the results.”

When it comes to getting fit, some people are just more motivated than others – just the sort of people you’d expect to volunteer for a wellness programme. Motivation, being an internal state of mind, is hard to measure objectively and thus control for – meaning that the participating group is likely to be biased to those most inclined to make the most of it.

Then there’s bias on the part of the organisers. If it’s your job to make a company wellness programme a great success, who are you going to encourage to take part – the health-conscious individual looking to get back in shape or the unrepentant, donut-scoffing flubba-wubba sat at the next desk?

There is a solution though: the randomised controlled trial – where participation is randomly assigned.

Carroll describes one such trial – of a wellness programme among university employees. This found no significant difference in outcomes between the intervention group and the control group. But here’s the twist:

“…the researchers also took the time to analyze the data as if it were an observational trial. In other words, they took the 3,300 who were offered the wellness program, then analyzed them the way a typical observational trial would, comparing those who participated with those who didn’t.”

When the trial was ‘de-randomised’ – i.e. based on simple observation of participants versus non-participants the hoped for differences in outcomes did appear. Furthermore, most of those differences remained even when factors like age and prior healthiness were controlled for.

Clearly, selection bias is a powerful thing.

Further reading

Wanted: a government willing to perform experiments on us

By Peter Franklin

There are some big implications here for public policy.

The idea that public resources should be allocated on the basis of evidence of effectiveness is a perfectly sound one. Furthermore, with the advent of ‘big data‘, the state is acquiring the ability to observe the outcomes of government intervention in greater detail than ever before.

However, the danger is that with so much data to play with, policy-makers will overlook the confounding factors of selection bias and other epistemic pitfalls.

Having left behind the dark age of pure guesswork, policy-making is set to enter a new age of false confidence.

However, they weren’t the first Earthlings to go extraterrestrial – that honour belongs not to mankind, but to tortoise-kind. On 18 September 1968, two tortoises, some flies and various other creepy-crawlies were launched into space aboard the Soviet Union’s Zond 5 mission – the first spacecraft to circle the moon and come back home again.

If life has evolved nowhere else in the universe, then that means the history of extraterrestrial life is a mere 50 years old – beginning with a couple of commie tortoises and their invertebrate fellow travellers.

Further reading

Mission to make money: a new space race is taking off

By Nigel Cameron

But how likely is it that life is unique to Earth? Not very, one might think. The universe is a big place – and one in which planets not unlike our own are thick as blackberries.

We don’t know how life first arose. We’ve yet to replicate the process in a laboratory. Yet, it must be possible, otherwise we wouldn’t be here. We also have evidence, albeit patchy, that the evolution of intelligence is possible. So, given the mind-boggling number of opportunities for it to happen elsewhere, it would seem exceedingly unlikely that we’re alone.

But how do we calculate the odds? In a fascinating piece for Vox, Liv Boeree writes about the main statistical tool at our disposal – the Drake Equation:

“In 1961, astronomer Frank Drake proposed a formula that multiplied seven ‘parameters’ together to estimate N, the number of detectable civilizations we should expect within our galaxy at a given moment in time.”

Even if you put in some very low figures for parameters like the fraction of planets that develop life, the chance of that life getting civilised, and the proportion of civilisations that send signals into space, the fact that key multipliers – like the number of planets – are so high should mean a universe positively cluttered with evidence of alien life.

Yet when we looked, we found nothing:

“In 1950, while working at Los Alamos National Laboratory, physicist Enrico Fermi famously exclaimed to his colleagues over lunch: ‘Where is everybody?’

“He had been pondering the surprising lack of evidence of other life outside of our planet. In a universe that had been around for some 14 billion years, and in that time developed more than a billion trillion stars, Fermi reasoned there simply must be other intelligent civilizations out there.”

Now, 68 years later, and despite numerous advances in astronomy and spaceflight, we’ve still got nothing.

Hence the Fermi Paradox – which juxtaposes what the Drake Equation would suggest is out there and the continuing lack of evidence that anything is.

Is the Drake Equation misleading us?

“…the Drake equation is ultimately vulnerable to the optimism or pessimism of whoever wields it. And this is reflected in previous scientific papers whose results give values of N ranging anywhere from 10 to many billions.

“As astronomer and SETI co-founder Jill Tarter eloquently put it in an interview with National Geographic in 2000: ‘The Drake Equation is a wonderful way to organize our ignorance.’”

Boeree writes about a recent update to the equation. This uses ‘Bayesian logic’ – a statistical method for revising estimated probabilities on the basis of subsequent observations. Along with another statistical tweak, the researchers used the Bayesian update to recalculate the probabilities:

“This two-stage process produced striking results: Based upon the current state of astrobiological knowledge, there’s a 53 to 99.6 percent chance we are the only civilization in this galaxy and a 39 to 85 percent chance we are the only one in the observable universe.”

Of course, this revised method is itself a “wonderful way to organise our ignorance” – albeit in a more sophisticated manner.

Nevertheless, the underlying Bayesian principle is a sound one – the longer and harder we look for evidence without finding it, the greater the likelihood that intelligent alien life is either freakishly rare or that we’re alone in the universe. 

In my view, proof of humanity’s unique status would be of infinitely greater importance than the discovery an alien civilisation. Indeed, it would be the most important scientific discovery ever made – one demanding an explanation that a lot of people wouldn’t like.

Still, at least they could comfort themselves with the thought that a tortoise won the space race. Aesop would be proud.

Further listening

Test article

By Charlotte Pickles, Nigel Cameron and Peter Franklin

Leaving aside very small countries such as Monaco and Luxembourg, what is the most densely populated nation in Europe?

The conventional answer is the Netherlands with 505 people per square kilometre, followed by England on 426.

And yet according to Alasdair Rae, the true answer is Spain. Rae may be a professor of urban studies and planning at Sheffield University, but how can he possibly be right? Spain’s population is smaller than England’s, while it’s geographical area is almost four times bigger. That surely settles the matter.

Or does it? Rae’s insight is that the Spanish population isn’t spread out over Spain, but rather a small part of it. This is how he puts his case in a thought provoking piece for the Conversation:

“A good way to understand this measure is to look at Spain. It has a population density of 93 people per km², giving the impression of a sparsely populated country. This is borne out in the map, where much of Spain appears to be empty; much more so than any other large European country.

“…Yet characterising Spain as a sparsely populated country does not reflect the experience on the ground – as anyone who knows Barcelona or Madrid can tell you.

“Spain contains within it more than 505,000 1km squares. But only 13% of them are lived in.”

By recalculating population density in each country using only the land area where people actually live he produces a measure of what he calls “lived density”.

By this yardstick, Spain’s population density leaps from 93 people per square kilometre to 737 – way above the equivalent for England and the Netherlands. From a human point of view, Spain is a densely populated country that happens to have a lot of empty countryside attached to it. Another good example is Iceland – which has a conventional population density of just 3 people per square kilometre, but a more convivial 187 when it comes to lived density.

All of this is an object lesson in the way the statistics can be demonstrably ‘true’ and yet completely misleading as to how people actually experience their lives. The political consequences of this phenomenon are explored in ‘Blindsided’ – a special series for UnHerd from our Flyover Country editor Henry Olsen.

Recommended reading

The elites missed populism's rise (but shouldn't have done)

By Henry Olsen

There’s a further point worth unpacking from Professor Rae’s detailed analysis of population density.

It’s often argued that because just 11% of England is built on, there’s plenty of room to build new houses while leaving most of the English countryside untouched. This is true (even if it does ignore the fact that the urbanising impact of development extents beyond its immediate ‘footprint’). However, it distracts from the fact that we could, if we were willing to do the redevelopment, pack a lot more living space within our cities – especially the global ones – which, crucially, is where the highest demand is:

“Spain has the most densely populated km² in Europe; more than 53,000 people inhabit a single 1km² area in Barcelona. France also has an area with more than 50,000 people in a single km², in Paris.

“There are 33 1km² areas across Europe with a population of 40,000 or more: 23 are in Spain, and ten are in France. England’s most densely populated km², in West London, has just over 20,000 people in it. “

As I’ve argued before, cities like London could densify – and furthermore do it sympathetically, even beautifully.

Traditional architecture and street patterns are not only compatible with density, they’re a time-tested way of sustaining it over decades and centuries.

If we want to push up ‘lived density’ then we must make density liveable.

However, that doesn’t mean that one shouldn’t question widely agreed assumptions – especially when they provide a foundation for government policy. It’s worth checking just how broad a basis of evidence the whole edifice rests upon.

Via Medium, there’s a great example from Ian Mulheirn, who scrutinises the widely accepted narrative for why we’re not building enough new houses. His focus is on the UK – England in particular – where he says that estimates for the number of new houses needed each year depend on projections of average household size.

Mulheirn explains why these projections are so important:

“The origins of the UK’s mythical housing shortage lie in the methodology used to forecast household formation. Only at census time do we really know how many households there are in England, and in between censuses we have to rely on forecasts based on past trends in the size of households of different types, combined with population estimates. The household size projection is the crucial element: the smaller the average household, the more homes are needed for any given population.”

Unfortunately, the UK Government keeps getting these projections wrong – assuming a continued decline in household size that has long since halted:

“During the last third of the 20th century, household size shrank rapidly. DCLG implicitly assumes that this trend is set to continue, basing their projections on census data starting in 1971.”

But the evidence we have suggests that English households — in keeping with most other developed OECD countries — stopped shrinking around the start of this century.

The UK Government does not deny that this has happened. It’s in the 2011 census data, so they could hardly do otherwise. However, according to Mulheirn, the Government is projecting a resumption of the downward trend.

Might this be a reasonable assumption? Could it be that the upward spiral in house prices has had an inhibitory – but temporary – effect on the formation of new households? Perhaps, but as Mulheirn points out, the halt in the trend towards smaller households goes far beyond the UK and its peculiarly overheated housing market.

In any case, one can hardly expect the decline in household sizes to go on for ever. There are natural limits – like the proportion of couples who actually want to split up or the average age at which young people want to leave the family home.

I’ve met some of the government analysts responsible for these projections. They are perceptive and conscientious professionals and I’d be surprised if they hadn’t recommended reforms to the methodology.

Recommended reading

If you don't 'get' statistics it's just as bad as not 'getting' Shakespeare

By Graeme Archer

The problem, however, is that the current approach, though statistically flawed, is politically convenient. After all, the argument that we need to match the number of new houses to the number of new families is seemingly unanswerable. Of course, what the politicians really want to do is restrain house prices – but that’s not something they’d dare say to the millions of voters who already own their homes.

Another reason why the politicians don’t want to get into a discussion about house prices, is that they reveal where housing is in greatest demand – i.e. usually where the most dynamic local economies are. It would make sense to allow as many people as possible to move to where they can use their talents most productively and hence earn the most money, but that would require the Government to ease key restrictions on new development – another politically contentious issue. It might even require the Government to have an explicit spatial policy – i.e. a geographically-literate economic policy.

Whether in Britain, America or elsewhere in the western world, housing markets are deeply dysfunctional. But there is one way in which they work exactly as markets ought to: through the price mechanism they provide a remarkably rich and detailed source of data about exactly where new housing capacity is most needed.

This is about as close to the economic concept of perfect information as we’re likely to get; unfortunately our politicians seem determined to ignore it.

You may not have heard of the most surprising scientific result of all time. It was published in the journal Psychological Science[1.  ‘False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant’, 2011. Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn], and it found that listening to ‘When I’m 64’ by the Beatles literally made you younger by nearly 17 months.

Not “made you feel younger”, or “made your brain younger”: the study demonstrated that listening to certain songs made you younger. And these surprising findings met the standard of “statistical significance” which most scientific journals demand.

In fact, the three authors of the study had set out to deliberately make a point. They wanted to show that, by using routine statistical methods[2.  Scott Alexander, ‘Two dark side statistics papers‘, Slate Star Codex, 2 January 2014], you could find apparently solid results that were not merely unlikely but patently absurd.

The problem is that the standard of “statistical significance” is too easily subverted. And to prove it, the three authors used tricks to find relevant-looking patterns out of pure, random fluke results. We might call their techniques “cheating”, but they are routinely used in almost all walks of science.

Say you’re running an experiment into whether or not a coin is biased towards heads. You flip the coin three times; it comes up heads three times. Does that prove your hypothesis? No: sometimes you get three heads in a row just by chance.

That’d be called a “false positive”. Your chance of getting three heads in a row on a fair coin is 1 in 8, or as a scientist would write it, p=0.125.

In 1925, the great statistician RA Fisher arbitrarily decided[3.  Statistical Methods for Research Workers, RA Fisher, 1925] that “significance” should be defined as p=0.05, or a 1 in 20 chance of a false positive. That definition is the standard that most research is still held to. Your three heads in a row wouldn’t do it, but five would: that’d be a 1 in 32 chance, or p<0.03.

The “When I’m 64” study had p=0.04, and thus met the threshold.

To prove the thesis, they did a simple test: they took 20 subjects, randomly divided them into two groups, and then played one group the Beatles and one group a control song, “Kalimba” by Mr Scruff. In theory, randomly dividing the two group means that they should be, on average, the same; the only difference should be the intervention, which song they listened to. And that means that, in theory, if you find a significant difference between the two groups, it has to be caused by that intervention. It’s a randomised controlled trial, just the same as is used to test cancer drugs.

And the study found that the “When I’m 64” group was, on average, 1.4 years younger than the control group, and found that this difference was statistically significant. So they had proved – to the standard most journals demand – that “When I’m 64” causes you to get younger.

Essentially, they gave themselves lots of chances to get that 1 in 20 fluke, then hid all their failed attempts at getting five heads in a row.

For instance, as well as making their 20 participants listen to the Beatles and Mr Scruff, they also asked them to listen to “Hot Potato” by the Wiggles. And they also asked 11 other questions, including how old they felt, their political orientation, whether they referred to the past as “the good old days”, and so on.

This allowed them to look at lots of different possibilities which might demonstrate their conclusion. Did the people who listened to “Hot Potato” happen, by chance, to be more politically conservative? Did people who listened to the Beatles say “the good old days” more often? No? Then the scientists didn’t report on that. There were dozens of ways to analyse the data, easily enough to get a 1-in-20 result just by fluke.

This behaviour is called “hypothesising after results are known”, or HARKing[4.  Norbert L. Kerr, ‘HARKing: Hypothesizing after the results are known‘, Personality and Social Psychology Review, 1 August, 1998]. It means waiting until you’ve got your data, then sifting through it in imaginative ways until you find something that looks like a result. It gives you a much higher chance of (falsely) reaching a “significant” finding: using tricks like this, collectively known as “p-hacking”, Simmons, Nelson and Simonsohn say that the chances of a false positive can rise above 60%.

Recommended reading

If you don't 'get' statistics it's just as bad as not 'getting' Shakespeare

By Graeme Archer

And this happens all the time, in real papers. A study by the Oxford University Centre for Evidence-Based Medicine[3.  The Centre for Evidence Based Medicine Outcome Monitoring Project (COMPare), Ben Goldacre, Henry Drysdale, Carl Heneghan, 18 May, 2016] found that the top five medical journals in the world regularly publish studies which change what they’re measuring after the trial begins, a practice related to HARKing.

It isn’t that scientists are frauds; it’s that the incentive system in science is flawed

-

It isn’t that scientists are frauds; it’s that the incentive system in science is flawed. As a community, we want science to build up a reliable body of knowledge. But as an individual scientist, you are incentivised to publish lots of papers showing exciting, novel results. And when the individual’s incentives don’t align with wider society’s, the individual incentives will win out almost every time. The trouble is, this wastes resources, skews results and creates fake narratives.

Chris Chambers[4.  Chris Chambers, ‘Psychology’s registration revolution‘, The Guardian], a professor of psychology at Cardiff, tweeted recently that he’d had a paper rejected for having conclusions that were “not necessarily novel”.

Short thread. Had a paper accepted today at @royalsociety Open Sci & another rejected at Appetite.

Here’s some review comments from Appetite, not as a whinge (I’m too old & leathery to complain about reviews) but as a reminder of the challenge we face reforming the field /1

— Chris Chambers (@chrisdc77) January 11, 2018

“We could have been more novel,” he wrote. “We could have HARKed this paper up the wazoo or p-hacked it. We didn’t.”

The trouble is a deep-seated flaw in the process of “peer review”. “They judge the work,” Chambers said, “not just on how important the question is and how well the study is designed, but also whether the results are novel and exciting.”

Everyone has their own set (I hypothesise); there are three, in my case:

• Gosh, what do you eat at Christmas? (Not meat, you savage.)
• Heavens, were you the only Scottish Conservative? Ha! Ha! (This is asked less often post-Ruth, thank goodness.)
• And, time after time in the workplace: you’re a statistician … but you also write, about politics? Isn’t that … rare?

This last enquiry channels Doctor Johnson’s remarks about dogs on their hind legs. The miracle for a statistician is that you can communicate in English at all, is the subtext, let alone string words together well enough to have someone pay to publish them. Conversely, in the eyes of my (political) editors I often see a half-suspicion that I’m making up the statistics career. Either (words)/or (numbers), is our default setting. Competency at both is sensed as bizarre.

Here’s the first seeming-paradox. There is no “either/or” between words and numbers; at the same time, of course, there is. CP Snow’s two cultures exist, most clearly in the words/numbers dichotomy, but only because we structure them into existence through conscious decisions about how to educate our young. It’s a human choice, not a law of the universe. Good with words or good with numbers is a story we tell about ourselves, and like all stories it has its heroes, and villains.

Good with words, and you can read law or PPE, become a barrister, then an MP, then Prime Minister. Not your only option: good with words, and you can read novels for three years, become a TV producer, then commission dramas about how awful the politicians are.

Good with numbers, on the other hand (and therefore bad with words), and… well. You won’t need to worry about competitive sports, at least, and there’s always the solace of Bach.

How else to explain the fake-boasting. “I was never good at maths!”, they trill, while looking over your shoulder. They mean “Thank God I never wasted my time on that rubbish.”

Here’s the epitome of the either/or value system, Simon Jenkins, breathing his lofty disdain for the very concept of maths-at-school: “The prominence of maths in the curriculum is education’s version of Orwell’s imaginary boot, ‘stamping on your face … forever’.”

According to Simon, encouraging children to be good at maths is like a totalitarian regime that pivots on the destruction of romantic love. Beggin’ your ‘umble pardon, sir, an’ I knows I’s only a statistician, but I thinks you’ve misread your Orwell.

This matters, politically. A few years ago, the then Prime Minister, David Cameron, wanted to improve facility with English among Britain’s immigrant communities. He was chastised by his Left-wing opponents for focusing efforts on women with (mainly) Bangladeshi and Pakistani backgrounds.

In my teabreak one day, I looked up the official statistics which broke down English ability by ethnicity and geography-of-origin, and calculated the odds of lacking English in each of those categories, and compared these between women and men.

It did indeed appear that women with Indian, Bangladeshi and Pakastani backgrounds were between four and five times as likely to lack any English ability, as compared to their menfolk. But I listened in vain to hear mention of such statistics in any political or journalistic commentary.

Government statisticians would have calculated any of those odds ratios for ministers, had the ministers asked for them. But my unofficial calculations took a few minutes prodding with an excel spreadsheet: it shouldn’t require a government department to help guide ministerial or journalistic intuition. The point is that such a basic facility with numbers should be taken for granted amongst the people who rule us.

This widespread political innumeracy means that the numbers which do make it into the public domain are treated with awe, like holy relics in the age before Luther. “Never mind how poor you feel. Look at the GDP [and ignore the tails of the income distribution].”

In Britain, Gordon Brown took advantage of statistical ignorance to fight and win a useless “war” against poverty. He defined an arbitrary point on the income distribution as “poverty”, and spent shedloads to move people from one side of it to the other. If poverty is defined as “income lower than £10,000”, and your income is £9,999.90, and Gordon gives you 11 pence, you are no longer poor, according to Gordon.

Journalists (good with words not numbers) repeated his claims about the numbers of people “moved out of poverty”, without once, to my memory, asking for the average improvement in income for such people, or its impact on the shape of the income distribution. Lacking statistical ability, most journalists don’t even know how to ask such questions. Meanwhile, after Gordon spent all our money, the objectively poor continued to shiver in misery at the extreme left-hand tail of the income curve.

There are hidden costs as well as these obvious ones. If you don’t know how to interrogate data, you’re unlikely to know how to design studies to test policy efficacy (all those pointless “pilot implementation schemes”, that produce data unfit for decision-making, rather than prospectively planned, randomised trials). This increases the chance, obviously, of bad policy decisions (and is nothing to do with Left vs Right.) From educational methods to poverty reduction, lack of properly planned trials (with randomisation) handicaps the development of evidence-based politics. And few politicians or journalists even appear to notice.

The drug development industry is obsessive about missing data, because the things we cannot measure (think of patients choosing to leave a clinical trial, and count their reasons for leaving) have a fundamental impact on the assessment of drug efficacy and safety. Here, too, there is an analogue for the price we pay for privileging words over numerical thinking: as Henry Olsen wrote in his excellent piece recently, “nobody” saw the rise of populism coming, mainly because “nobody” who writes about politics lives in the areas that gave Trump and Brexit their victories.

All those nobodies would have done better had they applied some basic statistical reasoning. “Everyone I know supports Remain/Clinton. But I can see from the polls that a significant chunk of the country doesn’t. What am I missing?”

Statistical thinking, in other words — not expertise in complex methodological topics, but basic statistical thinking — should be a desirable characteristic of a well-rounded citizen, whatever their field of work. There should be no either/or, and though (I’m only a statistician, remember) it’s a cliché to say so, it remains true: the person who can’t interrogate data is as limited — perhaps more so — than the citizen who doesn’t “get” Shakespeare.

We have the politicians and the journalists we deserve, of course: a depressing thought. So here’s a game to take your mind off things. Imagine three doors in front of you: A, B and C – behind one of them, I’ve parked the car of your dreams; behind the other two I’ve left photographs of my beautiful cat. (She’s alive, whatever Schrödinger says – it’s just her photo, and this isn’t a trick.) All you have to do to win the car is pick a door — and I’m going to give you two chances.

You pick door B – it’s in the middle, why not, principle of indifference, right? I open door A, and show you a lovely picture of Kitty (awwwww.) And I ask you, now we know that door A isn’t the car: do you want to stick with door B? Or switch to door C?

Practically everyone — this is one of the most famous “paradoxes” of them all — says “It doesn’t matter whether I switch or not. There’s a 50:50 chance the car is behind door B or door C.”

In fact, the chance of finding the car increases, from 33% (before the first door was opened) to 67%, if you switch. Lots of people become angry about this. It’s not the imaginary car, really, that they’re angry about. It’s having been forced to choose either(words)/or(numbers) at far too early an age. There are too many obvious substitutes for “car”, after all, and the “game” is a bit close to real life for comfort.

If you think anger is an incorrect response to the unalterable laws of probability; if you agree that dismissing “all that numbers stuff” is a defect, not the mark of a civilised human; if you’re fed up with politicians rattling off statistics without journalists deconstructing them: then my request of you is small.

When your children are picking subjects for school, encourage them not to drop maths. For as long as they’re in education, in fact, encourage them not to drop maths. Don’t perpetuate the either/or myth (or look over my shoulder at parties). Statistical thinking is as important as the ability to spell — neither Alexa nor Siri will shout “sloppy reasoning” when they read out the morning news headlines. And if you disagree with my conclusions, then I’m sorry: but I don’t think your numbers add up.