A couple of weeks after the switch, a bill of around £1,000 arrived on her mat. She had only been living in her new home for a few days. For weeks, then months, Jan emailed and phoned. She spoke to customer service bots, payment teams, even engineers. She created accounts and new passwords. At one point a new customer account offered hope, before another £1,000 bill was autogenerated. The Bill remained; nothing could shift it.

The case is unresolved. One computer-literate family member has dedicated dozens of hours to The Bill, but still it looms. Nothing and no-one can win an argument with the automated system. Months of Jan’s life have been consumed by this; and her confidence in dealing with any kind of technology is now irreparably damaged.

Jan is a victim of “techno-admin”: it’s a pervasive phenomenon, whereby we customers are forced into infuriating, confusing, absurdly time-consuming and bleakly unrewarding tasks by a machine. You probably have a similar story. How many incorrect bills, unprocessed address changes, reminder notices printed in error? How many sleepless nights? It is the scourge of our age.

It’s all over the news, too. The energy company EDF recently issued artist Grayson Perry with a £39,000 monthly electricity bill, which to a machine seemed like a perfectly normal £38,700 increase on the previous one. Perry spent three tedious hours fixing it. And of course, there is the Post Office Horizon scandal: techno-admin at its cruellest. At the sharp end, postmasters were incorrectly told by Fujitsu’s accounting software that they were in arrears, and the Post Office subsequently hounded them, ruining hundreds of lives, careers, and reputations. At the softer end, thousands endured hours of stressful, fruitless phone calls and complaints. All because the people in charge found it easier to trust a machine than ask difficult questions of their own organisation.

More from this author

This is how democracy dies

By Jamie Bartlett

We are all sub-postmasters these days: each of us daily dealing with computer systems which make our lives harder. How much of your workday is taken up with tasks like the following: filling in a lengthy online form, which crashes just as the finish line draws near; spending hours trying to cancel an online subscription; coming face-to-face with the dreaded “Schrödinger’s account”: you try to sign into an it using your email address, only to be told there is no such account; you try to create a new account with the same email address, and you are told one already exists.

These are rarely the consumer’s fault. Companies make errors, which you end up paying for. When Hackney Council was the victim of a ransomware attack a couple of years back, the service disruption was well-documented. Less discussed is the amount of techno-admin imposed on the borough’s 250,000 residents as a result. The fact that legitimate organisations routinely make such enormous mistakes makes it a lot easier for criminals to operate. I know of one person whose number plate was ‘cloned’: someone is driving around with his registration on their car, and this poor chap has spent hours appealing fines because ‘the system’ can’t or won’t recognise that it is not actually his car.

And last week, I spoke to a fraud professional who told me his son-in-law’s identity was stolen by SMS fraudsters who pretended to be his phone provider. They re-registered his number, got into his mobile banking and tried to take out loans in his name. After two months of effort, he still hasn’t got his old mobile number back. It’s unfair to blame the victim: many banks insist on some text-based communications, although they aren’t as secure as people assume. It won’t be long before everyone is terrified of answering their phone, lest it be a sophisticated scam.

Suggested reading

Why is my chatbot hitting on me?

By Kathleen Stock

Could the UK’s productivity problem, which has flatlined since 2010, be partly caused by the surge in techno-admin? Large firms use automation and digitisation to cut staff and reduce overheads, especially when it comes to customer service — but what they have actually done is outsource the admin work to us, the customer. We are the ones now form-filling, changing passwords and fixing mistakes. We are the ones paying to sit in a telephone queue for an hour. And don’t forget techno-admin has to be carried out during work hours, because lines close at 6pm.

Forcing us to do techno-admin doesn’t even save the companies much money. One business processes specialist I spoke to explained that the moment things are automated and digitised, complicated software is involved. Which means that whenever there is a problem, you need high-skilled and, therefore, well-paid staffers. Or you need to call on the software providers — the Fujitsus of this world — which just adds more layers and delays and expense. This is probably why the UK’s chief auditor recently claimed that the government could save £20 billion by — among other things — modernising its IT systems. In the NHS, outdated IT systems are thought to have contributed to multiple deaths.

Suggested reading

The horror of deepfake nudes

By Nina Schick

“Automation applied to an efficient operation will magnify the efficiency”, claims Bill Gates, but “automation applied to an inefficient operation will magnify the inefficiency.” In other words, automation mostly works when you can standardise inputs and outputs. But humans aren’t a Ford factory production line: we’re messy and things go wrong all the time.

In the coming years, more decisions and tasks will be outsourced to ever smarter AI: decisions relating to spending, sentencing, military tactics, housing allocation. But ever fewer people will understand how any of these systems actually work. Staff won’t dare to question them.

Picture the frictionless future of ubiquitous smart machines: you head downstairs in the morning to make a coffee when — oh! The smart coffee machine needs a software upgrade! Unfortunately, you’ve lost your account details, and cannot recall which of your 25 email addresses you signed up with in the first place. But no time to fix that now, because you are struggling to log into the kids’ remote homework platform. Apparently, your laptop needs a new app, which will only work if you download some other software first. In the meantime, the bank’s been in touch with an urgent fraud warning — at least, the voice on the line sounded like the bank, but who can really be sure these days — so now you need to change all your passwords.

It won’t be long before simply living will be a full-time job. And at that point, UK productivity will finally hit zero. Then, and only then, when the economy has collapsed into a black hole of rage and wasted time, might we rebel against the pointless, painful scourge of techno-admin.

The reason these annoying little pop-ups exist at all is thanks to a recent-ish law which helps you understand what’s being collected about you online. It’s something that probably deserves a moment of your attention. But we’re spending more and more time online; who wants to waste it checking 20 tedious Ts&Cs a day?

The future of the internet will come down to tiny things like this: subtle decisions made at the engineering or regulatory level — stuff which most of us ignore because we can’t be bothered to find out what it’s all about. Are you curious about “digital object architecture”? Do you want to know who’s behind the “Internet Corporation for Assigned Names and Numbers”? A colleague of mine once spent weeks tracking this stuff — only to conclude that internet protocols are “simply too impenetrable and boring to feel angry about”. Nobody cares. Nobody, except the autocrats of the world, who are quietly trying to wrestle control over various working groups and technical decisions — to make sure they can tame the digital beast.

Given our abject dependency on it, it’s scary to think of the internet as a flimsy experiment. But it’s already changed shape several times over its short life: from a government-funded research project for military scientists, to an academic network, to a vehicle for e-commerce and now a platform for social content. Through it all one thing remained constant, in theory if not always in practice: the idea that the internet was a single open network which anyone could join, and where everything was connected via a standard set of protocols and rules.

Suggested reading

The biggest threat to your online freedom

By Carl Miller

According to a new book, that’s about to change. Four Internets argues that the era of a single internet might be drawing to a close, replaced by a balkanised network of different versions living alongside each other. Scholars Wendy Hall and Kieron O’Hara reckon four internets in particular are already taking shape.

The “Silicon Valley” model is open and libertarian — anyone can join the network, no single authority is in charge and there are few rules about what sort of information can be transported across the network. Its modus operandi is to ask for forgiveness rather than permission. This is the original vision and remains just about still on top.

The “Washington DC” version, by contrast, prioritises data collection and corporate interests. It doesn’t mind big tech firms collecting massive amounts of information on citizens, providing it generates profits and excellent products for consumers to enjoy.

Then there’s the “Brussels bourgeois” model which is “ordered, respectful, polite, decent, well-behaved, well- mannered, and considerate” (and also a little dull). Preferred by most EU countries, it is quick to launch anti-trust cases and punish companies that “move quickly and break things”. It is the Brussels vision that created “GDPR” privacy rules, the “right to be forgotten” and those damn cookie laws.

Finally, there is the fast-developing “Beijing” internet, which is paternalistic and tightly controlled by the government. Back in 2000, Bill Clinton told a Chinese trade delegation that cracking down on the internet would be like “trying to nail jello to the wall … good luck”. But over the last 20 years the Chinese government has gradually brought the internet to heel through its doctrine of “cyber-sovereignty”. They’ve done it slowly and cleverly: China’s “deep packet inspection” technology checks what’s coming in from the rest of the world; IP addresses they don’t approve of are turned away. All internet content providers are held liable for what appears on their websites, under an insidious “you yourself decide” code, which encourages everyone to turn into draconian self-censors to avoid trouble.

More from this author

This is how democracy dies

By Jamie Bartlett

Where does the UK stand in all of this? It’s hard to imagine our government would have either the competence or the inclination to go as far as China’s, but what if the Beijing model is the future? In his recent book The Great Firewall of China, James Griffiths argues the Chinese model is starting to spread. Most visibly in Africa, where various governments are happy to buy up cheap, decent Chinese technology — and in some cases the surveillance or AI technology that can help them stay in charge. Although as some analysts have argued, it’s not always Beijing pushing the Beijing model — Western companies are also happy to assist snooping autocrats with cheque books. At home, Westminster is liable to fall for the allure of control. While we might not import Chinese technology itself — especially after the furore with Huawei and the 5G network — the promise of being in charge of the internet is irresistible.

And so, our internet is getting increasingly authoritarian. You may recall the Investigatory Powers Act 2016 (aka the “Snoopers Charter”), which, among other things, required internet companies to keep a record of the websites you visit for a year. Earlier this year it was revealed that the government was testing a creepy new tool as part of the Act — surveillance technology that could track and record everything that every single person in the country does on the internet. Funnily enough, the Chinese government cited the Snoopers Charters when defending their own draconian surveillance techniques.

There also seems to be a running game of cat and mouse between the police and campaign groups over the use of facial recognition technology. Last year the Court of Appeals deemed that South Wales Police’s use of live facial recognition technology was unlawful — a win for campaigners. But this year some forces started trialling retrospective facial recognition technology instead.

Then there’s the proposed Online Harm Bill, currently inching through Parliament, which aims to make companies more responsible for user safety on their platforms. It will force tech firms to take on a “duty of care” and remove “harmful content” while keeping up “democratically important” stuff. There’s certainly plenty of bad stuff online, but phrases like “duty of care” and “harmful content” are terrifyingly broad — and reminiscent of China’s “you yourself decide” code. If the British bill passes, it will invariably mean proactive and excessive censorship — with government setting the tone.

Suggested reading

The Chinese future isn't bright

By Peter Pomerantsev

And it probably will pass, because the truth is the public seem to quite like the Beijing model, even if they wouldn’t call it that. According to a recent YouGov survey, 78% of us want users to disclose their real identity when signing up on social media — a petition about this hit almost 700,000 signatures recently. It’s easy to understand the instinct — this followed the controversy over racist attacks on England footballers after the team lost the European Championship final, much of which came from anonymous social media accounts. Labour MPs Margaret Hodge, Angela Rayner and Jess Philips have all argued that de-anonymising users will force them to be accountable for their actions.

If the goal is noble, the method is not. For one thing, it’s not clear that such a move would deal with the problem, given a decent chunk of online abuse comes from anonymous users overseas (should we ban them too? How?) Besides, cyber-disinhibition is also caused by the lack of face-to-face, immediate contact, which wouldn’t change — unless of course the government forced us to hand-deliver our tweets and posts in person to the recipient. And besides, after everything we know about Facebook, are we sure we really want to hand our passports or driving licenses over to Zuck? What of those who have good reason to stay anonymous online — the whistleblower, the undercover journalist, the teen in a self-help group?

Anonymity used to be seen as a benefit of online life: a chance to say what you really think, to safely explore new ideas and new identities. “On the Internet nobody knows you’re a dog,” went the famous New Yorker cartoon, and it wasn’t meant as an insult. Now Fido and Patch are a menace to society.

When you add this all up, it’s possible to see the UK drifting accidentally into its very own authoritarian internet. Not up to Beijing’s Olympian standards, perhaps, but where censorship in the name of harmony trumps all and where surveillance is built in to the rules and the tech.

As the internet becomes more embedded in our everyday life, that will take on an even more sinister meaning. Any minor infraction in our self-driving cars will result in a system override and an automatic locked-door ride to the nearest police station (“in the interests of pedestrian safety!”). Smart fridges will hector you about meat consumption levels and simply refuse to open if you’ve been recorded saying anything rude or offensive (“in the interests of the nation’s health!”). The internet will become a far more sanitised, controlled and monitored place than it already is — and by extension so will all of us.

Forget for a moment the flashier signals of technological process: AI-powered personal assistants, Boston Dynamics back-flipping robots or blockchain cheerleaders. The two most important trends in the field of technology are quiet and relentless: increasing volumes of data and declining cost of computing power. In the long run they mean machines will, despite frequent hiccups, keep improving. They already outperform humans in a small but growing number of narrow tasks, but it’s unlikely we’ll see general artificial intelligence any time soon — much less the AI-goes-rogue scenario. Still, machines will gradually take over more and more decision-making in important areas of life, including those which have ethical or political dimensions. Already there are signs of AI drifting into bail conditions, warfare strategy, welfare payments and employment decisions.

The problem isn’t whether machine decisions are better or worse — that’s often a question of values anyway — but whether it’ll get to the point where no one will be able to understand how decisions are made at all. Today’salgorithms already deal with millions of inputs, insane computing power and calculations far beyond human understanding. Frankenstein-like, most creators no longer really understand their own algorithms. Stuff goes in and stuff comes out, but the middle part is becoming a mysterious tangle of signals and weighting. Consider the example of AlphaGo, the AI system that astonished the world by thrashing the world’s best Go player, before astonishing it a second time by thrashing itself. Aeronautic engineers know precisely why their planes stay in the air; Alpha Go’s inner workings were and are a mystery to everyone. And by 2050, Alpha Go will be fondly remembered as a child-like simpleton.

There will be seminars, lessons, bootcamps, and online training courses about how to work with The Algorithm. Don’t worry yourself overly, human! Singletons: Learn the best combination of words to secure your dream date! Join our “beat the algo” seminar where you will learn how to ensure your CV outwits the HR filtering systems. Use our VPN to trick web browsers into thinking you’re from a poorer neighbourhood to secure a better price! A few months back a handful of bootcamps opened, where parents pay $2,000 for experts to teach their kids how to succeed on YouTube. Some scoffed, but I suspect similar courses will soon be the norm. These will be the warning signs of a confused and frightened society.

Imagine a 21-year-old happily bouncing through life in the 2050s. His entire life will have been datafied and correlated. His sleep patterns from birth captured by some helpful SmartSleep ap; his Baby Shark video consumption aged 2 safely registered on a server somewhere. All those tiny markers will help guide his future one day: his love life determined by a sophisticated personality matching software, while his smart fridge lectures him about meat consumption (insurance premiums may be impacted you know!); his employment prospects determined by a CV checking system 100 times more accurate than today’s. His cryptocurrency portfolio automatically updating every half nano-second based on pre-determined preferences. His political choices and opinions subtly shaped by what pops up on his screen controlled by AI-editors using preference algorithms that have been running for 50 years.

More from this series

Is this the end of the world?

By Douglas Coupland

It sounds bad, but not apocalyptically bad, right? But imagine, now, that our 21-year-old is so impudent as to question or object to what these brilliantly clever systems are offering him up. There would probably be no obvious number to call with a complaint. He might try to sue the CV-checking software designed for the subtle discrimination he suffered — but the judges will throw the case out because the software designer has been dead for 30 years and they still don’t really understand what an algorithm is anyway.

The problem with such a machine-dependent world, then, is not what you might think. AI theorists spend a lot of time worrying about something called “value alignment”. It is a hypothetical future problem where a hyper-powerful AI takes instructions literally, with disastrous results. The most famous example is the “paperclip maximiser” where an unsuspecting factory owner asks an AI to make as many paperclips as possible — and it ends up turning the entire universe into paperclips. But I doubt you’ll need to worry about paperclips: you’ll be too busy on the phone to machine-like bureaucrats who can’t help with your application, because the machine has made a decision and the person who okayed it is off sick and the person who built the tech now works in Beijing and…

Confusing machines will annihilate accountability, which is one reason powerful people will like them. A couple of years ago UK health secretary Jeremy Hunt told the House of Commons that “a computer algorithm failure” meant 450,000 patients in England missed breast cancer screenings, and as a result many as 270 women might have had their lives shortened as a result. Who was responsible for this murderous and despicable “computer algorithm failure”? The tech guy who wrote the software, in good faith, years ago? The person who commissioned it? The people feeding the data in? Unsurprisingly, a subsequent inquiry into all this found that “no one person” was to blame. Nothing has been done in response, and nothing will. More recently, Boris Johnson blamed a “mutant algorithm” for the A-level fiasco — how convenient! Expect algorithms to become every politicians’ non-apology apology by the 2030s.

Around this time, the first casualties from driverless car accidents will start arriving in A&E. The subsequent enquiries will conclude that “no one person” is responsible for the deaths, either. It will instead be the fault of “unforeseen system incompatibilities” and “data corruptions” that make no sense, and offer no comfort, to anyone.

Suggested reading

The narcissism of apocalyptic thinking

By Sam Leith

Presumably all this will be accompanied by a mild identity crisis. Some of us will pray to these God-like systems in the hope their mysterious inner workings are good to us. (An Uber driver was recently overheard muttering that, “The Algorithm has been good to me today”.) The less sanguine will presumably try to smash them to pieces. That will be destined to fail because, unlike the Spinning Jenny, software can’t be destroyed with a bat or arsonist. It’s somewhere you can’t reach it.

What will our leaders do about it? When people aren’t held to account, they tend behave worse — especially if someone or something tells them it’s OK. In his infamous experiment on the nature of authority, Stanley Milgram asked people to (they believed) electrocute other participants, which they generally did if a man in a white doctor’s jacket told them it was OK. He called this “agentic shift” — the process by which humans shift responsibility to abstract processes and systems, and in the process lose their own sense of right and wrong. People are worryingly good at following orders without question. Adolf Eichmann, the chief bureaucratic mastermind behind the Holocaust, is history’s most infamous rule-follower, but there were thousands like him inside the Nazi machine, telling themselves that they were only following orders, and so they were not to blame.

Suggested reading

The calculated evil of Adolf Eichmann

By Alexander Faludy

The Adolf Eichmanns of the future will be hip, jean-wearing technologists and bureaucrats who confidently assure everyone that they need to follow the complicated data models and respect the analytics. Outsourcing morality to a machine, writes Virginia Eubanks in her book Automating Inequality, gives the nation:

“the ethical distance it needs to make inhuman choices: who gets food and who starves, who has housing and who remains homeless, and which families are broken up by the state.”

Some form of ‘ethical distance’ is probably necessary for fair and objective government, but if it goes too far, the result is decision-makers who see little relationship between their decisions and the effect on people’s lives. Smart machines will likely make things worse because rather than just following rules and making sure your little jigsaw piece fits, bureaucrats will have a machine to rely on, an intelligence apparently smarter and wiser than they’ll ever be. The ultimate form of deniability.

If, one day in the future, a world-ending cyberwar breaks out — the most likely form the bureaucalypse might take — it won’t be caused by SkyNet going rogue. It will be initiated by a group of well-dressed and well-meaning civil servants who lack the courage or conviction to disagree with the machine-modelling and AI-Strategists which told them that overall well-being would be improved by 13.2 percentage points, that the risk of retaliation was minimal. Having spent the previous decades relying on machine advice for everything from music choices to cancer diagnosis, disagreeing with the supercomputers will seem impossible, maybe even immoral.

Obviously, we humans are too thin-skinned to give up on the idea that we’re the ones in charge, so we’ll still have the plebiscites, the MPs, the Select Committees and the opinion pages. But the whole point and purpose of democracy — to hold powerful people to account, to ensure well-informed citizens are ultimately in charge — would be reduced to a charade. Real power and authority will become centralised in a tiny group of techno-geniuses and black boxes that no-one understands.

If anything, as the range of problems politicians can actually solve shrinks, the fabricated outrage and manufactured disagreements will grow. Around the same time machines get to decide the most efficient tax rate, politicians will be literally throwing themselves onto pyres over survey question options or toilets signs. While, in the real world, algorithms will sort us by intelligence, ambition and attractiveness, politics will become at best an empty ritual, at worst a form of entertainment, like a WWE wrestling match. And the scariest thought of all is this: a world run by machines and rubber stamped by humans who’ve forgotten how to think — all divorced from a democracy that has been reduced to pure content — might not worry people at all. In fact, plenty of us will probably quite enjoy it.

Over the last couple of years, people have wondered whether social media facilitates the growth of fascist ideology. It’s a reasonable question, given the spread of racist tweets, shock-jock cranks and anti-Semitic conspiracy theories. But that misses something bigger. Racists on Twitter will come and go: but social media is making all of us – even those who detest the ideology – adopt that fascist style of politics. If Eco were to design a communications system to encourage the fascist style, I suspect it wouldn’t look too dissimilar to some of our popular social media platforms.

More from this author

How the tech lords privatised the public square

By Jamie Bartlett

Take for example the idea of action. Fascists have always worshipped action for action’s sake, because to think is to emasculate oneself with doubt, critical analysis, and reasonableness. “Action being beautiful in itself,” explains Eco, “it must be taken before, or without, any previous reflection”. It would be difficult to write a better definition of a mad rampaging Twitter mob than this. I suspect the majority of people who demanded Jamie Oliver apologise for his jerk rice, or indeed demanded that he not apologise, hadn’t really thought too much about the matter before confidently pronouncing on it. I’m not saying further thinking would have, or should have, changed their minds. It’s the mode of action that I’m interested in: absolute, instantaneous, and without reflection.

I’m not blaming Zuck or Dorsey by the way. It’s simply that certain technologies lend themselves to certain behaviours; which then become normal in all aspects of society. In a print-based society, for all its flaws, there is at least a cultural predisposition for an ordering and coherence of facts and ideas, something the linguist Walter Ong called “the analytic management of knowledge”. It lends itself to reflection.

Social media platforms however are built to a very different logic: an endless, rapid flow of dissonant ideas and arguments, one after the other, without obvious order or sense of progression. It’s designed for you to blast out thoughts or ripostes over breakfast, on the move, at the bus-stop. What’s on your mind, Jamie? Facebook asks. Usually an ill-thought through response to what some other idiot has just said, I reply. Instant response is the only reasonable behaviour in such a world. I’ve noticed people rush to get their denouncements and public displays of outrage in quick, lest anyone beats them to it. All the shares and retweets come to those who move fast.

This is related to another of Eco’s characteristics of fascism (there were 14 in all): that disagreement with the leader equates to treason. The leader is the voice of the people, a representation of all that is good in them. This is important, since ‘the people’ are mostly just a theatrical prop anyway. Isn’t it odd that, despite this apparently being an age without deference, there is a newly found hero-worship and total leader loyalty in certain quarters?

It is the return of tribalism, which is the natural state for social media. This is because in our hyper-connected, over-saturated yet under-informed world, we are encircled by enemies and protected by fellow travellers. Joining a tribe is the only way to survive. And online there is always a fact or a comment or a hot take to prove your side is right and the other side is utterly wrong. When was the last time you actually changed your mind after discussing something online? I’ll answer that for you: probably never, because who has time online for the long, careful, respectful discussion necessary to see the other side of it?

In this world, opponents can’t merely hold principled differences of opinion, they must have sinister motives. We’re all guilty of it. Brexiteers that denounce any criticism or concern as traitorous and treasonous is one example – but so too is Labour’s anti-Semitism / Munich row, where critics of Jeremy must be incoherent babblers, sinister Blairites, Trump supporting Zionists, and who knows what else. All tribes need tribal leaders, who in turn need loyalty. Followers of Corbyn and Trump will both detest the comparison, but note how both have the merch, the chants, the hagiography. They’re radically different, but both are products of the tribalism that social media has accidentally brought about.

Incidentally, the ongoing denigration of the hated mainstream media – the detested MSM – by both the Left and Right, can be understood in this light too. The now common complaint about the MSM is that it’s not reporting this or that story. It’s usually expressed as: ‘here’s what THEY don’t want you to know’. The mainstream media certainly deserves criticism and gets things wrong all the time. But the fascist style turns errors and misjudgements into sinister motives and dark establishment plots against ‘the people’.

This too is incentivised by the new media eco-system because when you’re constantly surrounded by personalised feeds, it’s easy to believe that the ‘real news’ (i.e. what you’re seeing) is being suppressed by the corrupt elites. Just because it’s not on your feed, doesn’t mean it’s not out there. A quick search usually finds these supposedly hidden stories were, in fact, there all along. Trump calls the press “the enemy of the people” and we all shake our heads, recognising the fascist trope. Yet some of the recent criticism I’ve seen of the BBC from apparently enlightened people isn’t all that far behind.

In her masterpiece The Origins of Totalitarianism Hannah Arendt warned that if citizens float around like corks in a stormy sea, unsure of what to believe or trust, they will be susceptible to the charms of demagogues. This is where the fascist style finds fullest force. When you venture online, it sometimes feels like there’s an undirected anxious rage on all sides – frenetic, purposeless and habitual.

Scroll down your newsfeed or timeline. There are of course plenty of uplifting stories, but you’ll likely wind up angrier than when you started. You won’t be entirely sure why, or what to meaningfully do about it either: Police brutality! Austerity! Hypocrisy! Incompetent Ministers! A criminal who’s an immigrant! A stupid Trump tweet! A smug liberal! It’s hard to apportion the anger correctly: Jamie Oliver’s poorly named rice dish garnishes roughly the equivalent horror as the fact that the earth is on fire and life as we know it might end.

Further reading

How the Left conquered social media

By Helen Lewis

The info-overload is overwhelming, contradictory and constant, rendering people simultaneously outraged and impotent. This is a very dangerous combination. When she wrote Origins in the 1950s, Arendt never imagined the digital world, but she would surely recognise a bewildered and furious mob demanding a tribal leader who could channel the rage, fix the world’s ills and bring order to chaos. Is it all that surprising therefore that social media is helping politicians that embrace this style? Or that surveys find growing taste for authoritarian leaders? Is it all that surprising that, in these conditions, truth – Munich wreath laying, saying ‘would’ rather than ‘wouldn’t’, sticking £350 million on a bus – appears less important than loyalty to the side you’re on?

To say one side is worse than the other isn’t helpful, and it’s not my point anyway. As the old communications theorists like Marshall McLuhan used to say: the medium of the platform itself – its design, incentives, structure – is more important than the conent of the messages it conveys. “The clearest way to see through a culture” wrote Neil Postman several years back in his brilliant anti-TV missive Amusing Ourselves to Death, “is to attend to its tools for communication”.

I don’t blame the tech companies who’ve made our new tools – they generally hope and believe their platforms can help fight fascism. There’s plenty of examples of where they have, too. But the medium they’ve built is – unintentionally – encouraging a certain fascistic style of politics that’s affecting us all, subtly changing our behaviours and expectations. Combined with actual fascist ideology it’s obviously catastrophic, but even on its own it won’t lead anywhere except the political extremities.

Related listening

Test article

By Oliver Bullough & Jamie Bartlett

Chief among these is Lewis Mumford’s two volume Myth of the Machine (1966 and 1970) which examines the relationship between technology and culture from early man to now. This is everything Yuval Noah Harari has written in Sapiens or Homo Deus, and more (although Mumford is harder to read). Mumford was the pre-eminent tech critic of the 1960s and saw it as a dehumanising system of control.

In a similar vein, check out Neil Postman’s Amusing Ourselves to Death (1985). He argued that Huxley (distraction and entertainment) not Orwell (surveillance and boots) better predicted our dystopian future. The core idea is that the medium of communication dictates the sort of ideas and culture that can be sustained in a society (he was a student of philosopher and media theorist Marshall McLuhan after all). His focus on – and critique of – cable TV in the US is brilliant, and more relevant for social media than he could have predicted.

More from this author

How social media makes fascists of us all

By Jamie Bartlett

I find the radical fringes of thought are often a source of unexpected insight that mainstream writers dare not think or say. To that end, take a look at – and hold your breath for – the Unabomber’s Manifesto (1995). Ted Kascinski is a convicted terrorist who killed three people in a wave of anti-technology violence, and Manifesto is a rage against technology, which is in places violent and racist. But tucked inside are also fascinating speculations and predictions, some of which look surprisingly insightful twenty-five years on. (And don’t rule out a new wave of luddist violence – something I predicted here).

In a similar vein – Timothy C. May’s book-length blog ‘Cyphernomicon’ (1994) is a wild, bizarre, yet visionary, set of predictions about how encryption would transform society: crypto-currencies, dark net markets, ubiquitous encryption are all in there. May believed this would one day fatally weaken governments. He celebrated these trends; many of us will not.

Both of these works will shock and annoy you, but if you read them sceptically and carefully you’ll get a lot from them. And they’re both free to download.

More Christmas Reading

A survivor's guide to trauma

By Bidisha

Looking ahead rather than behind, two recent books set out a big problem heading our way. Both Cathy O’Neil’s Weapons of Math Destruction and Sayifa Umoja Noble’s Algorithms of Oppression (who both had decent stints in the private sector before writing these books) show how an increasing number of decisions are taken by smart machines and algorithms – but that doesn’t mean they will be less biased. In fact, they might simply replicate, or even worsen, existing economic, racial, gender or other inequalities, because they will appear objective and credible.

This problem – invisible algorithmic bias through search engines, or data discrimination – will become a major public preconception very soon, perhaps as soon as next year. The sooner we start thinking about it the better.

Over the last couple of years, people have wondered whether social media facilitates the growth of fascist ideology. It’s a reasonable question, given the spread of racist tweets, shock-jock cranks and anti-Semitic conspiracy theories. But that misses something bigger. Racists on Twitter will come and go: but social media is making all of us – even those who detest the ideology – adopt that fascist style of politics. If Eco were to design a communications system to encourage the fascist style, I suspect it wouldn’t look too dissimilar to some of our popular social media platforms.

More from this author

How the tech lords privatised the public square

By Jamie Bartlett

Take for example the idea of action. Fascists have always worshipped action for action’s sake, because to think is to emasculate oneself with doubt, critical analysis, and reasonableness. “Action being beautiful in itself,” explains Eco, “it must be taken before, or without, any previous reflection”. It would be difficult to write a better definition of a mad rampaging Twitter mob than this. I suspect the majority of people who demanded Jamie Oliver apologise for his jerk rice, or indeed demanded that he not apologise, hadn’t really thought too much about the matter before confidently pronouncing on it. I’m not saying further thinking would have, or should have, changed their minds. It’s the mode of action that I’m interested in: absolute, instantaneous, and without reflection.

I’m not blaming Zuck or Dorsey by the way. It’s simply that certain technologies lend themselves to certain behaviours; which then become normal in all aspects of society. In a print-based society, for all its flaws, there is at least a cultural predisposition for an ordering and coherence of facts and ideas, something the linguist Walter Ong called “the analytic management of knowledge”. It lends itself to reflection.

Social media platforms however are built to a very different logic: an endless, rapid flow of dissonant ideas and arguments, one after the other, without obvious order or sense of progression. It’s designed for you to blast out thoughts or ripostes over breakfast, on the move, at the bus-stop. What’s on your mind, Jamie? Facebook asks. Usually an ill-thought through response to what some other idiot has just said, I reply. Instant response is the only reasonable behaviour in such a world. I’ve noticed people rush to get their denouncements and public displays of outrage in quick, lest anyone beats them to it. All the shares and retweets come to those who move fast.

This is related to another of Eco’s characteristics of fascism (there were 14 in all): that disagreement with the leader equates to treason. The leader is the voice of the people, a representation of all that is good in them. This is important, since ‘the people’ are mostly just a theatrical prop anyway. Isn’t it odd that, despite this apparently being an age without deference, there is a newly found hero-worship and total leader loyalty in certain quarters?

It is the return of tribalism, which is the natural state for social media. This is because in our hyper-connected, over-saturated yet under-informed world, we are encircled by enemies and protected by fellow travellers. Joining a tribe is the only way to survive. And online there is always a fact or a comment or a hot take to prove your side is right and the other side is utterly wrong. When was the last time you actually changed your mind after discussing something online? I’ll answer that for you: probably never, because who has time online for the long, careful, respectful discussion necessary to see the other side of it?

In this world, opponents can’t merely hold principled differences of opinion, they must have sinister motives. We’re all guilty of it. Brexiteers that denounce any criticism or concern as traitorous and treasonous is one example – but so too is Labour’s anti-Semitism / Munich row, where critics of Jeremy must be incoherent babblers, sinister Blairites, Trump supporting Zionists, and who knows what else. All tribes need tribal leaders, who in turn need loyalty. Followers of Corbyn and Trump will both detest the comparison, but note how both have the merch, the chants, the hagiography. They’re radically different, but both are products of the tribalism that social media has accidentally brought about.

Incidentally, the ongoing denigration of the hated mainstream media – the detested MSM – by both the Left and Right, can be understood in this light too. The now common complaint about the MSM is that it’s not reporting this or that story. It’s usually expressed as: ‘here’s what THEY don’t want you to know’. The mainstream media certainly deserves criticism and gets things wrong all the time. But the fascist style turns errors and misjudgements into sinister motives and dark establishment plots against ‘the people’.

This too is incentivised by the new media eco-system because when you’re constantly surrounded by personalised feeds, it’s easy to believe that the ‘real news’ (i.e. what you’re seeing) is being suppressed by the corrupt elites. Just because it’s not on your feed, doesn’t mean it’s not out there. A quick search usually finds these supposedly hidden stories were, in fact, there all along. Trump calls the press “the enemy of the people” and we all shake our heads, recognising the fascist trope. Yet some of the recent criticism I’ve seen of the BBC from apparently enlightened people isn’t all that far behind.

In her masterpiece The Origins of Totalitarianism Hannah Arendt warned that if citizens float around like corks in a stormy sea, unsure of what to believe or trust, they will be susceptible to the charms of demagogues. This is where the fascist style finds fullest force. When you venture online, it sometimes feels like there’s an undirected anxious rage on all sides – frenetic, purposeless and habitual.

Scroll down your newsfeed or timeline. There are of course plenty of uplifting stories, but you’ll likely wind up angrier than when you started. You won’t be entirely sure why, or what to meaningfully do about it either: Police brutality! Austerity! Hypocrisy! Incompetent Ministers! A criminal who’s an immigrant! A stupid Trump tweet! A smug liberal! It’s hard to apportion the anger correctly: Jamie Oliver’s poorly named rice dish garnishes roughly the equivalent horror as the fact that the earth is on fire and life as we know it might end.

Further reading

How the Left conquered social media

By Helen Lewis

The info-overload is overwhelming, contradictory and constant, rendering people simultaneously outraged and impotent. This is a very dangerous combination. When she wrote Origins in the 1950s, Arendt never imagined the digital world, but she would surely recognise a bewildered and furious mob demanding a tribal leader who could channel the rage, fix the world’s ills and bring order to chaos. Is it all that surprising therefore that social media is helping politicians that embrace this style? Or that surveys find growing taste for authoritarian leaders? Is it all that surprising that, in these conditions, truth – Munich wreath laying, saying ‘would’ rather than ‘wouldn’t’, sticking £350 million on a bus – appears less important than loyalty to the side you’re on?

To say one side is worse than the other isn’t helpful, and it’s not my point anyway. As the old communications theorists like Marshall McLuhan used to say: the medium of the platform itself – its design, incentives, structure – is more important than the conent of the messages it conveys. “The clearest way to see through a culture” wrote Neil Postman several years back in his brilliant anti-TV missive Amusing Ourselves to Death, “is to attend to its tools for communication”.

I don’t blame the tech companies who’ve made our new tools – they generally hope and believe their platforms can help fight fascism. There’s plenty of examples of where they have, too. But the medium they’ve built is – unintentionally – encouraging a certain fascistic style of politics that’s affecting us all, subtly changing our behaviours and expectations. Combined with actual fascist ideology it’s obviously catastrophic, but even on its own it won’t lead anywhere except the political extremities.

Related listening

Test article

By Oliver Bullough & Jamie Bartlett

In a strange way Jones is a natural mutation of the internet business model itself – more outrage equals more clicks equals more sales. Perhaps one reason Google and Facebook ditched him was because he reminded them too much of themselves.

The Jones banishment is part of a concerted effort. The big tech firms are under increasing pressure to make the place more hostile for people like him: over the past few weeks and months several high profile radical right-wingers have found social media – for a long time their haven – is no longer so welcoming. Gavin McInnes, Vice Founder and alt-liter, was also expelled. Milo Yiannopoulus was acrimoniously dumped some time ago. Behind the scenes the companies are upping their monitoring procedures and removing significantly more content than even just a year ago – especially radical Islamist.

Very few liberals were overly worried about the Jones affair. To anyone who expressed a concern about free speech, the stock response was: Facebook and co haven’t denied Jones his freedom of speech. They haven’t stopped him from hosting his own blog, from shouting from his own rooftop or writing his own newsletter. They have simply stopped him spewing his babbling drivel on their private platform. Private platforms can do as they wish.

This is exactly correct, and I’m glad they’re acting more responsibly. But it is also exactly the problem. Regardless of the specifics of the Jones case, it illustrates the way tech companies increasingly decide what ideas people access, what’s in the public domain and what’s not, and therefore the shape and contour of the public sphere.

Not in a strictly legal sense: Alex Jones could indeed start up his own newsletter, and he probably will. But in a functional sense the public sphere has become a private enterprise, a set of servers owned by Californian companies.

I don’t need to recite the stats about how much time people spend there, or how majorities get their news from social media. You all know that Facebook (including Instagram), Twitter, Google (including YouTube), and Apple (iTunes) is where people receive their information about the world, and where the issues of the day are thrashed out. Amazon will join them soon enough, because personal assistant devices will become de facto info curators (‘Alexa – tell me the news’). You may be vaguely aware that these are all private profit-making companies, tucked along the West Coast with shareholders and profit targets and advertiser pressure.

Suggested reading

Review: "The People vs Tech", the threat at the heart of the internet

By James Ball

I don’t envy these companies by the way. They would rather not make determinations like this. They don’t want to get tangled up in controversial debates about what should or shouldn’t be hosted, but their size and influence has made it unavoidable. Whatever choice they make – whether about politics, offensive images, nipples – someone will get upset when their stuff is up or down on the great algorithmic wheel of fortune.

In any case, this transformation – public space into private server – is far bigger even than Mr Jones’ over-inflated digital footprint. There are plenty of ways the public debate can be subtly controlled, accidentally or through design. Algorithms are designed to serve you content that you’re likely to click on, as that means the potential to sell more advertising alongside it. For example, YouTube’s ‘up next’ videos are selected based on a sophisticated analysis of what is most likely to keep a person hooked in.

There are lots of reasons why politics has turned into a hot angry mess, but public discussion being subverted to the needs of advertisers ranks high among them. And before you say ‘But Rupert Murdoch…’, newspapers and broadcasters are still subject to more stringent regulation than social media. Not to mention that the YouTube algorithm alone shapes what over a billion users are likely to see, which is more than every newspaper in the world combined.

It gets worse. According to Robert Epstein, a psychologist at the American Institute for Behavioural Research and Technology, based on the win margins in national elections around the world, Google could determine the outcome of “upwards of 25 per cent” of them based on how it displayed search results.  There is no evidence that Facebook or Google have or would intentionally do such a thing – but how would you ever know?

Perhaps the Alex Jones decision was the right one. Having watched a lot of his stuff, no doubt it breaches all sorts of terms and conditions. But in the end these companies were surely also motivated by pressure, by fears about advertisers jumping ship, by PR, by popularity. (Twitter, to its credit, seemed more open, and said it would stick to banning accounts based on its rules).

Suggested reading

The cracks appearing in the digital economy's foundations

By David Z Morris

Ultimately, the transformation of the public sphere into a private server means that what we see will be increasingly motivated by advertiser needs, by reputation – cue the heckler’s veto, where whoever shouts the most gets their way – or by profit.  That’s not a good basis for such decisions. And whatever the motivating reason for removing content, those decisions will be taken by a very small group of people with similar world views, which isn’t good for political plurality.

So cheer away! It’s hard to be saddened if fantasists have a smaller public platform. But bear in mind that one day opinion might turn against your ideas and pet concerns. Or perhaps the bosses who run the giant platforms might change. You could suddenly vanish too. You’ll still have your free speech! But as you scream from the wilderness of deep cyber-space, no-one will hear you.