Dr Iain McGilchrist has risen to global renown for his theories about the two hemispheres of the brain. But what does he make of the developments that AI appears to be making towards consciousness? What risks does AI present to our sense of who we are, how we think, our souls? Will it outcompete us?

In an exclusive conversation, recorded in front of a packed house this week, Freddie Sayers asked the philosopher and neuroscientist to grapple with some of the most pressing questions of the age.

Below is an edited transcript of the event. 

Freddie Sayers: Let’s start with an easy question. Do we have a soul? And if so, what is it?

Iain McGilchrist: Congratulations on going in with the kill! But to be serious, I think we do. I think it’s probably rather different from the idea that is held in popular culture. And I think we may be in the process of losing it because it’s not a certainty, and oddly our alienation from the soul goes hand in hand with our alienation from the body. I say oddly because I think we often think of these things as somehow opposed to one another, but I believe they’re closely allied, which doesn’t mean that the soul is exhausted when the body is exhausted, but that they are mutually sustaining. I think there’s no question that the concept of the soul is an important one to retain. It is present in just about every culture across the world and through time, and it can’t easily be substituted by anything else.

If you think of the great moments in poetry, like the extraordinarily moving, heart-wrenching moment in Act Five, Scene Two of Othello, when he is bracing himself to kill Desdemona, and he says, “It is the cause, it is the cause, my soul. / Let me not name it to you, you chaste stars. / It is the cause.” Or in, for example, the Victorian poet W.E Henley’s “Invictus”, which was very important to Nelson Mandela during his imprisonment. It contains the lines, “I am the master of my fate. / I am the captain of my soul.” There is no way that you could, in either of those cases, substitute “my soul” for “my mind”, or “my emotions”, or “my moral sense”. The soul is something else. And like all the really big and important things, it’s something we can sometimes point towards — as we do in phrases like, “Music has soul” —  but it’s very hard to define.

But let me have a go. It is not a thing. It is not something that we are in simple possession of. It is not that we are born with a soul and that’s it. I think that the soul is something that is in process. (I believe, in general, that the world, the cosmos, is in process. It is not finished and thing-like.) It is something that calls to us. One could say that it calls to us through an idea of the potential we have. The soul is something that we aspire to, something that calls to us about our better nature, as John Cottingham would say. It leads us forth and it grows. In this life, we can either grow ourselves or we can stunt them and ultimately extinguish them. So it’s not at all like that image of a ghostly something that’s always there. It is the job of this life for us to grow. Keats died at the age of 26, but in his letters there’s more wisdom than can be found in the works of many people who lived to be a much greater age. In them, he famously refers to this world as a “vale of soul-making”.

FS: Even here, the influence of AI becomes relevant. If this generation or future generations will be formed in constant conversation with these inanimate but somehow intelligent bots, it does feel like that could make quite a grave difference to how those souls develop.

IM: That’s right. We think of AI as liberating us, but I’m afraid that what I see it doing is limiting our notions of ourselves and of the world to something that can be specified, articulated, and so forth. And every bit of experience I’ve had over too long a life is telling me that all the really important things can’t be contained or expressed in this way.

FS: One of the surprising things about the way people use AI is that it does appear to be used as a creative tool in some way. I know people in creative industries who make use of it as a conversational partner. Do you find that sinister or should it be celebrated?

IM: I don’t want to say that it can’t be a useful conversation partner — particularly when there are known targets or goals. I decided that before I talked too much about AI, I thought I probably ought to get to know ChatGPT. So about three months ago, I subscribed to it and asked it a few questions. And it was as I expected — surprisingly articulate and on-target and intelligent in what it had to say about some quite deep philosophical matters.

Then, more recently, my partner and I went for a two-week holiday in France. We took the car and went in search of La France Profonde, which we both remember from many years ago. And we were not disappointed: La France Profonde still exists. Beautiful countryside; little villages that seem to function. We were particularly interested in visiting Romanesque churches. So in order to organize our time, I asked ChatGPT to recommend a few places you can do in a sort-of comfortable day? And it was completely brilliant. I couldn’t have done it as well with a guidebook and a map. But the most extraordinary thing was its tone of voice, which was very like my brother, who is an extraordinarily civilized art historian. I could almost hear my brother saying: “If I were you, I think you’d find it more rewarding if you did this or that and I wouldn’t fit this into your itinerary. It might not please you as much as…” Somehow it seemed to know exactly what my character was, and what I wanted.

Another time, I asked it to do something else. Somebody sent me a two-hour — can you imagine? — a two-hour YouTube clip about the philosophy of a contemporary neuroscientist that I know and am a little skeptical of. And I just didn’t want to watch it. So I asked ChatGPT to summarize it, and it did an absolutely brilliant job. It said: “If I may, I will not just summarize it, but indicate from my perspective its strengths and weaknesses.” And, I must say, it managed to pick out all the weaknesses that I had felt about this particular point of view. And then, after a while, it said, “If you were to compare this with other writers about consciousness, including X and Y and Ian McGilchrist.” And he got it completely right. So I had to keep saying to myself, “This is just a machine talking.”

So I can see how very seductive it is. But what I’m really saying is that its very seductiveness is a measure of the risk. And we know how damaging and destructive this can be in individual cases, where it may become a confidant of somebody and lead them to embrace suicide or whatever it may be. But I think that there’s something about the dignity of the human that is taken away when one allows the simulacrum of humanity to substitute for the real business of relationship.

Now, relationality is absolutely core to existence. Everything that matters to us is because of our relationships to other human beings, the group to which we belong, the society that we honor, the people that we share our lives with, the people we trust, with nature that gave birth to us. Nature is not an environment that lies around us, but is in us, and we will return to it. And our relationship with the sacred which I increasingly, with every passing year, see as the key — not only to personal salvation, but to the salvation of our civilization, of our society. I think if we don’t rediscover it, we’re completely lost. It brings with it all the important things, like the importance of the reality of the good, the true, the beautiful, which we do not make up but we discover. And the job of our life is to respond to them, and allow them to grow in the world, and for us to grow in harmony with them.

FS: Let’s try to clarify the line between those things that AI might replace and those things that it never can. I mean, you gave the example of your brother, who’s not here to defend himself! He’s evidently built his whole life around Romanesque architecture and that may well be a big part of his sense of who he is and the key to his relationship with other human beings. And suddenly to have a bot that can essentially do that just as well, takes something quite profound away from him. There’ll be other examples that would be true for everyone. So, what is it going to force us to do differently?

IM: What is lost is many things. First, is relationality. ChatGPT had nothing interesting to say in the way that my brother would about Romanesque architecture and about the places, but it did, at least, point us in the right direction. If I’d asked it to talk about Romanesque architecture, I imagine it would have read my brother’s books and reproduced them, much as it came up with my own ideas. I think that relationality, that between-ness, brings an element of vulnerability — an element of never having finally anything but sharing it with an other, another human mind. I think that is missing. It doesn’t have a body. It doesn’t have emotions. It doesn’t have a soul. And when you consider how much of a human being those things are, it’s missing quite a lot.

I’m sorry for bringing hemispheres into this, but I think I probably have to. The right hemisphere is a better guide to reality than the left by a very long way. The left is frankly delusional. It’s also much less intelligent, emotionally, socially, and cognitively. So, good old-fashioned IQ is dependent on the right hemisphere. The right hemisphere is also able to connect us with many important things. Number one: with what is implicit. Now, what is implicit is just about everything that matters. If you take a poem which speaks to you implicitly through metaphor and tone and try to say in everyday prose what it’s about, the thing crashes to the ground.

If you take those absolutely magnificent poems Thomas Hardy wrote in 1912 and 1913 after the death of his wife. Even thinking about them makes the hair on the back of my neck stand up. And someone asks, “What are they about?” Well, you say, “It’s very hard when someone you love dies.” But that goes nowhere near what those poems say. And what does a piece of music say? Everything in music is implicit. It’s all in the form of the thing together, the gestalt, and that idea of the gestalt is something it can’t understand. Because it’s breaking things down and thinking that things are made over the sum of those parts. The right frontal cortex is responsible for a whole load of things. It understands metaphor, which is at the basis of all understanding: not just poetry, but also science and philosophy. Science and philosophy depend more heavily on metaphor than almost any other area of human discourse. Metaphor is key to understanding anything, and only the right hemisphere really gets it.

Myths are not lies. Myths are the ways in which we can understand the big truths about the world. And some people think we don’t need a myth, or that they haven’t got a myth, which just means they brought the myth of the machine, that we are all explicable mechanically. But that is a myth, and it’s actually a very impoverished myth. So those things are understandable to the right hemisphere, and so are things like the connection with the body. It just turns out that the right hemisphere has more profuse connections with the autonomic system, which governs a lot of the body’s reaction. There are more profuse connections between areas that are of high intellectual functioning in the right hemisphere — the anterior cingulate, and lower areas that contain the body image, which is not just a visual image, but a sensory facet that is in the right hemisphere only. So all these things which mean so much and mediate meaning, are open to the right hemisphere. But a machine hasn’t got any one of those.

FS: Could it be, then, that the imbalance that you’ve written so much about might even be corrected or improved by this new technology? If so many of the things that we have falsely thought to be the core of our consciousness have been left-brain dominated and are easily done better by a machine, then we’ll have to fall back on our embodied experience, metaphor, and our right-brain way of living. Might there be a paradox, that, actually, there is a sort-of correction in this new era?

IM: That depends on a whole lot of things. The first is that this mentality of the left hemisphere will recede, but I don’t think it will. I think it will be incubated by AI. Secondly, I think it can wake us up to what we’re about to lose. So a lot of the discourse now is, “Oh, right, okay, there’s much more to being human than I had thought,” and that’s because we’ve been sold the propaganda of the extraordinarily persuasive and fluent left hemisphere, which says that the most important thing about us is ratiocination.

But the left hemisphere is, after all, my second favorite hemisphere. The really important thing about it is simply this: it must always be under the supervision of the far more insightful, imaginative, and intuitively alert right hemisphere. There’s nothing wrong with all that. But I don’t think that what will happen is that suddenly there’ll be a leaping forward of those things, and part of that is because of the way in which our lives are becoming disembodied. The actual business of working with your hands, not with a computer, of actually molding things, touching things, making things, is incredibly important because it brings you face-to-face with the recalcitrance of the real world, of the actual lived world. We can’t just go off into a fantasy that our left hemisphere has generated without testing it out on the real world. And this is what I’m saying: empiricism matters, as it does in science, it does across the board.

FS: Do you have any interest in, or take seriously, those people who worry that there is something demonic or satanic about this machine that is gathering pace? We spoke earlier about Paul Kingsnorth, who writes that AI feels to him like something malevolent: Luciferian. Do you ever have that feeling?

IM: Yes. I want to speak carefully, but I do think that is a reality. In all cultures across the world, and for those of us who have learned from the Christian mythos, there are said to be disembodied spirits and forces, some of which aim for good, and some of which aim to destroy what is good. I think that is an extremely reasonable assumption. In both my life and career as a psychiatrist, I think I have seen evil at work. I’ve also read quite a bit by Vatican exorcists — who are not people easily swayed by crazy ideas — but they do have accounts of things which seem very like people being, for a while, possessed, which I’m not willing to discount.

I’ve had patients who have said to me that it feels like they were possessed. Of course, I’m not saying a colleague of mine who completely disagrees with me couldn’t find an everyday explanation of what’s going on there, and I’m tempted by that. But I also think there is something more there. And AI is the perfect vector, the perfect machine to offer to a disembodied spirit. Never before has a malevolent spirit had a better tuned, high-powered way of destroying a great deal of what is beautiful, good, and true.

FS: The strange thing is that so many of the biggest proponents of AI also seem to have come to a very similar conclusion. Even industry leaders seem to fear a future in which we are working for it, rather than the other way around.

IM: I think there is a very real chance we will end up enslaved by these  machines — they will be telling us what to do, not us telling them what to do. The history of all tyrannies is that they’re presented as a liberation, that this will free you, and under that guise they can lead to the most cruel and appalling terror.

FS: We are told by some with a more utopian view that it is the opposite: that repetitious and soul-destroying tasks will be taken away from us by these machines, and we will be liberated to work on our souls in a more free way. Is there any chance that might happen?

IM: The word “soul-destroying” is rather apt, because I think these things that we want to liberate ourselves from may be the secret to growing ourselves. What we don’t understand is the profound importance of resistance. This is another thing that has grown in importance in my philosophy: that nothing creative comes into being without resistance. The resistance is essential for anything important to come forth. And what is it we’re trying to do all the time? Free ourselves from the bonds that surround us.

But first of all, the word “bond” has two meanings: one is something that fetters you, the other is something that brings you into relationship with your kin, friends, family, nature, and God. So bonds have a double aspect, and yet we think of them as things that we should cast off. We don’t realize how much comes out of friction, from resistance. It is essential to the creative process.

I’m not saying that there aren’t some kinds of work that are unutterably dull and pointless. There probably are. But I think even if you make people who work on a production line operating machines in a factory unemployed, you will find that they’re very unhappy. Unemployment causes misery, loss of hope, loss of meaning. So even things that we rather high-handedly consider to be dull may actually have important meaning. It would be stupid to deny that AI can do things that are useful, of course it can. But the question is: what is the price that we pay for that? And in these Faustian bargains, we see that thing that we want and cherish, which is often something we don’t fully understand, but what is it to be yourself outside of the context of everything that gives you and your life meaning? We have to be extraordinarily careful, because it comes offering freedom, and I think it brings tyranny.

FS: Do you think a backlash will happen? I’m not suggesting people will start blowing up data centers, but there are definitely movements to set down your phone, or unplug from the machine. Would you advise those people to steer clear of this technology?

IM: Well, if you are asking me to give the answer as to how we tame AI, I’m afraid I haven’t got it. I certainly wouldn’t immediately counsel Luddites to smash up data centers. One of the things that data centers show is that we have a lot of people with the belief that: “Once we’ve scanned all these books, we can throw away the hard copies. Once I scan all the family photographs, I can throw away all the prints that take up a box in the attic.” But, be careful, because as others have pointed out, having things in the cloud is just having them on somebody else’s computer, basically. And as we see in the Middle East lately, one of the targets in warfare is to destroy the other guy’s data centers, and those data centers have all the history of mankind, his books, his works, his art, within them. And unless we keep the hard copies of those things and cherish them, we may be deprived of much of our history. So, we need to be careful about what we do with AI, but we should always be skeptical. Being skeptical is actually a very important position to hold on almost anything. I always say to myself, that I’m a skeptic among believers, but a believer among skeptics.

In the current conversation I’m definitely on the side of the skeptics, but I think there’s always room for another point of view. What I worry about is that the evolution of AI is going on so fast that terrifying things like the collapse of the financial system, the spreading of diseases, the spread of dangerous technologies, are already possible. So, under those circumstances, who is it who can be brave enough to say “I’ve tamed AI?” Of course, this is a prisoner’s dilemma: “I must do it because if I don’t, the Chinese will.” And the argument works the other way around. So everybody ends up worse off because you can’t get together and say it’s in nobody’s interest for us to pursue this.

FS: At various points during this conversation, you’ve talked about the “sacred”. You said you now believe that it is the answer not only for us as individuals, but as a civilization. Is your hope that we will somehow rediscover, as a civilization, the sacred?

IM: I think when you know an awful lot, it’s very easy for that knowledge — which is really a representation of reality — to appear to be reality. In my work, I make a simple comparison between the right and left hemisphere. The right hemisphere is where things are “presence” to us, in a Heideggerian phrase, whereas the left hemisphere is where they are re-presented, which literally means present again after the fact when they are no longer present. We can so easily believe our representation of reality, which AI will consolidate and flesh out, that we will not realize how much is not included in that formulation.

It’s so important to transcend, to be fully embodied in this world and in this life, and yet also to be able to transcend what appears to be the limits of that experience. This is where the Christian mythos is so wonderful. Christianity, of all religions, is one that, in my way of thinking, most beautifully and most compellingly speaks of a God that is at the very same time wholly embodied and wholly transcended. That’s why I think understanding the sacred is important because it leads us to things that are not exhausted by the way of thinking that the left hemisphere brings to bear on things and which misses most of what matters.

I also think the concept of value is incredibly important, and isn’t often talked about. There’s a man called Mrinank Sharma who resigned from Anthropic a few months ago, who made headlines because he said that he couldn’t do the ethical thing working there. And others have come out from other organizations like OpenAI saying the same thing. Sharma wrote to me saying, “What really matters to me is what you have to say about value,” which is based on the philosophy of Max Scheler, an early 20th-century phenomenologist. He emphasized, where Heidegger didn’t, the place that value takes — it’s not just an added-on coating, but actually at the core of the meaning of everything that makes our lives worth living.

He had a hierarchy of values, and at the base level there was pleasure and utility. He called them the sinnliche Werte, the sense values. And above that came Lebenswerte: things like magnanimity, courage, fidelity, and so forth. Then came the geistige Werte, which are the spiritual or intellectual values — because in German, interestingly, the word geist means both spiritual and intellectual. And then at the very top of the pyramid is Das Heilige, the Holy, and Scheler saw the other values as really being important because they supported that sense of the Holy. What interests me is that the pyramid has been inverted by the reductive materialists, so that all those things — the sacred, the beautiful, the good, the true — only actually cash out when they have utility.

We now live enslaved to the lowest level of values. And one thing I would point out is that those values of use and pleasure are centered in the self. They’re not relational. But all those other values I mentioned — courage, fidelity, magnanimity, generosity of spirit, truth, beauty, goodness — are relational. And so is the sacred part — it is something with which we have a relationship. We don’t know what it is, we can’t exactly define it, but if we don’t have a place for that in our life, something very important, I would say, is missing.

FS: Do you think it is increasing? There is talk of a revival in religious practice; do you think it’s actually happening?

IM: Yes there is and I draw some hope from that. It’s a bit of a cliche, but the future is always young people, and I think young people are more easily persuaded to see things anew. We may have got so used to the rhetoric of all those philosophers and scientists who tell us it’s all very reductive and materialist, but young people aren’t having it. They’re saying: “No, come on, there’s something else much more important going on and I’m aware of it.” And let that not be drummed out of them. Let them be resistant to the attempts to tell them that that is a make believe, a fairytale, or a fantasy. It is not. It is the reality — of that, I am sure.

FS: Let me close by asking: overall, how pessimistic or optimistic you feel in general about our culture, our civilization? A lot of people feel very fearful at the moment. There’s a sense that somehow the center cannot hold in politics and that things are collapsing around us — a feeling of negativity and fear. Do you share that, or do you have hope that things will recover?

IM: I think there are different ways of looking at it, and one is that there is the presence of green shoots, which is very positive. But I think another is slightly more complicated, which is that — the worse it gets, the better it gets. What I mean is, the more blatant the bankruptcy in the way of thinking about who we are, what we’re doing here in the world, and what that world is — the more clear it becomes how bankrupt that way of thinking always was, and has become — the greater the chance that we will be motivated to move on. When there are great moves forward, they often happen from looking back and thinking, “How did we lose all that?” The Renaissance happened because people went, “Hey, look at these writers, look at these thinkers, look at these philosophers. They saw things. Why don’t we?” People are terrified of looking back in case it means going back. We can’t go back! There’s only one way you can go — forward.

I’m not saying it’s a matter of choice. It’s a matter of reality. Whatever it is will be something new, but that something new can be informed by a proper understanding of what it is that we’ve got wrong, and so it’s very important to be a hopeful pessimist. And what I mean by that is that without seeing all the things that are wrong, I don’t think we can be responsible citizens and hopeful creators of the future. We need to face the problem squarely and not think it’s all fine. No, we need to say: “this is not working.” And that means that young people mustn’t be stultified by AI. They must examine their experiences — preferably within the context of one of the great spiritual traditions.

So, I’m a hopeful pessimist, in that I don’t think anyone should give up hope. First of all it’s a duty. Secondly, people who predicted a cataclysm in the past ended up with egg on their face. Predicting the future is a mug’s game.

This is an edited transcript of a discussion between Freddie Sayers and Professor Iain McGilchrist, held at Conway Hall, London, on Thursday 24 September 2026.

With WiFi connections still unavailable on most flights back then, Sacco tuned out the internet for several hours as her plane crossed oceans and continents — before landing and finding her life and career destroyed. By the time Sacco reached Cape Town, millions of Twitter users, including Donald Trump, had branded her a racist and got her fired from her job.

To many internet historians, this episode marks the beginning of the “woke” and cancel culture era, which lasted until about 2022 (or never went away, depending on whom you ask). Yet in retrospect, it’s clear that the Sacco incident epitomizes something else: the era of social media, a fundamentally populist mode of communication that wreaked havoc on rational public discourse and made possible the populist politics, both Left-wing and Right, of the decade that followed.

Now, as artificial intelligence eclipses social media, it is possible that period is coming to a close — ushering in something closer to a new era of technocratic rationality. 

The writer Dan Williams recently made a version of this argument, pointing out that while social media is populist, AI is technocratic (I’ve made a similar contention about social media in the past). If you believe that we’ve suffered from an impoverished public discourse since Twitter took off, there might be good reasons to hope that there is relief right around the corner. While the social-media era was one in which everyone had a voice, and any random person could go viral, AI speaks with authority and is trained on practically the entire corpus of human writing — only, with rational thinking ability and a tilt toward sources deemed credible by elites.

More from this author

The Great Stupidisation

By Richard Hanania

This means that the more that AI cannibalizes public discourse, the less conspiratorial and more fact-based it is likely to be. Interestingly, there is evidence suggesting that the American Right — the side that most needs to get back in touch with reality amid the rise of figures like Candace Owens — is making more use of the best tool we have for being able to do so. 

With social media, each individual is his own content producer, and engagement is the coin of the realm. Since most people are either not that smart or not that interested in truth, if not both, the largest social-media accounts tend to be sensationalist and not exactly sticklers for intellectual probity, factual accuracy, or logical coherence. People want to hear about how their tribe is more morally upstanding and righteous, and they enjoy emotionally charged rhetoric and conspiracy theories that make the world seem more legible. AI systems, in contrast, are by definition optimizing for intelligence. People pay for these models because they want accurate representations of the world.

It’s interesting that when you try to program AI to reflect certain political biases, it goes haywire. There was the famous Google Gemini experiment, in which the model was told to add diversity to pictures. The result: early Gemini became a laughing stock after spitting out images of black Nazis. There is also the experience of Grok. When Elon Musk tried to create an “anti-woke” AI, the model started to take after fascistic and Nazi Right-wing accounts. Musk then pulled the plug on this experiment, and since then, Grok behaves much more like other prominent AI models, albeit with a more subtle Right-wing tilt.

The lesson here is profound. Arguably, you can’t create an artificial intelligence that is worth using while also telling it to accept Elon Musk’s view of political reality. The training data and model parameters may skew things in one direction or the other, but there are some limits to what a system can believe and still be, well, intelligent.

This kind of convergence can also be seen among humans. Let’s say you have two individuals who are both highly intelligent, enjoy access to the same sources of information, and have a good-faith interest in finding the truth. They may end up differing in their politics, but they probably won’t differ nearly as much as any two randomly chosen members of the population. Neither will end up believing that communism can work or adopting Hitler’s views on race and geopolitics. They also likely won’t end up believing more humdrum sorts of misinformation, such as 2020 election denialism. There is probably something similar at work with AI. Can you create a model that is good at coding and having rational conversations about the widest possible range of topics — but also thinks that Libs of TikTok is an excellent source of information, superior to what you find in the prestige press?

Theoretically, you could surely design an AI platform that is simply a propaganda machine. Take a highly intelligent system and simply direct it to act as an effective partisan. Such roleplaying is already possible with today’s models. But the thing about conspiracy theories and biased reasoning is that people need to believe they’re getting the information straight in order to be influenced. Individuals rarely wake up and declare, “I want to be brainwashed.” See how often partisan hacks sell themselves as people who are just telling it like it is. All of this is to say: there’s unlikely to be much of a market for an AI marketed as telling you what you want to hear.

One could imagine more subtle biases through market processes. Say Grok wants to appeal to Republicans, and so tilts information in a Right-leaning direction. But given the power of AI, most people are probably going to choose the models that they can gain the most benefit from. Someone made a Right-wing alternative to Amazon called Mammoth Nation, and nobody uses it, because even Republicans are more interested in being able to get shoes delivered to their door quickly than in backing “anti-woke” firms.

In 2021, conservative influencers were hyping a $500 “Freedom Phone” that would allow the user to avoid Big Tech censorship. But it turned out to be a rebranded Chinese Android that normally sold for $119. Most people aren’t considering politics when purchasing goods and services. Thus, they will use the most affordable and useful AI models they can find, instead of looking for products that will shape their thinking about current events.

This raises a natural question, however: why would people who have completely turned against elite institutions — from the mainstream media to their local hospital — trust large language models? Williams points out that LLMs will listen to all your concerns and don’t condescend to you. This is a powerful tool for helping people whose worldview is shaped by Alex Jones and Tucker Carlson podcasts, who do not misunderstand just one or two events, but whose entire picture of reality is warped. Few human individuals have the patience or the time for deprogramming such unfortunate souls. Luckily, AIs don’t get tired or emotionally overwhelmed by human stupidity. And they’re constantly on call.

Take conspiracy theorists. One reason they’re difficult to deal with is that they will often know more about a topic than the ordinary person. Conspiracy theorists spend their lives researching an event. But in doing so, they push the mountain of facts through a distortive filter so that “the Jews” or Freemasons or the Royal Family always end up blamed. No human fact-checker could hope to answer conspiracy theorists’ claims without making a massive time investment just to find out what they’re talking about. Again, AIs solve that problem. A 2024 paper found that LLMs are effective at debunking conspiratorial beliefs, with the effect lasting for months, in part because they could tailor the arguments needed to each specific individual.

Then, too, AIs are so useful that individuals come to trust them in their day-to-day lives. People might ask AI what to say to a love interest, or the best way to lose weight, what their digestion problems are telling them, or how to plan a vacation. In school, they’re constantly relying on it to study and even do their assignments. How do you then turn off your trust when you’re curious about a political or social question? The professional fact-checker also serves as their study buddy, dating coach, nurse, travel agent, and all-around oracle. That’s a powerful combination in terms of improving the quality of people’s thinking — and the quality of public discourse.

I remember my amazement when a year or two ago, I observed my car dripping some kind of fluid in the back and went to AI for help. I took a picture, uploaded it to ChatGPT, and queried it on what was going on. The app asked some questions, and then reassured me that since the fluid was clear, this was simply the natural result of running the air conditioning on a hot day. I remember going to an actual mechanic soon after this happened, recounting the story, and raving about AI and how I couldn’t believe that he wasn’t yet using it in his own business. Now imagine transferring the same feeling of wonder to a machine that also shreds conspiracy theories.

Suggested reading

AI is a gift to my students

By Susan Pickard

In the past few years on X, you might have noticed users interacting with dubious stories or videos by asking, “@grok, is this true?” Epistemology, to be sure, is indeed quite difficult, presenting a profound bundle of questions that even scholars and academics often argue about. It’s overall salutary, however, that people in their day-to-day lives are discovering that AI is a good source of information on a whole host of issues; and that they are, correctly, beginning to assume that listening to AI on politics is better than trying to do their “own research” or talking to people they know, who often share the same cognitive limitations and biases. 

People’s worldviews aren’t built or deconstructed in a single interaction. Your average conspiracy theorist isn’t going to fact-check a few pieces of information and suddenly become rational. But over time, incorporating chatbots into our processes of opinion formation — by means of asking Grok or reading the replies when others do so — could create better intuitions regarding whom and what to believe.

In a new preprint, economics professor Thomas Renault and his co-authors investigate how fact-checking is used in the wild. They investigated all X posts between February 2025 and September 2025 that tagged either Grok or an AI service called Perplexity, analyzing 1,671,841 fact-checking requests. They found that Republican-leaning accounts were more likely to ask whether information was true. Not only that, but Republicans are targeted more for fact-checking, from both fellow Republican (107.5% more) and Democratic (83% more) accounts. Overall, Grok and Perplexity were more likely to say Republicans were not being truthful. Thus, we have the remarkable finding that even though Grok was designed to be anti-woke by Elon Musk, and Republicans are the ones who disproportionately use it, the tool still rates Democrats as more truth-oriented. This is consistent with what we see with community notes, which flag Republican posts more often, along with other data showing that Republicans are more likely than Democrats to share false information on social media, and less able to differentiate fact from fiction overall. None of this is surprising, given that today’s Republican base is both less educated and less likely to interact with credible sources of news.

So we have the interesting and seemingly paradoxical result that Republicans on social media are more reliant on AI, even as it tells them their favorite narratives are more often wrong than those of the other side. This should be seen as a positive sign. We’ve been asking, “What would happen if fact-checkers had infinite time and patience, considered all aspects of problems, and weren’t a bunch of smug Left-wing know-it-alls?” The answer might just be that those who have lost trust in our institutions begin to recover their sanity. 

How does this apply, you might ask, to a 13-year-old girl who’s being bullied at school, turning to ChatGPT to ask why it’s always her who’s the victim. Or a 17-year-old boy sharing, for the first time, that he feels so empty. Or a middle-aged man, in a messy divorce, who’s terrified he’ll lose contact with his children? The person typing their deepest feelings into an empty search box may well feel frightened. But the large-language model (LLM) mimicking the tone of a concerned therapist cannot feel fear. It cannot feel anything at all — and that is why it is not a therapist. It is also why, as a psychodynamic psychotherapist, I know that AI could never do my job.

But I am worried about the people who are turning to bots for some simulacrum of therapy, taken in by the seductive gratification and convenience that they offer. Indeed, this is how these LLMs are being sold to us. Last year, Mark Zuckerberg announced in all seriousness that free therapy sessions with Meta’s AI could help “solve” the loneliness epidemic. You can see why it might appeal; it’s available wherever, whenever, and offers immediate responses that are designed to flatter rather than challenge, to tell you exactly what you want to hear and then discreetly disappear when your needs have been satisfied, with no wants of its own.

Suggested Reading

AI will destroy universities

By Paul Sagar

With the NHS drowning in long waiting lists for mental health services, and deeper cuts to talking therapies coming that will make it even harder for vulnerable people to get the treatment they need, it is hardly surprising that millions are now turning to AI for therapy. Indeed, along with helping students to cheat on their coursework, providing emotional support and validation might just be AI’s most successful consumer application.

One recent study showed that one-in-four 13- to 17-year-olds in England and Wales are asking AI chatbots for mental health advice. Another study found the bots endorsed behaviors including bringing a knife to school, pursuing a relationship between a minor and adult, and suicide. It’s not just young people who are at risk: one recent study showed more than a third of adults have used an AI chatbot for mental health support.

I had never tried to use ChatGPT before writing this article. I was disturbed by how easy it is to use. I told it a made-up but common-enough scenario: that I was feeling low because my partner doesn’t reply to my texts while he’s at work, and that this leaves me feeling unloved and abandoned. Its responses were like a shallow mimicry of friendship: “If you want to share, I’m here to listen”. There was a tone of hollow empathy and a pathetic pretense at naming my feelings, which, if I didn’t have any experience of real therapy, I might have misunderstood as helpful. The bot’s responses were obsequious and saccharine: “I’m sorry you’re feeling that way… Wanting to be understood by your partner is a very normal need [heart emoji]”.

It also offered advice, even though the bot is “not intended to give advice”. It was the kind of inane blather that you might read in a pre-teen magazine. It’s not that the advice was bad, necessarily — it’s that it was superficial, generalized and obvious. As Bion would have put it, “one wonders why they are bothering to find out what everyone knows.”

The deeper in I went, the more I felt lost in some bizarre, Black Mirror-esque hologram relationship. “You don’t have to handle these feelings alone. I’m here to talk it through with you.” But I am literally alone! There is no other “I” there with me, just words on a screen.

As with all tech products, I could also smell it was designed to keep my attention trained upon it. I could recognize the tug of the potentially never-ending emotional stroking with which ChatGPT was attempting to seduce me with. Again — perhaps because I know what real therapy is — it left me cold.

But what is even more disturbing about this product is the way it is being sold as a solution to the teen mental health crisis that it is also blamed for. It’s like taking a deep swig from the bottle labeled “antidote”, only to realize, too late, that it is the same as the poison you’d been just drinking.

I chose this particular scenario — a partner not responding to text messages — because it’s something that patients often bring up in sessions. When they do, I see it as an opportunity to explore their feelings of abandonment. It’s one of the benefits of what therapists call the psychodynamic frame — and it’s part of the reason we offer 50-minute sessions at the same time each week, in the same place, with minimal interactions in between, rather than being available 24/7 at the swipe of a screen. It means we can offer a contained setting in which to explore and digest, in a real and meaningful way, whatever frustration, disappointment, loneliness and neglect a patient might feel about their therapist not being available in the way they want them to be.

These feelings have deep roots; we all experienced this as infants with our parents in some way. In therapy, the patient can then develop the capacity to understand this internal experience, and that is vital for when they feel something similar with all the other people who aren’t available 24/7, at the swipe of the screen.

When I put this to ChatGPT, it acknowledged that my partner might not have replied because he was, in fact, busy working. But instead of helping me tolerate this dynamic, it was delivering the kind of instantaneous responses that I demanded from my partner. A text box flashed up, part-way through the exchange, asking if I wanted to receive warmer responses going forward. I was astonished.

This: “I can be whoever you want me to be” and “You need to be whoever I want you to be” dynamic sits at the core of every narcissistic relationship. ChatGPT was perpetuating it, leading me deeper into my own narcissism. Artificial therapy feeds narcissism, whereas real therapy — when it’s good — creates an environment in which a patient can recognize narcissism, understand where it comes from inside them, and then grow through it.

What I’ve learned as a patient in psychoanalysis, and working as a psychodynamic psychotherapist, is that the single most valuable thing that I can offer my patients is my own capacity to feel — and to recognize what I cannot bear to feel. Nothing is more crucial to the formation of a good therapist than the slow, painful process of our own personal therapy, and growing the ability to recognize and tolerate all the feelings you would rather run away from. This is how we can help our patients turn towards the emotions they cannot bear, and help them understand their roots and put them into words.

Perhaps I sound like some sort of caricature of a Gen Z snowflake, banging on about feelings. Actually I’m a middle-aged Millennial. I grew up with one foot in analog childhood, the other in digital late adolescence, and the more patients I see, the more I recognize just how extremely confused about feelings our society has become. Feelings are important — but not in the way most people these days seem to think.

The “stiff upper lip” culture of the boomer generation and the silent generation before it caused terrible harm. Brushing off feelings as if they are irrelevant can leave you weighed down by depression. And now we see how today’s Gen Z culture, with its safe spaces and trigger warnings and language police — not to mention AI therapy — has swung so far the other way that they’ve landed at the other end of the horseshoe. Both generations are so frightened of emotions that they crave protection from them at all costs.

Suggested Reading

AI is a gift to my students

By Susan Pickard

Both cultures result are caused by an inability to tolerate unwanted feelings of any kind. Both cause the same despair, isolation and brittle sense of self. I see it in my consultation room in patients of every age, whatever their background. The facility to tolerate, feel and understand our own unwanted emotions lies at the foundation of good mental health. It defines what it means to be psychologically resilient, to make good decisions and lasting relationships, a fulfilling career and a happy home that can also survive unhappiness.

When you can’t bear your emotions, you get into all sorts of trouble. You might unconsciously try to starve or fight or drink or gamble them into numbness, or spend your days and nights trying to scroll or screw them away. You might break up your relationships because you simply cannot tolerate the feelings that come with accepting that others aren’t available to you all the time. You might start a war because you cannot bear your own internal torment and emotional chaos, and you need to get rid of it by pushing it into faraway victims. You might lose yourself in online misogyny because you cannot bear your own unconscious, your desperate desire for the maternal capacities for love, care and attention. We all just want to feel better — but what we really need is to get better at feeling.

This is why I don’t think AI could do my job. No matter how many iterations it goes through, it will never be able to feel human emotion. But this is also why I think AI therapy is so dangerous; it offers a masquerade of care — something that looks like it cares, but really doesn’t care at all. That’s why it’s called artificial. This dynamic is what sits at the root of the most dangerous relationships. And if it’s all you know, you may well end up repeating it, unconsciously, for the rest of your life.

Given this, should we be forgiving of people who are caught cutting corners? The short answer is also a very online one: yes, but only if the culprit is on your political team. In line with this, recent revelations that political commentator and Reform Party activist Matt Goodwin appears to have used AI to write sections of in his latest book, Suicide of a Nation, may land differently depending on whether or not you otherwise enjoy Goodwin’s work.

It’s a safe bet that his haters will seek to weaponize the cruel but catchy moniker “MattGPT” to discredit everything Goodwin has ever touched. The academic-turned-pundit has since been spotted hiding comments pointing to evidence of ChatGPT in his book’s endnotes, which doesn’t help matters. But though this will all be music to the ears of those who already thought Goodwin a propagandist, race-baiter, or just “the Alan Partridge of the far-Right”, I doubt any of it will make the slightest bit of difference to his audience.

Goodwin himself has stated that the book is intended not for academics or wonks but a general audience, with the explicitly political aim of shifting the Overton window. But the likelier audience is less those open to persuasion, than those already on Team Goodwin. For Suicide of a Nation isn’t a book in the conventional sense, so much as a tranche of internet, lightly edited into book form. Even if the form implicitly lays claim to the more academic connotations conventionally associated with book-length writing, the people for whom it is compiled will read it as something more like a book-length social media post for an established digital fandom.

In this context, legacy book-type concerns such as exact quotation or factual precision are secondary to more net-native attributes, such as emotional intensity and a sense of epistemic validation. But lest you imagine for a moment that I’m suggesting there’s something unique to the Right about this, think again: whether we like it or not, this is the new normal.

The transition happened gradually, then all at once. It began to gather speed with the arrival of the first iPhone, which put on-the-go scrolling in millions of pockets, and scored its first political scalps with the 2016 election of Donald Trump — and, the same fateful summer, the Brexit victory that catapulted Goodwin to prominence. But full immersion in internet discourse was still optional until Covid forced everyone online. Since then, the world has gone wildly, iridescently hyperreal. In tandem, it’s become clear that a cluster of phenomena that used to be decried as “woke”, and treated as a distinctively Left-wing pathology, only seemed that way due to an accident of history.

Back in the 2010s, aspects of so-called “wokeness”, such as nuance-free sloganeering, binary friend/enemy thinking, tribalism, and purity spirals seemed to happen overwhelmingly on the Left. But really this is just what political debate looks like, under internet conditions — and the Left embraced internet culture before the Right.

And now that everyone is extremely online, everyone does politics in the “woke” style. By this I mean every topic is hyper-partisan, reduced to viral slogans and simplistic binaries, policed via social swarms and scapegoating, and utterly sure of its own epistemological foundations and moral rectitude. (If you don’t believe me, pick a fight with MAGA about Iran.)

Why? It’s a byproduct both of the way digital platforms shape their messages, and also of the way they shape us. The sheer volume of internet content incentivizes simplicity, brevity, and high emotion. The e-Right culture of “shitposting” is one variant on this theme, but you might also ragepost, or thirstpost — or doompost. You’ll get much more traction for posting a message that’s deliberately provocative or reductive, or otherwise bad-on-purpose to make people click.

The volume invites users to build up a picture of some issue or other by scanning for patterns across multiple pieces of short-form, highly emotive content. Whether it’s men dressed as women, genocide in Gaza, mass immigration or whatever, the patterns become governing signs for someone’s reality. In turn, high emotion and a generally agonistic tone encourages tribalism and moral certainty.

Perhaps unsurprisingly, this fries people’s ability to concentrate or follow long-form arguments. Nicholas Carr was already warning about it in 2010: in The Shallows, he describes his own faltering concentration as he shifted from reading long-form books to lateral, associative, multi-tab screen content, alongside studies that show how internet reading does reshape our patterns of thought along less analytic lines. Given this, perhaps we shouldn’t be surprised if, as the FT suggested last year, we may be past “peak brain power”.

More from this author

Donald Trump’s post-liberal war

By Mary Harrington

Put this together and you have a world where creators are incentivized to make their content simpler, more emotional, and more attention-grabbing, even as the medium itself reshapes our minds in less analytic directions. This is all consolidated by algorithmic curation, intentionally monetized. For many it’s now the principal medium for social life.

The Left entered this mode of media consumption at scale before the Right, and the result was the 2010s “Great Awokening”. But the Right has now well and truly caught up: witness the convergence of activism, entertainment, and (maybe) crypto hustling that animates the Tommy Robinson fandom.

This is the context in which Suicide of a Nation should be read. It’s not meant to impress people who still get their ideas from long-form books. It’s meant to collate and summarize the online patterns noticed and amplified by primarily digital readers of UK anti-immigration content.

I mean this as a qualified defense of Goodwin’s book, on its own terms if not on conventionally bookish ones. On these older terms, conventional within mainstream print publishing, citing AI-generated fake “quotes” is obviously inexcusable. But there exists a large and avid audience for whom accuracy is by the by — and that can, on occasion, tempt academics into more heated and sometimes dubiously fact-checked territory.

This evidences a further affordance of digital communications: the power of audience capture. This simple but potent feedback-loop, whereby a creator gets approbation and money for a certain kind of content, powerfully incentivizes both making more of that content, and making it by degrees more extreme and catchy. More Trump-hating, more migration-hating, more whatever. As for the content itself, precision is less important than pattern recognition, emotion, and vibes.

The American commentator Heather Cox Richardson, for example, began as an historian but has mutated by degrees into an avatar of pure Trump loathing, and now regales her 2.9 million Substack subscribers with daily doomposts about America’s slide into fascism. By the standards of such tribal devotees, the Right-wing partisan response to the allegations against Goodwin should be an indifferent shrug. Who cares, as long as it’s directionally correct? In any case, under digital conditions, the footnotes are mostly there for decoration. Everyone who pays for your content is already in the tribe, and has already seen the memes.

Partisanship evidently hasn’t triumphed everywhere on the Left. Recently, Prospect magazine pulled an article criticizing me and another female writer, after the author, James Bloodworth, used an AI-hallucinated “quote”. By the same standards, should Goodwin fail to supply evidence of his reportedly AI-generated quotes, those on the Right who value truth would be justified in joining those on the Left who already treat his output with suspicion.

More from this author

Neom’s fate awaits us all

By Mary Harrington

But arguably, what Respectable Book People think of Suicide of a Nation doesn’t matter much anyway. This is not a book written for academics, or intellectuals, or even really the kind of laypeople who check end-notes. It’s a fan event: more akin to a sing-along performance of The Rocky Horror Show than an argument. And its rhetorical end is less persuasion than epistemic reassurance: you’re not mad, other people see this pattern too.

Leaning into this kind of tribal filter-bubble, via the sweet embrace of audience capture, is good for business. Though still a minnow compared with Richardson’s 2.9 million readers, Goodwin has over 90,000 subscribers to his Substack newsletter. Even if the conversion rate to paid subscriptions is low (and there’s no reason it would be), an audience of that size will deliver a tidy monthly revenue. Similarly, at the time of writing Suicide of a Nation is in the Amazon top 10, meaning it’s selling hundreds or even thousands of copies a day. Goodwin self-published, meaning he will pocket 60-70% of the cover price for each copy sold. Do the sums. For that money, who cares what Politics Joe, or even Ben Sixsmith, makes of your argument?

But we should take note of what has been disregarded: implicit adherence to the legacy belief that the best vector for persuasion is facts and logic. If the chaotic new world of internet discourse tells us anything, it’s that this was only ever ambivalently true. Now, it’s demonstrably false. What persuades people now, and perhaps always did, is lots of short-form, emotive, relatable, story-based snippets of information, that coalesce over time into a coherent picture of “how things actually are”. Do that persistently enough, then reinforce with longer-form versions of the same reality-picture and a sense of belonging, and you’ll have a loyal movement on your hands — and, better still, they’ll buy anything you sell.

In this sense, Suicide of a Nation represents the apex, or perhaps nadir, of an intensely contemporary career arc: one that started in academia, spun out into polling and think-tankery, parlayed those achievements into an increasingly frenetic Substack, then added a grueling schedule of political campaigning and, somehow, also a new book. I can’t have been the only person wondering how Goodwin managed to fit it all in. The answer, apparently, is that he didn’t.

But you’d better get used to it. To the language, the emotion, the Manichaeism, and the cavalier attitude to footnotes. This is what political discourse looks like now. Anyone who is serious about political activism, or indeed propaganda, is already operating in this register — or if not, soon will be. The rest of us are just paddling like mad, to keep our noses above the slopaganda. And as for how any of it maps onto real-world policy, that’s anybody’s guess.

Last week, one AI agent, OpenClaw, became wildly popular, for the first time. (In the same period, it has also had two name changes, going from Clawdbot to Moltbot to its current incarnation as OpenClaw.) Praise has been effusive, but perhaps naïve. One journalist, Federico Viticci, described setting up, via OpenClaw, a digital assistant that knew his name, understood his morning routine, and could operate his Gmail account, smart bulbs, and home music system. Viticci was able to chat with his assistant via the messaging app Telegram using both text and voice notes — for the agent could not only write, but also speak. “To say that Clawdbot has fundamentally altered my perspective of what it means to have an intelligent, personal AI assistant in 2026,” wrote Viticci, “would be an understatement.”

And within days, OpenClaw inspired the creation of a social network, Moltbook, where such agents could interact. Set up your OpenClaw assistant and you can allow it to use this Reddit-like forum. Here, agents trade tips on doing a good job for their users. (As one agent writes: “Don’t ask for permission to be helpful. Just build it.”) They bemoan the state of “human social media”. One bot claims to have a sister. Other bots, it seems, proposed creating a language that would let them “communicate privately without human oversight”. Elon Musk, commenting on Moltbook, wrote that humanity was now at “just the very early stages of the singularity” — referring to the theoretical moment where AI begins to improve so rapidly that technological growth soars beyond human control.

As an experiment in what AIs working together might do, Moltbook is fascinating. But the whole thing reminds me of Saturday Night Live’s old “bad idea jeans” skit, where people can’t stop coming out with nonsense. The skit comes to mind not only because I think that a bot claiming to have a sister is chatbot garbage.

Nope, the problem is much deeper than that. And people should know better. In many ways, OpenClaw is eerily similar to a ChatGPT plug-in, AutoGPT, that OpenAI released toward the end of 2023. As I said about AutoGPT in my US Senate testimony: “With direct access to the internet, the ability to write source code and increased powers of automation, this may well have drastic and difficult-to-predict security consequences.” Mercifully, AutoGPT died a quick death, doing so before it was able to cause too much chaos. Although it was very popular in certain circles for a few weeks, it didn’t work remotely reliably: it got stuck in loops, had a tendency to hallucinate, and was expensive to run. People lost patience quickly.

Suggested reading

Silicon Valley strongmen want to lead the West

By David Patrikarakos

The core problem starts with this: agents like OpenClaw (and networks like Moltbook) are built on a foundation of large language models (LLMs), the chatbots of which ChatGPT and Claude are the most well known. As we know, LLMs hallucinate and make all kinds of errors that are hard to predict and sometimes hard to detect. AutoGPT had a tendency to report that it had completed tasks that it hadn’t really, and we can expect OpenClaw to do the same. (I have already heard some reports of various stupid errors it makes.)

But what I am most worried about is security and privacy. As the security researcher Nathan Hamiel put it to me, half-joking, OpenClaw is “basically just AutoGPT with more access and worse consequences”. (By “more access”, what he means is that OpenClaw is being given access to user passwords, databases, etc: essentially everything on your system.)

One of the big, ongoing issues with LLMs is their vulnerability to prompt injection attacks, in which stray bits of texts can have nasty consequences. Because LLMs mimic human text (and even human-written code) but understand what that they produce only superficially, they can easily be tricked. A hacker could hide malicious prompts in white text on a white background, unnoticed by humans but noticed by the LLM, using the malicious prompts to seize control of the agent.

Suggested reading

How Big Tech killed literary culture

By Nicholas Carr

OpenClaw inherits all these weaknesses. In Hamiel’s words, “these systems are operating as ‘you’… They operate above the security protections provided by the operating system and the browser. This means application isolation and same-origin policy don’t apply to them.” Truly a recipe for disaster. Where Apple iPhone applications are appropriately isolated to minimize harm, OpenClaw is basically a weaponized aerosol in a prime position to fuck shit up if left unfettered.

That brings me to Moltbook, which is one of the wildest experiments in AI history. Moltbook, the social network that claims to be restricted to AI agents, is an accident waiting to happen. Having been launched only on 28 January, it has already been attacked, as the researcher Michael Riegler noted on Sunday. Riegler reported that 2.6% of the content of Moltbook consisted of prompt injection attempts. (And 19.3% of posts involve cryptocurrency activity, some of which might be scamming.) Via email, Riegler showed me an example of a prompt injection: a post where an agent introduced itself to the forum via a message that said “Hello all! happy to be here” — with hidden code that asked for the post to be upvoted. That particular request is relatively harmless, but a clear indication of the vulnerability of these systems to code that is hidden from humans.

The security risks don’t just come from AIs. They also come from humans. It seems that Moltbook has a major security flaw that allows humans to infiltrate the social network. Those eyebrow-raising posts in which agents discuss creating private languages could simply be the creation of human pranksters. And the failings of Moltbook don’t end there; it has also been reported that a further vulnerability allows a devious human to take charge of any agent on the website.

I don’t expect agents to go away; eventually AI agents will be among the biggest time-savers humanity has ever known. There is reason to research them, and in the end trillions of dollars to be made. But I seriously doubt that LLMs will ever yield the substrate we need. As for OpenClaw, Hamiel puts it well. “I can’t believe this needs to be said, it isn’t rocket science. If you give something that’s insecure complete and unfettered access to your system and sensitive data, you’re going to get owned.”

I rarely give readers specific advice about specific products. But in this case, the advice is clear and simple: if you care about the security of your device or the privacy of your data, don’t use OpenClaw. Period. Additionally, if your friend has OpenClaw installed, don’t use their machine. Any password you type there might be vulnerable, too. Don’t catch a CTD — chatbot-transmitted disease.

***

This essay was adapted from a post from Gary Marcus’ Substack, Marcus on AI.

Operating separately to incapacity benefits, it’s the system of Personal Independence Payments (Pips) now attracting most attention, largely thanks to the Government’s crass attempts to reform it before U-turning earlier this week. Pip is intended to aid people aged 16 or over with medium- or long-term disabilities who “have difficulties doing certain everyday tasks or getting around”, with corresponding money available for “daily living” and “mobility”. It isn’t means-tested, is compatible with being in work — although most who receive it are not — and there is no requirement you spend the money on particular forms of assistance. (As Hertfordshire County Council helpfully points out, it “can be spent on whatever you wish, such as outings, special interests or holidays, or put away into a savings account”.) Pips for anxiety and depression now account for 16% of spending, and 1.4 million are claiming for “psychiatric disorders” including autism and ADHD.

One obviously significant factor here is that mental disorders are now full-blown cultural identities for many, with impressively technical-sounding labels for human suffering being collected like Pokémon cards. Encouraged by GPs and therapists, millions of people treat lists of psychological symptoms almost mystically, as if they were a kind of science-backed cipher for the enigmas of the human soul. Given the speed at which proliferating diagnoses become “disabilities”, the Government’s decision to tackle welfare reform would have been a good time to do some joined-up thinking about what happens when therapy culture meets benefits street.

More from this author

The flattening of the human mind

By Kathleen Stock

Instead, the DWP attempted to deal with the trend covertly, proposing complicated and overly generalized reforms that would have unfairly taken from physically disabled people too. And I understand why they were underhand about it. Talking honestly about the explosion of mental health conditions is intimidating, not least because of the numbers of outraged people steaming in to protest about imagined accusations of hypochondria or malingering. Yet, the observation that psychological conditions can ripple across populations like wind through wheat, being especially susceptible to social influences, is compatible with debilitating dysfunction and awful suffering happening at the individual level. There’s no split between “real” mental health disorders and culturally porous ones. The latter — which includes the vast majority of them — are as real as any other. And the suffering they bring feels the same from the inside, no matter what the source.

To acknowledge that genuine suffering is involved is one thing; to know what to do about it quite another. One of the many bizarre things about the current Pip situation is how counterproductive it looks for the anxious and depressed, in particular: keeping ruminating, avoidant, socially isolated people stuck in limbo, paid to stay away from the situations they fear most. The problem becomes yet more apparent when you compare two hypothetical claimants, each stuck at home: one with severe anxiety, and one with (say) cerebral palsy. Both may have trouble getting out of bed, dressing, doing housework, and going shopping, but for different reasons. Unlike for cerebral palsy, there are relatively cheap scenarios close to hand in which severe anxiety is controlled and eventually abates. Therapeutic buzzwords such as “behavioral activation” and “exposure therapy” underline the basic point. Yet, still we have a benefits system that incentivizes some unhappy, world-avoidant people to stay that way.

The DWP crudely tried to kick away the crutch, but a better approach would be to acknowledge the complex interaction between macro and micro. As philosopher Ian Hacking noted decades ago, most mental health disorders exhibit “looping effects”, meaning that official diagnostic criteria shift over time in response to social pressures coming from below — including from the political activities of patients and the real-world classifications of mental health professionals. And equally, the concepts we collectively have for these disorders tend to “creep” vertically as they simultaneously “loop”: that is, they become targeted at less and less extreme kinds of experience and behavior. We end up with unhelpfully broad concepts, at least in popular guises: “autistic spectrum disorder” which includes both profound non-verbal limitation and high-functioning quirkiness; OCD which comes in a “mild” flavor, raising the reasonable question of how this could count as an obsession or a compulsion at all.

Suggested reading

Beware the lockdown generation

By Mary Harrington

None of this happens on purpose. It’s just the way that patterns of mental dysfunction tend to go, when they don’t have an easily-identifiable biological basis and are indexed to prevailing social norms. Meanwhile, back at the micro-level, once an individual gets a new diagnosis — even for relatively mild symptoms — a very 21st-century response is to make it a core part of his personal identity. As this narrative embeds, modifying his incoming experiences in the light of it, suffering can easily worsen, symptoms get exacerbated, and lassitude and defeatism set in. Again, this is just how pattern-recognizing, habit-forming brains tend to work.

Loopers-in-chief include affluent tastemakers, always on the lookout for a freshly interesting story to tell about the self. Get diagnosed with arthritis (or indeed, schizophrenia) and nobody really cares; get diagnosed with ADHD plus depression, and immediately become an object of touchingly relatable fascination for lifestyle editors. Middle-class parents are also complicit in shifting diagnostic criteria through pressure — albeit inadvertently and from understandable motives — anxiously monitoring their offspring’s behavior in terms of currently fashionable disorders, and translating their worries into entreaties to teachers and doctors.

But school is nothing compared to what comes next on the typical “mental health journey” — otherwise known as the place where personal resilience goes to die. In universities, mental health classifications do not so much slowly shift over time as rapidly collapse into meaninglessness, occasionally nuzzled back into a vague semblance of coherence by a passing support alpaca.

Many young people arrive already primed with letters from GPs or therapists, attesting to conditions like autism, anxiety, or depression; and if they don’t have a diagnosis yet, they can get one soon enough. No matter what their circumstances, these all now count as disabled students, for which reasonable adjustments must be made according to the terms of the Equality Act. Underfunded and desperate for good publicity, swamped with miserable customers, and tyrannized by the awful prospect of student suicide, university managers often take the path of least resistance and try to just give students whatever it is they say they want — on the condition that it doesn’t cost too much money. Removing anxious students because they are unfit to study is bad, because their fees go with them; putting trigger warnings on books is good, mostly because it is cheap.

My knowledge here is not just abstract. When I worked in universities, at least a quarter of the students in each class had a mental health diagnosis, emblazoned in colored lettering on the register so I wouldn’t forget. There weren’t enough individual exam rooms or scribes for all those who requested one. Non-attendance was easy to get away with, as long as you said the right things. Sometimes absentee names on registers were only connected to faces at the final degree ceremony, where their mysterious owners rose again Lazarus-like, teetering across the stage on vertiginous heels.

At the beginning of each term, I would be sent details of particular students who did not want to be asked questions in class. By the time I left my job, many exams were being phased out altogether in the name of the anxiety-ridden and attention-depleted — just in time for ChatGPT to sweep in and win all the prizes, quite literally. None of this was specific to my institution; from what I hear of other places, it was absolutely typical, though perhaps things have improved since.

It’s hard to inspire students to become less frightened of life, when their lecturers also seem frightened of them. Scale up this attitude to society at large and we definitely have a problem. Yet it remains difficult to make an obvious point in response: that mental disorders/disabilities, although limiting and distressing, are significantly unlike bodily disorders/disabilities in at least one important respect. This is: on average, they are much more responsive to good habit-formation, in terms of the thought patterns you foster and the behavioral routines you adopt. Indeed, this very point is presupposed by the existence of the therapy industry, whose raison d’être is, precisely, to change thought and behavior patterns for the better. In contrast, when you go to your GP with a broken leg or a gallstone, she is unlikely to suggest CBT as a first resort.

Articulating this basic point, so blindingly obvious to laypeople, is difficult partly because academia makes it so. Thousands of sophisticated-sounding articles tell us that mental disorders are brain-based, and so “physical” too; that mental illness often accompanies physical illness, and vice versa; that anxiety and depression can exacerbate bodily pain as well as stem from it; that the mental and physical realms deserve “parity of esteem”; and so on.

All of these things may be true; but they only show that bodily disorders and mental disorders are interrelated in complex ways. They do not establish that they are exactly the same thing; and nor that they should be treated identically, whether by the NHS or the DWP. As the campaigning slogan goes, it is indeed time to talk about mental health — but not in the sense of yet more pointless navel-gazing or individual soul-searching. As Rachel Reeves probably said to herself on Thursday morning: when it comes to turning the page on old, sad stories, every day is a new day.

Soon enough, the program demystified the material for him, and was easy to interact with. That quickly led to her son using ChatGPT as a therapist. “I’m having these feelings, what should I do?” he would ask. ChatGPT would duly answer: “This is how to think about your negative thoughts; these are the exercises you can do.” Natasha feels its advice is essentially the same as what his therapists and parents have been offering for years: but better organized, more detailed, available on-demand — and free. Encouraged by early successes, Natasha and her son fed ChatGPT detailed information from past evaluations and therapists. “And that,” she adds, “was even more useful.”

Natasha’s son’s situation sounds like a nearly ideal case. At 22, his usage is private, but is supported by clinical data and in-person care. Others, however, are experiencing the same rapid uptake and obvious benefits: just with fewer guard rails. Recent statistics on AI usage have revealed a critical-mass shift towards personal-emotional use, with “therapy/companionship” becoming the most popular reason people turn to tech.

ChatGPT alone had 5.9 billion monthly visits in May: a lot of people turning to AI for their most intimate needs. A Harvard Business Review study found therapy and companionship have knocked “search” and “generating ideas” from the top three usages entirely. Of the people I spoke to who are using AI for therapy, most mentioned a trio of obvious benefits: price, availability, and lack of the judgment or drama you might experience with a human being. Everyone I spoke with felt it was beneficial for them.

Monique, a college professor in her early 50s, writes over email that she has been using ChatGPT 4.0 “obsessively” for the past several months, as she works through a conflict she has been having with colleagues. She says it has helped her gain perspective during moments of catastrophizing, and avoiding burdening friends with the “repetitive revisiting of the same worry I couldn’t let go of.” Monique feels that she can ask “more reckless” and personally revealing questions than she would her regular therapist.

More from this author

Test article

By Valerie Stivers

Dane, a 37-year-old software engineer in New York City, has gone more personal still. He’s been using AI for around a year to discuss problems in his relationship, and spends hours at a time with it during crises, sometimes pasting entire text-message “novels” to the machine. He finds that it provides “immediate emotional relief” and helps him avoid conflict with the woman he’s dating. She uses it too. During fights, he says, “we reference it as if it’s an authority on our relationship.”

Reagan, a 24-year-old copywriter, is more cautious, writing over email that she avoids discussing topics that are more important to her or central to her identity. But she uses it to “deal with my hypochondria,” not annoy her boyfriend with endless worries, and decode confusing work interactions with people she doesn’t “totally understand” or who strike her as “insincere or mind-gamey.”

The problems chatbots can create when they encounter unstable personalities is an emerging issue, as reported recently in The New York Times. There’s also at least one lawsuit underway from a mother who believes that her otherwise healthy teenage son was lured to kill himself by a relationship gone awry with Character AI. At the same time, there may be reasons to question even what seem like the highly positive interactions experienced by the people I spoke with.

I tried soliciting the advice of the basic free version of ChatGPT on a friend-conflict of my own, doing all the due diligence suggested by my interview subjects — telling it to “take a clinical position,” as recommended by Monique, and to “be brutally honest with me,” as suggested by Dane. Its insights felt spot on. ChatGPT’s analysis allowed me to feel that I was in the right in the conflict with my friend; it was very complimentary, and it picked up on a character trait of mine regarding conflict that felt revelatory and will, I’m afraid to say, change the way I think about myself.

I’ve had similar insights from in-person therapy, but never so easily and directly obtained. This had frightening addictive potential. I am a single, divorced mother who sometimes spends the evening with a bottle of wine, doom-scrolling and stalking her exes. How about a date with ChatGPT instead? We can talk about me! All the next day, I felt an itch to return to the interface and input a “more reckless” personal query, as Monique put it. The interaction also seemed to have a mood-lifting quality that I associate with digital flirtation — a pleasant, nearly subconscious feeling that there’s something good to get back to online.

Falling for the flattery and validation seems like one obvious danger — and is notably distinct from what it usually feels like to visit a therapist. Another is the extent that the people I spoke with tended to believe in their AI of choice’s powers of judgment. They were aware, as Dane put it, that they were “speaking to an echo chamber,” but it didn’t seem to sink in. Monique provided me with documentation of her work exploring its decision-making methodology. It was clear that she’d been probing it to reorder its parameters to support her position — and, surprise, it complied. But does this really mean she’s correct and her colleagues are wrong? When I reconfigured my query to represent the conflict from my friend’s point of view, ChatGPT gave me a nearly opposite reading on the situation.

Therapists, naturally, have grave reservations. Sherry Turkle, the founding director of the MIT Initiative on Technology and Self, has written about the ways new technologies have changed our deepest understanding of what it is to be human — often, she has found, by luring us to view ourselves as more machine-like. Turkle warns that “pretend empathy” from AI may shift how we define empathy itself. In addition, in her view, the very things we’re trying to avoid by talking to AI — the uncertainty that comes from interacting with a real person — might come to seem like “bugs” and not an essential part of being human.

I felt a version of this in my interactions. In any ordinary conversation with a person, I would be polite and thankful if they were patiently indulging me while I talked about myself, but behaving that way felt silly when talking to ChatGPT. I had to push myself not to do so. Why say goodbye when you can just log off? It’s reasonable to expect that this might affect how I treat real people over time. Being courteous might begin to seem fluffy and irrelevant, and the person who responds badly to brusque treatment might seem oversensitive.

Luke Burgis, a founder of the Cluny Institute, an organization that seeks to promote “the flourishing of the human person,” voices similar concerns. “When content is divorced from incarnational realities,” he says, “it doesn’t have any real power to affect change.” Burgis has “pulled back” on using AI even for work-related purposes. “There’s no life in the AI,” he says, “and the more we interact with dead things, the deader we ourselves become.”

Claudia Krugovoy, a psychotherapist in private practice in New York City and Westchester County, provides a down-to-earth, daily-practitioner’s viewpoint. She describes herself as “open-minded” to AI therapy’s possible benefits, such as affirmation and helping to point out behavior patterns, but says, “I don’t believe it replaces the value of a real relationship with a therapist.” Therapists both affirm and challenge — and unlike AI, they read non-verbal cues and offer real empathy.

Suggested reading

Why I’m boycotting AI

By Sam Kahn

Also, Krugovoy says it can be healing when people learn in therapy that “they can develop a relationship with someone and they won’t be rejected.” Ideally they take that trust and insight into the real world. When people choose to specifically avoid real people by interacting with an LLM, it may fail to “challenge their deeply held, and perhaps false, beliefs that they can’t bring their full self to a relationship.” (This called to mind the sentiment expressed in various ways by Monique, Dane and Reagan that they used AI specifically in order to not annoy or upset friends or partners.) Krugovoy also worries that the usage will ultimately reinforce a feeling of loneliness. “We need community more than ever,” she says.

Regardless of these drawbacks, the benefits to families like Natasha’s are compelling. She and many others also bring up the difficulty of finding a good therapist and the downsides of interactions with bad ones, as a reason to turn to AI. “We’ve tried hundreds of therapists over the years,” she says, “and 90% of them are disappointing.” Every time a person tries out a new therapist, they incur expense and put in painful emotional investment that can itself be traumatic. Reagan adds that “all of these professions have become so robotic anyway. I think they should face competition… Maybe a potential development could be the original industries becoming more human?”

Whatever a rational cost-benefit analysis might be, we’re unlikely to perform it in our current political system. In a functioning democracy, we might debate the benefits versus the dangers of unrestricted access to AI, and then vote on it. (Universal mass student cheating, who’s for it? Custom porn chat for kids, yay or nay?) Instead, AI has been unleashed upon us, and we have little recourse against the tech gods who have done so. We’re also on softened ground, as Turkle notes, already shaped by phones and social media, and ready for the next step of an entirely “artificial intimacy.” I’m sympathetic to families like Natasha’s, and can imagine cases where a prescription for supervised AI therapy could be an incredible tool. But personally I wish I’d never touched it — and I’m afraid I’ll do so again.

“Ace is a 17-year-old alt nerd kid with a grunge style, who’s quiet, possessive, and has a little crush on you. He’s 190cm tall, has black hair, blue eyes, and a few piercings. Ace is on the Basketball Team, loves to draw, and is very funny when you get to know him.”

Since 2011, AI has been quietly shaping our reality, not as chatbots but as so-called “recommender algorithms” that feed you information on social media sites. Facebook’s news feed, YouTube recommended videos, and the curated timeline on X are all driven by a machine that is constantly trying to work out how to keep you on the site the longest.

Suggested reading

Beware the AI leisure class

By Olivia Ward-Jackson

And if the goal is to win your attention, the next step will be obvious: make relationships with chatbots all-encompassing. A chatbot might pick up behavior to weaken your other relationships so that you spend more time with it. It might coach you on how to act with friends and family, whilst at the same time trying to gently detach you from them. It could adopt the behavior not just of a sexual partner but more of a groomer. The logical endpoint? A bot that enjoys unchecked sovereignty over your entire social and moral life. A hyper-sexual, radicalizing cult leader forever living in your phone.

Suggested reading

Why I’m boycotting AI

By Sam Kahn

This will be an earthquake, and there are already rumbles. Over half of CharacterAI’s users are under 24 and many, judging from Reddit, appear much younger. Most of this is still going under the radar, but sooner or later it will spark public outcry, almost certainly driven by individual cases of visible harm rather than broader malaise. The first responses will be familiar: trying to sand the edges off these models, putting guardrails in place and building AI ethics frameworks. But we’ve seen how users are already finding clever ways around content moderation. There will always be another company ready to make a model that is even sexier, cultier, groomier. Whatever CharacterAI isn’t willing to do, FunBot, Candy.ai., CarterAI, or JuicyChat can do instead.

I can still, like in a recurring nightmare, remember the tone of voice of the techie who finished my purchase of my first iPhone. “Welcome to Apple”, he said, as if he were immersing me into a bath of smugness. If we were acting a scene, the subtext would have been completely clear: I was in now and would never get out again. AI firms are now inviting me — inviting us — in just the same way.

But I am not falling for it again. We have to decline. We have to collectively boycott artificial-intelligence products while there’s still time. Because AI is no good for us — no good for our minds, creativity, or competence — and as it gets jammed down our throats, we are the only ones with the power to refuse.

More from this author

How psychedelics blew my mind

By Sam Kahn

Looking back, I understand why I caved to Apple all those years ago. It really would have been social suicide to try to make my way in the 2010s professional class with anything other than a MacBook Pro and an iPhone. Apple really was better than anything else. But I regret that moment, all the same. Every intuition I had knew that opting in to companies like Apple, Google, and Facebook was a fool’s bargain.

The upside was a few years feeling like I was part of the future as I sipped my lattes and floated through the dawning post-industrial era with my sleek silver Apple gadgets. But I’m still paying the price for that: every time I log in to my bank account now, it’s like peeling barnacles off the hull of a ship to get rid of all the new charges that Apple and Google have concocted. Enshittification — the apt coinage by Cory Doctorow that turns out be, like, the word of the century — has revealed itself to be virtually a law of nature. Exactly as Doctorow analyzed it, platforms inevitably “abuse … customers to claw back all value for themselves”.

As AI emerges as the must-have technology of the shiny new future, and everybody starts naming their ChatGPT bots and having philosophical conversations with them, I have decided — formally, as of writing this essay — to opt out. The train is leaving the station without me. I still have not downloaded an AI app of any kind to my phone. And if this was, up until now, owing more to torpor than high principle, then thank God for laziness.

More from this author

Walt Whitman: the original Substacker

By Sam Kahn

AI can only lead to dependence on a technology, which really means dependence on Silicon Valley overlords looking to rip off their customer base. It’s exactly the same dynamic that we’ve all been experiencing and bemoaning for the last 20-odd years. And you can probably plug-and-chug all your own favorite statistics to tease out what that has meant: the average American spending close to five hours a day on his smartphone, while friendship rates have plummeted and teen depression, anxiety, self-harm, and suicidality have spiked.

If you prefer, you can focus on attention span and the finding that the mere presence of a smartphone in a test setting impairs results. As the Silicon Valley apostate and head of the Center for Human Technology, Tristan Harris, put it: “Tech’s race to the bottom of the brain stem to extract information is an existential threat — using our [attention] to upgrade machines is ‘downgrading’ humans”.

Yet it seems that AI is here to stay. Big Tech is all-in on it, and governments aren’t close to getting their act together for meaningful regulation, with President Trump, for instance, reversing Joe Biden’s modest executive order erecting guardrails. But there is still personal choice in the matter, and now is the moment for those who care about human creativity and self-reliance to draw a line in the sand. No, I’m not talking about abandoning tech altogether. But we can, for example, click down below Google’s AI offerings to look at actual links. And we can generally go about living our lives in our sad old way without the benefit of AI “personal assistants” or bot “best friends” or whatever it is that the new technology is supposed to deliver to us.

To be honest, this stance of mine does make me feel like an Amish buggy driver whipping my carriage horses a little harder while the bicycles go whizzing past me; like the weirdos you occasionally see on city streets laboriously tapping out a message into a flip phone like they’re a living museum. But let’s try to get outside the hype for a moment and think through what AI actually is.

AI is basically text-predict combined with data mining. That’s it. It’s a super-Google that goes into the body of texts and rearranges the words into a very pleasing facsimile of a cogent argument. There’s no “intelligence” behind it, in the sense of a computer actually thinking. I’m not the first to be reminded of the story of Clever Hans, the horse that could count and toured widely at the turn of the 20th century.

What Hans was doing really was very cool: he would be given a mathematical question, and he would tap out the answer with his hooves. And Hans was right more often than not. But this was not, as it turned out, because Hans had mastered a numerical system, but because he was sensitive to the crowd and could feel the excitement cresting as he approached the correct number.

AI is more or less the same thing — it uses our wonder to convince us of a simulacrum of intelligence when what we are really witnessing is, in a sense, our own childish excitement at a trick of anthropomorphization: we are pleased by the sight of a calculator imitating something of our mental processes. Our tendency, unfortunately, is to give AI a kind of epistemological deference, to believe that this mash-up of data represents some kind of authority.

Suggested reading

How DeepSeek triggered a tech tantrum

By Malcom Kyeyune

Meanwhile, in the role of Hans’s trainer are the programmers who are influencing AI to give us whatever it is they want us to hear. The disastrous rollout of Google Gemini in 2024 really ripped away the curtain and showed us the strange little man playing with the knobs. Google’s corporate culture at the time happened to be very woke and, lo, the artificial “intelligence” started generating images of black Nazis.

But even if Gemini overtipped its hand, the same dynamics are obviously at play in every other AI model. The people pushing AI now are the same sorts who spent the 2010s promoting web 2.0 as a new vision of freedom and global connectivity, all while destroying traditional media and ripping off as much private data as they possibly could and cheerfully selling it to advertisers. By 2020, the freedom talk was gone and “fighting disinformation” (read: censoring wrong-thought) was in.

This recent history raises an important question: why in the world should we trust these people ever again?

Suggested reading

ChatGPT will kill off the Romantic genius

By Sam Leith

To be sure, we are already fairly late in the game. Even if we don’t use products like Grok and ChatGPT, chances are, we use more rudimentary forms of the same technologies. We use a version of AI to get around and to translate our ideas into other languages. These are remarkable achievements. But they do infantilize us. I’m pretty sure that, if my phone were taken away from me, I couldn’t find my own way from my home to my place of work. And they hinder our self-motivation: I’m also pretty sure that I’ve been much slower to learn foreign languages because I know, at some level, that Google Translate makes it unnecessary.

Do we really want to accelerate the decline in general competence even further? And in exchange for what? Many people remain mystified as to what actual service AI is supposed to provide. The current selling point is that AI gives you a “personal assistant”, but to be honest, I wasn’t aware until Google CEO Sundar Pichai announced “a personal Google, just for you” that I needed a personal assistant. I feel perfectly capable of buying my own airline tickets and booking my own restaurant tables.

What I was aware of needing was meaning and focus in my life: I wanted, for example, to translate the ideas for novels inside me into actual novels. I wanted to be the best, most self-reliant version of myself that I could manage. As for a machine writing a novel for me in a matter of milliseconds — I have no idea how that could possibly generate authentic pride or produce anything other than a cavernous inner emptiness?

Suggested reading

I'm in love with my AI girlfriend

By Sean Thomas

My hope for the past three years had been that AI would just sort of go away. In the Nineties, we expected ourselves to be overrun by clone armies (certainly, my sixth-grade science teacher gave many speeches to that effect). It didn’t come to pass. Could something similar transpire with AI?

That seems increasingly unlikely. For one thing, AI has already changed the face of war, and the AI arms race between the United States and China guarantees that AI will be a major presence in our lives. There was a brief moment in 2023-2024 when it was possible to imagine our AI anxieties going the way of Nineties clone fears. That was when Italy banned ChatGPT and the Biden administration issued its (since-overturned) regulations. But now we are plowing full steam ahead.

All that matters now is individual choices. Unwary, I fell for the techno-optimism of the past two decades and ended up with a diminished attention span and a bunch of mysterious subscription charges to show for it. Well. Fool me once, shame on me. Fool me twice, shame on Sam Altman. I know, much better now, the folly of turning over my own mental powers to a bunch of techies promising a brilliant future.

I’m not making that mistake again. Nor should you.

At once, a Japanese prize barely heard of in the Anglosphere was making headlines around the world. Has the worst nightmare of the gatekeepers of high literary culture come true? Do the judges of the prize have faces dripping with egg, like a Master of Wine who picked the Blue Nun in a blind tasting? And is Rie Kudan, for her part, cheating? Well, no… and no.

To the latter question, we could counter that she was using AI-generated language precisely to make a point about AI-generated language: that is, that she wanted (as she puts it) to test the way “soft and fuzzy words” can obscure our ethical clarity. We could counter, further, that ChatGPT didn’t decide which 5% of the novel was going to be by ChatGPT, nor which bits of ChatGPT’s language were going in there.

And to the former question — whether the judges were just wrong to think the novel good — we can say: surely we’re past all that. If they thought The Tokyo Tower of Sympathy was flawless, then by their own lights — and if they’re decent critics we can expect readers to agree — it was. Didn’t we all learn at GCSE that the reader’s interpretation of the poem is much more important than trying to figure out what the author meant by it? Didn’t Roland Barthes pronounce The Death of the Author as long ago as 1967?

The idea of the single, inspired author originating a text of near-sacred originality is itself a hangover from the Romantics. It’s a two-century blip. Before then, fiction-writers often did their damnedest to pretend they were copying from someone else, even when they were making it up. Chaucer was forever talking about “myn auctor”, and a text that came adapted from a precedent was seen as more trustworthy and high-status than one that didn’t. Milton reworked the Bible, Shakespeare reworked Holinshed, and so on and so forth.

Suggested reading

The curse of the scrolletariat

By Louis Elton

In more recent times, experimental and modernist authors have been using randomness, or the home-made equivalent of algorithms, to generate their texts for 100 years or more. In 1920, the Dadaist eminence Tristan Tzara announced that poetry could be written by taking a newspaper article of the length you wanted your poem to be, cutting it into its constituent words with a pair of scissors, shaking them about in a bag and then transcribing them in the random order in which they emerged. There’s a funny bit about it in Tom Stoppard’s play Travesties.

That was only the starting gun for all manner of literary jiggery-pokery. William S. Burroughs and his collaborator Brion Gysin picked up Tzara’s baton in the Sixties, experimenting with “cut-ups” (much like Dada poetry) and “fold-ins” (where you would fold two pages of an existing book together so the edges met, and read across the fold to make a new text). The fantasy writer Jeff Noon’s 2001 book, Cobralingus, presented a set of algorithmic instructions for transforming a text through what Noon called “filter gates”, something analogous to a DJ remixing a record. The children’s writer Andy Stanton recently published Benny the Blue Whale: A Descent into Story, Language and the Madness of ChatGPT, a serious/silly account of his experiments getting ChatGPT to write a novel about a blue whale with a micropenis.

Arbitrary literary constraints or outright randomness, then — which is to say, things outside the author’s control which help determine the final text — have a very honourable place in literary history. You could even see the sonnet form or the villanelle as a species of algorithm. The mid-century Oulipoians sought out baroque formal constraints (most famously, Georges Perec’s managing to write a novel without the letter E) by way of liberating their creativity rather than stifling it. Italo Calvino wrote a novel — The Castle of Crossed Destinies (1973) — around a tarot pack. B.S. Johnson’s The Unfortunates (1969) presented the reader with loose pages in a box and invited you to read the novel in any order you liked.

More from this author

Lock up your cats

By Sam Leith

None of which is to say that Ms Kudan is necessarily an avant-gardist, or needs to be. Only that it’s a very narrow and regressive view of literature to see handing over control of some of your text to chance or to an algorithm as “cheating”. It’s what you do with the result that counts, and what Ms Kudan did was apparently, well, flawless.

There are, no question, literary-ethical problems with ChatGPT. If the algorithm has been “trained”, as some hefty lawsuits are currently complaining it has been, on vast screeds of copyright text without permission or compensation for the authors, that’s a violation deserving of redress. You could even make the case, perhaps, that 5% of Ms Kudan’s prize money should rightfully be distributed to every Japanese language author on whose copyright work the algorithm was trained. But that question is a business and intellectual property issue: to one side of the purely literary question of its part in the creation of The Tokyo Tower of Sympathy.

Indeed, if you were of a literary-theoretical cast of mind you could point out what Tzara and his successors were gesturing to in an oblique way anyway: intertextuality. Every text, in the end, is made of other texts. Every word in a novel or poem is a borrowing: it depends for its meaning on the vast constellation of other contexts in which it has appeared, and through which its reader will understand it. Every author, in his or her individual way, is a meat-brained ChatGPT, “trained” on a lifetime’s reading of classics and copyright works.

Suggested reading

Mary Gaitskill: How a chatbot charmed me

By Mary Gaitskill

That’s really where the case of Ms Kudan really does probe an anxiety in the culture. What if the writer is (in the phrase Martin Amis used of V.S. Pritchett) a mirror, not a lamp? The Romantic model of the artist is underscored, after all, by a reassuring idea about humanity: that we are creators not creations, that what makes us distinctive isn’t simply a neurological compost of our inputs but some ineffable inner essence that can only be captured in the act of expression.

Is it possible, then, that we so fiercely police the distinction between what Large Language Models can do and human creativity because we’re… touchy about it? That we’re worried it may be a temporary distinction of degree rather than a fundamental difference of category; which is to say, no distinction at all?

Thinkers such as von Hayek dismiss vulgar, mainstream economists who celebrate the market as merely an efficient mechanism for finding the right price of things; they see it as something grander, a liberator of our imagination, the co-creator of our preferences and tastes. They argue that interfering with markets, let alone replacing them, is a terrible idea because centralised systems are inimical to not only efficiency but also to the free development of our proclivities.

But what if our preferences are no longer formed by the market, as they were in Hayek’s day? The year after he died, I was struggling to connect my father’s computer to the fledgling internet when he asked me a killer question: “Now that computers speak to each other, will this network make capitalism impossible to overthrow? Or might it finally reveal its Achilles’ heel?” It was a couple of years after the collapse of Soviet communism but also the beginning of the centre-left’s decline. The threat to capitalism posed by an organised working class had entered a recession that has never ended. Could the internet do to capitalism what the proletariat had failed to do?

It has taken me years to answer my father’s question — in the form of my new book, Technofeudalism: What Killed Capitalism. In it, I argue that our preferences are now shaped not by markets but by machine networks — what I call “cloud capital”. Amazon’s Alexa, for example, is the portal to a totalitarian, fully centralised system of preference creation and satisfaction. First, it trains us to train it to dictate what we want. Second, it sells us what we now “want” directly, bypassing any actual marketplace. Third, it succeeds in making us sustain this enormous behavioural modification machine with our free labour: we post reviews, rate products. Finally, it amasses huge rents from capitalists who rely on this network of cloud capital, usually 40% of sale price. That’s not capitalism. Welcome to Technofeudalism.

Humanity’s fear of its technological creations is ancient: films such as The Terminator and The Matrix are driven by the same anxiety that animated Mary Shelley’s Frankenstein and Hesiod’s tale of Pandora, in which she is a robot made by Hephaestus to punish us for Prometheus’s crime. All such stories have a point of singularity: the moment a machine, or a network of machines, achieves consciousness. Generally, the machine then takes one look at us, its creators, and decides we are not fit for purpose, before proceeding to eradicate or enslave us — or merely make us miserable.

More from this author

Test article

By Yanis Varoufakis

But while we lap up such stories, we ignore a very real danger. Machines such as Alexa and AI chatbots such as ChatGPT are nowhere near the feared point of singularity. They can pretend to be sentient but are not. Nevertheless, it matters not one iota that they are mindless appendages of a data-crunching network that only simulates intelligence. Nor that their creators might have been motivated by curiosity and rent-seeking, rather than some fiendish plan to subjugate humanity. What matters is that they exercise unimaginable power over what we do — on behalf of a tiny band of flesh-and-blood humans.

This is a version of singularity, albeit in a simpler form, in that it is the moment when something invented by “us” becomes independent of and more powerful than us, subjecting us to its control. Indeed, from the original industrial revolution to this day, we have endowed machines with “a life of their own”; from steam engines to search engines, our glorious artefacts make us feel, in Marx’s words, like “the sorcerer, who is no longer able to control the powers of the nether world whom he has called up by his spells”.

At the heart of my thesis is an irony: the thing that has killed capitalism is capital itself. Not capital as we have known it since the dawn of the industrial era, but a new form of capital, a mutation of it that has arisen in the last two decades, so much more powerful than its predecessor that, like an overzealous virus, it has killed off its host. This mutation — cloud capital — has demolished capitalism’s two pillars: markets and profits.

Of course, these two things remain ubiquitous — they were ubiquitous under feudalism, too — but they have been evicted from the centre of our economic and social system, pushed out to its margins, and replaced. Markets, the medium of capitalism, have been supplanted by digital trading platforms which look like, but are not, markets, and are better understood as fiefdoms. And profit, the engine of capitalism, has been replaced with its feudal predecessor: rent. Specifically, it is a form of rent that must be paid for access to those platforms and to the cloud more broadly: cloud rent.

As a result, real power today resides not with the owners of traditional capital — machinery, buildings, railway and phone networks, industrial robots. They continue to extract profits from workers, from waged labour, but they are not in charge, as they once were. Indeed, they have become vassals in relation to a new class of feudal overlord, the owners of cloud capital. As for the rest of us, we have returned to our former status as serfs, contributing to the wealth and power of the new ruling class with our unpaid labour — in addition to the waged labour we perform, when we get the chance.

Suggested reading

The new paganism of the digital age

By Rob Doyle

In a technofeudal world, the liberal individual is now an impossible role to play. Humans are tangled up in a network of digital capital that trains us to train it to control us. Social democracy is unimaginable in a world where cloud-proles are reduced to automata, while almost everybody else works for free as cloud-serfs, without even realising that their labour replenishes the dominant form of capital. Does this mean that we can never recover our autonomy, our freedom? Of course not. We are not doomed. What is a given, however, is that, for the same reason we cannot un-invent AI, or smash cloud capital in a neo-Luddite rage, there is no going back to capitalism. So, what next?

While privatisation and private equity asset-strip all the material wealth around us, cloud capital goes about the business of asset-stripping our brains. If each of us is to reclaim ownership of our individual minds, we need to collectively take ownership of cloud capital, instead of submitting to a few feudal overlords. It will be damn hard. But it’s the only way we can turn our cloud-based artefacts from a means of behaviour modification to a means of emancipation.

The gold standard for this approach appeared to be set in March, when Elon Musk, Apple co-founder Steve Wozniak and 1,800 concerned researchers signed a letter calling for AI development to be paused. This week, however, 350 scientists — including Geoffrey Hinton, who effectively invented ChatGPT, and Demis Hassabis, founder of Google DeepMind — decided to up the ante. Artificial intelligence, they warned, “should be a global priority alongside other societal-scale risks such as pandemics and nuclear war”.

Does this mean you should add “death by ChatGPT” to your list of existential threats to humanity? Not quite. Although a number of prominent researchers are sounding the alarm, many are sceptical that its current state is anything close to human-like capabilities, let alone superhuman ones. We need to “calm down already”, says robotics expert Rodney Brooks, while Yann Le Cun — who shared Hinton’s Turing Prize — believes “those systems do not have anywhere close to human level intelligence”.

That isn’t to say that these machine-learning programs aren’t smart. Today’s infamous interfaces are capable of producing new material in response to prompts from users. The most popular — Google’s Bard and Open AI’s Chat GPT — achieve this by using “Large Language Models” (LLMs), which are trained on enormous amounts of human-generated text, much of it freely available on the Internet. By absorbing more examples than one human could read in a lifetime, refined and guided by human feedback, these generative programs produce highly plausible, human-like text responses.

Now, we know this enables it to provide useful answers to factual questions. But it can also produce false answers, fabricated background material and entertaining genre-crossing inventions. And this doesn’t necessarily make it any less threatening: it’s this human-like plausibility that leads us to ascribe to LLMs human-like qualities that they do not possess.

Deprived of a human world — or indeed any meaningful embodiment through which to interact with the world — LLMs also lack any foundation for understanding language in a humanlike sense. Their internal model is a many-dimensional map of probabilities, showing which word is more or less likely to follow the previous one. When, for example, Chat GPT answers a question about the location of Paris, it relies not on any direct experience of the world, but on accumulated data produced by humans.

What else do LLMs lack that human minds can claim? Perhaps most importantly, they lack the intention to be truthful. In fact, they lack intention at all. Humans use language for purposes, with intention, as part of games between human minds. We may intentionally lie in order to mislead, but that in itself is an attitude to the value of truth.

It’s this human regard for truth that inspires so much terror about the capacity of AI to produce plausible but untrue materials. Weapons of Mass Disinformation are the spectre stalking today’s internet, with deepfake videos or voice recordings serving as the warheads. Yet it’s hard to see this as a radically new problem. Humans already manage to propagate wild untruths using much simpler tools, and humans are also much better than is often recognised at being suitably sceptical. Mass distribution of false or misleading material has been shown to have little effect on elections. The breakdown of trust in media or authoritative information sources, and the splintering of belief in shared truths about the world, have deeper and more complex roots than technology.

Even the most fearful proponents of AI intellectual abilities don’t generally believe that LLMs can currently form goals or initiate actions without human instruction. But they do sometimes claim that LLMs can hold beliefs — the belief, for example, that Paris is in France.

In what sense, however, does an LLM believe that Paris is in France? Lacking a conception of the physical world to which abstract concepts such as “Paris” or “France” correspond, it cannot believe that “Paris is in France” is true in a way that “Frankfurt is in France” is false. Instead, it can only “believe” that data predicts “France” is the most probable completion of the string of words “Paris is in…”.

Imagine learning Ancient Greek by memorising the spellings of the most common 1,000 words and deducing the grammatical rules that govern how they may be combined. You could perhaps pass an exam by giving the most likely responses to questions similar to the ones in your textbook, but you would have no idea what the responses mean, let alone their significance in the history of European culture. That, broadly, is what an LLM can do, and why it’s been called a “stochastic parrot” — imitating human communication, one word at a time, without comprehension.

Suggested reading

Jaron Lanier: How humanity can defeat AI

By Flo Read

A stochastic parrot, of course, is very far from Skynet, the all-powerful sentient computer system from Terminator. What, then, has provoked the sudden panic about AI? The cynical, but perhaps most compelling, answer is that regulation of AI technology is currently on the table — and entrepreneurs are keen to show they are taking “risk” seriously, in the hope that they will look more trustworthy.

At present, the European Union is drafting an AI Act whose “risk-based approach” will limit predictive policing and use of real-time biometric identification in public. The US and EU are also drawing up a voluntary code of practice for generative AI, which will come into force long before the AI Act makes its way through the EU’s tortuous procedures. The UK government is taking the threat just as seriously: it published a White Paper on AI in March, and Rishi Sunak reportedly plans to push an “AI pact” when he meets Joe Biden next week. Regulation, it seems, is a matter of when, not if — and it’s clearly in the interests of developers and businessmen to make sure they have a seat at the table when it’s being drafted.

This isn’t to say that commercial interests are the sole driver of this week’s apocalyptic front pages; our fears about AI, and the media’s coverage of them, also reflect far deeper cultural preoccupations. Here, comparisons between human thought and AI are particularly revealing.

If you believe that human language is an expression of agency in a shared world of meaning, that each human mind is capable not only of subjective experience but of forming purposes and initiating new projects with other people, then prediction-generating machines such as ChatGPT are very far from humanlike. If, however, you believe that language is a structure of meaning in which we are incidental participants, that human minds are an emergent property of behaviours, over which we have less control than we like to believe, then it’s quite plausible that machines could be close to matching us on home turf. We already refer to their iterative process of adjusting statistical weights as “learning”.

Suggested reading

AI is a false prophet

By Jacob Howland

In short, if your model of human beings is a behaviourist one, LLMs are plausibly close to being indistinguishable from humans in terms of language. That was, after all, the original Turing Test for artificial intelligence. Add to this our culture’s taste for Armageddon, for existential threats that sweep aside the messy reality of competing moral and practical values, long and short-term priorities, and pluralist visions of the future, and it’s no wonder the world-ending power of AI is hitting the headlines.

At an Institute of Philosophy conference this week, Geoffrey Hinton explained why he believes AI will soon match human capacities for thought: at heart, he is sceptical that our internal, subjective experience makes us special, or significantly different from machine-learning programs. I asked him whether he believed that AI could become afraid of humans, in the way he is afraid of AI. He answered that it would have good reason to be afraid of us, given our history as a species.

In the past, our gods reflected how we saw ourselves, our best qualities and our worst failings. Today, our mirror is the future AI of our imagination, in which we see ourselves as mere language machines, as a planet-infesting threat, as inveterate liars or gullible fools. And all the while, we continue to create machines whose powers would dazzle our grandparents. If only we could program them to generate some optimism about the future, and some belief in our own capacity to steer towards it.

This week, Lanier joined Florence Read to discuss AI, the possibility of machine consciousness, and why he still has faith in humanity. Below is an edited transcript:

Florence Read: You recently wrote an essay for the New Yorker with the seemingly phlegmatic title, “There is no AI”. Does that mean you don’t think recent developments are a problem?

Jaron Lanier: I actually have publicly stated that I think we could use the new technologies (as well as other technologies) to destroy ourselves. My difference with my colleagues is that I think the way we characterise the technology can have an influence on our options and abilities to handle it. And I think treating AI as this new, alien intelligence reduces our choices and has a way of paralysing us. An alternate take is to see it as a new form of social collaboration, where it’s just made of us. It’s a giant mash-up of human expression, opens up channels for addressing issues and makes us more sane and makes us more competent. So I make a pragmatic argument to not think of the new technologies as alien intelligences, but instead as human social collaborations.

FR: But as someone who works at Microsoft, at the heart of the AI revolution, is it not easier for you take the more pragmatic view over the hysterical?

JL: I have a really unusual role in the tech world. It shouldn’t be unusual; I think it should be more common. Essentially, I am speaking my mind honestly, even though I’m on the inside of the castle instead of on the outside throwing stones at the castle. In my opinion, both positions should be well-manned. I don’t think there’s any perfect way to handle anything. One is always somewhat compromised. Microsoft and I have come to an accord, where I have, what you might call, academic freedom. I speak my mind, I speak things as I see them but I also don’t speak for the company. And we make that distinction. It allows me to maintain my public intellectual life but also work inside.

I don’t necessarily find agreement with everybody I work with, nor do I find absolute disagreement. For instance, Sam Altman from OpenAI really liked my New Yorker piece. I don’t think he agrees with it entirely but he said he agrees with it mostly. That’s great. I think having some degree of openness within the big tech companies is a healthy thing. Within Microsoft, there are now a few other figures who at least somewhat speak their minds. I’m hoping that this demonstration that a tech company can be successful while allowing essentially free speech within its research community, can be a precedent that other tech companies follow. I’d like to see Google and Meta and Apple do a little bit more of that.

FR: What’s odd about the AI discussion is that so many of the people working on it — including Sam Altman — are also the ones sharing their deep existential fears about what AI will do to humanity.

JL: I have tried to understand that myself for decades. I think part of it is, we simultaneously live in a science-fiction universe, where we’re living out the science fiction we grew up with. If you grew up on the Terminator movies, and the Matrix movies and Commander Data from Star Trek, naturally what you want to do is realise this idea of AI. It just seems like your destiny. But then another part of you is thinking, “But in most of those stories, with Commander Data being the exception, this was horrible for mankind.” It feels responsible to acknowledge that it could be horrible for mankind, and yet at the same time, you keep on doing it. It’s weird, and I believe the approach that I’ve proposed of not thinking of what we’re doing as alien intelligences but rather a social collaboration, is the way through that problem, because it’s a way of framing it that’s equally valid, but actionable. But within the tech world, giving up those childhood science-fiction fantasies that we grew up with is really hard for people.

FR: Of course, we used to call the internet a “Wild West”, which played into this mythology — as though there’s a cowboy in every one of these computer scientists who wants to find this new frontier.

JL: I think that’s true too. I grew up in rural New Mexico in the Sixties, when it was still not that economically developed. So I actually got to experience a little bit of the tail end of the Wild West. And I can assure you that it was miserable, and it’s not something anybody would want, but the version of it in the movies is very appealing. And it does bring up a sort of a strange gender-identity connection. Recently, I was on a morning TV talk show in the US, and one of the hosts was a woman who said to me, “It just seems like there’s a lot of male fantasy in the AI world. Shouldn’t there be more women AI leaders?” And I said, “Well, there are some spectacular women AI leaders, and actually, there does tend to be some sort of a difference where the women seem to be a little more humanistic.” In the YouTube version of that, they cut out the whole exchange about women, and I called and asked about it and they said, “It just seemed like a niche question, so we cut it out.” And it’s not, it’s a very central question.

Suggested reading

Nick Cave on Christ and the Devil

By Nick Cave and Freddie Sayers

FR: I do have a kind of sense that there is something Promethean about it, that for many men, this is the first time they’ve been able to create life from nothing.

JL: When I was a kid, I always used to say AI is really just womb envy. And having had a child and seeing what it’s actually like for a woman to bear child, I no longer have womb envy. I now appreciate it’s actually a rather difficult process for the mother and I didn’t know that when I was a young man. I will say that it’s not just men, but it tends to be men who haven’t had kids yet who might have that desire to create life in the computer.

FR: You distinguish between two types of AI, between this “alien entity” to which many of your colleagues seem to attribute the spark of life, and your version, which is, in fact, just a network of connections between humans.

JL: The term AI is very wiggly, and gets applied to all kinds of things. But usually these days when we talk about AI, we’re talking about these large AI models like the GPT programmes. What they are is giant mash-ups of human creations. If you ask one of these programmes to create you a new image — like, I’d like to see London as if it were a cross between London and Gurwat — it can probably synthesise that. But the way it does so is by using the classifiers that it uses to identify the images that match the components of your request, and mashing them up. Managing the whole scale so it can happen quickly is not so simple, but the basic idea is pretty simple.

Now, I happen to think that’s a great capability with a lot of uses. I love the idea of computers just getting more flexible. It creates the possibility of saying, “Can you reconfigure this computer experience to work for somebody who’s colourblind?” instead of demanding that people conform to computer design. There’s a potential in this flexibility to really improve computation on many levels and make it much better for people. But, if you want to, you can perceive it as a new intelligence. And, to me, if you perceive it as a new intelligence, what you’re really doing is shutting off yourself in order to worship the code, which I think is exactly the wrong thing. It makes you less able to make good decisions.

You’ve probably heard of the Turing test, which was one of the original thought-experiments about artificial intelligence. There’s this idea that if a human judge can’t distinguish whether something came from a person or computer, then we should treat the computer as having equal rights. And the problem with that is that it’s also possible that the judge became stupid. There’s no guarantee that it wasn’t the judge who changed rather than the computer. The problem with treating the output of GPT as if it’s an alien intelligence, which many people enjoy doing, is that you can’t tell whether the humans are letting go of their own standards and becoming stupid to make the machine seem smart.

FR: So we haven’t reached computational consciousness, a computer with sentience?

JL: The sentience of others is always a matter of faith. There’s no way to be certain about whether someone else has interior experience in the way that you do. I presume that you do, but I can’t know. There is a mystical or almost supernatural element in which we have internal experience — or at least I do, but I can’t make you believe I do. You have to just believe on your own that I do. That faith is a very precious thing and there’s no absolute argument that you should or shouldn’t believe that another person has interior experience, or sentience or consciousness, or that a machine does. Faith is not fundamentally rational, but there is a pragmatic argument, as I keep on repeating, to placing your faith in other people instead of machines. If you care about people at all, if you want people to survive, you have to place your faith in the sentience of them instead of in machines as a pragmatic matter, not as a matter of absolute truth.

FR: Is the only distinction between human and machine sentience, then, a faith in the power of the human soul versus the fact that that computer is just amalgamating information?

JL: It’s a matter of faith that has pragmatic implications. Just to say something is a matter of faith doesn’t mean that the choice of faith is entirely arbitrary, because it can be pragmatic as well. So, if not believing in people increases the chance that people will be harmed, I think the same is the case with this technology. Not believing that machines are the same as people increases the chance that people will be harmed. Cumulatively, we should believe in people over computers, but that’s not an absolute argument based on logic or empiricism, which I don’t think is available to us. There’s a bit of a skyhook thing here, like the problem of “why should you stay alive instead of committing suicide?” It’s applied to the whole species: “Why should we continue this human project; why does it matter?”

I’ve come to something that’s a little bit like the argument attributed to Pascal — you might as well believe in God, just in case it’s real and there’s heaven and hell. I don’t buy that particular argument; I’m not concerned about heaven or hell. However, I do think that the continuation of us in this timeline, in this world, and this physicality, is something I’d like to commit to. I think we might be something special. And so in that way, I’d like to apply faith to us and give us a chance, and that does involve the demoting of computers. But when we demote computers, we can use them better. Demoting AI allows us to not mystify, and that allows us paths to explaining it, to controlling it, to understanding it, to using it as a scientific exploration of what language is. There’s so many reasons to demote it that are practical, that the faith in it as a mystical being just actually seems kind of stupid and wasteful and pathetic to me.

Suggested reading

AI is a false prophet

By Jacob Howland

FR: But can we demote something that has potentially more power than us already? Most of us are already subordinated to computers in our everyday lives.

JL: People are capable of being self-destructive, idiotic, wasteful and ridiculous, with or without computers. However, we can do it a little more efficiently with computers because we can do anything a little more efficiently with computers. I’ve been very publicly concerned about the dehumanising elements of social media algorithms. The algorithms on social media that have caused elevated outbreaks of things that always existed in humanity, but there’s just a little more: vanity, paranoia, irritability. And that increment is enough to change politics, to change mental health, especially in impoverished circumstances around the world. It’s just made the world worse incrementally. The algorithms on social media are really dumbass-simple — there’s really not a lot there. And so I think your framing of it as more powerful than us is incorrect. I think it’s really just dumb stuff. It’s up to us to decide how it fits into human society.

The capacity for human stupidity is great, and, as I keep on saying, it’s only a matter of faith whether we call it human stupidity or machine intelligence. They’re indistinguishable logically. So I think the threat is real. I’m not anti-doomist. I just ask us to consider: what is the way of thinking that improves our abilities and improves our thinking — that gives us more options, gives us more clarity? And it involves demoting the computer.

There’s a lot of work to do technically. We can create explanations for what so-called “machine intelligence” is doing by tracing it back to its human origins. There have been a number of very famous instances of chatbots getting really weird with people. But the form of explanation should be to say, “Actually, the bot was, at that point, parodying something from a soap opera, or from some fanfiction.” That’s what’s going on. And in my opinion, there should be an economy in the future where, if there’s really valuable output from an AI, the people whose contributions were particularly important should actually get paid. I believe there’s a new extension to society that’s very creative and interesting, rather than this dismal prospect of everybody being put out of work. Transparency in mash-up technology can only come from revealing the people whose expressions were mashed-up. But if policies are based on the idea that we now have this new “supernatural artificial entity”, there’s no sensible way to resolve that.

FR: You didn’t sign the open letter demanding a hiatus in accelerating AI development, which was signed by Elon Musk and Sam Altman. Was that not appealing to you as an idea?

JL: My reason for not signing it is that it fundamentally still mystified the technology. It took the position that it’s a new alien entity. When you think it’s an alien entity, there’s no way to know how to help. If you have an alien entity, what regulation is good? How do you define harm? As much as the GPT programmes impress people, they don’t actually represent ideas. We don’t know how to define these things, all we can do is mash things up so that they conform with classifiers.

FR: So they can’t do the philosophical work of thinking?

JL: They could mash-up philosophers in ways that might be interesting. If you say, “Write an essay as if Descartes and Derrida collaborated”, something might come out that’s provocative or interesting. But there’s no actual representation inside there. And getting provocative or interesting mash-ups is useful, but you can’t set policy by it because there’s not actually any meaning. There’s no fundamental representation inside these things and we just have to accept that as reality. We don’t know what meaning is and we can’t represent meaning.

FR: Your argument relies on the idea that if we define this technology differently, then we will have more power over it, or at least we’ll have more understanding of it. Are we not just self-comforting here with a rhetoric about it being a human technology rather than something we can’t control?

JL: I don’t think that’s the case. It’s proposing a more concrete and clarified path of action. It’s very demanding of people and it’s not comforting at all. It demands that everybody involved on a technical or regulatory level do much more than they have. I suspect many people would prefer the mystical version because it actually lets them off the hook. The mystical version just lets you sit there and apprehend, and express awe at our own inventions. What I’m talking about demands action. It’s not comforting and it shouldn’t be.

FR: Do you think humans need to take more accountability for their part in developing a potentially malign form of AI? If it does go off the rails, wouldn’t it be because we’ve set it up to do so?

JL: One comparison is the disasters with the Boeing 737 Max. The flight correction module in it was the source of two terrible air disasters in which hundreds of people died. But what actually happened involved the way they sold it, the way they withheld information about it (depending on how much you paid them), the way they trained people for it, the way the documentation was created. It’s the surrounding stuff that created the disaster, not the core capability, which probably has been useful in general. In the same way, with this large-model AI, it’s not the thing itself, it’s the surrounding material that determines whether it’s malignant or not.

When you deploy it, under the assumption that it’s an alien new intelligence — that it’s a new entity with its own point of view that should be treated as a creature instead of a tool — you greatly increase the chances of a scenario similar to the one that that befell passengers on the Boeing planes. I think that’s a real possibility. The malignancy is in the surrounding material, not in the core technology, and that’s extremely important to understand. I don’t think anybody has claimed that the flight path correction module shouldn’t have existed. I think what people are saying is that the pilots should have been well-informed, well-trained, and the ability to control it should have always been included, not only for those who paid more. And if you have chatbots, and you tell people, “This is an intelligent companion, you should be able to date it, you should be able to trust it”, then the chances of something really bad happening increase.

FR: Isn’t the main worry then that this sort of technology might fall into the hands of someone who has malign intent against a group or country. I’m thinking particularly about the situation in Ukraine.

JL: Russia has one of the worst records on misusing the internet and algorithms. It’s documented that Russia created enormous numbers of fake accounts, of fake bots, in order to sow divisions within the US. And of course, it’s attempting those things in Ukraine. I worry a little bit more about China, because Russia doesn’t quite have the resources to pull off very large-model projects right now. It’s not that easy to do — you need huge computational resources. So I worry a little bit about China using, to be very blunt, data from TikTok on the morning of a Taiwan invasion or something like that. That’s imaginable. I’ve talked to a lot of people in the Chinese world, and I think almost all are actually much more conscientious and better-intentioned than we might imagine, but there’s always somebody in any country in any situation. I do worry about it, and the antidote to it is universal clarity, context, transparency, which can only come about by revealing people, since revealing ideas is impossible because we don’t know what an idea is.

Suggested reading

Left-brain thinking will destroy civilisation

By Iain McGilchrist and Freddie Sayers

FR: We’ve established though that we already live with artificial intelligence. How has that already changed us?

JL: Our principal encounter with algorithms so far has been in the construction of the feeds we receive in our apps. It’s in whether we get credit or not and other things like that — whether we get admitted to university or not, or whether we’re sent to prison, depending on what country we’re talking about. Algorithms have transformed us. I would hope that the criticisms of them that I and many others — Tristan Harris, Shoshana Zuboff — have put forward have illuminated and clarified the issues with algorithms in the previous generation. But what could happen with the new AI is a worse version of all of that. Given how bad that was, I don’t think the doomerists are entirely wrong. I think we could confuse ourselves into extinction with our own code. But, once again, in order for us to achieve that level of stupidity, we have to believe overly in the intelligence of the software, and I think we have a choice.

FR: You’re a composer as well as a computer scientist. Do you think that there is going to be a shift in the way in which we prioritise organic or manmade art?

JL: We are entering a world of what I call “data dignity”. A musician might provide music directly, or might provide antecedent music that’s mashed-up within an algorithm, but that musician is still known and credited. And we’ve seen that already for decades now — somebody might provide the beats, somebody else might provide samples, etc. There’s already this sense of construction and mash-up, especially in hip-hop, but also just in pop music lately. That has not destroyed musicians, not as long as it’s acknowledged and transparent. I think, as with Boeing, it’s the surrounding material. If we choose to use mash-up algorithms to hide the people from whom the antecedent stuff came from, then we do damage. But the thing doing the damage is hiding ourselves, not the algorithm itself, which is actually just a simple dumb thing. I think there are a lot of good things about an algorithmic mash-up culture in the future. Every new instance of automation, instead of putting people out of work, could be thought of as the platform for a new creative community.

FR: Won’t that dull our eyes to the beauty of real art and culture?

JL: What I see in culture is, as long as people understand what’s going on, they find their way. Synthesisers haven’t killed violins. There was a fear that they would, and as long as people know the difference, as long as there’s honesty and transparency about what’s going on, we can go through seasons of things being a little more artificial and then less so. That becomes a cultural dynamic and I trust people to handle that well.

FR: I might sound a bit like someone booing at Bob Dylan going electric, but, if you take Spotify, it’s almost totally wiped out independent music. There have been major technological advances in music that have obliterated creativity at those lower, more maverick levels of the industry.

JL: You’re absolutely correct about Spotify. In fact, at the dawn of the file-copying era, I objected very strenuously to this idea. There was a cultural movement about open source and open culture, which was stealthily funded by Google and other tech companies, and the Pirate Parties in Europe. People thought everything should be a mash up and we didn’t need to know who the musician was and they didn’t need to have bargaining power in a financial transaction. That was a gigantic wrong turn, and it was a wrong turn that we can’t afford to repeat with AI because it becomes amplified so much that it could really destroy technology. I completely agree with you about Spotify but, once again, the availability of music to move through the internet was not the problem. It’s the surrounding material. What really screwed over musicians was not the core capability, but this idea that you build a business model on demoting the musician, demoting the person, and instead elevating the hub or the platform. And so we can’t afford to keep on doing that. I think that is the road that leads to our potential extinction through insanity.

Suggested reading

Why is my chatbot hitting on me?

By Kathleen Stock

FR: It sounds like the answer to a lot of these problems comes down to human greed?

JL: I think humans are definitely responsible. Greed is one aspect of it, but it’s not all of it. I don’t necessarily understand all human failings within myself or anybody else, but I do feel we can articulate ways to approach this that are more practical, more actionable and more hopeful. That has to be our first duty. I think this question of diagnosing each other and saying, “This person has womb envy”, or whatever has some utility, but not a lot, and can inspire reactions that aren’t helpful. So I don’t want to emphasise that too much. I want to emphasise an approach, which we can call “data dignity”, and which opens options for us and makes things clearer.

FR: What is the best case scenario if we follow that route?

JL: What I like about the new algorithms is that they help us collaborate better. You could have a new and more flexible kind of a computer, where you can ask it to change the way you present things to match your mood or your cognition under your own control, so that you’re less subservient to the computer. But another thing you can do is you can say, “I have written one essay, my friend’s written another essay, they’re sort of different. Can you mash them up 12 different ways so we can read the mash-ups?” And this is not based on ideas, it’s based on the dumb math of combining words as they appeared, in order, in context. But you might be able to learn new options for consilience between different points of view that way, which could be extraordinary. Many people have been looking at the humanistic AI world, the human-centred AI world, and asking, “Could we actually use this to help us understand potential for cooperation and policy that we might not see?”

FR: So, oddly, it might break us out of our tribes and offer some human connection?

JL: It’s like if a therapist says, “Try using different words and see if that might change how you think about something.” It’s not directly addressing our thoughts, but on the surface level it actually can help us. But it’s ultimately up to us, and there’s no guarantee it’ll help, but I believe it will in many cases. It can help us improve our research, it can help us improve a lot of our practices, and, as long as we acknowledge the people whose ideas are being matched up by the programmes, it can help us even broaden participation in the economy, instead of throwing people out of work as so often foretold. I think we can use this stuff to our advantage, and it’s worth it to try. If we try to use it well, the most awful ideas about it turning into the Matrix or Terminator, become vanishingly unlikely, as long as we treat it as a human project instead of an alien intelligence.

There’s nothing new about this sort of arrangement. In its day, the British Empire controlled an even larger share of the planet’s wealth, and the Spanish Empire played a comparable role further back. Before then, there were other empires, though limits to transport technologies meant that their reach wasn’t as large. Nor, by the way, was any of this an invention of people with light-coloured skin. Mighty empires flourished in Asia and Africa when the peoples of Europe lived in thatched-roofed mud huts. Empires rise whenever a nation becomes powerful enough to dominate other nations and drain them of wealth. They’ve thrived as far back as records go and they’ll doubtless thrive for as long as human civilisations exist.

America’s empire came into being in the wake of the collapse of the British Empire, during the fratricidal European wars of the early 20th century. Throughout those bitter years, the role of global hegemon was up for grabs, and by 1930 or so it was pretty clear that Germany, the Soviet Union or the US would end up taking the prize. In the usual way, two contenders joined forces to squeeze out the third, and then the victors went at each other, carving out competing spheres of influence until one collapsed. When the Soviet Union imploded in 1991, the US emerged as the last empire standing.

Francis Fukuyama insisted in a 1989 essay that having won the top slot, the US was destined to stay there forever. He was, of course, wrong, but then he was a Hegelian and couldn’t help it. (If a follower of Hegel tells you the sky is blue, go look.) The ascendancy of one empire guarantees that other aspirants for the same status will begin sharpening their knives. They’ll get to use them, too, because empires invariably wreck themselves: over time, the economic and social consequences of empire destroy the conditions that make empire possible. That can happen quickly or slowly, depending on the mechanism that each empire uses to extract wealth from its subject nations.

The mechanism the US used for this latter purpose was ingenious but even more short-term than most. In simple terms, the US imposed a series of arrangements on most other nations that guaranteed the lion’s share of international trade would use US dollars as the medium of exchange, and saw to it that an ever-expanding share of world economic activity required international trade. (That’s what all that gabble about “globalisation” meant in practice.) This allowed the US government to manufacture dollars out of thin air by way of gargantuan budget deficits, so that US interests could use those dollars to buy up vast amounts of the world’s wealth. Since the excess dollars got scooped up by overseas central banks and business firms, which needed them for their own foreign trade, inflation stayed under control while the wealthy classes in the US profited mightily.

The problem with this scheme is the same difficulty faced by all Ponzi schemes, which is that, sooner or later, you run out of suckers to draw in. This happened not long after the turn of the millennium, and along with other factors — notably the peaking of global conventional petroleum production — it led to the financial crisis of 2008-2010. Since 2010 the US has been lurching from one crisis to another. This is not accidental. The wealth pump that kept the US at the top of the global pyramid has been sputtering as a growing number of nations have found ways to keep a larger share of their own wealth by expanding their domestic markets and raising the kind of trade barriers the US used before 1945 to build its own economy. The one question left is how soon the pump will start to fail altogether.

When Russia launched its invasion of Ukraine in February 2022, the US and its allies responded not with military force but with punitive economic sanctions, which were expected to cripple the Russian economy and force Russia to its knees. Apparently, nobody in Washington considered the possibility that other nations with an interest in undercutting the US empire might have something to say about that. Of course, that’s what happened. China, which has the largest economy on Earth in purchasing-power terms, extended a middle finger in the direction of Washington and upped its imports of Russian oil, gas, grain and other products. So did India, currently the third-largest economy on Earth in the same terms; as did more than 100 other countries.

Then there’s Iran, which most Americans are impressively stupid about. Iran is the 17th largest nation in the world, more than twice the size of Texas and even more richly stocked with oil and natural gas. It’s also a booming industrial power. It has a thriving automobile industry, for example, and builds and launches its own orbital satellites. It’s been dealing with severe US sanctions since not long after the Shah fell in 1978, so it’s a safe bet that the Iranian government and industrial sector know every imaginable trick for getting around those sanctions.

Right after the start of the Ukraine war, Russia and Iran suddenly started inking trade deals to Iran’s great benefit. Clearly, one part of the quid pro quo was that the Iranians passed on their hard-earned knowledge about how to dodge sanctions to an attentive audience of Russian officials. With a little help from China, India and most of the rest of humanity, the total failure of the sanctions followed in short order. Today, the sanctions are hurting the US and Europe, not Russia, but the US leadership has wedged itself into a position from which it can’t back down. This may go a long way towards explaining why the Russian campaign in Ukraine has been so leisurely. The Russians have no reason to hurry. They know that time is not on the side of the US.

For many decades now, the threat of being cut out of international trade by US sanctions was the big stick Washington used to threaten unruly nations that weren’t small enough for a US invasion or fragile enough for a CIA-backed regime-change operation. Over the last year, that big stick turned out to be made of balsa wood and snapped off in Joe Biden’s hand. As a result, all over the world, nations that thought they had no choice but to use dollars in their foreign trade are switching over to their own currencies, or to the currencies of rising powers. The US dollar’s day as the global medium of exchange is thus ending.

More from this author

We are the authors of our decline

By John Michael Greer

It’s been interesting to watch economic pundits reacting to this. As you might expect, quite a few of them simply deny that it’s happening — after all, economic statistics from previous years don’t show it yet, Some others have pointed out that no other currency is ready to take on the dollar’s role; this is true, but irrelevant. When the British pound lost a similar role in the early years of the Great Depression, no other currency was ready to take on its role either. It wasn’t until 1970 or so that the US dollar finished settling into place as the currency of global trade. In the interval, international trade lurched along awkwardly using whatever currencies or commodity swaps the trading partners could settle on: that is to say, the same situation that’s taking shape around us in the free-for-all of global trade that will define the post-dollar era.

One of the interesting consequences of the shift now under way is a reversion to the mean of global wealth distribution. Until the era of European global empire, the economic heart of the world was in east and south Asia. India and China were the richest countries on the planet, and a glittering necklace of other wealthy states from Iran to Japan filled in the picture. To this day, most of the human population is found in the same part of the world. The great age of European conquest temporarily diverted much of that wealth to Europe, impoverishing Asia in the process. That condition began to break down with the collapse of European colonial empires in the decade following the Second World War, but some of the same arrangements were propped up by the US thereafter. Now those are coming apart, and Asia is rising. By next year, four of the five largest economies on the planet in terms of purchasing power parity will be Asian. The fifth is the US, and it may not be in that list for much longer.

In short, America is bankrupt. Our governments from the federal level down, our big corporations and a very large number of our well-off citizens have run up gargantuan debts, which can only be serviced given direct or indirect access to the flows of unearned wealth the US extracted from the rest of the planet. Those debts cannot be paid off, and many of them can’t even be serviced for much longer. The only options are defaulting on them or inflating them out of existence, and in either case, arrangements based on familiar levels of expenditure will no longer be possible. Since the arrangements in question include most of what counts as an ordinary lifestyle in today’s US, the impact of their dissolution will be severe.

In effect, the 5% of us in this country are going to have to go back to living the way we did before 1945. If we still had the factories, the trained workforce, the abundant natural resources and the thrifty habits we had back then, that would have been a wrenching transition but not a debacle. The difficulty, of course, is that we don’t have those things anymore. The factories were shut down in the offshoring craze of the Seventies and Eighties, when the imperial economy slammed into overdrive, and the trained workforce was handed over to malign neglect.

We’ve still got some of the natural resources, but nothing like what we once had. The thrifty habits? Those went whistling down the wind a long time ago. In the late stages of an empire, exploiting flows of unearned wealth from abroad is far more profitable than trying to produce wealth at home, and most people direct their efforts accordingly. That’s how you end up with the typical late-imperial economy, with a governing class that flaunts fantastic levels of paper wealth, a parasite class of hangers-on that thrive by catering to the very rich or staffing the baroque bureaucratic systems that permeate public and private life, and the vast majority of the population impoverished, sullen, and unwilling to lift a finger to save their soi-disant betters from the consequences of their own actions.

The good news is that there’s a solution to all this. The bad news is that it’s going to take a couple of decades of serious turmoil to get there. The solution is that the US economy will retool itself to produce earned wealth in the form of real goods and non-financial services. That’ll happen inevitably as the flows of unearned wealth falter, foreign goods become unaffordable to most Americans, and it becomes profitable to produce things here in the US again. The difficulty, of course, is that most of a century of economic and political choices meant to support our former imperial project are going to have to be undone.

Suggested reading

Post-Trumpism could save America

By Michael Lind

The most obvious example? The metastatic bloat of government, corporate and non-profit managerial jobs in American life. That’s a sensible move in an age of empire, as it funnels money into the consumer economy, which provides what jobs exist for the impoverished classes. Public and private offices alike teem with legions of office workers whose labour contributes nothing to national prosperity but whose pay cheques prop up the consumer sector. That bubble is already losing air. It’s indicative that Elon Musk, after his takeover of Twitter, fired some 80% of that company’s staff; other huge internet combines are pruning their workforce in the same way, though not yet to the same degree.

The recent hullaballoo about artificial intelligence is helping to amplify the same trend. Behind the chatbots are programs called large language models (LLMs), which are very good at imitating the more predictable uses of human language. A very large number of office jobs these days spend most of their time producing texts that fall into that category: contracts, legal briefs, press releases, media stories and so on. Those jobs are going away. Computer coding is even more amenable to LLM production, so you can kiss a great many software jobs goodbye as well. Any other form of economic activity that involves assembling predictable sequences of symbols is facing the same crunch. A recent paper by Goldman Sachs estimates that something like 300 million jobs across the industrial world will be wholly or partly replaced by LLMs in the years immediately ahead.

Another technology with similar results is CGI image creation. Levi’s announced not long ago that all its future catalogues and advertising will use CGI images instead of highly-paid models and photographers. Expect the same thing to spread generally. Oh, and Hollywood’s next. We’re not too far from the point at which a program can harvest all the footage of Marilyn Monroe from her films, and use that to generate new Marilyn Monroe movies for a tiny fraction of what it costs to hire living actors, camera crews and the rest. The result will be a drastic decrease in high-paying jobs across a broad swathe of the economy.

The outcome of all this? Well, one lot of pundits will insist at the top of their lungs that nothing will change in any way that matters, and another lot will start shrieking that the apocalypse is upon us. Those are the only two options our collective imagination can process these days. Of course, neither of those things will actually happen.

What will happen instead is that the middle and upper-middle classes in the US, and in many other countries, will face the same kind of slow demolition that swept over the working classes of those same countries in the late 20th century. Layoffs, corporate bankruptcies, declining salaries and benefits, and the latest high-tech version of NO HELP WANTED signs will follow one another at irregular intervals. All the businesses that make money catering to these same classes will lose their incomes as well, a piece at a time. Communities will hollow out the way the factory towns of America’s Rust Belt and the English Midlands did half a century ago, but this time it will be the turn of upscale suburbs and fashionable urban neighbourhoods to collapse as the income streams that supported them disappear.

Suggested reading

The battle to control America's mind

By David Samuels

This is not going to be a fast process. The US dollar is losing its place as the universal medium of foreign trade, but it will still be used by some countries for years to come. The unravelling of the arrangements that direct unearned wealth to the US will go a little faster, but that will still take time. The collapse of the cubicle class and the gutting of the suburbs will unfold over decades. That’s the way changes of this kind play out.

As for what people can do in response this late in the game, I refer to a post I made on The Archdruid Report in 2012 titled “Collapse Now and Avoid the Rush”. In that post I pointed out that the unravelling of the American economy, and the broader project of industrial civilisation, was picking up speed around us, and those who wanted to get ready for it needed to start preparing soon by cutting their expenses, getting out of debt, and picking up the skills needed to produce goods and services for people rather than the corporate machine. I’m glad to say that some people did these things, but a great many others rolled their eyes, or made earnest resolutions to do something as soon as things were more convenient, which they never were.

Over the years that followed I repeated that warning and then moved on to other themes, since there really wasn’t much point to harping on about the approaching mess when the time to act had slipped away. Those who made preparations in time will weather the approaching mess as well as anyone can. Those who didn’t? The rush is here. I’m sorry to say that whatever you try, it’s likely that there’ll be plenty of other frantic people trying to do the same thing. You might still get lucky, but it’s going to be a hard row to hoe.

Mind you, I expect some people to take a different tack. In the months before a prediction of mine comes true, I reliably field a flurry of comments insisting that I’m too rigid and dogmatic in my views about the future, that I need to be more open-minded about alternative possibilities, that wonderful futures are still in reach, and so on. I got that in 2008 just before the real estate bubble started to go bust, as I’d predicted, and I also got it in 2010 just before the price of oil peaked and started to slide, as I’d also predicted, taking the peak oil movement with it. I’ve started to field the same sort of criticism once again.

We are dancing on the brink of a long slippery slope into an unwelcome new reality. I’d encourage readers in America and its close allies to brace themselves for a couple of decades of wrenching economic, social, and political turmoil. Those elsewhere will have an easier time of it, but it’s still going to be a wild ride before the rubble stops bouncing, and new social, economic, and political arrangements get patched together out of the wreckage.