The writer who coined the word ‘enshittification’ tells us why AI will never deliver what it promises – and why it still appeals so much to those in powerA “centaur”, in automation theory, is a person assisted by a machine, and a “reverse centaur”, hero of Cory Doctorow’s new book,
Zoe Williams
A “centaur”, in automation theory, is a person assisted by a machine, and a “reverse centaur”, hero of Cory Doctorow’s new book, The Reverse Centaur’s Guide to Life After AI, is a “human who is conscripted into acting as an assistant to a machine”. Every warehouse worker who ever had to urinate in a water bottle because they couldn’t otherwise meet the fulfilment targets set by an algorithm is a reverse centaur. Reaching into the future, everyone who has to sit in a self-driving truck to make sure it doesn’t crash, presumably on minimum rather than truck-driver wages, is a reverse centaur; as is every lawyer no longer on lawyer’s money checking Gemini’s command of precedent, every indie band scraping a living doing covers of AI-generated hits, and so on. That, anyway, is the promise: AI is coming for your job, and it is coming for your kids’ jobs, and there is no point fighting it because the future’s already here.
Wiping out the world of work, and with it our ability to sustain ourselves and live autonomous lives, is only the beginning, if you listen to AI’s architects. Elon Musk has called it the single greatest threat to human civilisation, Sam Altman has said it will “most likely lead to the end of the world” and Dario Amodei, CEO of Anthropic, memorably forecast that AI would come to see us the way we see animals: cute to have around but ultimately a resource to be exploited. “AI people claim they’re about to create God, by teaching words to a word-guessing programme,” Doctorow says. “It’s grandiose.”
Doctorow bursts into our video-call conversation as if we’re already two hours in, his delivery puckish and urgent. He’s not happy with his new AI camera, which “supposedly tracks your face, and then doesn’t, and just points in arbitrary directions. I mean, this is where my camera thinks my face is,” he says, and indeed, it is nowhere near his face. “Thanks, camera. Glad I’m wearing trousers.”
AI cannot and will never render us obsolete, Doctorow says. “It’s a conjuring trick. That’s probably the most important thing to get across.” A machine has been invented that is really good at building sentences by predicting what word would usually come next, and we invest it with meaning, insight, omnipotence. But we’re “imputing intentionality to this thing that intends nothing. It’s not because, objectively, it seems intentional, but because, in a state of nature, we don’t encounter sentences that don’t have sentence writers, we don’t encounter images that don’t have painters, and so on.” We marvel when it does things right, and conveniently ignore what it gets wrong, or indulge its “hallucinations”, which is just a fancy word for “errors”.
“Where I think the word ‘hallucination’ is useful,” he says, “is not to describe what the AI is doing, but what we do when we encounter a word salad, and we impute a writer to the word salad.” If you think AI can become conscious, he suggests, it’s because you’ve forgotten what consciousness is.
But that doesn’t mean there’s no threat. This technology can absolutely wreak global havoc; it’ll just be of a very old-fashioned kind. A vast amount of investment has gone into AI. “When I wrote this book [last year], it was a $700bn bubble. It’s a $1.4tn bubble now. The only thing worse than a $1.4tn bubble is a $2.4tn bubble, which we’re headed for,” he says. Nine tech companies in the US account for 35% of its entire stock market valuation, which was illustrated rather sharply when the war in Iran had a greater impact on European and Asian stock markets than America’s – they were “insulated” by the dominance of the tech sector, people said at the time.
But insulated might not be the right word. Doctorow describes “two poles in finance law. One is Stein’s Law: anything that can’t go on for ever eventually stops. And the other is Keynes: the market can remain irrational longer than you can remain solvent. So it’s hard to predict when bubbles are gonna pop. But it’s easy to predict that bubbles will pop.”
Doctorow, 54, is in Los Angeles; he divides his time between there and London (his British wife, Alice Taylor, runs the BBC’s AI Creative Lab, which isn’t ironic but it would slow us down to explain why not). He came to mass attention as a tech writer with his book Enshittification: Why Everything Suddenly Got Worse and What to Do About It; it’s hard to believe it was only published last year, and impossible to remember what we did without that word. People now use “enshittification” for everything, from the degradation of public services post-austerity to climate-crisis-related chaos events, when in fact Doctorow’s proposition had quite a specific use regarding tech: giant platforms lock you in and then make your experience worse on purpose. “I’m not frustrated by that at all. I think it’s glorious. My first two languages are English and Yiddish, languages that don’t have language academies, where dictionaries are descriptive, not proscriptive, where words change meaning.”
He’s Canadian, his early life dotted with signs of an independent spirit and tech-curious mind: he dropped out of high school and graduated from Seed, an “alternative” school in Toronto; he spent time in the 00s setting up OpenCola, a free peer-to-peer software company, then editing Boing Boing, which was vastly popular for a time as an online muster point for discussing tech, futurism and the left. He has an honorary doctorate in computer science from the Open University, has held numerous academic residencies and is professor-at-large at Cornell University.
Alongside his nonfiction, he’s an extremely prolific sci-fi writer, three-time winner of the Prometheus award (for young adult novels Little Brother, Pirate Cinema and Homeland). He writes graphic novels, authored a children’s picture book named after his daughter, Poesy, who is now 18, and publishes a newsletter almost daily. I wonder whether he’s dismissive of AI’s takeover because he finds writing, creating and communicating so effortless that maybe he doesn’t realise that for many people it’s like pulling teeth, and they wish a machine would do it. Absolutely not, he says – but more important than all the things AI can’t do, and will never be able to do, from call centre operations to radiology, is to understand the motivation behind all the feverish claims for it.
Why, he wonders, are capital allocators allocating so much capital to this? It’s because of a promise as old as the loom: that bosses will be able to replace their workers with machines. This isn’t just about money – often, when the machine is ultimately not as good as a human, or needs so much supervision from a human that it becomes more expensive, there’s still tremendous appetite for automation.
“The one thing a boss does not want is co-determination. Bosses are haunted by the knowledge that even though they fancy that they’re driving the car, if they don’t show up, everything continues to work. Whereas if the workers don’t show up, everything shuts down. And so perhaps they’re in the back seat with a toy steering wheel. AI is the promise of wiring that toy steering wheel directly into the drivetrain of the car. It’s products without product designers. Workplaces without workers, screenplays without screenwriters, movies without actors, hospitals without doctors and nurses. This is the promise.”
Once that promise is made, of course, it feeds an insecurity in the workforce, because if we all really were obsolete, that would be pretty consequential, and we can already see harbingers in our youth unemployment figures and our self-service checkouts. But then we become our own (second) worst enemy, because of what Doctorow, quoting Lee Vinsel from Virginia Tech, calls “criti-hype” – critique that both feeds off and feeds into the hype, and makes the doomsday scenarios more plausible. This may never have been more pronounced – government ministers have spoken openly about all the time and money they’ll save once routine tasks are automated.
“Do you not remember when they said cryptocurrency would replace all of the world’s financial systems? They told us that the metaverse would be the default, that we wouldn’t have tourism or sex any more,” Doctorow says, with vaudeville outrage. “We have such poor object permanence!” (This is fancy, child-development vocabulary for “really bad memories”.)
Another powerful AI critic, the journalist Karen Hao, has argued that when apocalyptic claims are made for AI, there’s often a veiled threat behind it: “Let us experiment as we wish, have our datacentres, because otherwise Chinese companies will get there first.” Doctorow agrees. “You don’t want a Confucian God. You want an Old Testament God. Different smiting.” Essentially, though, he thinks the big talk has a simpler motivation: anything to keep the investment flowing.
Whenever an exercise in automation fails and is abandoned – you may remember Amazon’s staff-less grocery stores, which actually required three people to be constantly watching each shopper on CCTV and guessing what they were putting in their baskets – it never dents the AI cheerleaders’ confidence that they’re speeding towards a post-worker world; nor, so far, the enthusiasm of investors. It’s got to the point where it’s more important to keep the narrative afloat than to consider whether it’s realistic.
“You don’t have to believe that a face cream is gonna make women look younger to believe that there will be women who will buy a face cream marketed to make them look younger. It doesn’t have to work. Are investors investing in the face cream because they think the face cream works? What I’m saying is maybe they’re investing in the face cream because they’re familiar with patriarchy.”
The problem is, people can see AI all around them, working better today than it did yesterday – turning out more convincing AI slop, answering their personal problems, turning fully clothed women into naked ones, analysing 18th-century poetry. It feels as if it can do anything, but none of that is the material basis for the investment. “I think it’s very important to be critical of things like Elon Musk’s Grok AI that allow users to turn out child porn [in January, Musk’s Grok AI was found to be capable of generating non-consensual sexualised images of people, including children, and xAI was rolling out ‘technological measures’ to restrict Grok’s ability to undress people in photos]. But we can’t pretend that there’s anyone who invested in his AI company, who looked at the prospectus and said, ‘I see you have a line item here for the expected revenues for child porn – that’s great, that’s why I’m here.’” In other words, go ahead and ban child porn – “we should do that anyway” – but no one’s going to take their money out. “If we want to actually target the harm that Musk is doing, we have to make his investors think he can’t make money.”
Fundamentally, this is a Marxist analysis, I suggest: that labour and capital are elementally at odds, and the latter will exploit the former even if all value is lost in the process. “I don’t necessarily disagree, but that’s not the argument I’m making. The argument I’m making is that bosses resent and work relentlessly to end co-determination as a class.” But that’s the same, I insist, and he shrugs, as if to say: we can bicker about Marx after we’ve stopped this runaway train.
What Marx didn’t have to contend with is the trillionaire, even in old money. “There is something about being very rich and insulated from the consequences of your actions that makes you solipsistic. You cannot make billions of dollars without hurting lots of people. And you can’t hurt lots of people without, in some sense, believing that they’re not really people.”
He raises Musk’s publicly acknowledged ketamine use. “I have a chronic pain condition, and I’ve had ketamine administered, and one of the things about ketamine is that it feels like the whole world is a thing you imagined. Like you’re the only real person in the world. I don’t think it’s a coincidence that Elon Musk calls the people who disagree with him NPCs, non-player characters, because he doesn’t think they’re really real.”
The larger problem than Musk’s personality, or even your average billionaire’s, though, is that bubble: “The more of the stock market there is wound up in it, the more economic harm there will be in the blast radius of AI, which is not to the capital allocators who’ve given them $1.4tn to play with – it’s everyone else. We have a quarter of a century’s experience now with popped bubbles, and we use them as an excuse to do the austerity that our politicians dream of doing anyway.”
