A new book by Nick Bostrom is a major publishing and cultural event. His 2014 book “Superintelligence” helped to wake the world up to the impact of the first Big Bang in AI, the arrival of Deep Learning. Since then we have had a second Big Bang in AI, with the introduction of Transformer systems like GPT-4. Bostrom’s previous book focused on the downside potential of advanced AI. His new one explores the upside.

“Deep Utopia” is an easier read than its predecessor, although its author cannot resist using some of the phraseology of professional philosophers, so readers may have to look up words like “modulo” and “simpliciter”. Despite its density and its sometimes grim conclusions, “Superintelligence” had a sprinkling of playful self-ridicule and snark. There is much more of this in the current offering.

Odd structure

The structure of “Deep Utopia” is deeply odd. The book’s core is a series of lectures by an older version of the author, which are interrupted a couple of times by conflicting bookings of the auditorium, and once by a fire alarm. The lectures are attended and commented on by three students, Kelvin, Tessius, and Firafax. At one point they break the theatrical fourth wall by discussing whether they are fictional characters in a book, a device reminiscent of the 1991 novel “Sophie’s World”.

Interspersed between the lectures are a couple of parables. One is told in letters from Feodore the Fox to his Uncle Pasternaught. Feodore and his porcine friend and mentor Pignolius are in a state of nature, and groping their way towards an agricultural revolution. Despite this, Feodore’s letters are written in a highly educated, erudite style, and he has a decent grasp of the scientific method, using data and experiments.

The other parable concerns Thermo Rex, a domestic space heater whose very rich owner dies, and dedicates his large fortune to its maintenance and well-being. This causes the heater to be upgraded and granted superhuman intelligence, and also consciousness. Despite the echoes of terrifying dinosaurs in its name, it refrains from intervening in human life.

An assumption and a swerve

So much for the structure; what about the content? Two of its most striking features are a huge and controversial assumption, and a huge swerve.

The assumption is that in the foreseeable future we will find ourselves in what Bostrom calls a “solved world”, which is “technologically mature”. This means that all significant scientific problems have been resolved, and humanity is calmly spreading out into the cosmos, its population expanding exponentially as we go. We enjoy enormous abundance, and pretty much all sources of conflict have been removed. The central project of the book is to determine whether this state of affairs would be enjoyable for humans (or post-humans), and whether our lives could be meaningful.

(Personally, I find this assumption implausible. In the past, every time we solved a challenge it revealed several new ones, and although the past is an unreliable guide to the future, I strongly suspect this pattern will continue until we discover who really constructed this particular simulation. Hence I prefer Kevin Kelly’s idea of Protopia to the notion of Utopia. A protopia is a situation in which everything is very good, and day by day it keeps getting a bit better.)

Shallow and deep redundancy

Bostrom’s first task is to decide whether in a solved world, humans and post-humans will be redundant. He makes the helpful distinction between shallow and deep redundancy. In shallow redundancy, there are no jobs for humans because machines can do everything we do for money cheaper, better and faster. He suggests that certain jobs could not be automated if consumers wanted the practitioner to be conscious, and other jobs might require the person doing them to have moral status. However, it would become impossible for humans to hold down even these recondite jobs if conscious machines arrive. Nevertheless, in shallow redundancy, humans can live worthwhile and indeed meaningful lives being creative, having fun, and doing work that they enjoy but are not paid for.

In a state of deep redundancy, there are no tasks, including pastimes, that it is worthwhile for humans and post-humans to undertake. In this situation, AI makes leisure activities like shopping, gardening, browsing and collecting antiques, and exercising feel pointless. Even parenting could become deeply redundant as robots could be better parents, and anyway parenting could not take up enough of a person’s (now very long) life to provide its purpose.

If the Utopians have what Bostrom calls plasticity and autopotency – the ability to modify their own mental states – they could escape despair from uselessness. But although they could abolish boredom, they could not abolish boringness. Bostrom cites the example of Peer in Greg Egan’s 1994 novel “Permutation City”, who has re-wired his brain to exult in the accomplishment of carving perfect chair legs, even after he has finished hundreds of thousands of them. Peer is not remotely bored, but his life is profoundly boring, and lacking meaning.

The meaning of life

And so the good professor heads off in search of the meaning of life. Spoiler alert, this is where the huge swerve happens: he does not provide it. To be fair, he gives us advance warning, in what is for me one of the book’s best passages: “Asking someone the meaning of life is like asking their recommendation for shoe size. This is especially clear if we entertain the radical possibility that we are not in a simulation.” (Channelling Douglas Adams, he adds that the best shoe size is ten.)

The lens which Bostrom chooses for his analysis of meaning is a theory developed by a South African philosopher, Thaddeus Metz. This theory stipulates that in order to be meaningful, a life should follow an arc of overall improvement, and include elements of originality and helping others. It is an objectivist theory, which means that meaning cannot simply be what each of us decides we want it to be. Subjectivist ideas of meaning could be satisfied by simply tweaking your psychology, and could include the kind of life which the American legal scholar Richard Posner warned us about: “brawling, stealing, over-eating, drinking and sleeping late.”

For Metz, a meaningful life must also have an encompassing, transcendental purpose: it should absorb a lot of a person’s time and energy, and it should serve a purpose beyond their mundane lives. But Bostrom spares himself the problem of giving a definitive answer about the meaning of life by having his Dean abruptly terminate his final lecture. His students comment that the answer “got lost in the literary upholstery.”

Upside potential, and jokes

Given that Bostrom’s avowed reason for writing “Deep Utopia” was to alleviate some of the doom and gloom surrounding AI at the moment, and perhaps offset the alarm raised by his earlier book, it is frustrating that it lacks much description of the technology’s upside potential. His own 2008 “Letter from Utopia” demonstrates that he is perfectly capable of providing it: “There is a beauty and joy here that you cannot fathom. It feels so good that if the sensation were translated into tears of gratitude, rivers would overflow.”

Instead we are left with the jokes and the epigrams, and for me at least, these are worth price of admission. Even if many of us are presently doomed to be “homo cubiculi”, our species shows promise: “Between the sunshine of hope and the rain of disappointment, grows this strange crop we call humanity.” One of our best features is our capacity for aesthetic appreciation. With enough of that, “a duck’s beak can fascinate for weeks. Without it we are like the patrol dogs at the Louvre.”


Related Posts