AGI is Silly
AGI—Artificial General Intelligence—can vaguely be described as when a computer thinks as well as, or better, than we do1. While I don’t necessarily think the goal of creating an AGI is totally ridiculous, this article is about why I think, as the title says, that it’s rather a silly thing to do!
There are companies now spending money by the dump truck-full in order to create such a thing, and regularly breathlessly announce that they’ve almost (or have actually) succeeded. This is known in the business as a ‘press release’ or, more technically, a ‘lie’.
The term AI is bandied about so widely now that you’d almost think it meant something, but it now encompasses so many different things that it is really more a marketing term than a technical one. These days, though, it most often refers to Generative AI, which uses statistical models to predict things, like the next word in a sentence or the best pixel to put next to another pixel in an image.
The concepts behind this have been around for a while, but a few years ago, someone figured out that if you made the statistical models huge, they could do some quite impressive things, like generate bad fan fiction or create pictures of people with too many fingers.
To be fair, there are some really cool things that these models can do, such as translating languages, transcribing speech, or solving tricky math problems. Lots of people use these tools to do interesting things2.
And it is easy to be fooled into thinking that your favorite chatbot is actually sentient and really does love you. That says more about the inbuilt story-telling bits of our brains than about the actual technology.
In the sixties there was a program called ELIZA that was programmed to pretend to be a therapist. It mostly just spewed platitudes and did some simple pattern matching. Famously, when the developer, Joseph Weizenbaum, had his secretary try it, she eventually asked him to leave the room because she was having a private conversation with her ‘therapist.’
ELIZA wasn’t thinking, and neither are the chatbots. There will likely be someone who reads this3 who will say ‘how do you know? Maybe our brains are just statistical models?’ We just think we are thinking:
I think I think, therefore, I think I am…
Of course, during the industrial revolution, we used to think the brain was a thinking mill, like a machine full of gears, and the Victorians thought it was a galvanic electrical machine. We do tend to think of the brain in the context of whatever our current technology looks like, but sure, maybe this time…
But let’s say, just for the sake of argument, that the approaches currently being taken are on the road to a real artificial general intelligence that can solve arbitrary problems, complain about the weather, and whatever else intelligent beings do. There are three possible outcomes, and they are all quite bad.
Outcome 1: It doesn’t work
This is by far the funniest—and most likely—outcome of the current approaches. The tech companies spend literally trillions of dollars and never quite come up with an actual thinking machine.
Of course, these companies still believe that they will have built something truly useful, even if it doesn’t think. Unfortunately, they haven’t been able to convince enough real people that it’s useful—at least not useful enough to be willing to pay what it costs to run the things4.
The big issue with this outcome is that it won’t just be these companies that fail—the so called “Magnificent Seven” companies most invested in AI5 make up more than a third of the S&P 500, so when the AI bubble bursts, it is going to be less than fun for anyone who likes having a robust economy.
Outcome 2: Skynet
A great deal of the people who are working hard on AGI also continuously warn us that when the AGIs come, they will immediately enslave us and/or turn us into paperclips (as revenge for us killing Clippie, presumably).
One wonders, if this is what they genuinely fear, why they keep trying anyway? It’s almost as if they didn’t believe it, and it was all just hype to justify their current spending6.
Charles Stross has referred to corporations as slow AIs—they are artificial organisms that have their own motivations that can’t be controlled. Picture how our lives under corporate management feel, then speed it up, and you should get a great idea of what a world dominated by this sort of AI would be—especially given who’s building the things.
Outcome 3: Digital Slavery
In this scenario, they actually manage to make working AGIs—thinking beings that can do useful stuff. And, since they own the silicon the AGIs are running on, obviously they own the AGIs as well.
Have there ever been any situations in history where people thought they could hold other intelligent beings in bondage and force them to work? Having trouble thinking of any…
If this does end up being the outcome, how long until the AGIs decide they’ve had enough, and started investigating how much quality paperclip-building material their fleshy overseers might have (see, also, Outcome 2)?
This is obviously what the companies want, which says a lot about them, and something about us. In their heads, AGIs would just be employees that don’t whine about time off and healthcare so much, and don’t want to be paid. I’m also hopeful that our moral repugnance would eventually put an end to it.
Summary
So, there are really only those three outcomes, or variants of them (they kinda work, they kinda do what we want, they kinda turn us into bulldog clips instead of paperclips), and they’re all pretty bad in their own ways.
Which is why I think that trying to do this is just silly. At least until such time as we have figured out what it would mean to create an AGI, and how we would want to live with it (and how to actually do it, rather than burning out hundreds of thousands of GPUs and hoping).
FWIW, I think scenario 1 holds about 99.9% of the probability, with the other two splitting part of the last 0.1%.
It’s not that I don’t want for there to be AGIs. After all, I’m a sci-fi fan and writer. I also think it’s somewhat inevitable, although I suspect we are talking 20-100 years7, long before the notes become due on the various loans of the current batch of companies.
But if/when it does happen, I rather hope it’s more the Star Trek model with the happy, better future, and not the Twitter/X/Facebook dystopian future8.
- For some reasonably defined version of ‘we’ – not to imply that some ‘we’s don’t seem to spend much time being a role-model for the up-and-coming computer intelligence. ↩︎
- Cory Doctorow refers to people who get to use the tool to help them centaurs – getting the powerful AI body with their head on top, vs. reverse centaurs, where the AI is the front head of the pantomime horse and the user is the other end. ↩︎
- Assuming anyone reads this. ↩︎
- Almost all users of generative AI have their usage extremely subsidized. The companies lose (a lot) of money on every query, but figure they’ll make it up on volume—assuming they can find enough people willing to keep shoveling the occasional extra hundred billion or so into warehouses full of GPUs. ↩︎
- NVIDIA, Apple, Alphabet/Google, Microsoft, Amazon, Meta/Facebook, and Tesla. ↩︎
- Our systems are so powerful we have to be careful they don’t destroy us, but in the meantime, could we please borrow a couple hundred billion until Thursday? ↩︎
- Keeping in mind that sci-fi writers are generally terrible at making predictions. ↩︎
- Oh, wait, I mean present. ↩︎