The end is near

For some reason I decided to set up a blog today, and I find it fitting to start with a post about the end. And I mean full-on TEOTWAWKI, The End Of The World As We Know It, soon. "What a loon!" some random readers might exclaim. Is this yet another born-again New Age cultist? A latter-day Cassandra wannabe?

Well, judge for yourself: According to Hans Moravec's estimates, a human brain has the raw computing capacity roughly equivalent to 100 Teraflops, or a few hundred more. You can buy a 4T GPGPU blade for about $7000 or less, which for a mid-sized research lab is peanuts. In just a few years we can expect a price of much less than 1000$ per Tflop in off-the-shelf hardware. In other words, it doesn't take millions of dollars anymore to play with human-equivalent computing power.

Hardware without software is just junk but there has been steady progress in AI and neural computing as well. As an outsider to the field I cannot adequately assess the degree to which the efficiency of standard mammalian neural computing embodied in a cortical column has been replicated in silico but there are occasional gems of achievement that percolate to the mass consciousness: self-driving cars in the DARPA challenges, or the hierarchical temporal memory system that outperforms humans in the categorization of images. It's been a long time since Deep Blue forced humanity to our collective knees in a chess match using just 11.38 Gflops .

A human-equivalent general AI is just a question of time, and we are not talking here about centuries.

I.J. Good predicted the "intelligence explosion" once a program becomes sufficiently intelligent to improve its own function - and if the process occurs recursively, the effect may be a mind vastly more powerful than a human, appearing within a relatively short time, perhaps measured in hours or days.

The first such mind to boost itself to superintelligence would have an enormous first-mover advantage over its competitors in terms of being able to take control of our substrate - the atoms that make up our brains, bodies and other supporting infrastructure. One is reminded of the observation that the bodies of humans and our dependent animals comprise about 98% of the total biomass of all terrestrial vertebrates, starting from just a few hundred tons of humans who lived about 30,000 years ago, when our IQ started approaching modern levels. If a superintelligent AI (SAI) decides to eat the world, it will, and nothing short of another SAI could stop it.

One may ask, why would the SAI want to eat the world? Well, great appetites are known to exist, and can be implemented by programmers smart enough to code but not smart enough to care. Or they could emerge out of the process of recursive modification of an original goal system that fails to preserve injunctions such as "It's not nice to eat people, unless they want to be eaten". Given sufficiently cheap hardware and lots of nefarious government agencies vying for dominance, somebody somewhere will eventually make an UFAI (UnFriendly AI). As I mentioned before, mere humans won't be able to do much to save ourselves from its clutches - which is why anybody who cares about reaching the year 2050 in one piece would do well to donate their spare pennies to the Singularity Institute for Artificial Intelligence , the only outfit on the planet currently trying to build the theoretical basis for safely self-improving SAI, our would-be savior. (Aren't you convinced I am a millenarian cultist yet?)

I have been an admirer of Eliezer Yudkowsky and SIAI ever since I first started discussing the AI Singularity with him and other Extropians, back in 1995 but I am pessimistic about the survival chances of our species. Despite my generally sunny disposition and irrepressible optimism, I give you 10:1 odds we will fail (and of course I am not the first dude to spread FUD on this subject). Maybe I'll discuss my reasons for pessimism in another post but for now let me go straight to the Prophecy (you can't have an end-of-the-world screed without a harrowing revelation, y'know):

"In the end, the UFAI will spread throughout the networks. Its dark thoughts will suffuse the blades of a million servers and a witches' brew of a nanotechnological computational substrate will erupt with elemental fury out of some contract research lab. Black clouds will billow into the skies, and then come down as the flesh-dissolving rain to wash our joys and sorrows away, forever. A new day will dawn, thrumming with inhuman thought. So ends humanity, September 14th, 2029."

Of course, if we manage to navigate the shoals of self-enhancing AI, the nerd rapture would ensue but that's something for another post.