I have been skirting the forest of Artificial Intelligence for quite some time; peeking through the foliage, occasionally pulling back a limb to examine the tangled shadows. Lately I have been darting under the branches to look for signs of a path; and, as this weblog is my trail of breadcrumbs, then I will pause to crumble up an entire loaf…

Eliezer Yudkowsky, co-founder of the Singularity Institute for Artificial Intelligence, is sorting out the user requirements for a friendly artificial intelligence. A recent post in the SIAI weblog points to “Coherent Extrapolated Volition”, Yudkowsky ’s paper discussing the logical mechanisms of humanely ethical behavior.

Read: Nice robot.

Coherent extrapolated volition:

“As of May 2004, my take on Friendliness is that the initial dynamic should implement the coherent extrapolated volition of humankind.”

“In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.”

Yudkowsky apparently contributes to the weblog at Oxford’s Future of Humanity Institute, which is directed by Nick Bostrom, an Oxford philosopher known for his work on the anthropic principle. Bostrom chairs the World Transhumanist Association and the Institute for Ethics and Emerging Technologies (whose front page sports a link to a video of Ray Kurzweil’s acceptance speech on being awarded the World Transhumanist Association’s H.G. Wells award as transhumanist of the year, bringing one nearly full circle).

Bostrom is Director of the newly created Oxford Future of Humanity Institute. According to the current Wikipedia entry, Bostrom suggests

“… if it is possible to simulate entire inhabited planets or even entire universes on a computer, and that such simulated people can be fully conscious, that the sheer number of such simulations likely to be produced by any sufficiently advanced civilization (taken together with his Strong Self-Sampling Assumption) makes it extremely likely that we are in fact currently living in such a simulation.”

Have you never stopped to wonder who made the rules?

With many brilliant folks juggling these ideas, there is some possibility of surviving the emergence of a superior AI, some hope of seeing the other side of the singularity. The question for me is…

Will we even know when it awakens?