Supermodelling

This post is the first in a brief series where I plan to touch on some general points about AGI, in preparation for re-abandoning AGI as the self-described focus of my playtime research.  My interest in AGI has not waned… I am signed up to attend AGI-09, which is the next conference dedicated to the topic, and I’m really looking forward to it.  I’ll continue to follow some of the online AGI discussions and blogs, though I skim most of it now.  I’ll keep reading books from authors in the many “AGI-related” academic fields.  There’s a huge library of brilliant books out there.  Right now I’m reading Steven Pinker’s The Stuff of Thought.

Problem is, AGI is still too blurry for my taste.  I like to get my hands dirty, building things and experimenting, making specific arguments, looking for answers to relatively precise questions.  Unfortunately I have not been able to find justified ways to do any of these things applied to “AGI” as a general topic.  It is possible to write code, but the relevance of the code is very doubtful.  It’s possible to debate principles, but the debates almost always end up being about the definitions of words, which wouldn’t be so bad if they ever got pinned down as a result.  It’s possible to prove theorems, but there’s no math that actually captures intelligence in anything but trivial ways (that is, it’s almost entirely math for its own sake, rather than math that usefully models general intelligence).

So to make myself happy I need to pick some plausibly AGI-related topics and focus on those instead of more general issues.  For a while, at least.

It seems clear to me that we are a long ways away from “solving” AGI anyway.

Before refocusing this blog, I do have a couple of things to muse about, though.  The first is my opinions about the current state of AGI as a field of study and research.

The biggest problem, and the underlying source of my discontent, is that there are no clear questions to be answered and no clear goals to be pursued; this reflects a complete lack of consensus about the fundamental issues that should define the field.  In a way, that makes AGI a bleeding-edge field — but in another way, it makes AGI not a coherent field of study at all, when there is scant agreement even about basic things like what the G and I mean.

Given this incoherence, the only way to assess “progress” toward the vague goal is just to look at projects underway, and there are only a few of them really.  The 800 pound gorilla at the moment is Ben Goertzel’s Novamente project (and its open-source offspring OpenCog).  Goertzel has written a great deal of material about his ideas and his model of intelligence, but amazingly it still isn’t enough for me to grok exactly what he’s up to.  So far to me it looks like a large pile of technology choices tied very loosely together with an underlying philosophy of intelligence but I cannot see specifically how any of it addresses the fundamental things about mind that interest me.  Also, the main underlying knowledge representation (probabilistic logic networks) are a really nifty expert-system-style modelling formalism but in a deep sense shares the “suggestively named graph node” criticism that all other such knowledge representation schemes fall prey to — since there is no specific story about concept formation, modal reasoning, semantic grounding, and so on.  I imagine that Goertzel would disagree with me on that point and say that the specifics are coming as the implementation proceeds.  I hope that’s right, and I look forward to seeing what comes from the effort, but at the moment I don’t see any reason to think that the grab bag of algorithms and data structures will add up to emergent dynamics capturing general intelligence.

Venerable old AGI-like projects (SOAR and CYC in particular) are still alive and moving forward after decades of work with no particular end in sight as far as I can see.  I can’t think of any reason that the next few decades will lead to fundamental breakthroughs but as usual maybe I just can’t see the big picture.

Other interesting and attention-grabbing projects such as Blue Brain and Numenta may have AGI-related implications down the road but at present they don’t seem to really count as AGI research.

A number of individuals have interesting AGI projects going…  Steve Reed’s Texai project appears to have a clarity of purpose that is impressive.  Vladimir Nesov appears to be building some experimental foundations based on ideas that seem interesting, although most of the time I can’t really understand his explanations; perhaps his thoughts are too deep for me to follow.  Pei Wang’s NARS system is highly regarded.

None of these efforts appear to me to be anywhere near the stage where a convincing impressive demonstration of general intelligence is on the horizon.  I don’t see how another decade is going to make much difference either… but I do wish them well and will follow their progress with great interest.

Some other writers have very interesting ideas with potential (such as Richard Loosemore and J Storrs Hall) but they don’t write very much about what they are up to.

I respect all this stuff and think it’s really great.

The “AGI community” is another matter.  The publications and conference are more or less passable, but unfortunately the less formal fora are victims of their own subject matter.  It’s a rather distressing mix of tedious self-styled critics, delusional wingnuts with theories and prototypes that are clearly not anywhere near the target, people who mistake databases or simple data structures for knowledge and work diligently on batches of code with no discernable purpose or depth, people who apparently cannot even tell the difference between plot devices dreamed up by science fiction writers to move stories forward and actual technologies, deranged lunatic fringe paranoiacs trying to save humanity from imminent destruction by “superintelligent” rogue AI systems (despite absolutely no evidence of actual progress in the field)…. and plenty of other idiosyncratic frustrating viewpoints.  Adding yet another ridiculous idiosyncratic frustrating viewpoint (mine) to the mix doesn’t seem particularly necessary, noble, or fun.

This is all inevetable I suppose given the hyperbolic end goal that all of us AGI folk share… but the disconnect between what is actually real and important today vs the chaotic noise of uncritical (or supercritical) fantasyland is fun in small batches as an occasional diversion, but quickly becomes ugly and tiresome.

So I’m going to stop discussing and thinking about general AGI issues pretty soon in favor of more tangible related things, which I will write about.

1. 
   QuantumG said...
   5:54 am - October 30th, 2008

   Well written and accurate summary. I like it, and I’m prettimuch in the same boat, the only really difference being that I already gave up, came back and am watching the OpenCog stuff with interest. They really do have a big grab bag of algorithms and a coherent idea of how to put them together to make something that can infer stuff in complex environments. Whether or not the whole virtual embodied learning will pay off is not completely clear yet, and there’s the complete wild card of Linas’ work on natural language processing.

2. 
   Vladimir Nesov said...
   2:34 pm - October 30th, 2008

   We don’t know enough to be confident that all hell can’t suddenly break loose, and so can’t discard this scenario as currently irrelevant. For all I know, AGI (the lethal kind) might have a very simple underlying principle, easy to implement. Someone might just stumble upon it in any of the variety of contexts. Maybe it doesn’t intuitively look plausible, but being sure that it can’t happen isn’t justified.

   Another question is how useless most of the current chat about it is (which is usually too shallow), but building adequate awareness about the problem, especially among AGI researchers, might be valuable.

   You didn’t mention Yudkowsky’s Overcoming Bias book-in-progress, which might be a very important step towards establishing foundation of thinking in this field, solving the very problem you pointed out: disarray in fundamental approaches, when it isn’t clear where various projects are going, when researchers don’t understand what they are building, when optimistic interpretations of sophisticated and bloated implementations or theories don’t connect to the real-world results.

3. 
   Derek said...
   3:39 pm - October 30th, 2008

   QuantumG: I also am curious and eager to see where OpenCog goes. I wanted to find a justification for joining that project but haven’t yet been convinced.

4. 
   Derek said...
   4:17 pm - October 30th, 2008

   Vladimir, I agree that we cannot disprove the “all hell breaks loose” scenario. It is not impossible. Similarly, people playing around with chemistry might pop gray goo out of their test tubes, nosy reality hackers might find a bug in the simulation we all might be living in, a huge asteroid might wipe out life on earth, the population explosion and expansion of leisure time might lead to an idel thinker thinking a thought that somehow destroys the universe just by being thunk. For me, “you can’t prove it’s impossible” is is insufficient justification for me to spend the hours of my one precious life fretting about apocalyptic visions. I am afraid of those who are serious about it, though. Some lunatic trying to multiply infinity times zero could easily come to the conclusion that murdering AGI researchers is a net positive for humanity.

   Still, if I thought I had a magic bean core algorithm that could grow into something with super powers, I would not just forge ahead heedlessly. I’m so far from understanding any of the basic foundational issues of cognition that a monkey at a keyboard is roughly as likely as me to create such a thing.

   As for Yudkowsky, I think he’s a brilliant mind that flits from the surface of one topic to the next but so far has made no deep impact on the issues that are important to him (unless he is keeping it secret). It seems a bit early for him to be writing his autobiography. I do read OB, though, because he’s a great writer with interesting ideas even though I don’t agree with too many of them.

   I have made substantial contributions to SIAI (I’m currently listed as a sustaining donor) specifically to fund Yudkowsky’s effort to formalize Friendliness and self-improvement… but there hasn’t been any result for years now and although his OB stuff is entertaining, I think I’ll give $30 to buy the book rather than $1000 to fund nontechnical work.

5. 
   Vladimir Nesov said...
   5:30 pm - October 30th, 2008

   Existence of a magic core algorithm doesn’t look nearly as a priory unlikely as other ridiculous possibilities you listed (note that grey goo would need to survive in the wild). Even if you don’t know how to do it, someone else might.

   Material Yudkowsky compiled can act as safety rail for researchers, explaining the importance of FAI, cautioning against pitfalls and blind alleys. It can easily save someone from spending whole life on misguided activity. It’s pointless to start doing technical work if you don’t understand what you should work on, and also minuscule amount technical understanding can sometimes heal nontechnical intuitions in a way that separates efficient road to success from certain failure. It’s important not to confuse precision with formality.