Supermodelling

A memory: splashing down from the boat into warm turqoise water off the coast of Belize, hand holding my regulator in place, a chaos of refracted sunlight and the familiar sound of going under the surface.

Our lives seem bound by these events; we guard them jealously — even the painful ones… they arise unbidden to remind us of who we are, who we were.

Perhaps these episodic memories are flash-frozen snapshots of concept activations, chained together in sequence, formed during times of novelty or stress… neural pathways recording grist for the mill of metaphor. Causal traces stored against future need.

I’m not sure we can be human minds without them.

But: can there be minds without them? Minds in general? Minds in the abstract?

There is a tension between thinking about our own minds (the only minds we know of), and trying to puzzle out exactly what the necessary core of minds must be. Perhaps there are many different approaches to mind, different ways of collating data and putting it to use in predicting and controlling the world.

Shall we be heavily inspired by human minds, hopefully grasping at sufficiency, or shall we discard all but reasoned necessity?

Is metaphor just a convenient mechanism? A shortcut mapping of similarities taking advantage of evolution’s empirical observation that form often does imply function? That crude similarities lead to better than random guesses? Are such tricks good ideas for computer minds? Are they necessary for effective communication with human beings? If we scrape away all the tricks, is anything left?

There is very little work available on the abstract approach to mind. I’d like to find more, but most of the reading material at hand involves vague descriptions of human psychological phenomena. We take what we can get I suppose.

Here’s something that bugs me and that I need to study along with these root puzzles about concepts and models: why are automatic theorem provers and automatic computer program generators so pathetic? It seems as if the distilled rationality of formal logic (probabilistic, nonmonotonic, purple, whatever) should capture the essence of abstract mind… but we apparently have not or cannot codify physics in a useful generative way. In fact, it seems we cannot even do much of a job of formalizing that most formal of subjects: mathematics itself — at least not in such a way that anything interesting comes from the effort.

Why is that? I wonder.

Having discussed how the physical and abstract universe contains things (and abstractions and events and relations, which are also things), it seems clear to me that the core funciton of mind is modelling these things.

But what is modelling? Searching for an answer to that question is part of what I’m trying to do, but basically by modelling I mean representation and the manipulation of representations.

A representation is a partial mind state that refers to the thing being represented. Partial mind state is a rather vague term whose meaning depends on the mind substrate; it might refer to properties of biological neural networks, it might refer to data structures in a computer memory. Reference is the key deep issue. Coming up with a clean way of thinking about reference is one of my major long-term goals. It seems certain that the degree to which a representation refers to a thing depends on the usefulness and accuracy of the modelling manipulations performed on the representation. As a quick example, a representation for “flower” refers to physical flowers (in part) to the degree that perceptions of the world allow accurate recognition of flowers.

In practical terms it seems as if a causal relationship between the things being represented and the representation itself is necessary, but it might be that such a relationship is only extremely helpful. That is, the details of a flower category representation could have come from some process independent of flowers themselves (say, from Deep Thought about the implications of the laws of physics, or pure random representation generation) – but observations, descriptions, or other causal links to actual flowers seem more practical.

Following what I think is fairly normal terminology, I will refer to these representations as concepts from now on.

Note that concepts are themselves things. An AGI should be able to work with pieces of itself just like pieces of the external universe, and constructing concepts about concepts is a big part of that. The details, however, are murky to me at this point. What are the “parts” of a concept? What does it mean for one concept to be related to another, and where does that relationship come from? I hope that eventually, after thinking long enough about concepts, ways of thinking about these questions will appear.

The important nuts and bolts place to start, I think, is to focus on the “manipulations” rather than the representations — the point of concepts is that they are the way minds deal with the universe, and we should be able to develop a set of requirements for that.

Some obvious examples: Concepts have to be created somehow. They are used in some way to predict the behavior of the universe. They are used for logical and metaphorical reasoning (although whether minds must use metaphor is unclear). They must be used for recognition. They may be given labels for use in linguistic communication. And so on.

The next phase of my research will involve coming up with this requirements list in a more detailed form. I really don’t want to end up with fifty unrelated bullet points; I am hoping that many of these requirements will end up being different ways of describing a small set of underlying mechanisms… but it’s too soon to start inventing mechanisms without knowing what they need to do.

Besides elaborating on a list I have developed myself from thought and general reading, I want to dive into an analysis of several different works related to this subject. My initial reading list is:

• Minsky: The Society of Mind
• Lakoff and Nunez: Where Mathematics Comes From
• Lakoff: Women, Fire, and Dangerous Things
• Yudkowsky: Levels of Organization in General Intelligence
• Fauconnier: The Way We Think

I expect this requirements phase to take quite some time. In my earlier research I was too quick to jump to hopefully-cool representation mechanisms, but I’m in no particular hurry.  A particular interest of mine is trying to simplify the apparent complications inherent in incorporating time into a conceptual framework.  I am greatly puzzled by this subject.

Besides this work I will continue to think about interesting AGI-related topics.

Why do I want to help build a thinking machine?

Because death is bad.

We bravely talk about death being natural; about mortality charging up our limited hours with meaning; about meeting God in the afterlife… but until the time comes when our suffering becomes unbearable, we do not want to die. We do not want our loved ones to die, we do not want the stranger on the street to die.

And there are other bad things. Hunger is bad. Excessive pain is bad. Lack of freedom is bad. And so on.

In theory, in a grand theory born of audacious yearning, these things can be drastically reduced because they only require moving stuff around. The trick is knowing which stuff to move, building the stuff to precise specifications, and moving it around properly. Stuff like food. Stuff like chemicals. Stuff like cellular repair robots.

Alas, not all bad things can be so conveniently dispatched. People seek power over others, act on hatred and fear, cause harm to achieve desired ends. Just moving some stuff around in a consensual way does not help with that, though motivations for harming others may be reduced somewhat when material benefit is always more easily obtained in other ways.

Still, ending physical suffering, hunger, poverty, and even death is desirable. If only we could move that stuff around properly….

As smart and agile and organized as we can be at our best, we are not that smart, not that agile, not that organized. We need help, and since we’re not going to find it sitting around, we need to build it. We need AGI.

Looking at it purely from this perspective, what I want to build is a computer system that fills the following roles in a “superintelligent” way:

• Scientist, to answer factual questions about the universe and the things in it.
• Inventor, to find creative solutions to problems and new applications we don’t even know we want yet.
• Engineer, to design and build things.
• Manager, to coordinate resources.
• Process Controller, to manipulate the physical world.

In all of these roles, our AGI must be able to fluently communicate with us and understand what it should do (and what it should not do).

Hopefully, this pragmatic view of a thinking machine lets us put aside certain tangential issues, such as consciousness, qualia, and so on. However, for our AGI to effectively communicate with us and understand us it cannot be completely independent of human nature.

If we are going to give an AGI the ability to act, we need to be pretty sure it acts in a good way, and will continue to do so. This so-called Friendliness problem will be the subject of some posts as time goes by. For the moment it is not a large concern although it will be someday. I can muse on the nature of cognition without endangering anybody.

If this is the vision for the far-off desired future, we have to figure out how to get there from here, so I want to start thinking about an intermediate step which is in the right direction and provides sufficient challenges to focus thinking about cognitive architecture and act as a testing ground for theories.

This post doesn’t really have much to do with AGI.

One of the most celebrated examples of the astonishing emergent thinginess of the abstract universe is the Mandelbrot Set. Everybody who plays with it has been struck by its wonder.

It’s really a simple formula and it’s very easy to write a computer program to compute it.

In 1985 I wrote a little program to plot points of the Mandelbrot Set on my new Amiga 1000 computer. By modern standards its display was dreadful and at 0.1 Megaflops it wasn’t exactly “interactive”, but it was fun.

In the early 1990s I was lecturing computer science courses at a big 10 university and winding down my graduate studies (”slowly giving up” is an okay way to put it). At that time a Thinking Machines CM-5 was installed in our building, which was a pretty big deal. In the very first top500 supercomputer survey it ranked as the 63rd most powerful computer in the world. I had the chance to play with it a little bit just after it was installed (it was summertime and the real research did not yet monopolize the machine) — I wrote a Mandelbrot program for it, which was pretty cool.

This machine could do 8 Gigaflops: 80,000 times faster than the old Amiga. The bottleneck was not computing the images, it was getting them to the rather primitive display hardware.

Now on my desk I have a quad-core PC capable of doing 16 simultaneous floating point instructions per cycle; on this task using 32 bit floats I think it’s about twice as fast as the CM-5 was. Nice progress for 15 years.

So I wrote another Mandelbrot program just for fun. It’s only for Windows and you’ll want a decent monitor/videocard and the faster the processor the better, the more cores the better (up to 8), and you can download it here if you want. This version uses double precision arithmetic so it is half as fast as a single precision version would be — but the single precision math loses coherency at a low zoom level so it’s not as much fun as double precision.

Moving the mouse pans the view when a mouse button is down. Left mouse button zooms in. Right mouse button zooms out. Hit escape to exit.

For the curious, the source code is here.

I’ll probably play around with this again at some point; there are some features I’d like to add:

• Smoother control
• Better and alternate colorings
• Dynamic alteration of iteration count

An upcoming generation of video cards will support double precision math; when that happens I’d like to write a version for one of those — this is almost a perfect SIMD application, and it should really fly.

One point that is important to me for musing about building AGI: if possible, build a mind on an abstract substrate whose implementation makes good use of the hardware. For now and the forseeable future, that means vectorized floating point math. Interesting.

The previous observations about the universe were fairly straightforward, but the universe is more than just physical objects.  In particular, the universe includes nonphysical things.  The first type of nonphysical thing that interests me is a category — an abstraction that refers to groups of things, making a new thing not through spatiotemporal proximity or other physical relation but rather through abstract relationships.

What exactly is an abstract relationship?  I don’t think I even have the right modes of thought to begin thinking about an answer to that question but there seem to be lots of different important aspects to it.  For example, we could say that a planet is “round”.  Round might be thought of as a “property” of a thing or we could say that the planet is categorized as a “round thing”.

But where does “round” come from?  What is the existential status of “round”?

I wish I had a useful answer to these questions.  I do know that this abstraction process, the invention of categorical and other abstract relationships (and their manipulation) is the most important core thing done by minds.  My obsession at this point in my musings on AGI could easily be summarized as:  Where does “round” come from? Wherever the true nature of these category-things may be, it is clear that just like other things discussed previously, they are in general fuzzy in nature.  This likely derives at least in part from the inherently fuzzy nature of the things being categorized.

Whereas physical systems obey the laws of physics, abstract systems obey their own arbitrary rules.  It seems to me that there is an infinity of possible abstract systems, many of which in their elaboration doubtless reveal amazing thinginess.  Conway’s Life is one trivial example, and chess is another, but I imagine superintelligent creatures of the future will take great delight in pondering abstract systems to uncover the beautiful things they contain.  Perhaps there will be a general theory of abstractions, a level of meta-musing about conceivable universes that I cannot even imagine.

Homo Sapiens as individuals and as a species have some limited ability to deal with abstractions — a degree beyond other animals, which also have some such ability.  For practical reasons, these abstractions tend to be closely related to the physical nature of our universe (and thus ourselves) — low-level category formation from sensory input, a type of abstraction built into creatures by evolution, is probably the simplest such ability, and many “hardwired” elaborations on that plan seem to be part of brain function.  Humans have found some hard-won abstractions that are more sophisticated, though — good examples are truth and number.

Truth leads to logic and statistics, number leads to mathematics.  In all of such cases, complicated things are built from simpler things in astonishing ways, using abstract relationships and properties instead of physical ones.  The many many human lifetimes spent slowly expanding these ideas were worthwhile primarily because these abstractions map very well onto the physical world.  We even build abstractions out of the process of building world-modelling abstractions, to arrive at science and rationality.

One point that colors my thinking about building AGI:  Abstract mathematics include interesting categories like “prime numbers” about which astonishing things can be proved and subsequent simple knots of thinginess emerge.  These categories are different than most — “prime number” is not vague.  It is precise.  While that allows long chains of perfect reasoning, most of the things we as thinking beings living in our physical world deal with are vague and not suitable to similar methods.  Despite the apparent truth of reductionism in the sense that all physical reality arises from some root precise formal system, things at larger scales do not have effective formal descriptions (pure reductive descriptions are not effective because of both their unknowability and unwieldy size).  For this reason I am skeptical of approaches to AGI that are based on formal logic and deduction; such a method for defining and referencing things is not expressive enough to work with actual things and seems bound to leak meaning like a sieve.

Nevertheless, when trying to come to grips with the questions “where does ’round’ come from?“, “where does ‘knot’ come from?“, “where does ‘democracy’ come from?” the question “where does ‘prime number’ come from?” does appear as if it could be relevant.

Well, this blog post concludes my brief notes on the nature of the universe, although not in a really satisfactory way.  It’s rather vague.  This may be inevitable to a degree given that the things of our universe appear to be inherently vague, but apprehending things (both physical and abstract) and then working with the results is the core thing that minds do.  It’s what thinking is, and to the extent that we can build machines that do it effectively, we will have built machines that think.

So then, how to go about it?

The next really interesting blindingly obvious observation about the universe is that it changes. Things come into being and go away. Things change. Relationships between things change. I’ll call these time-related phenomena events. More specifically, purely in terms of the physical universe, an event consists of spacetime-localized non-uniform non-random changes in universe-stuff.

Just as space is so interestingly filled with physical things, time is interestingly filled with events. And, just as things are disturbingly vague, so are events. Consider for example a “supernova” event. The extent of this event in time and space is unclear. When did it start? When exactly is it finished?

Events occur at a huge range of space and time scales, and they often encapsulate other events occurring in sequence or in parallel.

Described this way, events sure seem intuitively similar to things. In fact, it is easy for us to think of them *as* things… one of the interesting capabilities of human minds is their ability to apparently effortlessly work with representations of both events and timeless things, almost as if they are in some way the same.

How could that work? I consider this one of the biggest foundational puzzles to be worked through as part of building a mind — to what extent can things and events be modeled in a unified way?

First, let me say that I claim no particular original contribution to the knoledge of mankind in anything that I write in this blog. It could be (and is in fact likely) that others have said the same things I do. My failure to cite such restatement of positions others have discovered is due to faulty memory. In most cases I don’t know whether my ideas are original or not. That would be important if I were angling for some special status as an original thinker, but I’m not. I’m just looking for understanding.

Now, a couple more observations about things. The universe contains things at many different scale levels — from subatomic particles up to galaxy clusters. Things of pre-scientific human experience range roughly from bits of dust up to stars (or, since naive observation can give no reasonable size estimate to astronomical objects, we could say “up to mountains”). This is a very large range, and all parts of that range are populated with different things. For some reason this completely delights me.

Almost all things (those that are non-”atomic”) are made of other things and/or unthinglike stuff. This means that compositional hierarchy is an inherent feature of the universe.

Note that “unthinglike stuff” really is a collection of things (so our science has revealed).

Here’s a puzzle that will need unraveling — a region of space with no things in it is easily conceived by our modern minds as a region of vacuum, which is itself a thing. Later, when I start thinking of things as more than just the physical contents of regions of space, the same phenomenon will arise. It’s interesting.

Critically (to me), things are almost always fuzzy. The sun is a thing, but exactly which things count as part of it and which don’t? Which region of space? There are a variety of ways to attempt to make that explicit, but to me they all seem justified more by the desire to formalize the thing than to explain the actual thingness apprehended by my mind.

An even better illustration of this is the picture of the crater from the last post. Which exact parts of that landscape are part of the crater and which aren’t? There really doesn’t seem to be any natural or even justifiable boundaries for this thing and certainly no obvious ones in use by me when I apprehend the crater.

I can think of very few things that this principle does not apply to. That is, such maddening vagueness regarding the “definition” of things, even when the things are just individual physical objects, seems to be the rule rather than the exception.

I may just be an idiot, but this to me is a very deep and troubling observation — that I cannot say exactly what my mind’s apprehension of a cloud refers to in the universe.

I don’t yet want to dive into “solving” this issue from the perspective of what artificial minds must do. I am just probing for what seem like the fundamental issues, and there are lots more of those still to find.