Disclaimer
The opinions expressed herein are my own personal opinions and do not represent anyone else's view in any way, including those of my employer.
© Copyright 2008
The opinions expressed herein are my own personal opinions and do not represent anyone else's view in any way, including those of my employer.
© Copyright 2008
Critics sometimes see the AI enterprise as merely the application of the latest tech fad — given a shiny new toy like a computer, we just try clumsily to make a metaphorical connection: mind is like a computer.
That is of course completely backwards: actually, computers are like the mind — and purposefully so. Arithmetic, algebra, logic, and algorithms of various sorts were dreamed up long before computers came along to automate them. Our ancestors invented them hundreds and thousands of years ago to provide quantitative models of bits of the world they cared about, usefully predicting and explaining things.
Since then, we’ve found other uses for computers — as media for communicating and presenting information, for example — but modelling the world and modelling mental processes have always been a major motivation behind the technology.
The sheer number-crunching capacity of computers makes it feasible to model the world with methods not directly derived from introspection — physics simulations and whatnot. And some methods start out with introspective inspiration but quickly get sharpened to a narrow point to such an extent that they no longer bear a close resemblance to their conceptual and linguistic origins.
Consider chess as an example. Starting from the introspective view that a person might think: if I do this then he’ll do that and then I can counter with such-and-so… This one little observation about a small part of the mind’s modelling of the game led to interesting techniques for tree search, and then got taken to an extreme… in this case culminating in Deep Blue and the defeat of Garry Kasparov.
When this narrowing and optimization of mind-inspired methods occurs, we typically say it is no longer AI, which actually seems appropriate to me.
Whether the source of computer models is introspection, physics, statistics, or ad-hoc empirical observation, modelling is the pervasive basis for much of the usefulness of computers. Consider a word processing program: much of its value derives from models of paper documents and other old structured comunication forms, models of language, models of printing machinery (printer driver interfaces), and so on.
Years ago I did some work for a friend of mine who owned a business that developed and sold a complex software package to help manage fleets of concrete trucks (the big ones with the spinning drums on them). Such a package is full of models, from models of general business activities like invoicing and accounting to models of delivering megatons of concrete on a customer’s desired schedule — which involves geography and roads, time required to load and unload the trucks, etc. These models make forecasting and resource allocation much more accurate.
These kinds of things are the meat and potatoes of computer applications. Web browsing, playing games, and presenting media like music and movies are probably bigger uses for most people these days, but when you scratch the surface you’ll frequently find modelling underneath: HTML is a document model, and game graphics are probably the clearest example of computer modelling you’ll ever see.
The first “real” computer, Eniac (1946), was built to calculate artillery firing tables (a pure modelling task). Today’s most powerful computer (the IBM Roadrunner, over a million gigaflops) is used primarily to simulate how nuclear materials age, modelling the safety and effectiveness of nuclear weapons (not the happiest purpose, but better than testing by exploding them).
All told, computer models comprise a hugely important industrial technology. And building them is hard, a labor-intensive and specialized hugely expensive process. The universe reflecting itself is not a trivial accomplishment! There are many clear economic benefits from: specific model construction, the invention of new modelling techniques, and improving the modelling process. Society can be served, and money made, by digging deeper into the roots of how it all works.
Our minds are really amazing modelling engines. So in a practical sense it isn’t far off to say that Artificial Intelligence == Automatic Modelling. That’s why — besides the interesting philosophical investigation of our own nature — many of us think AI is such a fascinating task. Not because we think “Wow, brains are like computers!” but because modelling modelling may be the key to the future.
I’ll end this post by listing a few computer modelling methods:
If, as is widely reported, AI researchers (those scurvy dogs) began barking up the wrong tree a long time ago, I wonder when exactly that was and what scent led them astray. They started the chase promisingly… I think a lot of the foundational work of the Old Masters was really quite brilliant.
Case in point: The Physical Symbol System Hypothesis.
If you’re inclined to speculate that a computer could in principle be “intelligent” (I am so inclined; if that turns out to be wrong, then, well), it would be helpful to say some more about what is meant by a “computer”. If the speculation is phrased as “the UNIVAC I on the third floor could be intelligent,” that seems overly specific. So the natural thing would be to express it using one of the (equivalent) abstractions for “computation”, such as the Turing Machine, and just say that intelligence is computable and leave it at that.
In 1976, Allen Newell and Herb Simon (Newell’s doctoral advisor) chose a different way of putting it, a more detailed hypothesis that emphasized what they considered to be some actual fundamental issues of cognition. They called their abstract computer a physical symbol system. So:
The PSSH: A physical symbol system has the necessary and sufficient means for general intelligent action.
To simplify a bit, I’ll take “physical” as given and (as a nuts’n'bolts engineering-type guy) I’ll leave off the “necessary” part. Thus: a symbol system has the sufficient means for general intelligent action. So what’s a symbol system? Let’s start with some of their definitions:
Although that seems straightforward enough, people view the world through their own set of biases, which produce a number of (in my opinion) erroneous views of the PSSH and, by extension, rather odd characterizations of the AI enterprise. For example:
Viewpoint 1: 0 and 1 are symbols and machine instructions are processes that manipulate structures of bits, so it sort of follows that the PSSH is really only saying the same thing as “computers could be intelligent.”
Although this is a comforting way to respond to critics who attack the AI enterprise by enumerating the inadequacies of particular symbol systems, it ignores one of the central concepts of the PSSH: designation (which is not part of the basic definitions given above but is nontheless part of the PSSH). According to Newell, a symbol structure designates a thing if the system can use the symbol structure to affect the thing itself or behave in ways dependent on the thing. That is, some sort of “access” to the object is provided by the symbol structure, and this relationship between symbols and the universe is critical, although it is of course limited (the map is not the territory, after all…). This gives rise to the usual way of thinking about symbols as “standing for” things.
Viewpoint 2: Symbols are atomic tokens such as hotdog. Although much work in AI has proceeded from this kind of assumption, it leads to significant difficulties and conundrums. Specifically, if hotdog is just a label floating around, where does the designation to an actual hotdog come from? This is one trivial reflection of the famous symbol grounding problem, which is a very subtle but crucial challenge to the “Viewpoint 2″ way of looking at the PSSH. In reaction to this, some folks start thinking of some elements of an AI system (for starters, data related directly to sensor readings such as a camera image) as being subsymbolic, a suggestive word implying that symbols are constructed out of nonsymbolic entities. But there is no room for semantically loaded “nonsymbolic entities” in symbol systems. Furthermore, ugly issues arise from this “duality” — what makes some tokens symbols and some subsymbols? Where does the symbolness come from? Perhaps (the thinking goes) the very concept of symbol as used in the PSSH is wrongheaded if it leads to this kind of mystical dualism and complication.
Viewpoint 2b: Going even further, if symbols are clean labeled tokens, it is convenient for many purposes to provide semantics by assigning truth values or, more generally, probability values to symbol structures, and then choosing the processes of the symbol system to be truth-preserving transformations. This leads to symbolic logic and it dates back at least as far as Aristotle. Its productivity has made logic fundamental to many AI approaches, though sometimes you have to dig a bit to reveal how a particular system relies on this foundational way of carving up the world.
Note, though, that there is no inherent reason that symbols have to be simple. One more time: there is no inherent reason that symbols have to be simple. One more time: … oh, never mind.
Newell viewed the designation issue through the idea of a “representation law” — suppose that X is some sort of situation in the world, and T is some sort of transformation. For example, X = me holding a hotdog, and T = me eating the hotdog. The representation law says:
decode[encode(T)encode(X)] = T(X).
Here, encode is a translation from the “external” situation to an “internal” representation, and decode is a translation in the opposite direction. So the law basically says that there is an internal process representing ‘eating a hotdog’ (encode(T)), and an internal symbol structure representing the situation of me holding a hotdog (encode(X)). Applying the process to the structure should produce a symbol structure that corresponds to the external situation after the hotdog is eaten. For example, both in the real world and in the internal representation:
This corresponds exactly to the intuitive sense of what I expect from a model, so perhaps the representation law could be the foundation for a theory of modelling. Unfortunately, though, it is kind of frustrating because it sidesteps all of the important practical issues by defining them away. It implies that the causal relationship between the hotdog and the representation of the hotdog is unimportant — as long as the representation law holds, it doesn’t matter where the representation came from. So basically symbols are defined as the things that do what symbols should do. This facile embedding of designation in compartmentalized encode and decode procedures trivializes the process — for evidence, notice that AI researchers talk a LOT about T and X and the symbol structures themselves, but rarely about encode or decode. Still, having this representation law as a starting point is better than nothing!
Of course, these early steps (however profound and crucial), as embodied in their shortcut Viewpoint 2 form, are not The Answer; mistaking this path for the destination led to an overly-optimistic implicit view of denotation as primarily residing in the relationship between things and indivisible symbol tokens. The result: brittle structures too leaky to hold much meaning.
But what of it? Taking that torch and marching with it down the road is our job, and let’s get to it. My little trail: designation follows naturally from the principle that things themselves exist only as their descriptions, so the invention of modelling substrates and simplicity-biased induction methods on those substrates is the natural direction to take. Easy to say! But if we wean ourselves off of an exclusive reliance on truth-preservation as the justification for model transformations, what can replace it? Reductively deducible laws (e.g. F = ma), certainly, but those are rare. Can simplicity + empirical correctness fill the gap? I don’t know.
In 1992, just five years after the William James Lectures which were later published as the magnum opus Unified Theories of Cognition, Allen Newell passed away at age 65, one of a hundred billion tragic triumphs for the great enemy. SOAR is the ambitious and still-active software system embodying the ideas of Newell and his many academic descendents. Check it out, it’s cool.
The other day I was walking around the house and, as happens to me occasionally, I noticed that I’m really good at it. You’re really good at it too… try it! Walk around a little bit! The smoothness of movement, economy of action, flexibility of adapting to circumstance, variety of execution, exquisiteness of balance — you’re like a ballet dancer. Stunning!
It’s a seemingly straightforward physical task, and almost all of us (including our pets) easily and completely blow away the finest and most high tech engineering on the planet. That’s always been very attention-grabbing for me. If we could even get close to making devices which move as flexibly, robustly, and accurately as we do, an industry that could rival that of the automobile would open up.
In the summer of 2000 I decided to experiment with a simple legged robot named Bing. I knew absolutely nothing about electronics or mechanics. I got the legs mostly constructed, but did not finish the robot. The geometry was awkward, the linkage mechanism was stupid, the motors were not strong enough, and the joints were poorly designed and wobbly. I had great fun, though, and learned some things. Read about it here if you want.
Years later, in 2004, I discovered the Japanese Robo-One competition and that rekindled my interest in robots that walk. Following the generally-used design ideas for Robo-One competitors, I designed and built a little humanoid using a lot of hobby servos. Again I learned a lot about electronics and mechanical things. I got Bing 2 to take a few steps and perform a few other maneuvers Here’s a couple of very short videos:
Unfortunately, there is a huge problem with this type of robot — it has no sensor feedback, so the only programming you can do is play back canned sequences of movements and hope they work properly. The severe impact of this limitation became clear to me as I experimented with the device, and I put it aside.
Another two years went by and I decided to have a go at a third generation. Bing 3 was designed to have a lot of sensor feedback — pressure-sensitive pads on its feet, position and electrical power feedback from each joint, and a “guidance system” based on a set of accelerometers and gyroscopes. In addition, I built a computer-controlled milling machine to help me build the parts. Once again, I had great fun constructing it, learning a lot in the process — my skills with mechanical and electrical stuff are now at the “not bad” level. I documented the project in a very detailed thread on a hobby robot forum.
I did some interesting programming on Bing 3, including sensor conditioning / integration, and a 3-D geometric representation of the robot’s body that let me compute things like the theoretical center of mass and whatnot. The model included inverse kinematics to help control the limbs.
But two things caused me to stop working on Bing 3: The first and most devastating — which I should really have discovered as early as my first work in 2000 — is the terrible inappropriateness of “hobby servos” for complex control of articulated robots. For those who don’t know, the interface to servos like these is to give it a desired position, and then a controller on a computer chip inside the servo works as hard as it can to achieve that position as quickly as possible and maintain it at all costs. This is a very rigid and frustrating way to control a “simulated muscle”, and I was unable to achieve any sort of fluid, efficient, or flexible motion. This continual “clenching” also caused me to ruin a couple of motors.
The second problem was the sheer complexity of the task. Controlling an articulated robot with sensor feedback is very hard. Bing 3 has 19 motors and a total of 52 channels of sensor feedback. Figuring out a walking method for such a complex device starting from scratch proved to be too difficult for me.
Now it’s 2009 and I’m once again thinking about this task. Time to build more robots. Because of the complexity problem I’m not going to build another “humanoid” though — I want to try to come at the problem from a much simpler perspective, which means starting with just a few degrees of freedom, with a goal of masterful control of a simple robotic system. I will keep the sensor-rich approach from Bing 3, but get some motors that I can operate in “torque mode” — instead of telling the motor what position to go to, just tell it how hard to push.
Additionally, since I am so interested in AI and modelling and learning these days, I want to start thinking about control systems somehow in a more adaptable and less ad-hoc way. Hopefully this change of persepective will be fruitful — instead of viewing the task as a walking problem, I will think about it as a modelling problem, which should also shed some light on the more general issues of how to build effective models.
Since Microsoft has chosen an unfortunate name for its new search engine, I need to pick a new robot name: Zing.
Stay tuned for gory-detailed build reports as the summer moves forward!
I still have some rather obnoxious and academic preliminaries to slog through before I feel comfortable moving on to the specific technology of applied philosophical engineering. Even though this process is both abstruse and fumbling, it is necessary and I can only hope somebody somewhere thinks it’s as interesting as I do.
In an earlier post, I marveled at the thinginess of the universe. But what does it mean to say that things exist? I don’t mean to question the existence of the water molecules making up an excellent tubular ocean wave; I am wondering about the existence of the wave itself as a thing. The purpose of this essay is to answer that question from what I hope will eventually turn out to be a highly practical and useful perspective.
Let’s call the appearance of thinginess emergence.
Now that word (emergence, emergence, emergence!) has a rather poor reputation these days, especially among those with a reductionist world view — what good does it do to hide actual causal compositional mechanisms behind such a mystical and impenetrable label?
Emergence, because it’s slippery and interesting, is debated academically by Philosophers, who view the concept from several different perspectives. Alas, in my view (as I have been explaining), most of the things in the universe (including concepts such as emergence itself) are fuzzy enough to defy such fine description, and so detailed philosophical analysis of this type boils down to ultimately meaningless distinctions between subjective viewpoints. Not to say that all non-mathematical analysis is worthless… but I think it usually can only go so deep before it diffuses into a cloud of imprecision and personal preference. Your mileage may vary, though, and if you’re interested, there are some relatively well thought-out and detailed modern philosophy papers on emergence. Here’s one.
Emergence could be said to be the core of the interdisciplinary science-ish field of Complex Systems because typically emergence is characterized as a difficult-to-predict result of nonlinear interactions between components. From that perspective, the most sexy and interesting sort of emergence involves the creation of “complexity” from simplicity — for example, auto-catalysis and evolution producing complex living creatures from boring chemical soup. John Holland has written an excellent book on this subject, and there are many other researchers justifiably fascinated by the complexity-generating aspect of emergence.
For me, the idea of emergence, beyond the peculiar fact of its ubiquity, is important because it is central to the way I look at minds and modelling. I don’t think this view is particularly uncommon, but I will try to explain my take on it as carefully as I can because many essays to follow will rely (usually implicitly) on this basic viewpoint:
The usual way of thinking about emergence revolves around the idea of irreducibility — for example, even though life is certainly in one sense nothing more than molecules banging together, it is impractical, impossible, or just silly to express the interesting concepts about living creatures in terms of bouncing chemicals, which makes one question the feasibility of reductionism as an ultimately fruitful methodology.
I personally am not so hung up on this point, though: “Prime number” is a perfectly valid bit of emergent structure even though it is completely and simply reduceable to specific “lower level” elements. The important thing is usefulness of the description — that is, the existence of a productive ontology, not so much the peculiar details of specific ontological properties.
Going back to some examples from previous essays, when an asteroid smashes into the moon, the crater that is formed is an emergent entity. When gravity causes interstellar glop to gather into a huge ball which gets so dense and hot that nuclear fusion commences and a star is born, that star is emergent. Democracy as a social structure is emergent from human nature and other aspects of our world.
Going further, all of the things in the universe — from bees to thunderstorms to love to death to money to the category we label “coffee cup” — are emergent.
It might seem that if (literally) everything is emergent, there is no point to the concept. But there is no emergent structure in uniform randomness (maximum entropy) nor in uniform invariance (minimum entropy). For something to fall between these extremes, it must be possible to capture the thinginess of a bit of emergent structure with a description — that is exactly the sense in which things exist: by virtue of their describability.
These descriptions are also things. That the universe can reflect itself in this way is glorious and astonishing. The fact of it is the basis for minds, and that’s why it matters.
We “see” the universe through our own descriptions — sensory descriptions, conceptual descriptions, linguistic descriptions, mathematical descriptions, logical descriptions. But the descriptions available to us through inbuilt capability and training are only a subset of the possible descriptions that could be applied to the universe. I wonder how much of the “real” emergent structure of the universe we can “see” — and how much we miss, hidden in plain sight by the limits of our descriptive ability.
I will call a description of an emergent thing a model. This leads me directly to the next steps in elaborating modelling:
These and other questions will focus the ongoing project of understanding and applying modelling, and in so doing perhaps advance the miraculous history of the universe reflecting itself in the mirrors of mind.
Postscript:
Altnough I think of emergence in this way myself, it is probably not smart to broaden the definition of a perfectly good word in this way for the purpose of conversation. Therefore I use the more standard-style defintion of with no currently-apparent tractable reductionist theory when talking to others.
More Options ...
Categories
Tag Cloud
Blog RSS
Comments RSS
Void (Default)
Life
Earth
Wind
Water
FireLightweight
Last 50 Posts 
Back