Supermodelling

It seems to me fruitful to think of intelligence as having three deeply-interrelated components: modelling, language, and “weird stuff” — where “weird stuff” is consciousness and free will and so on: the things that writers put in novels and movies to make interesting plots, and what gets discussed on mailing lists and frightens those with weak ties to reality. Much of that anthropomorphic material makes it seem like AI researchers are writing programs to create golems or voodoo dolls.

I am still fascinated by all aspects of AGI, and probably always will be, but since I want to focus on things that are not quite so broad as “AGI”, this is a good place to start chopping. Besides, I’m not really all that interested in the weird stuff any more.

I am interested in language, but not enough to focus on it.

That leaves modelling. What do I mean by that word?

Modelling is the representation of things, and methods for manipulating those representations which, when interpreted, can be put to good use. Probably the biggest such use is prediction. If the model is accurate, we can extrapolate its parameters over time to predict what the modelled thing will do in the future. We can predict the effect of performing some operation on a thing by performing an analogue of that operation on a model of the thing.

Besides prediction, there are other sorts of reasoning we can do using models — we can make guesses about the origin of a thing by manipulating models of other things (such as components). We can also model abstract sorts of things like categories by which means we can figure out how to recognize them.

And there is much more to the story, I’m sure of it. I’d like to spend time thinking about the question “What is modelling?”. It seems so much more answerable and useful than asking “What is intelligence?”.

The most interesting question of all — one that I do not yet feel qualified to even begin addressing — is how to model modelling itself. This kind of situation where questions and methods wrap around on themselves leads directly to some of the “weird stuff”… and if ever I do work up the chutzpah to study the bizarre things, it will probably be by sneaking up on the idea of modelling modelling.

But not today. Today I am curious about all the ways we have of modelling things on computers. What are they good at? What are they bad at? Why? Do the methods support construction or adjustment of models automatically or must they be completely pre-specified? How efficient are the methods? Can things be modelled using multiple techniques? If so, what is the relationship between the models?

One interesting distinction seems to be whether something is modelled from the inside or the outside. Outside models are based and judged purely on external observations of the subject. A painting is an outside model (though most good artists aspire to be inside), and a neural network (or other statistical regression technique) is usually probably an outside model as well.

Inside models address the “why” behind the subject — a prediction using an inside model is based on interactions of sub-models of things purportedly making up the subject. It is interested in causal relationships.

Outside models do not require deep understanding, focus on surface features, and can often be easily learned. Inside models reflect fuller understanding, reflect a subject’s internal structure, are much more interesting, and are usually very difficult to acquire automatically.

I am tempted to say that animal brains only make outside models; humans have some ability to form inside models. But I haven’t thought about it that much, and it’s unclear that the conclusion is worth anything even if true.

So I’m going to learn more about modelling, and probably model a few things myself.