Revolution now?
Something that seems to have flown under the radar of the mainstream tech news sites was OpenAI’s announcement of plugins for GPT.
There’s been an explosion of cool stuff created with GPT and, although it’s perhaps too early for projects to be using plugins (they’re behind a wait-list), I expected to see more discussion of its implications in, e.g., HackerNews, or the big tech news sites. Particularly, there’s a point that I’ve not seen anyone make.
The main issue I have with any AI hype that goes beyond “this is a really powerful tool to solve textual problems” is that current models lack an internal, deterministic world model. Obviously there’s some sort of world model (the big parameter matrices mean something, otherwise there’d be nothing for us to be excited about right now) but I hold that this model must be symbolic, because the world is made of entities, where most of them are best modeled by discrete symbols and rules. An apple dropped on Earth will always fall down, not just with 99.999999% probability, and the process that gives us the former answer is very different from the process that gives us the later.
Without a symbolic deterministic world model, your AI algorithm is stuck in number space, not entity space, and so you can only do things with numbers. These can have very impressive results, as we’ve seen with GPT, but it doesn’t matter how big your parameter matrices are, they will always just model the probability of a word given the previous word.
One of the plugins announced for GPT is Wolfram|Alpha’s. This gives direct access to a language of real world entities which can be manipulated as symbols. Although not directly a part of GPT (it’s a plugin, after all), this directly adds real world semantics and connection to the purely statistical models, where their conjunction is something capable of transcending both.
If we want AI agents capable of reasoning (caveat), it’s not enough to rely on statistical models alone (however effective they might be — and they are). It’s very hard to reason about (but easy to compute with) , and it’s rather easy to reason with ; if it’s as easy to compute with the later as it is with the former, then the future of AI is very bright indeed.
Tags: #ai #philosophy