Computational Conversations EP#1 | How do Al, machine learning and LLMs relate?

If you are data rich and understanding poor, that's the time you use machine learning. But the converse was also true, that if you are understanding rich and data poor, machine learning wasn't the way to go.

Hi and welcome to our podcast. This is a new departure for us. We're testing out whether we can talk about things that we have talked about for, in some cases, up to 30 years working together, but do it on video as a podcast because it may be interesting how we discuss and argue things and we kind of know our views, but then we don't know our views on various topics. We really want to start this one with an explainer, and I was going to ask John to kick this off. How do AI, machine learning and LLMs relate? Are they the same? Are they different? What should we be thinking differently? Often these words are used interchangeably, but I don't think they ought to be.

Okay, well, it's a good topic, so let me start with. Hopefully I can just pick one off straight away, but perhaps you'll disagree. AI, I considered somewhat meaningless. It's a great marking term and it's being attached to every product in site at the moment. It's usually defined as computers doing things that. Or machines doing things that humans are needed for their intellect to be able to achieve. And now can be done on a computer.

And so it hits in the name, right? Artificial intelligence.

Yes. But what we mean by AI now and yesterday, it's constantly changing. So when mechanical adding machines came up, I'm sure if the term existed then it would have perfectly applied that. The idea that suddenly there's something that, like adding that only humans can do can be done by a machine. So it's an artificial intelligence. So it's a constantly shifting goalpost where it's all used just to mean the latest stuff. Because we've been doing a. For at least since the 1940s. What we mean by it and what we dismiss is that's not AI, that's just doing arithmetic. Well, that changes over time and we'll continue.

But do you actually think that really. Because you could argue that when you really want to affix the term AI, I mean, clearly it's being used in all sorts of ways, Right? So I agree about that. But you could argue it's to do with being quintessentially human. And when you feel like somebody in. Almost obviously there's the Turing Test and other similar kinds of tests of whether you can tell something's a machine or a human. Isn't it that. That it's hitting at now it may fail or succeed and people may have fixed it in the wrong cases. But isn't that the fundamental difference between things that have little bits of intelligence as opposed to things that sort of demonstrate that they have apparently a human breadth?

Yeah, but if you look at it from my viewpoint, then what is quintessentially human changes over time because we take it for granted now that computers can count better than we can when it first happening or when computers could first play chess or could first play Go. You know, these things seem like essentially human things that suddenly this amazing magic machine can. I agree the only dismiss it now is not quintessentially human because we're so used to it.

Yeah, but is there in fact a qualitative point in the, you know, line in the sand where you say when you interact with this machine it does seem like a human and for all the interactions, not just for one very specific task like playing chess. And you're right, the people have often been amazed. I mean I'm sure this was true for previous industrial revolutions altogether, not just for ones involving the mind, but also brawn as well as brain, so to speak. But I know, doesn't this feel qualitatively different when you get to the point? I mean, and we've seen that this year but with. And perhaps we'll come on to this LLMs things like ChatGPT where you send it something and many people, at least at the outset, may not be clear whether they've got a human coming back or a computer coming back. It seems extremely unobvious. Well, I think a wide range that.

Rather depends on how you set the experiment up because if you can see it, then you can tell that LLM isn't human because you're looking at a screen, not a flesh and blood person you can reach out and touch. If you go back to my kind of counterexample when computers were first doing arithmetic, if you were sent an answer through the post that came out of the ENIAC machine, you couldn't tell whether that number was added up by humans or not. And at the time that you were amazed, you might think, well it's four digits that absolutely must be human. Added two four digit numbers together. What machine could do something that complicated? So I think if you it's constantly evolving the closer to being able to pass through a kind of human interface like real time conversation. Now I think the interface perhaps is.

More the significant piece as to whether it seems whether you can interact with it directly.

Well, that's just part of the interface. You know, in the 19th century, we didn't interact with humans a lot of the time face to face because you sent mail and it would take two days to get an answer. And so a machine that could answer in, you know, in a day would be. Must be a human if it could write a letter. So, you know, we're, you know, we get into the world where you start having kind of, you know, robotics and synthetic humans. So we actually really can't tell there's no interface barrier anymore. Then you could claim maybe that really is crossing some threshold. But then you're in Blade Runner territory rather than where we are now. So the whole act of chatting through the, through a chat interface is a way of hiding, putting the curse nut between you and the AI. So you can't tell, but if you could see, you could see the lights flashing on and off, you'd know. So I think that's just.

So you're saying that that mode of interaction is in a sense disguising the fact that it doesn't look like a human, et cetera, et cetera. Let's go back to our original explainer, which I diverted off somewhat. Diverted you off somewhat. So you were saying AI was easy to peel off as one of these things between AI, machine learning, LLMs, et cetera.

That's lack of real technical definition and abuse in the marketplaces makes that slightly different. In my mind, we're getting somewhere slightly more technical. When you start thinking about machine learning versus LLMs or generative AI more generally. And actually to explain, I'd like to go back a step and talk about modeling first, because I think there's this interesting historical arc that, that this all fits into that when we first had. Well, even before computers, but when we had maths on pencil and paper, and then that became moved into an automated way. In computing, we have models for the world. We had models for how the sun orbits the earth and when the moon will next be a full moon and things like that. We built intellectual models. And we did that through thinking about them. We as humans decided what the rules of behavior were, whether they were equations or differential equations or some other way of writing a formula or a process for calculating something. And what computers did was to raise the automation of that. So now we didn't actually have to get the pencil and paper out and do that ourselves. Suddenly the machine could do that automatically, but the rules came from humans. We said that the differential equation for projectile motion is this. The computer works out the consequences, but it's human led ideas. So then as the models get more complicated and as data proliferated, a new paradigm emerged. And that's where we get into the machine learning world, which is, what if we could take a problem like projectile motion loss, maybe something we don't understand the rules for, but we got lots of data for. Can the computer figure out the rules? So for the first time, the humans were setting a kind of framework for thinking, but not the rules, and the computer was working out the rules for behavior. So you could take some problem like can you recognize whether this is a picture of a cat or a dog, and you could build the rules without having to understand them yourself. You know, as a human, I might try and write down cats have pointier ears. But actually that kind of failed as an approach because the rules are in the real world, pointy ears as well. But I never remember that's true.

I'm not a sufficiently dog person.

Well, but you illustrate the point, which is that we as humans can't even write down the rules ourselves. We know it when we see it. But when you actually try and write down the rules for computer vision, that kind of failed because they're so hard to write general sets of rules. And having the computer crunch data allowed it to come up with the rules to solve the specific problem. Now, so in a way, that's using the computer to go one level higher from doing, executing the rules to making.

The rules move up a layer of automation, in a sense.

So the way I see generative AI in LLMs, for now at least a lead example, although I think the multimodal models are going to sweep them aside fairly quickly there, the paradigm is slightly higher level for the computer. Again, which is rather than looking at a problem and trying to have the computer figure out the rules for it, we have the computer look at the world and try and figure out rules for the world, regardless of the problem. And then having built that rule set, we can say, ah, and the particular thing I'm interested in is what's in this picture? Or, you know, what's the answer to this question. But the model is more general. And so that's why a lot of people think that it's a big leap toward general AI.

But in one point you're investing. Just to be clear, AI does not equal LLMs. So a lot of people are using these terms interchangeably. In my view, AI and however we define it, is an aim. We're trying to somehow have an artificially intelligent machine, a machine that apparently exhibits intelligence. However, you Think that is, LLMs are a process by which you may get to that which seems to have produced a lot of results which have become very apparent this year. But they're by no means the only mechanism by which you might get artificial intelligence. So that's a fundamental difference in the use of these terms, I think.

And of course behind the scenes, LLMs are machine learned models, but they've just reached a level of generality and scale that they.

So you're defining something further, which is that machine learning behind LMS is kind of the basic. What you're saying is that machine learning is kind of a basic underlying way to do things that has enabled specifically LLMs to be helpful for generating artificial intelligence style of results.

So if you look at it on a more technical level, an LLM is really not doing anything remotely like artificial intelligence as we might conceive it as humans, although we could speculate about whether we're as intelligent as we think we are and whether we're actually just running a bunch of rules. But what they're doing is a very simple task. The LLM's being given this general thing of understanding language and what words come next. It's doing no more, no less, that given a sequence of words, what are the words that are likely to come next? But the breakthrough has been that they are so good at that, that the words that come next that we could achieve five, 10 years ago were things like, well, it's about time for a noun or a verb in the sentence, and we've had eight words, it's time for a full stop. So they had very short range understanding of language. The very likely next word that came, you know, if you just had a verb, then maybe an adjective would be a likely word to come next. And what we now have a model for is how the words at the beginning of a story relate to words at the end of a story, or how words relate to a question, or how the sequence of actions that one might write down follow a set of instructions and that sort of combination of short and long range. The short range gives you the very human, like it's a proper sentence. But the long range gives things that start appearing to have a kind of narrative arc or an understanding of question and answer or instruction behavior properties. That's where this kind of emergent behavior starts happening. But in the end they're doing exactly the same model as we've been doing in machine learning for the last 10 or 20 years. And the other thing is, we reached a level higher.

We oughtn't to forget, in my view. And by the way, we should wrap this soon, but we oughtn't to forget. There will be a place for deliberate algorithmic modeling, if you want to call it that. Right. Just because there's a new kid on the block and machine learning is not new, but certainly this use of machine learning, et cetera is with LLMs doesn't mean we should completely jettison everything where we deliberately set up a model. And I think actually it'll be the hybrid of these two that's delivering many aspects of truly artificial intelligence, so to speak, as we move forward and we don't know quite where they are. And by the way, there's a sort of parallel which is that when computers, before computers came in at all, mathematics was all a lot about symbolically representing things algebraically because that was the way you could minimize calculating. Then computers came in and meant that you could calculate stuff very quickly that was cheap relatively and got much, much cheaper. So then people often forgot about any symbolic calculation. Just in numerics, everything was numerical building my plain wing profile is numerical rather than symbolic because it seemed to work much better. I think we're now in an era where you can do much, much better by doing some symbolic work, some numerics work, because the computer can do both. And I think it's the same will be the case with traditional, I don't really want to say traditional, but deliberate modeling versus sort of machine learning style.

Well, it's the right tool for the job. And I've had a line I've been using for years which I now I'm trying to update for the new LLM world because for a long time I would explain to people when I talked about projects that we would do with customers that if you are data rich and understanding poor, that's the time you use machine learning. But the converse was also true, that if you were understanding rich and data poor, machine learning wasn't the way to go. Classical modeling, the kind of old fashioned AI as we been doing since the 1980s was the way to go. So if you know the physics of a situation, use equations or whatever the modeling is, if you have lots of data but you've no idea what the rules are, then machine learning is the way to go. And now I think we've got this third paradigm which is a little bit harder to put into words. But I think it's if you can say what you want or you kind of know it when you see it, then generative AI is quite a good solution. So there are things where you could say well here's four examples of the kind of thing I want it to do but I don't really know what I mean but I know it when I see it that LLM's very good at being driven by examples or if you can describe it in the way that if you would sit down an 18 year old, perfectly sensible but completely inexperienced worker at a desk and say okay, I've got three minutes to explain your job to you. If they could get a three minute instruction then LLMs could probably have a stab at that as well. So we've now got these three competing paradigms but the start point is what's my understanding of the problem and what am I trying to achieve? Not leaping to LLMs that are the solution to everything because they're not.

Well, very good. I hope that's been helpful. Our aim to give you an idea of the difference between AI machine learning, LLMs and hope to see you back soon on a different topic that we find ourselves discussing.

See you next time everyone.