Business, Innovation, and Managing Life (November 1, 2023)

Hello, everyone. Welcome to another episode of Q and A about business, innovation and managing life. This is actually the middle of our annual technology conference, but it is online, so I get to also spend a little bit of time with, with you guys here. All right, let's take a look at some of these questions Mali is asking. Will startups survive using AI and ML technology? AI and machine learning technology. If so, how to compete with big industries? Absolutely, they can survive. They got to do the right things. But I think the main thing to realize is when it comes to LLMs, large language models, when it comes to other types of machine learning, they're sort of the core development of the foundation models, in the case of LLMs, the sort of core machine learning systems. And then there's the application of those things to lots of different areas. Most of the value is going to come in the application to different areas. It'll be like computers in general. Some of the value of the computer industry is people making the hardware, people making operating system software. But an awful lot of the value is in the application layer of how does that get applied in particular areas. And so it will be with, and already is with machine learning and will be with large language models. That the kind of, the, the idea of a startup where people say, oh, there's this large language model, let's immediately do a startup that does this. Like, like for example, we saw. Well, right now we see a bunch of LLM education startups where it's like we can just connect this to sort of interacting with students and it will be good. Well, it won't actually. It's a more complicated problem and we've been looking at this quite a bit. And the number one thing to say is it's actually pretty complicated and there'll be an awful lot of technology and engineering that has to be done to make a truly usable AI tutoring system. With LLMs, just the connect together the wires and press the go button isn't going to work. What happens when a new area opens up? There are usually lots of startups that kind of frothily jump into that area where people say it's going to be easy to do this, but in the end it's not going to be easy. It never is. Happened with blockchain, happened with lots of different areas. Everybody says it's going to be easy. To make something that's really valuable is never that easy. And I think the thing that happens is sometimes the people who jump in and say it's going to be easy, they have their act together. Enough and they have enough tenacity that they make it work even when it isn't so easy. And others are just like, we thought it was going to be easy. It's not easy. Oh, my gosh, it's collapsing. We have got to give up. So I think that's the thing to say about that. Now, in terms of the extent to which it could be the case that a small motion on the part of a big player in the field kind of squashes everybody in their path, I think that is becoming decreasingly an issue. I mean, yes, it is true that you're running LLMs on other people's servers for now. I think the actual core LLM technology will become more and more generic and more and more something that you could in principle run on your own servers or at least on some standard cloud service is servers and so on. That's already to some extent true. So I think the risk of, oh my gosh, we're going to get blown up because somebody's going to change the rules how to use the LLM, that's a decreasingly significant risk, I think. I mean, there may be some things that will happen even in the course of the next few months on that, but I think that will be a decreasing risk in my view. I think the thing that is, you know, there's a certain frothiness risk of just people thinking things are easy and running around and saying, we can do this and that. And the other risk is lots of venture capital going into that field because people are saying, it's going to be easy, it's going to be amazing. It's, you know, it's going to be different this time. We're really going to be able to make a billion dollars in three weeks type thing. Well, you know, it turns out these things, they're always a little bit different than last time, but there are a lot of things that are the same. And, you know, I do think that the. Well, it's sort of interesting to see because you hear about a lot of, oh, my gosh, this company that's, you know, two kids and a dog just got $200 million in some or just got valued at $500 million. And it's like, usually the true story is a little bit more sober than that. It's in everybody's interest to make the sort of the public story be as amazing as possible. But the true story will often be less amazing, even if it is still kind of irrational. And many of those bets, sort of throwing large amounts of money at small Teams doing things, many of them will not pay off. You mostly won't hear about those ones. You'll hear about the 1 in 50 that does pay off. And then that fuels a whole cycle of kind of, well, we should do this both on the part of entrepreneurs, on the part of investors. Look, this one paid off. Even though it was 1 in 50 and the aggregate was a total lose, it still looks so sparkly that people want to chase after it. I've sometimes thought that that was partly a feature of, and I say this with all sort of positivism, a feature of the American character, so to speak, the kind of frontiers type character. One's always hopeful. And there are other parts of the world where people are much more dour and much more cynical about what's going to happen. Whereas I think America tends to be the. The has been kind of a, a land of hopeful, so to speak, where people are always like, you know, look, we can see this one great example. Let's go, let's go chase it as well. Let's see. Brady is asking, how often do you find yourself stuck on what to do next? How do you decide on what project to move forward with? Well, so there's several different cases. One case is there are no projects in prospect. Another case is there are too many projects in prospect. And for me, at least, there's a ripening that happens with projects. I imagine a project, maybe it's decades ahead of when I can do it. I imagine a project and I gradually start accumulating knowledge about it. I start accumulating capabilities around it, and then at some point it ripens to the point where, yes, there's a project I can actually do. And sometimes that happens very gradually, that it sort of gradually gets to the point where the world has moved in that direction or there's starting to be enough kind of capabilities to do that and so on. Sometimes it's more sudden. And once the project is kind of ripe, the question is, should I do it now? And actually right now. So I have had in the past these periods of time when I'll do big projects. They last a couple of years, maybe longer, and then they'll be done, finished, or at least they'll be in maintenance mode of some kind, or they'll be delegated to other people primarily. And then I have sometimes described myself as having interproject gaps where I haven't yet locked into the next big project. I used to have interproject gaps. In these years, probably the last decade or more, I really haven't had interproject gaps because there's so many projects bubbling up that I want to do. That's kind of like there's never a time when I'm, like, looking around and saying, hey, what am I going to do next? I have to say, usually the moment of starting a project I find a lot of fun. I find it. Well, I could find it quite daunting because it's like I'm making decisions that are going to have this big effect over the next several years of my life. But somehow I think I'm enough of an optimist or something that I just am like, well, I'm just going to go forward and the project's going to get done real quickly. And these aren't going to be these sort of mega decisions that actually they often turn out to be. But right now I have multiple projects that are kind of vying for my attention. Some of them are kind of more, I can get this done. Kaboom. It's rather finite size. Some are much bigger, Some are much more speculative. Some take me off in directions very different from the ones I've been in so far. And I'm actually quite kind of thinking right now, oh, I can tell you my project. I mean, I've got this thing to do with our physics project that I call observer theory. And it's kind of an important thing. It's an important piece of abstract science. It will have many consequences. I'm not sure how well people will understand it, but for me, at least, it's very important to work it through. It is also something where it has been gradually ripening, and there's a part of it that I understand fairly well. There are other parts that maybe I can factor off and do later. They don't understand so well yet. And maybe over time I will understand them better. But I may not get to that understanding unless I go through and try and systematically write down what I already know. So that's one project. There's another project. There's some other kind of more expositional projects that I have that are seemingly quite small that I think might be useful because they will help solidify my understanding of some kinds of things. There's also projects to do with figuring out implications of our physics project for practical cosmology, astrophysics, quantum mechanics. These, to me, are rather attractive projects because I think that the thing that really will kind of propel our physics project forward at a much more rapid rate is having real experimental evidence that, for example, evidence of the discreteness of space, even effects where you can just say, go look for this effect and where, you know, there's a definitive thing to do. And the problem with that project is I haven't done kind of connect to real physics and you know, work out the details of exactly what angular resolution this telescope needs and what the rate of this thing is. I haven't done that kind of physics in a long time. I only, I've only done that kind of a, in a scattered way. I, I, I think I used to be quite good at it. I think I still know how to do it. And I've got, over the years, I've obviously built tools in Wolfram language and so on that help lots of other people to do those things. So, but on the other hand, I feel like the details of what has to be worked out there are things that I probably shouldn't do myself. I probably should delegate them just because I could do them. They might be quite fun. But it's kind of like writing code. It's something which I can do it, but I don't think it, once it's defined, it doesn't necessarily need me to do it. So I'm kind of trying to figure out how I get the right kind of team and people and so on to do that project probably, rather than diving into mostly doing it myself. And then a big project that's kind of looming. Well, another project about the physics project has to do with this thing we call infogeometry, infra calculus, kind of nailing down the formal structure of the physics project better. That's a project other people are working on. Hopefully, hopefully that will make good progress now. And then another big project that's a looming project for me is writing this big book and course about computational thinking, which I think is pretty important. I think it's important for the world. I think I have sort of, my conception of what it is has kind of whittled itself down in a sense, or expanded itself up to the point where I'm probably the only person who can write the thing that I imagine. And although it is in a sense a service project, I had good experience this year with my ChatGPT explainer project, for example. And so that's sort of a motivating force behind trying to really do this exposition project, which I think again, it's going to be important for kind of defining how people think about the world computationally. So that's the, that's, that's my own personal story of projects to do now for other people. It's so hard because you know, I've had a lifetime of kind of learning about myself and learning how I do projects, learning kind of what Runway I need to be able to do a project of a certain size. I know, for example, for me, I'm really a sequential project doer. I really. It's very difficult for me to have more than one project that I'm really intensely doing. I mean, I can be managing and watching lots of other projects, but in terms of my intense focus, it tends to be one project at a time. And then there's the. You know, what happens is a certain Runway. Like, I have two months before I am distracted for a month, let's say. And, you know, is it going to fit in that time? If it's not, and I'm going to have it, you know, I'm going to have to come to a screeching halt and restart it later. For me, personally, that's very difficult. For other people, that's not so difficult. But I think. I mean, in terms of choosing what project you should do, I always feel like you get. I don't know. As I try and internalize what's involved in a project, I either get more excited or less excited. And there's a point at which I get so excited, I'm actually going to do the project. And there's a point at which I'm kind of like, well, I don't know. I don't really feel like it right now. Now, having said that, I've kind of defined in my life a whole bunch of projects to do. And I have to say, every one that I haven't done, I've kind of regretted not doing. And I've often come back years later to finish it. Like, I just worked on a project about the second law of thermodynamics that I started 51 years ago. Now. I just worked on a project about recursive evaluation of expressions that I worked on, let's see, in 1980, so 43 years ago. And I just am working on a project that was in a footnote that I wrote in the early 1990s where I really wanted to work it out in more detail. And I'm now getting to do that. So at least for me, it really is, you know, whenever I put a project aside, whenever I haven't taken the project as far as I feel I could take it, that always kind of sort of, it eats at me, and eventually I try and do the project. At least that's true of intellectual projects. That's not as true for me of organizational Projects. It's kind of a thing where it's like, well, I could do that, I could organize this, but it's like, well, let me not do that. But for intellectual projects, I tend to have this feature that I really want. The eye that I more or less drew but didn't have the dots on top of. I come back and put the dot on top. Even though my timescale for doing that seems to be in the many decades. Jamie is asking, how do you prepare for conference talks? Do you ever get nervous or stage fright? I just gave a talk for our technology conference, the keynote for our technology conference that I don't think was my best talk. And here's the thing that happens with talks that I give. I've gotten experienced enough now that many times I really am not preparing at all. I mean, many times I'll maybe have one or two key ideas that I think about maybe in the five minutes before I'm giving the talk that are going to be sort of the anchor for what I'm talking about. But the rest is really off the cuff. I mean, very much like these live streams which are 100% off the cuff and perhaps these live streams even help me to get better, I hope, at doing kind of off the cuff speeches and so on. I found what I do when it's off the cuff is quite good usually. Well, it's quite good if it's a match for the audience. And one of the reasons for doing things off the cuff is the sooner before the talk you are kind of thinking about what you're going to do, the more you know about what the audience is like, the more you know about what your mood is that day, what your, you know, what you feel excited about that day. The more you, the more you kind of have a way into the talk. Now, I have to say, generally what I find is once I have done the first two or three minutes of a talk, I have defined a, a kind of a. Not a theme, but I've defined a way of doing that talk which then I can carry through for the rest of the talk. And if it's the right way of doing the talk, if it's the right kind of approach to the audience, and so on, it works well. If it's wrong, it's bad. But again, the more I know, the more I can get a feeling for the audience, the more likely I am to have that match. And I find when I'm giving a talk, particularly in person, that when there's an audience there, you can kind of Sense the reaction of the audience. At least if the environment isn't too big and blobulous, you can kind of feel the energy in the room, so to speak. And I have to say that I know that if I feel positive energy in the room, the talk gets better, talk gets more energetic. I kind of am more alert and really, really paying attention, and it gets better. So that's something which, again, if I prepared exactly what I'm going to do, I don't have that opportunity. Now, what I tend to do as a practical matter is I'll usually have a slide farm, I usually call it, of a big collection of all these different slides I've used for different purposes. And I'll usually have those on a secondary screen. And when I travel around, I'll use a laptop and I have a, a little attached screen that I can attach that I'll use as my slide farm. And then I'll project from the main screen of the laptop or whatever. And then when I want to make the next point, I'm looking around in my slide farm and I'm finding the slide I want to use. So I haven't put them in a definite order. Now, having said that, a large fraction of talks I give are done either, well, purely verbally or they're done with live demos where I'm actually typing things into Wolf Notebook or whatever else. And those, again, I'm not preparing those, I'm doing those in real time. I think that's a very useful form of communication that kind of shows, you know, an actual person actually typing can interact with computational language. It's not just a pre prepared, pre canned kind of thing. So anyway, the, the, that's my typical mode of giving talks. I have to say, I've been doing talks and public speaking really kind of all my life. And so I don't think at various times when I first was wearing various biometric things, I thought I'm going to figure out does my heart rate go up when I walk in front of a big audience? And the answer is not really. And it goes up if things go wrong in the, you know, some demo I'm giving. Things like that, but not the audience itself doesn't do that. It only, it only does it by virtue of me walking around on a, you know, exercising myself walking around on a stage or when I get really energized when I'm, when I'm giving one of these talks, I think it's one of these things where somehow the only thing that threw me for a loop and back in years ago was when you give these talks to large auditoriums and it's completely black and you're out there, you know, there's a thousand people there, but you can't see any of them at first. When that first happened to me, I found that very confusing. And things happen all the time. When you actually give talks in person, crazy stuff will happen. Like I was just giving a. Well, doing an interview discussion with some people at a. Oh, gosh, when was this? Two weeks ago. And new and different thing happens. The acoustics is so bad that although probably the people in the room could hear fine, I couldn't hear what the person who's asking me the questions, who's sitting, you know, five feet away from me was saying. So that was kind of a new and different. Wait a minute. I got to deal with this. How do I deal with this type situation? I have to say that sometimes it's very rare when I will prepare talks. For example, the talk that I gave this keynote that I just gave, I did have to prepare that because there's so much material that I have to organize it. Otherwise there's no way I can kind of hit all the points I want to hit. And that's always very challenging for me. At least when I have some notes about what I'm going to say and some kind of free form, the interface between those things is bad. And so for me, it's much better to just have pure, very succinct notes and everything else is filled in on the cuff, off the cuff. But rather than to have something where I've got little fragments that I know I'm going to say, and then the rest is off the cuff. I have at times in the past, I haven't done this recently. I have at times in the past kind of written out the first three sentences of a talk just so that I get in the right groove at the beginning of the talk and then it's free form after that. I would say that the one exception to all of this, I did this talk for TED AI a couple weeks ago where I'm kind of following the TED rules. I'm really pretty bad at following these kinds of rules. But I thought, what the heck? It's sort of a useful thing to have a version of the story of what I'm doing crushed into 18 minutes. To do that, I had to write the whole talk out. And I know some people kind of go to the trouble of, you know, memorizing the talk for these TED talks and so on. I'M not going to do that. I frankly don't have the time to do it. And also the talk is actually probably even too complicated to easily be able to memorize without really a lot of effort. But that was a talk where I did write out every word and I have no idea what the video looks like. But I felt like it was okay when I got to kind of essentially read the talk because I couldn't really do anything other than that because if I had had some sort of places where I wasn't quite reading the talk, I would have said things which were kind of over saying what I was going to say then. And then I wouldn't have been able to fit this whole sort of set of puzzle pieces together to fit it into that time constraint. But that's generally what I do now in terms of other people giving talks and so on, I think that the people can get very nervous about giving talks. I think it's more the look. All these people are looking at me. I should feel very self conscious and I suppose my point of view is, and perhaps it's an unreasonable one that I'm kind of, it's sort of their issue that they're looking at me. It's not my issue what, you know, what they might think, so to speak. It's kind of like I'm just going to do my thing because anyway, I'm really kind of interested in saying the things I want to say and it's, you know, their experience is up to them. Now, having said that, I will tell you one slightly funny story, which is when various times in the past I've been kind of exposed to photo shoots. Like, you know, oh, we need these pictures for some promotional, this, that and the other thing. There was a period in my life of probably 15 years or more where I just could not be in a photo shoot and not have the world's most forced expression. And good photographers have a whole patter that they're kind of using to try and get the person like me who is otherwise looking incredibly stiff and self conscious to just have a natural expression. Well, somehow that got solved. I think maybe, kind of, maybe it's because of the dynamics of how picture taking works and the fact that there was a period of time when if you were having sort of the professional, you know, media journalism picture taken, there were these bright lights and they would kind of make me feel like they would start feeling quite disoriented after a while just by being exposed to these bright lights. And that of course changed with high dynamic Range, you know, light sensors and so on. But so, I mean, I, you know, having said that, I don't get kind of weirded out by giving talks in public. I did have this personal glitch of having a really hard time with photo shoots. So that gives me perhaps more empathy for people who have a hard time with giving talks. And I would say that the thing that is difficult to do is if you're kind of nervous giving the talk and you write it out and you stand there and you're reading my name is so and so, and I'm going to give a talk about whatever, it really doesn't come across very well. It takes some effort to get to the point. And I don't know, like this recent TED talk I gave. I haven't watched the video, so I don't know how well or badly I did it, but somehow, even though I'm reading the talk there, I'm kind of saying it more or less the way I think I would say it if I wasn't reading the talk. Talk, so to speak. But there's a tremendous tendency, if you have a talk, to read that. You really read it in a very robotic way. And that's pretty deadly usually. And in fact, often one is better off, I think, trying to more wing it than to be in the situation where you're kind of reading robotically. But I suppose the question is really, how do you get to the point where you don't feel nervous about, oh, my gosh, something could go wrong? And I think somehow it's really a matter of confidence and, you know, and. And something of a matter of how you feel psychologically about the audience and how you feel about your. What you. Why you're doing what you're doing. And I think the. So I'm not sure that's great advice. I think people always have these things about, you know, sometimes another thing that can happen to me in giving talks is if the audience really obviously isn't getting it, that gives me a hard time. It's like, okay, how do I redirect this sometimes? I remember the first time, years ago now, where I gave a talk to a group of middle school kids. It was a flaming disaster at the beginning. I could tell they just weren't getting it. It was the wrong talk. So I kind of just said, at some point, okay. I said, let's do something different. And I just started a completely different talk, which worked great. But it was a case where I think that was probably the better move than to try and recover the Thing that was just going really badly. But sometimes you don't have that option because you're kind of partway in and you're realizing you've lost people and where do you go next? And that is challenging. And. It always helps if you have some props or slides or something or a demo to give where the demo is really cool and people can concentrate on that. I think that another thing to realize when you give demos, real time demos, they sometimes go wrong and you have to kind of. I would say that particularly I'm thinking of Teo Gray, who's a longtime person at our company, who developed the notebook front end for orphan language back 35, 36 years ago. I have to say he's as a demoer of user interfaces, it always used to amuse me back in the day, 30 years ago or something or more, he would be demoing things and he would have this little way of wiggling up and down on the document he was demoing. And why was he wiggling up and down? Because he thought there might be some detritus left on the screen by bad bitmap operations. And by wiggling up and down he was washing that detritus away. And so there are all these little tricks of demos like that, which I have to say I don't really end up using much. But it's kind of a thing where, you know, the main thing that happens if something goes wrong and you're like freaking out about the fact that the thing went wrong, then the audience is going to freak out about it too. But if you're like, eh, it went wrong, okay, there's a bug, let's report it on we go. Like for example, one thing I really don't like actually about demoing LLM related stuff is that that really is not reproducible and what happens will be different each time. And that actually is quite hard to demo. I was just realizing that today that I don't look forward to LLM demos because I look forward to demos where I'm just typing stuff and interesting things are happening. But these LLM demos, it's kind of like I'm typing stuff and it might be interesting or it might just go really splat and I might not be able to make the point I wanted to make. And it's not just a simple fix of let me change this word because I don't even know what I have to change. It's kind of a complete crapshoot. So I don't like that as much. Okay, Nick here is commenting about Deming saying that, that asked about a speech, responded, I know what I said, but I'm not sure what they heard. Ah, yeah, well, you know, to me, I think particularly when I'm trying to explain things, it's kind of my responsibility to try and get the people who I'm explaining it to to understand that's kind of, if I didn't do that, I can't hide behind. Well, they just didn't, you know, were, you know, too clueless to, to get what I was saying. I think it's my responsibility to make sure they understand it. And that's so. And, And I always consider it something of a puzzle. You know, you've got something perhaps quite complicated to explain. What steps do you have to go through? How do you build the different layers to get to the point where you can explain the final thing? But I think for me, it's a very interesting exercise. It's one that helps me understand much better. Bit is commenting that they bring their cats to talk so they can look cute if the talk bombs. That's amusing. It's certainly good for a, a zoom talk or something. I have seen a few times people I've been talking to where a cat appears. And yeah, you're right, that's a real win, I have to say. Early in the pandemic, a woman I was working with quite a bit had a boa constrictor. No, no, it was a python. Sorry, a python. And I like kind of all sorts of animals, but it was quite a. Kind of interesting to see the, the python comes on the screen and then it's wrapping itself around her neck and it's like, I'm, I'm, you know, is that really safe? Is it going to, Is it. You know, it's not going to get upset and I'm not going to see you kind of crushed by your neck. Crushed by the python. But then it's like, hold the python up to the, up to the camera. I want to say hello to the thing. It's kind of, kind of interesting. Unfortunately, we had a virtual bring your pets to work day at the company. I wasn't part of that, but just for folks at the company. And this particular snake led to a bit of a disaster in the bring your pets to work virtually. And the people who definitely didn't like snakes who ended up getting exposed to that. So that was a bit unfortunate. Nick is saying. Any thoughts on what a leader or manager can do to support team members to learn to manage stress? Well, the question is, why are you Stressed. So, for example, you might be stressed because things became very urgent. Why did they become urgent? Well, because you didn't plan and then they became urgent. It's also worth saying that for some people, many people actually, a certain amount of stress is actually a good thing. It's something where if it's like, oh, you're just lying around and there's nothing much going on and there's no importance to what you're doing, it's like, hey, why am I here? Whereas if it's kind of like this is important, really, you have to get this right, it's building something. And that creates a bit of, kind of, I would say, sort of positive stress of this really matters, I've really got to do this, et cetera. That's not a bad thing, I have to say. I think that often the greatest stress comes from things like, well, what does it come from? It's different for everybody, but it's things like, well, the people, where it comes from, I'm out of my depth and I don't really know what I'm doing and help and so on. That's a place where management can really help because it's like, do you know what you're doing? Do you understand what you're supposed to do? Have you been given a project where the steps you've been asked to go through are ones that you can actually execute and. Or have you been given a project where you just can't execute? Those steps are too big, it's too difficult, you'll drown on the way. Getting from one stepping stone to the next type thing. And that's a management issue to make sure that people know how to get started on something. The project has been right sized for that person and they know how to get started. I think that other times types of stress come from uncertainty, like, is the boss going to like my project or is the boss going to be in a bad mood and going to hate my project? That for many people is quite stressful. If you care about the project and you care about what's happening, to have that uncertainty is stressful. And I think. Being in a situation where you have direct access to feedback on your project is a good thing. And for management to give that direct feedback, I think is a good thing. Now people will criticize me for being probably too direct. If I think the thing is garbage, I'll say so. And if I kind of think the person did no work or whatever, I'll say so. And that has the feature that at the moment, when you're saying it, it's kind of awkward because it's like you're really attacking this thing. You know, I try to attack the thing, not the person, so to speak. But nevertheless, if it's a bad piece of work that somebody spent 15 minutes on and they didn't get it right, they didn't think about it, I'm not going to kind of let them off the hook. Now the positive side of that is that means if I say, hey, that's good, it actually means that I'm not just going to say that as a way to kind of make the conversation pleasanter, so to speak. And I think that's a useful thing that management can do is say what you think in a way that isn't. There's no point in kind of destroying the self confidence of the person. But on the other hand, you need to explain what you need to explain with enough clarity what you thought about what they did and help them to do it better the next time. I think being saying, look, I mean sometimes you kind of know this is never going to work. You got to move to a different job. This is just not going to work. You're not going to get there from here, at least not now, without more experience or whatever else. But sometimes it's just like you did. This wasn't like, for example, I had the following issue yesterday. Very good team people where there's a bunch of senior people in the team and they've hired a bunch of more junior people. And we were doing a monthly review of that particular project and the senior people were saying, oh, I didn't get much done because I was training the junior people all month, more or less. And I kind of gave them a hard time about this because I said it is a tricky thing. Well, I gave them a hard time because not a lot had gotten done on this project. That was the number one reason for giving them a hard time. But then I wanted to be constructive because I want them to do better and to deliver more next month and so on. So we really talked about this trade off between if you're a senior person and you know how to do something and you've brought in various more junior people and it takes you a certain amount of time to explain what to do and it's going to take them five times longer to do that task than it's going to take you, the senior person to do it. When do you do it yourself versus when do you try and get the more junior people to do it? But. And how do you Mentor those more junior people. And how do you balance things that you can do five times faster and just doing some of that with the management of the other people. And one of the things, for example, in that case, one thing I know many people find difficult is switching from individual work, individual sort of creative work to kind of management type work. And different people do this differently. Whether you have a very boxed in schedule of this time to this time I'm doing management type stuff. I have to say over the years for myself, I've gotten vastly better at just being able to switch sort of instantly between one kind of work and another kind of work. I had a hard time doing that for a long time. It took me a while and I had to kind of get myself into different things. If I was going to write something, for example, that's still true. If I'm going to write something, I usually will have a 10 minute period where I'm kind of getting myself oriented towards, okay, I need to be in a writing mode, I'm going to do that as opposed to I'm in a management talking quickly about projects kind of mode. Let's see, Nick is asking a lot of large companies that earn high profits have been scaling down their workforces. What are the sort of common drivers and motivators between these scaling down trends? Well, there's a certain amount of craziness in the tech industry because some companies have bulked themselves up with people, often very well pedigreed people, very expensive people, and they've just got lots of them. And there were times in Silicon Valley particularly where companies would be hiring people not because they wanted the people themselves, but because they didn't want their competitors to be able to hire those people. And so that led to very, very bloated workforces where it's like sometimes I look at what some software companies are doing and it's like, and I know they have, you know, 10,000 people working on something or in some particular area, a few thousand people. It's like I have no clue how one would use that many people to do this particular thing. I just don't know how you would spend, find the work for those people to do. And it's mysterious to me, for example, but sometimes you get into this dynamic where sort of in a company, particularly a large company, there are many layers of management where people kind of get the feathers in their caps by having more people that they are managing, not necessarily by the productivity of what's being made, but by like I've got a team of 50 people is much more impressive than I've got a team of five people. They just happen to be producing amazing stuff. And so there are these kind of somewhat perverse incentives that develop in very large companies. By the time if the people who are sort of really setting the strategy of the company have visibility to the trenches, so to speak, where all the work is getting done, or work is getting done, that tends to not happen as much. But by the time you're at 8, 10 layers of management, it's very easy for that kind of phenomenon to happen. And so then this question of, well, should we downscale, downsize? It ends up being often, well, we can kind of see our profits may go down in the future. When you've got these people, they're going to cost you a fixed amount of money. Let's let some of them go, because then it will be cheaper. And that's a very. This question of what you see, projection wise, for what your profitability is going to be going forward. That's always a tricky thing. And I'm not sure people have gotten that right necessarily. But I think it's often the case that once you start realizing, oh, I got too many people in this area, there is a tendency to just say, let's cut that area and go on from there. And it doesn't help that in the worldwide labor market there are such big disparities in costs in different places. And I think, for example, Silicon Valley, where a lot of tech industry is concentrated, is this place of just exceptionally high costs for everybody. Cost to have an apartment, cost to pay an employee, etc. So. And that, you know, that's another driver of these kinds of dynamics. But it's always, you know, I'm happy to say so far he says, you know, I don't want to jinx it, but in the last 35 years, we've never had to have a kind of, you know, at our company, we've never ended up in the sort of big layoff people type mode. Sometimes we've been like, let's put the brakes on hiring people. So sometimes put the brakes on refilling positions. But we haven't sort of had to come to the screeching halt type thing. And as a person who feels responsible for the people working with them, that's a good thing. And I consider it sort of part of our responsibility to try to make sure that doesn't happen, to try and make sure that we have kind of a stable environment for people to work in. Not all companies think that and not all people who do management think that, I mean, to some extent it's kind of like, well, we've got this blob of people. We can, you know, drop this blob of people. We can add this blob of people. It's a much more sort of, in a sense a much less human calculus, so to speak, of what to do. But different people feel differently about this, I think. You know, I suppose some people, myself included, tend to be kind of people oriented managers, CEOs where I kind of feel I'm responsible for the people rather than just the people are a plug compatible blob that does a particular thing. I was asking what's my favorite aspect of being a CEO? What's my least favorite. My favorite aspect of being a CEO is I get to have ideas and get them implemented. That's my favorite aspect. It's kind of like having this, this great machine for taking ideas I have and, and getting them executed. I also enjoy working with the people that, that we have. Just, it's a, it's kind of a, a, you know, if I was sort of all on my own or something, it'd be very, very different. It's, it's, it's very nice to have a group of people all sort of pushing in that direction. I think that you ask what's my least favorite aspect of being a CEO? I would say that, that for me, some of the sort of grindier. Well, I'm more interested in the product R and D strategy side of the company than I am in the kind of the, the details of the commercial side of the company. I would say that. But you know, as a CEO, you certainly have to pay attention to those things. I would say that, that sometimes there are, there are situations where you're, you know, there's some business development that's going on and it's like you're hearing about this and it's like there's a certain. I don't mind figuring out the strategy, but it is very sort of tiresome when it's kind of, when there's something going on and it's kind of like it's going to come back every week. It's going to be the next micro step and it's very frustrating to kind of watch that process. I would say it's a minor kind of beef about the story of CEOing. Another thing that happens with CEOing is you'll have, and this is a question of the culture of a company, but one thing that can happen with CEOing is you spend all your time as the complaints department. And that people, whether it's people from the outside world, customers, whether it's your internal people, they're always complaining, they're always telling you, this person's doing a bad job, this is the wrong thing, this is whatever. And I would say that for me, our culture, company culture, is probably such that we don't get as much of that as some people get. But the question is, are the complaints valid? Maybe they are, in which case you want to hear about them. Because the worst thing is for nobody to tell you that something is wrong. But then you have to kind of, you know, you parse through it, and I don't mind doing that and try and understand what's true. Sometimes you get frustrated because people are really kind of hiding stuff. And it's kind of like you really have to drill. You know, why did this happen? You know, who was responsible for that. And when you find out who was responsible, you kind of have to be civilized about that. Unless the person just did something totally outrageous, in which case, well, then you found out that out. But most of the time somebody just made a mistake. But it's really worthwhile to know where was that mistake made and how was that mistake made? Like, for instance, I found some goofy thing on some website two days ago, maybe yesterday, and it's like, how did we make this mistake? And turns out there was an answer, and the answer was kind of goofy, and somebody was trying to do the right thing, but they ended up doing the wrong thing. But what it really reveals is there's a process that was missing to check that the right thing was done. And so now we can put that process in place. And if I hadn't drilled and said, how did this mistake actually happen? We wouldn't have discovered this process failure, and we wouldn't have been able to build something around that would actually permanently fix it, so to speak. So I think that's. So. I don't know. I've been lucky enough to mostly CEO a company that's been doing. That's been doing well. I think if you're in a situation where you're kind of things are going badly and you're having to let people go and all those kinds of things, that's probably very unpleasant. I've fortunately never been in that position. I think that. I suppose if you ask. One of the more difficult things, at least for me with CEOing, is when you don't know stuff, when things happen in the company and you don't know it and As I say, sometimes there are all sorts of reasons of kind of why. Well, that thing, nobody's really coming forward, nobody's being upfront about what's going on, whatever. My all time, okay, I tell the story internally within our company way too often. This is a story about Wolf Malfur, actually, where I was really pushing, pushing, pushing. The website has got to be as fast as possible and it was kind of slow. And this went on for years and people would tell me, oh, it's your network connection. I was like, no, it's not, it's not my network connection. Sorry. And anyway, it turned out in the end that some programmer, through a piece of laziness had put in a pause somewhere into a piece of code to prevent some race condition from happening. And even though I had pushed pretty hard over the course of years, this was never found. And that was sort of, as I told everybody, sort of bad Pavlovian conditioning for me. But because when these things happen now I am much more likely to just say, show me the code. Let's drill all the way down. Let's not let there be something where somebody just didn't tell me because they didn't turn over that stone because nobody was really responsible for turning over that rock, whatever it is. And so nobody did. And it's kind of like that can be a frustrating thing when there's sort of stuff that's happening and it's like, oh, well, this was kind of goofed up, but you don't really see what's underneath it. You don't know why you can't drill down. And it takes a lot of effort to drill down. And sometimes it's a complicated trade off because, you know, it's going to take me hours of work to drill through this. And in the end you're going to find out what actually happened and then you're going to be able to potentially fix it. And that might be valuable or maybe sometimes it will just not be that valuable. It'll just be somebody made some stupid mistake and they made that mistake and big deal, and there's nothing really to learn from it. Let's see, Mac is asking, they, like I advice. They'll retire in three or four years. Do you think it's too late to start learning machine learning, data science, AI, all these kinds of things? They're saying, I was thinking of dedicating part of my day to streaming as a hobby, something to keep my mind active. I have to say I'm ancient enough that lots of people my age are retiring And I'm kind of horrified by the fact that all these people I knew and, you know, elementary school, high school, whatever, it's like they're saying to me, when are you going to retire? And it's like, I don't feel like retiring. I'm doing the things I like to do, and I hope to keep doing them for as long as possible. You know, I have to say, I think retirement is a funny kind of concept because in a sense, it's like, well, you know, you can get to the point where you just can't do the things you were doing for physical or mental reasons, but that's sort of a different story. But if you're kind of like, well, I put in my time and now I can retire, it's kind of like, well, then what do you want to do? You should do. Life is more fun, I think, if you're doing the things you want to be doing, and if you're lucky, the things that you do for work, so to speak, align with the things you want to be doing. If that isn't the case and you need sort of the. The hobby of there's something else which is the hobby that you really want to do. The real trick, I think, is can you align that thing you really want to do with the thing that is sort of a thing that you spend your time doing that might even be considered work and might even make a living for you? So I would say that in terms of. One of the things that it's often said is retirement is the most dangerous thing for one's health, that people who have kind of this sort of flow of, I'm trying to get this done, I'm trying to get this done, I'm trying to get this done, tend to be sort of better, sort of more vigorous and so on than people where it's like, I don't really have anything to do kind of thing. Sometimes people will say, I'm working so hard, I just want to, you know, chillax and do nothing. Not me particularly, but. And why not me? Well, not, I think, because I'm built any differently from other people, but more because I'm doing things I like to do. So if for me, it's kind of like, well, what would I like to do if I'm. What do I do to relax? Well, I do something which to other people would be considered that's just work, but I like doing it, so that's what I'll do to quotes, relax, so to speak. And I think that's a good Thing, if you can set it up. And for the person asking this question, I think doing streaming is a fine thing to do. Obviously, I have fun doing it. That's what I'm doing right now. And you should think of that as the thing you're doing. If you want to think of it as a hobby, great. But. But if you think of it as the thing you're doing, the goal is to have something you want to do that you spend much of your time doing. Whether things align that you get paid to do it or not is a different question. I mean, for example, I do all this live streaming because I like doing it. It doesn't make. I'm not doing it for a living, so to speak. I'm just doing it because I find it interesting and I feel good about the fact that hopefully I'm providing some useful information to people and that's why I'm doing it. And whether I was retired, taking a pension and doing this, or doing it as carving out an hour of the day when I'm doing other things for a living, it doesn't really make much difference in terms of the question of whether it's worth learning stuff when you're kind of towards the end of a career. You know, I think one of the important things about particularly modern careers and sort of people in general, I think keeping on learning throughout life is really worthwhile. I mean, I've been lucky enough that I've kind of been almost forced into that, although I think I would have done it anyway. Just keep learning more stuff. Keep learning more stuff because the more, you know, the easier it is to learn even more. And the more, you know, the easier it is to make interesting connections, do good things. And I think that, that. So I'm a big believer in learning more stuff now. If you're kind of learning it for, like, I'm going to learn this stuff so that I can get my certificate of whatever, so I can be a whatever and never have to learn anymore. Then clearly, if you're sort of coming towards the end of the career where you might use those things, it's not a good bet to learn that stuff. But I would say that the obvious thing that I would say is go learn. Kind of if you're interested in live streaming and you're interested in learning things, you can combine those and kind of learn and live stream, so to speak, and that becomes sort of the continuing objective. It isn't just, I'm learning, I'm learning for the next three or four years, and then oops my career has ended and that knowledge is now not useful to me. It's like if you have a sort of a plan that involves integrating those ideas, you have something which to me at least kind of makes sense. It's kind of like, do those Wolfram language live experiments or something that you want to do about AI. Do them as live streams, do them. Have fun doing that. And that becomes. Whether you do that AI thing for the sake of some particular thing that's needed in your job, whether working for a bank or whatever it is, doesn't matter at that point, because you're doing it for an objective. That's something you want to do. That's something sort of independent of the, oh, I'm now retiring from that particular position. Let's see. Okay. Nick is asking here about saying they're in the software QA and testing world and there's a challenge of getting decision makers to invest in doing testing and doing it early. You know, it's an interesting thing. Back when I started our company 36 years ago, there was this vibe in the software industry that the quality assurance folk, the testers and so on, that they were the enemy of the developers. That there was sort of a, you know, that that was the. Was kind of almost the cartoons of, you know, the QA person as the devil incarnate, so to speak, for the developer. That the QA person will be coming to the developer and saying your code is all garbage and, you know, complaining and so on. Somehow in the culture of our company, that never happened. Somehow the vibe ended up being much more. The QA group is kind of helping the development group and that the development group is kind of. And they're kind of pleased with the fact that QA is testing their stuff and coming back with, you know, with suggestions to make the thing that they're doing better. That dynamic was pretty important. It has been pretty important in the history of our company that QA is not the enemy now in terms of management and investment in kind of testing and so on. You know, it always helps when there are technical people running the company. I mean, again, I've had that situation because sort of I and our senior management are all kind of technically oriented people. And so it's sort of, you know, things that make technical sense are easy to have one understand make sense. I think, you know, computer security, like quite. Quality assurance is one of these things where until something goes wrong, you don't know you didn't do the right thing. And it's somehow you just have to have that experience base of seeing, hey, this went wrong. I mean, I would say if I were really trying to convince management of sort of why testing is important, I don't know, there's a long list of disasters that have happened in the industry from testing that hasn't been done right. But also it's a thing where the cost of losing a customer because something went wrong because of bad testing is quite high. And it's kind of like that's a thing that sort of you can rationally understand. I think more what tends to go wrong is the dynamic of the development groups who don't want the testing. They see that as being just sort of a drag on their ability to rush forward quickly. I mean, for us, I don't know, maybe it's just the culture of the company. It's kind of like, you know, we're about to put out a big release and nobody is like saying, oh, you know, let's. Let's cut the testing time down so that we can get the thing out quicker and, you know, cut those corners. Nobody. People are just used to the fact that it takes a certain amount of time to do that testing. Now, on the testing side, you know, we've obviously worked hard to make our overnight tests and so on and our test processing system as efficient as possible. So it isn't like. Let me give you a counterexample to that kind of thing. If you publish a book with a traditional publishing company, a very common situation is they'll say, give us everything for your book. The book will come out a year from now. It's like, what are you doing in that year? And, you know, a lot of it is kind of, I think, not very good processes. I know our record is. We got the ChatGPT explainer book that I wrote out in three weeks earlier this year. That's. That's a short time. Usually in the past, it's more like maybe a couple of months, three months, something like that. But that's really just time to get things printed and those kinds of things. The. I think that that is. So that's an industry where, where it's a kind of a. A rush, rush, rush, get your stuff done and then wait for us to grind through our processes. If that's the way people feel about QA in a company, it's not going to go well. If people feel QA is really efficient and it's coming back with things in an effective way, then it's kind of a. Then that creates a good sort of feedback loop and a Good vibe for the development team. And I think that's. So that's another sort of comment about that. Let's see. Well, okay, a couple of more questions then. I should probably wrap up for today. Sava asks advice on being a prolific. Being prolific or focusing as a college student? Specifically the trade off between open ended exploring and focusing, you know, well, several things. I mean, one of the things about modern studenting is people seem to be so incredibly busy and that kind of makes me sort of nervous because it's like you do have to have some time to actually think and kind of understand stuff for yourself, so to speak, rather than always being on this kind of do this, do this, do this, do this, do this. Okay. Talking about courses, for example, college is a time when you get the opportunity to just learn about all kinds of different, random things. Yes. Maybe you have some major, you're trying to get some, you know, some go in some particular direction, but you'll have the excuse to take a bunch of random courses. Take random courses, you'll never do it again. You know, at least in American colleges, different. In different countries where they're more directed. But in US Colleges, you can kind of take courses on all kinds of different things. If you're interested in something, take a course on it. It may turn out to change the direction of your life because you discover that's really the thing. I didn't know much about it, but I'm really into it. That's one thing to say in terms of I don't think in some cases people are like, I know what I want to do, it's this. And I want to just go in the shortest path to get to that endpoint. I was like that. But if you have any opportunity or predilection towards not being like that, take those extra random courses because you'll never do it any other time in your life. You'll never know about those things, probably unless you take that random course. But you have to want to take the course. If you're taking the course as a matter of kind of duty or something, you probably won't really be into it and you won't remember it. So I think one of the things is, you know, things that you're enthusiastic about, go do them. Things you're not so enthusiastic about, try not to do them because you won't have, you know, if you're like learning something, you're in some course, you won't actually get as much out of it. If you weren't enthusiastic, if you weren't into the subject matter and so on. I think that in terms of, you know, I have to say, I roll my eyes a bit at some of the things that some college students like. They. They sign up, they're freshmen in college or something, and immediately they're signing up for, you know, the consulting club that's going to take on consulting clients and go and try and be a baby consultant type thing. It's like, why are you doing that? You know, maybe it tiny bit increases your chance of being hired as an intern by this consulting firm and then hired a job four years later. But it's like, look, you know, maybe you'll spend your life as a consultant, and maybe that's a fine thing for you, but unless you're passionately keen to jump into that at the very beginning of college, it's like, why are you doing that? That's kind of why not do the things that you get an excuse to do when you're in college, of exploring lots of different things that you won't have an excuse to do later on when you're, you know, working 70 hours a week as a consultant, for example. So I think my feeling is if you've got a very definite endpoint you're trying to get to, well, then just go for that endpoint. But if you're not really sure what the endpoint is, it's really worthwhile to use college as this period of life where you kind of get to explore things and nobody's really telling you you've got to achieve this, this thing by this month, other than you're signing up for this class, do this class type thing, but you get to choose what you want to do. All right, I think I. Maybe one more question and then I should wrap up for today. Well, a lot of very interesting questions here, and I don't feel like any of them are quick to answer. I'll answer one here. Des is commenting. Think a huge amount of value of college is having informal discussions with small groups of people. And that's not compatible with being focused purely on your kind of GPA and so on. I agree. I think another thing, I've said this I think before, sort of a trick of colleges, Go talk to professors. At least half the time. They're really quite happy to talk to students and keen to talk to students. And you get a lot of value out of that. It's amazing how rarely students will actually go and talk to professors. I mean, don't go and waste their time. Go and kind of, you know, learn enough that you can ask interesting questions and have an interesting discussion and everybody will have a good time and you'll learn a whole bunch. I think that people also, the dynamic of interacting with other people in college is great. And particularly, you know, insofar as colleges sort of sought for people who will meet other people that they like to meet, find interesting and so on. That's a good thing about college. And I have to say an awful lot of people build up often lifetime networks when they're in college. I think sometimes that the extent to which people's networks are all college type thing, that's a negative statement about their ability to build other kinds of networks. I know in my life I've plotted this at various times, my acquisition of acquaintances, so to speak, as a function of time. And it's been sort of uniform except for the time, the decade when I was kind of a hermit working on my new kind of science book where I accumulated essentially no new acquaintances. By choice of being a hermit, I accumulated some children who were born during that time. That doesn't really count. But not sort of acquaintances that I just ran into because I wasn't running into people because I was kind of really, really a hermit working on this big project. But I think that it's a, for me at least it's been a very worthwhile thing getting to know more people through the course of my life as opposed to just saying I'm in college and I'm, you know, I'm, I'm thrown together with a bunch of people. These are the people I'm going to hang out with forever. You will meet interesting people, hopefully. And particularly because of the sort of sorting of colleges and things, there is a tendency for people to meet people. It's a good tendency for people to meet people who are somehow compatible with them. And you know, and so that's a good thing. But I think, and yes, I mean, of course there's the dynamic of, you know, people go to college to go crazy and you know, go do crazy parties and be totally sort of out of their minds half the time. To me that's like, that's an escapist story. That's a, hey, I don't really want to be in this life type thing. I'm going to just do random stuff that kind of, you know, take me out of being leading the life I'm actually leading type thing. And I think that's a shame. And it's probably a sign that you're, you're not doing the things you should be doing. If you if at least that's my point of view. Maybe I'm just a an unenlightened sort of stick in the mud old fogey type thing, but that's a certain amount of sort of have fun in college and meet interesting people and so on. But the let's just do these completely kind of things that young people can kind of more or less get away with doing and not destroy their health and so on, but kind of be out of it just seems like a lose to me. Anyway, Des is commenting Hermits acquire cats, not children. Yeah, okay, I admit that requires hermit plus wife equals children in some sense, but not you don't then they're not. It's children are not a solo activity and cats probably. So I have to say that the yeah, I don't have any cats or other pets. It's primarily because my wife and children have been allergic to various kinds of pet related things both physiologically and somehow. Sometimes it's going to destroy the whole house type story. So I've avoided that and now I only get to think a bit about what the alien mind of the pet might be like rather than having the interaction with them myself in any case. All right, I think we've degenerated now into a corner of things to talk about that's even more bizarre than most. But all right, I think we should wrap up there. So thanks for joining me and I see lots of interesting questions to save up for another time, so bye for now.