Business, Innovation, and Managing Life (May 17, 2023)

Sa. Okay. Hi, everyone. Welcome to another episode of Q and A about business, innovation and managing life. I mentioned one procedural thing, which is that I am doing kind of, I have two kind of slots for doing live streaming each week. And I have this slot where I've been sort of alternating between this series and History of Science and technology series. And I have a slot on Fridays, same time, where I have been doing a science and technology Q and A for kids and others every week. And I'm going to start alternating that with another Q and A about future of science and technology. So I just want to mention that because maybe there's some questions that come up today that will move to the future of science and technology. Question time. Let's see, what do we have here? Well, there's a question here from Ivan. From a leadership standpoint, what are the best learnings on how to lead with purpose? What is my leadership style? Well, I think the, the most important thing probably in leadership is figuring out sort of where are you trying to get to? What is the vision of where you're trying to get to? If that's kind of fuzzy for me at least, if it's like, okay, team, where do we think we should be going? That's not a good sign. I mean, I will typically try and come up with some hypothesis about what we're going to try to do, and then other people will, you know, I'll have kind of a vigorous discussion with other people where at first people won't understand what it is I'm, I'm trying to get to or sometimes they won't understand. A lot of people I work with have been working with me for a long time and, you know, they have a bit of, bit of a better chance to understand what I'm talking about. But, you know, right now we're in a very fertile period of thinking about LLMs and all the applications of LLMs. And, you know, it's kind of one serious new idea every few days. And there's sort of questions about can one, you know, like there's one I was just presenting to people yesterday. That was an idea I had probably. When was it? A few days ago. It's kind of an interesting idea. Not quite ready to talk about it yet, but the, you know, I had sort of a definite way of thinking about that idea. I kind of met with a bunch of people who I'm kind of used to working with, and they were like, it took them a while to get the idea. They didn't understand the idea at first. And Then we kind of refined it a bit, but now, from a leadership point of view, my role in dealing with that idea will be to clean up the idea as much as I can, often in discussing it with other people, and then sort of keep reminding people, this is the idea. This is where we're trying to get to. This is the set of goals. I would say that's one thing. I mean, one of the things that happens is when you haven't been clear enough about what you're trying to do and you just sort of, well, I'll just delegate all that to people and somebody will figure it out. Sometimes that will happen, but often it will not. And part of the art of doing that delegation is, is knowing what is realistic for people to be successful with in terms of sort of what kinds of figuring out they can be expected to do. It's no good to say, you know, I have this big idea for a project and I'm going to delegate it to quite junior people and they're going to figure out all the. All the details of what we're really going to do. It's not going to work. Sometimes, you know, I tend to be, you know, there are people, as I say, who I've worked with for a long time where I have a pretty good understanding of. Of what, you know, where if I give them a concept or project at this stage of development, they will be able to do this kind of thing with it. And sometimes people will pleasantly surprise me and I'll give them something and I think they'll need a lot more definition to make progress and they'll come back, you know, a week later and it's all kind of defined and they really made progress and they didn't need anymore. And sometimes the other thing happens where it's like, look, I think I've defined this project well enough for person X and they don't really. And they don't manage to get the thing to actually fly. But I think one of the important things is you've got to crispen up the vision, the strategy, what you're trying to do to the point where you are feeding it to people who will be capable of taking something at that level of definition and doing things with it. I would say for me, one of the things that I tend to do quite a lot of is sort of the thinking in public kind of thing I tend to do. I do lots of even live streams of internal meetings that we have designing software and things like this. I tend to be somebody who likes to, I work things out with other people kind of in real time. I don't typically go off and sort of say, well, I'm going to go figure that out sort of secretly and then come back and announce to people the results. Sometimes it will happen that I have ideas when I'm not in the middle of talking to other people. But my goal is much more to kind of think in public as a way to both get my ideas kind of reacted to and refined as quickly as possible and as a way to have other people kind of understand what, where it is that these ideas are coming from and help them to kind of align with those ideas. And also I think it's educational for people to see how things get figured out. I mean, I know in many generations of people, sort of at our company and beyond where kind of I've sort of exposed them to the decision making, the idea generation process. And that seems to be a really useful thing in terms of people understanding how to do those kinds of things for themselves. Another aspect of kind of leadership, I suppose I tend to be pretty direct with people and it's kind of like when I think something is good, I'll say I think it's good. When I think it's bad, I'll say it's bad. And I think sometimes there are people who don't react particularly well to that kind of directness. In our particular company and culture, we're sort of selecting for people who react well to that. I think, you know, I kind of expect the same thing from other people with respect to the things I'm saying or doing. And I think that's, you know, a feature that that works well. It's not one of these things where if somebody's done kind of a bad job on something, I will tell them I think they've done kind of a bad job. I mean, kind of my principle of it is, you know, never attack the person, but you know, if the work is bad, you can attack the work. And yes, I do. And I think that kind of defines kind of a notion of people can actually say what they think, which I think is very healthy, at least in our particular company culture. I suppose another thing is kind of I expect people to think at all times and not just drop drone away at, you know, oh, we had this plan, let's just keep executing this plan, even though it's obvious that this plan is just walking us off a cliff. It's kind of like I expect people to be kind of thinking at all times. And I try to do that myself. And you Know, one of the things that can happen is particularly like in a situation like the world of LLMs, where things are very fast moving and you know, things are changing every week and you have a hypothesis about what you should be doing. And there's sort of a question of do you. When you've. One of the tricky things let's do right now actually is you have an idea. Let's say you had an idea three months ago. It's a pretty good idea. We start pursuing it things. We realize that it isn't quite the way it seemed to be. Maybe things change in the outside world, but maybe we, because nobody's ever done something like this before, realize that, well actually the thing we thought was important wasn't so important. Something else is more important. It is, it's an interesting thing to, you know, the, the process of kind of changing that direction because you, you got a team all, you know, all fired up to go in direction number one, but then you realize that isn't quite the right direction. Actually. You know, the thing that was one of the things that, that, that you wanted them to do is something that can be sort of down prioritized and something else that has to be up prioritized. I mean, in my way of doing things, I'm really pretty direct about, you know, this is what we thought before, this is why we changed our mind. You know, it's kind of like there's not a lot of hidden state and not a lot of, kind of sort of pretending that it was really all the same thing or something like that. It's very much exposing my thought processes of how I got to where, where I now think we should be. Maybe it's different from what I thought three months ago and maybe that requires a shift of some kind. You know, another feature I suppose of leadership is, you know, in the end you're leading people and there's always a question of are the right people in the right places. And I always think it's kind of this, this sort of puzzle of you've got certain projects you want to have done, you have certain people available, so to speak, and can you match the projects with the people. And sometimes you're taking people outside of the things they were doing before and you're sort of connected, guessing that you can extend them in this way or that way. Sometimes there are projects you just can't do right then because you don't have the right people. And if you try and do them with the wrong people, they're just going to fail. And sometimes There are projects where you were kind of not sure you were going to do them, but somebody who's just exactly the right person for that project shows up and then you launch into doing that project. I do think that not only at the level of the CEO leadership, but at the level of leading projects, leadership matters. And a project with the wrong leader, you know, even if it's a pretty good project, and even if you've got a bunch of resources for it, it won't do nearly as well as a project with a good leader. So, you know, this whole process of sort of, can you match up the people? Can you know the people well enough to know what they're likely to be successful with and what they're not likely to be successful with? And I have a decent memory and I'm sort of interested in people. And so I do try to maintain some kind of, I don't know, mental model of the people that I'm working with to get some idea of what I think they will be successful with and not. I suppose one other thing I might mention in terms of sort of leadership ideas is one of the things that is always a question is when do you sort of fire and forget delegate, and when do you kind of stick your nose in the whole situation? And, you know, I tend to have a lot of projects. We do. We have kind of a regular review schedule and so on. And there's, I suppose. Well, yeah, saying about that. I mean, I think one of the things that tends to be one of my bigger roles in terms of leadership is, first of all, let's return to what the big picture was. You know, we're dived into some detail looking, reviewing some particular set of projects that are going on. Let's zoom out a bit and understand what the big picture is. But then also, how does this fit in with other things that we're doing at the company and so on. And I find myself very often saying, oh, you know, this is going close to something that this group over here is doing. You know, I have some reasonably global view of all the various things that we're doing at our company. And so a significant part of my role is just connecting sort of group or person A to person B and saying these things are actually kind of aligned. You guys should kind of meet and figure out sort of how to combine these efforts or whatever else it is. Or this thing we're doing over here is completely incompatible with this thing we're doing over here. I suppose the other thing tends to be, when it comes to a Lot of these more detailed management things tends to be a question of priorities. And it's always tricky to, you know, people will have pet projects, I'll have pet projects. One will not know which things are going to be successful, which things are not. One won't necessarily know how hard things are going to be, but one's always having to switch priorities around and that's kind of an art. But it also is something where you can't switch them too often because that leads to sort of horrible churning behavior on the part of the organization. And I think another thing that's important in our company at least, has been that we never just outright kill projects. I think in the whole 35, 36 year history of our company, there are very, very, very few projects where significant work got put in and the project was just killed instead. We do not infrequently hibernate projects, but we have a really great record in bringing those projects back to life again. Sometimes it takes a while, sometimes it takes many quite a few years before those projects come back. But the work that was done on those projects isn't work that just disappears. I mean, this is one of the things that I think is very dispiriting in, well, quite a lot of tech companies that there'll be. People will work really hard on a project and then, you know, some random management process will come along and oops, that project was killed. We have managed to set things up so that we don't usually get a long way in a project and have it be a silly project. It's usually a project that makes sense. It may not be a project that reaches the point where it is high enough priority to really put resources into doing it. Now it could also be that it's a project where we discover it's harder than we thought and maybe there's some ambient work that we're doing or that's happening in the world at large that will make that project a lot easier if we just wait a while and, and three years later that project will be quite easy to do. Now it's really hard to do. I mean, now it requires building our own version of some complicated infrastructure thing. But we know that there are sort of nascent versions of systems that other people are building that will solve those problems for us and we can just wait until those things are built and then plug them in and then it's easy for us. Now, I was going to say one more thing about delegation, which is that sort of a question of when do you delegate something and then Just leave people to get on with it. And when do you kind of, you know, what happens when things, when things get stuck and so on. And one of the things that I believed maybe 25 years ago more, I sort of believed that if you as CEO or something, stuck your nose into kind of projects sort of far down in the organization, even if they were stuck and even if you could know how to unstick them, that it was a bad idea because it would kind of make people feel like they shouldn't really take responsibility. Oh, the CEO is going to come and fix it if I don't do it. Oh, the CEO doesn't really believe in me because, you know, they're coming in and, and, and sort of doing my project for me, et cetera, et cetera, et cetera. But in fact, what I found is that when we get into situations where things are sort of stuck, where there's some crazy bug that's been out there for six months and it's kind of important and we just don't know how to fix it, or some operational thing that goes wrong, where something goes down and nobody can see what's going on, et cetera, et cetera, et cetera. And I dive into those things, and I'll dive into things where I'm really going many layers down in the management hierarchy, so to speak, and ending up working with people on some specific thing that's broken, you know, right then, or has been broken or whatever else. You know, I had thought that this was a bad idea. Turns out it's actually a really pretty good idea. I mean, it helps that I think I have touchwood. We'll see sort of almost 100% success rate in actually solving the problems when I, you know, dive in and try and do it. And I think that the, quite to the contrary of what I believed, you know, when you're dealing with, I don't know, a DevOps team or something, and, you know, it's like. And you're in there kind of in the trenches trying to solve something for an hour or two and you get it done. And then it's kind of, I think people appreciate that the CEO kind of understands about what they do, cares about what they do enough to actually spend time, you know, working with them. And, and also it's a little bit of, you know, it's kind of it. It. Since I, you know, have a really good record of actually solving the problem, it makes, I think, a little bit of okay, the CEO might know what they're doing. That's a Good thing. So, you know, I have some sort of breaking, you know, breaking, you know, poking one's nose into things that are sort of many layers down in the organization from time to time, occasionally when things are stuck and so on, I found to be quite, quite valuable. And it's much better to have something where, you know, if something is stuck and you can kind of solve it in an hour and it's been stuck for a while, why not just do it? And rather than trying to sort of puppet control people to say, oh, why don't you try this, why don't you try that? Just show them how you solve it, and then they'll see how you solve it the next time. They'll solve it themselves and they will have learned something and nobody will feel bad. Actually, people will feel good about it. A few thoughts anyway. Memes is asking, how do you handle making mistakes? Mostly I try to learn from them. I mean, I really do review things which were, you know, where I say that was a mistake and kind of I, I think back and I can kind of, I will tell people as sort of cautionary tales about mistakes that I made 40 years ago or something. As you know, they're things you can learn from in terms of. I mean, sometimes there are two paths. You pick the wrong one. I would say that's the rarer case where it's just a slam dunk, you did the wrong thing. It's more often, it's pretty complicated. It's kind of like, oh, it's probably the wrong thing, probably the wrong choice, but it's, it's something, something kind of, you know, it's not so clear. I think that, you know, first of all, in terms of the leadership management kind of thing, it's just, you know, typically you just want to admit you made a mistake. It can be a little bit kind of difficult sometimes doing that when you're dealing with fairly junior people who kind of, you worry, sort of freak out if you say, I, you know, we made a mistake, help, the ship is about to crash on the rocks type thing. And so I think that that's not necessarily the, the best thing to emphasize, so to speak. It's just, you know, well, we're going to change. I think the important thing to realize is in a lot of these leadership things is it really works a lot better if you just say, we're going to do this, and even if something went wrong and you have to talk about what went wrong, lead with, we're going to do this, this thing went wrong, and so we got to change course from this or that or the other. But the important message is we know what we're going to do, we're going to do this rather than, oh my gosh, something went horribly wrong. We're now lost at sea, we don't know what to do. That's not a great message, particularly if, you know, if your job is to lead, then if you're lost at sea, you're not doing your job. The other people's job isn't to figure out how to lead, it's to do what they're supposed to be doing. And you know, it's not really sort of fair to say to them, oh, help, we don't know what to do. You know, let's try and figure it out somewhere. If you're the leader, you better figured out what you want to, what you want to do. I think another feature of sort of making mistakes is I'm very much of a don't look back kind of character. You make decisions, you move on. I look back in the sense of learning that was a mistake and I use it as raw material to kind of learn how not to make that mistake in the future. But I'm not thinking about, oh my gosh, that was a mistake. I feel so bad about it. I, you know, I'm kind of sort of personally melting down about having made that mistake. I tend to be very much of a just move forward. And I suppose, you know, it helps me that I make a lot of decisions every day and so inevitably there will be some rate of making mistakes. And so it's, it's like there's always another decision to make and it doesn't need to, I don't need to agonize about the one decision out of, you know, let's say I made 50 decisions and one of them was really a big mistake. It, it doesn't that you know, you've got the other 49 that you, you know, are just moving forward with. And so it's not, you don't have really the opportunity to agonize about the one that got away, so to speak. Let's is commenting. It's impressive to see in live coding session pull open a 20 year old Mathematica document to refer to an earlier idea that you had. How do you manage the inventory of ideas and notebooks over 30 plus years? Do you create standalone notebooks or large massive ones? What I mostly do is I have a fairly well organized file system which is kind of indexed by the projects that I have done and the projects that I think about doing and there's a limited number of projects that I think about doing in a serious way. Maybe I don't know, 30 to 50 at any given time maximum. And the kind of burn down rate of those projects is really quite long, probably five years or more. There are projects that have been out there for decades that I've been steadily collecting material on and sometimes working on. But I've generally tried to projects I have done, they're often organized by things like things I've written where I know, oh, I wrote a piece about this and there's a bunch of research that went into that. I've got the notebooks for that research and so on. I tend to be very boring in the way that I set these things up. It tends to be I'll have a folder that's about a particular topic. I will often have something, you know, SW notes 01-SWnotes 02. Then sometimes I'll say SWnotes Dash 03 and then the topic and then the more specific topic within that folder, so to speak. So I find it useful to just have these files, you know, numbered by which one I did first, second, third. That turns out to be pretty useful. And then sometimes that sort of secondary piece to the name is a helpful thing for me to quickly find something that I'm looking for. Now you might say that's a very old fashioned, why can't you use tagging and search and feed the whole thing into an LLM and whatever else? I just found that the approach I just described works pretty well. And it's, you know, most of the time, much of the time I'll never need to look back to something I did before. So the. I don't want to have some elaborate system where every time I'm working on some file I have to think, well, if I wanted this file 25 years from now, you know, what would I do, etc. Etc. Etc. It's rather just put it in whatever form is easiest for me to do right then. And if at some time in the future I want to go back and try and find this thing, yes, it's going to take me a little bit more effort, but that's okay because the total effort spent is vastly less. If I spend almost no effort kind of putting the files into the file system, so to speak, and then in the rare case where I need one of those again, okay, it's going to take me five or ten minutes to go track the thing down. That's okay. And I tend to sometimes What I'll end up doing is I'll do a bunch of work and then I'll realize actually a lot of what I did was kind of a waste of time. I can say it much more succinctly. I've got a good summary of that and I'll make a blah blah blah, you know, 07 dash dash summary and I might have many layers of those summaries but then I kind of know that if I have that, I kind of, that's the file I'll look at first is the summary file and then only if that doesn't give me satisfaction will I go back and look at the other ones. So it's a, it's a pretty simple system but it's, it's worked pretty well for me. Let's see. Gosh, hold on a second. Okay. Aaron is asking, have you seen other people learn to not need to quotes, relax and to continuously work like I do? Aaron says I'm most satisfied when I'm being productive, but I find myself fatiguing or losing focus at some point. How do you maintain your work ethic? Well, mostly I maintain my work ethic by doing things I like to do. Now you know, I will admit to several glitches in my work ethic. You know, I can procrastinate. I like to study. Why do I procrastinate? Usually the reason I procrastinate is because I actually haven't figured something out. And the procrastination is actually a correct kind of self protection system because it's like, well, I could start working on this project but I'll actually be going in the wrong direction. That's a very common thing and, and where it's kind of like, well, you know, I'm just not quite ready to do that now. Something that goes wrong for me is I work on some project and I like to just, I get it all in mind. I like to just go through and get it done. And when I get blocked for some reason for more than a couple of days, I lose state and basically it's very painful to me to go back to that project. So right now, for example, in, you know, I'm working on a bunch of things related to LLMs and I'm writing a bunch of different pieces about things we're doing with LLMs and I'm frustrated because at least in the last week or so I've been repeatedly blocked because I'm basically trying to, I don't know, it's whatever it's write the documentation for the airplane while it's still being built and trying to fly it at the same time type thing. So it's sort of a, a crazy thing to try to do at some level, but I'm keen to. I know that writing about these things we're doing is important to help define and crispen up exactly what we're doing. So it's important to do that. It's also frustrating. I keep on getting blocked and you know, like, even yesterday I was having a hard time. I had at least an hour or two of procrastination of something I really should have been working on because I was sort of frustrated because I was blocked on various things. And I always find it very difficult to get back into things that I had sort of had been doing and then lost state on. But, but you know, and when I, when that happens for me, realistically, there's always email to read. You know, I'll have a, a horrifyingly large sort of backlog of mail. And you know, it's very rare that my overall backlog gets down to less than a thousand messages. And I usually get about 35 or 600 per day. And I'll usually grind those down over the course of a day. You know, keep a bunch of data on all these little sort of peaks and valleys and so on. And sometimes I'll let the email, you know, pile up for a while and then I'll have a few days of really grinding it down. But for me, if I'm really kind of sort of, oh, I can't, I don't know, I'm not really ready to jump into the major thing I'm supposed to be doing today. There's always email for me to grind down and write responses to and think about and so on. And that's for me, you know, I suppose other people might go and watch television or something. For me, you know, there's always that task of going through email and so on. I think it also helps from time to time in terms of maintaining sort of activity that I do work with other people quite a lot. And you know, typically in a day there'll be some periods of time where, when mostly in the evenings when I'll work primarily by myself or at weekends. But a lot of the time I'm working with people and in a sense that helps maintain. You know, if I get into a meeting, there are 10 people there, I'm going to make an effort and do succeed in being fairly focused on the particular thing that we're doing there and the fact that There are other people who are sort of also pushing in the same direction. Help helps me out in terms of maintaining that focus. So. And I think I find that it's things that I do that are productive, but maybe not in the sort of the main trunk of some particular set of projects I'm doing. Right. Then there's still things to do that are productive. And I find that when I've done one thing that has kind of energized me, whether it's grinding through lots of email, whether it's doing some. It's even doing one of these live streams or some other thing that I find kind of engaging to do, and I kind of get ramped up on that. That always helps me to sort of develop general momentum. That helps me in going into perhaps some project that I might have been procrastinating on. But I suppose if I'm to sort of draw a general conclusion for that, having things which are still productive and still things that I really need to get done, but aren't necessarily the things that require the most kind of intense kind of focus, having those things as a background is. Is, I think, a useful thing. And probably also the having the fact that I have lots of things that I do with other people that kind of help provide sort of collective energy probably is a helpful thing, I think. Also, I mean, I think it's. Well, I don't know. And, you know, I can usually. I can work for, I don't know, somewhere between one and a half hours and probably two and a half hours, just sort of sitting there typing on my computer or whatever else. And. And then I kind of have to get up and pace around or something like that. A. A recent innovation of mine was I have a terrible habit when I pace around. I always go get something to eat and I'm like, no, I'm not going to do that. So this is my. My exercise for the past week. So we'll see what. What happens as a result of that is, you know, be able to pace around and not get another piece of chocolate or another thing to eat, because I don't think that's particularly. I don't really need it. It's not like I'm hungry. It's a thing to do. And it's kind of like you have to define for yourself. I can pace around and not have that be my destination, so to speak. I haven't quite figured out what the new destination is going to be. But that's the thing that I still have to figure. There's a question from adjective here. Could you share your personal experience with how your intelligence has evolved as you've aged, particularly in terms of recall? Specifically, can you describe what it feels like for you when you take a brief pause of 0.2 to 2 seconds to grasp a concept while discussing complex topics and communication or video presentations? Gosh, you know, so far, touch wood, I still seem to be fairly alert and the brain still seems to be working. I think it helps me a lot. That's probably like sort of taking physical exercise. The fact that I really do quite a lot of thinking, you know, in every day, I kind of think helps me to sort of stay mentally agile in some way. I think that, I mean, I do notice people who I know who have kind of stopped being in a situation where they really do a lot of thinking every day. I have noticed that the sort of mental agility does tend to go down. And I think that's just the same thing, that if you're running every day or something, you're going to be able to run better than if you're not. I think that in terms of remembering things, you know, the main thing for me is I've just kept learning sort of all my life and I have a pretty decent memory. And it often surprises me that I can remember things that I learned 50 years ago, let's say, and I don't think I've thought about them in 50 years and yet I still remember them. And you know, we can talk about the science of how memory works and the fact that most memories are probably things where you remember remembering, not where that single neuron is still storing that data 50 years later. I think actually we really will learn a lot about how that all works from studying LLMs and really kind of studying sort of what it means to put the fine tuning of an LLM is maybe not so different from what happens with humans. And it's sort of important for the LLM to re review things that learned earlier rather than just feeding it a bunch of new data and only new data because it tends to sort of overlearn the new stuff. And I'm not sure that we humans are really that different from that. I don't know in detail. Maybe I'll figure something out about that sort of LLM science and the not too distant future. But I think that the thing that I've noticed is I happen to have a reasonably good memory and that really helps. But it also helps that learned a lot of different things. And when you learn a new thing after you've learned a Lot of other things. The new thing is much easier to learn because there's sort of places you can slot that new knowledge into. Oh, yes, it's just like this and that. I have to say, I think some aspects of how that works are kind of shockingly similar to how LLMs work and how they sort of successfully managed to get prompted to do all sorts of different things that were sort of somehow analogous to something that was in there, but it wasn't kind of not the thing that was literally learned. I find sort of the same thing for myself. And I suppose it's a weird thing when you work a lot with computers and AI and things like this that you kind of see analogies to the way that you yourself work in the things that you see in the AI systems and so on. And I think this thing about slotting in new knowledge as an analogy to something you knew before is a thing that both, both I and the AI, and no doubt most other humans and the AIs do. But the more you know, the easier it is to do that. And I have found, as I say, I, I, I mean, I don't know, it's certainly I, I, I use no memory tricks whatsoever. I don't know, you know, there are all these kinds of things, oh, try and remember this with that and the other. I don't do any of that stuff. I don't know any of those things. I notice sort of when I have certain kinds of, I mean, I know a decent amount about my memory and what kinds of things I remember and what I don't. And that's important to not be wrong about what you remember. Like, for example, for me, I'll remember facts and things that happened, but I couldn't necessarily place the order in which those things happened. So I can remember I did this. But if you ask me, did I do that 10 years ago or 15 years ago, did I do that before this thing or after that thing? I don't necessarily know. That's much harder for me to remember. But the thing I can remember and it's similarly like, for example, I don't know, it's just a way my memory works. Like, I meet people, I hear their name. If I get the whole name, I have a decent chance to remember it. If I get the face is hopeless for me. I'm really pretty bad at that. But names, I'm pretty good at remembering. They're a little bit easier for me to remember if they're names that are vaguely similar in type to names that I've heard before. More difficult if they're names that are just type of names that I haven't really heard that much and I kind of have to. Then I find that if I only get, oh, the person is called John or something, but I never get the second name. I actually find it harder to remember the first name than if I'd had the whole name, things like this. But now in terms of. At times when I, for example, to one of these live streams and somebody's asking some question and I don't immediately know the answer and I have to think. Can I describe what it feels like to think? Not really. It's just, I. I don't know, one just pauses and one just thinks about something. And for me, at least maybe like an LLM, I don't know that there's a lot of, you know, I'm. I'm talking as I'm thinking, so to speak. And I don't kind of feel it that I'm sitting, thinking, thinking, thinking, and then I'll say what I'm thinking, so to speak. Usually I find that I can say something that is even useful to me as I kind of externalize what I'm thinking that helps me to take the next step in thinking. Gosh, that's depressingly similar to an LLM, isn't it? Well, any case. But I think that's what I found. I mean, I would say that if I were to assess my kind of problem solving abilities now versus when I was younger, I would say I'm definitively better now for several reasons. One, because I know more. Another, because I understand what I'm good at better and what kinds of ways to approach things are likely to work. What kinds of ways to approach things? I know more about kind of ways to approach things. I don't know. Be an interesting question if in terms of the actual speed of figuring something out, I don't know, I don't feel like I'm slower at doing that. I feel like if anything, I'm a bit faster at doing that, but just because I know kind of more methods for figuring things out. Perhaps not because the single you got to solve this problem with this method that I don't know whether that's gotten slower or not, but that's my feelings about those things. Let's see. Okay, J.R. is asking, do you have advice for the new generation of college graduates entering the workforce? What's the best way to apply for jobs? How do you maintain those jobs for years to come? Well, good for you for thinking that you'll maintain those jobs for years to come. I think it's much more. I mean too much is kind of like, I'll do this for a year, I'll do that for a year. Maybe that's good for some people. Certainly. Well, I suppose maybe I speak partly as an employer, so to speak. It's, you know, one puts a certain investment into, into people and there's a certain sort of ramp up phase and if it's just like, oh, I'm onto the next thing, that's kind of disappointing. But in terms of applying for jobs, I would say that the number one thing is do some research, figure out, I mean, if you know what, what kind of thing you're interested in, you know, where that's done and you, you know, there's some company or something or whatever else and it's like you want to, you think you'd be a good fit there. What? Well, why learn more about the company, see why you would be a good fit and send them email and say here's why I think I'd be a good fit. Don't kind of over. There are times where people will send mail to us and they'll explain and they'll get it right and they'll say, look, here's a thing that they know enough about our company. They've used our products. They are like they can, you know, they paid attention to, you know, in the case of our company, for example, we're really open about a lot of things we're doing so it's really pretty easy to know, you know, here's the thing we're doing and oh, I could plug in here and that's a, you know, that's a, that's a good pitch if, if somebody can make it. I think that sort of the, the kind of. I read your page on Wikipedia and now I'm regurgitating pieces of it. That's a very bad pitch letter. It's really that when you really have something that is really kind of, you know, a good match on offer, that's an interesting thing. And then I would say typically you have to send that and maybe I, I don't know the most general case of this, but sending that to sort of execs is probably a decent idea now. You know, there will be things that you can't see from the outside about a company that, that may cause your great idea to just be a total non starter at that time. For example. I'm not sure. Well, okay. Another piece of advice perhaps is if you think there's interesting stuff going on and there's an interesting company and so on sort of getting in there. Even if you get in with a job that seems beneath your station, so to speak, is often a win, not always. Some companies you can never advance. It's sort of hopeless. You're trapped in this kind of low level place. But many companies, if you go in there and you do a fantastic job and the company is set up to sort of notice that, then that can be a good thing to do. It can be difficult when people come out of the education system and there are so many things that are great jobs but nobody ever knew they existed. And that's one of the challenges is, you know, I don't know, in a company like ours there are just tons of job categories where nobody went to school to do that particular thing. And I'm not sure that people would even know from the outside that those categories of jobs exist. But nevertheless, if you can find that and you know, those are good things to go for, I think, you know, one of the things that I don't really know too well is, well, okay, so there are companies of different sizes and their hiring practices will be completely different. Even among different companies of similar size, their hiring practices may be very different. I mean, there are companies where it's a big industrial system and it's like, do this test, satisfy this requirement, et cetera, et cetera, et cetera, then the HR department will process you through and otherwise it's hopeless. And there are companies, typically smaller ones, where it's much more of a hey, here's a talented person. Let's see if we can find a place for them. I'm not sure how common that is. I mean that's certainly something that's been a long time feature of our company. But I'm not sure how widely that happens. That is that somebody comes in and it's like, hey, this is a really talented person. I wonder if we have a place for them rather than just here are the jobs we've advertised. Do you fit into any of those jobs? I mean, for us in particular, we have a number of mechanisms for finding certain kinds of talent. And other companies have versions of this too. Like we have our annual summer school and lots of people, you know, that's about science and technology and those kinds of things. And that ends up being a fertile sort of hunting ground on both sides of us finding people and people finding that there are interesting roles that exist in a company like ours for them. Yeah, I think, yeah. I mean one of the things, okay, so I will say in terms of college graduates and the workforce and so on, at least for me, the COVID letter that starts, I just got a degree in X, therefore I am qualified for job Y that you advertised on your careers site, that for me is usually not a promising cover letter because it's like, well, can you do this or not? Somehow it's like, well, I just got this degree and I put a lot of effort into getting this degree. Now I've got this piece of paper and now, you know, that changes me as a person. Maybe I'm just different in that respect. But I don't really buy that. I mean, it's kind of like, well, give me evidence you can do the job. I don't really care that some university certified that you had this or that degree. Maybe that's an extreme position on my part. And certainly in HR departments of larger companies there are like, okay, is it a computer science graduate with a GPA more than X from this university, et cetera, et cetera, et cetera. But you know, and I think there are. Yeah, I mean, the other thing is in terms of what kinds of sort of job areas to pick. We are in a time of change in terms of sort of lots of kinds of technical, creative, et cetera, et cetera, jobs because of AI and so on. And I think we're also, you know, we're at a time when there's sort of the whole computational X thing is, you know, those engines are very much started and there are all these different fields that are sort of advancing based on that. I think it's a really cool thing when people can attach themselves to fields that are in a rapid growth phase and then they will be. That tide will lift them up over the course of a career. And, you know, you start off in some field where you're one of the first few people who does X and then decades later you are, oh yeah, that was one of the first people who does this field. And now you're a top person in that field. So that's always a good thing to think about. I think there are fields which are, you know, likely to contract, and those are probably not great things to get into unless you have a direct, specific passion for those particular things. Let's see, The Paula asks, you think we'll get to a point where AIs are in charge of interviewing? Could this be beneficial or harmful? I absolutely think AIs will get used for various kinds of screening, interview purposes. It sounds kind of dystopian and horrifying. I suspect it will be fairly useful because it's kind of like there's, you know, like, like for our company. And I unfortunately I very rarely on the front lines interviewing for anything other than pretty senior positions. But you know, we have for decades now asked, you know, some very basic questions on our job application forms, like, what's the largest project you've ever done? How do you think you could contribute to our company? You know, what do you know about, you know, have you used our products? What have you done with them? Those kinds of things and sort of, there's definitely certain patterns of responses which I, I don't think are beyond an AI to be able to untangle. And it's kind of like with those, there are certain kinds of responses where it's like, yeah, this is going to be for us likely to be a promising person or for us it's just not likely to be a promising person. And the thing to remember about applying for jobs is that it is a two sided business in the sense that you may be the most amazing person of all. And there just isn't a fit with this company because this company has a culture that just doesn't fit with the way that you are. And you know, I think that, that, and that can happen because you're a person who likes to be very creative and do lots of different things and do this and that. And it's a company that's very rigidly sort of focused and very sort of structured into, you know, go in straight lines and you won't be able to stand it. Or it's a place where you really like to be kind of told this is what you should do and like to be given, you know, clear expectations and so on. And it's a company that has much more hands off management, those kinds of things. Or it's a company where, you know, people are pretty direct with each other. And so, you know, there's, it's fairly intense and that, you know, you don't respond well to that or it's a place where you're fairly intense and fairly direct and the company will freak out because you'll get all these kinds of complaints. Oh my gosh, this person was way too aggressive in this meeting type thing. So, you know, there are lots of different dimensions in which there can be not a fit. And it's in no way a reflection on, oh, you know, you must be a terrible person who's a total loser because you don't fit with this particular company. And knowing what that company culture Is. Is sometimes, well, you know, meeting employees from the company and so on. That's always a good way to get some idea of that. And, you know, sometimes people will come in and. And be like, I'm really good for your company and I'm going to change it. It's not really, you know, unless you're hiring a CEO or something, that's not likely to end that well. In other words, companies have a certain culture, a certain way of operating, and, you know, it's the person who comes in as the energetic, you know, junior person who's like, I'm going to change everything around here. It's. It's very unlikely to work. I mean, you know, it is something where at our company, we're always very pleased to have people have lots of energy to do a lot of stuff. But the company does what it does, and that energy has to be kind of directed in a way that isn't just bashing into walls all the time. And I think those are some kinds of things to think about. Can you understand what the company is like? And I suppose for a company like ours, things like live streams and so on are a curious way to. To really get a view inside what's going on. I think we're pretty unique in having those kinds of resources available to people, but there are other ways. And certainly if you can, you know, be an intern at a company, that's a. That's a clear way in on both sides. You know, the company gets to know you, you get to know the company. It's kind of, you know, our summer school is kind of a miniature version of that. And also we have intern programs and so on that work that kind of way. Let's see. Okay. EVP is asking advice on how you lead when you sincerely know less about the subject than the people you're assigned to lead. You know, I always like to learn, and I always take the point of view that I can learn anything. And if I'm working with people where they know more than me about that particular thing, and I really have to. I have to try to figure out, they're going to say, we need to follow this strategy. Okay, do I believe that? You know, for me, it tends to be a. Okay, explain it to me. The person goes, blah, blah, blah, blah, blah. I don't understand a word of it. I'll try again. Can you explain it to me? Can you explain it to me? Et cetera, et cetera, et cetera. If it really fails, if it really just can't explain it to Me, I kind of have to give up because I kind of have low confidence that things are going to go right because I think it's a good. I like to think, as I say, that I can kind of learn anything. And it's like, you have to get a sense, I suppose you have to get enough of a sense of what people are doing that you can tell whether they know what they're talking about. And, you know, there are places where people are doing specific kinds of things where I don't know all the details of that. And. Yeah, so, I mean, the question is, how do you tell whether they know what they're talking about? And usually for me, it's, can they explain the pieces that I ask about? Can they explain them to me when they can't? For me, it's a really bad sign. When they can, it's a good sign. And I get to understand a little bit more as they explain it to me, get to understand a little bit more about how they think about things. And what I found is that, you know, every different field, every different kind of work has a slightly different pattern of how people think about things. And it's not, you know, it's like if you're going to be a research scientist, there are different kinds of ways to think about things. And if you're going to be a, you know, a DevOps person or a. Or a person or product manager for something, these are all. They all have a certain pattern of a rhythm in which you think about things. And you kind of have to, I would say, sync with that rhythm with a team. If you're working with people who know about something you just don't know about, you have to get that rhythm. You have to know kind of what can you expect because you don't have that rhythm, Then things will happen. Like, they'll say, well, you'll say, how long is this going to take? They say, well, we don't know. You know, it's complicated. We haven't been done before. It might take a week, it might take a month, it might take a year. Okay, well, if you don't have an idea of the sort of rhythm of what happens, the way of thinking about things, then as they tell you, well, we did this little piece. You have no sense of whether that means they're halfway through, whether that means they're just starting or what it means, or even whether they've actually thought about it enough to be properly engaged on the project. So. So I suppose that the. I mean, I've learned through working with people on things that I didn't know much about, I've learned a ton of stuff. And, you know, often I will find that there'll be a project I'll work on with somebody. They'll have a different way of thinking about things than I have, and I will absorb that. And the next time I have to do a project in that area, I'll know that way of thinking about things, and I'll be much more in a position to directly lead it and sort of figure out what we should do and than I was at the first time. And I think it's important to recognize this thing about different areas, have different rhythms, different ways of thinking about things. Because if you come in and you say, I know how to think about things because I think about this kind of thing, and you try and impose it on that other area, it probably won't work very well. You kind of have to let yourself be educated in that way of thinking about things. Now, I know that for me, when I'm dealing with projects where I profoundly don't understand the project, it's very hard for me to lead that well. And I certainly feel for people I know who run companies, and they're not technical people, and the companies have a lot of technical content. That is a very difficult thing to manage because you'll have people. The CIO will be telling you, oh, there's a this and a that and a. All kinds of funky names, and you've never heard about it. And, you know, maybe they'll try and dumb it down for you, and you just have no idea what's going on. And that CIO could be amazing and they could be delivering and the whole thing is going to be perfect, or they could be absolutely full of it, and you wouldn't know. Now, part of how you deal with that is with kind of just understanding people as people and getting some sense of who's talking sense and who's not talking sense. But. And you can do that up to a point. I always find that more difficult when there are sort of human cultural differences. It's always a little bit harder to tell whether this type of person with this cultural background is being very modest and, you know, and is going to. Is really going to be able to come through, whether they just don't have the wherewithal to do it, whether this other person sounds really confident and, you know, oh, they went to all the right schools and they learned to sound confident, but they actually don't know what they're talking about. It's Often when there's kind of a, when you haven't sort of seen as much kind of, of the sort of human culture that people come from, it's sometimes a bit more difficult to judge those kinds of things. It's easier when you get them to actually explain, you know, explicitly what are they doing, what are they talking about? You know, one of the things that happens to me, you know, we do a lot of technical stuff. I'm, I would say on the high end of technical awareness and, you know, people will be talking to me, oh, we're using some system, some buzzword, some this, that and the other. I'm going to learn every single buzzword. You know, it's, it's like somebody says this buzzword and they kind of, you know, occasionally when people are new at our company, they'll be like, oh, I'm going to try and sort of dumb it down and not just say it the way an expert would say it about that particular thing. And I'm like, please don't do that. Just tell me the kind of expert version of it. It may be that two words in the sentence I won't understand. Then I'll ask you, what do those mean? And then I'll get a sense of what's going on. Because among other things, the kind of dumbed down version is no longer kind of in the same rhythm as that field. You don't know. It's much harder to get a sense of sort of the pulse of what's going on if you're being given kind of a, a, you know, a watered down, dumbed down version of what's happening. If you, if you hear the expert stuff, even if it's a little bit of a struggle for you to understand it, and even if maybe you don't get all, all aspects of it, it's still very useful to just get the sort of cultural sort of sense of what the expert version of that is. Paul was asking, as a company operating worldwide, do you find language barriers to be an issue? Can AI eliminate those barriers with a universal translator? Yeah, I mean, we, we translate our stuff into a bunch of different languages. Yes. In fact, the sort of latest generation of AI looks very promising as a way to let us more cheaply translate a lot of kinds of things. I would say that in terms of doing business around the world, the languages are one thing, but maybe the particular business we're in, which is in kind of more techy types of things, where English tends to be a pretty widespread language, although not universal by any means. And certainly it helps our customers a lot, I think, when we translate things. But I would say that in terms of doing business worldwide, the sort of cultural differences in business in different countries are a much more serious issue than the straight language translation kinds of things. And you know, that can be very complicated, very difficult set of things to understand. And there are countries where, you know, we probably, we just don't understand that country and, you know, we don't understand whether it's one of these places where, you know, where the only way a deal ever gets done is to be in person, shaking hands and so on. Or when anything ever gets done, it's always a chain of this person knows this person knows this person, or whether it's something where there's no, the world is really not as uniform as one might think based on sort of supply chains and things like that in terms of actually doing business. I mean, back in the past you would be doing all of this kind of, oh, you're going to do business in this other country, you have to have these kind of referral letters and all this kind of thing to even be able to get started there. A lot of that has gotten a lot simpler. I mean, it's not. But even, even, you know, even something as simple as, you know, taking credit card billings in, in all countries around the world, that's not easy. You know, we've been in India, for example, you know, we've had, I think we've finally solved it, you know, taking credit card billings for Wolfram Alpha Pro and so on. But that's been a multi year, a very messy business with, with just, you know, it's very complicated. The, the requirements are complicated, the legal requirements are complicated, the actual ways of working with people are complicated, et cetera, et cetera, et cetera. But you know, I think, and there are times when there's, I mean, there are countries where just, I don't know, the expectations in terms of, oh, like who's going to be a straight shooter, who's going to be a total cheater. You know, when somebody says they're going to do a deal, are they actually going to do it? Is this, you know, how long does it take to get something done? If the thing's been dragging for six months, does that mean it's dead or does that mean that it's just, that's the way things work in that country? Does it, you know, do you just A lot of things like that that are, that are much beyond language translation, they're really sort of cultural translation and you know, in our company we have people from, you know, who work on things in different countries who are typically from those countries or know a lot about those particular countries. And it's. There's quite a bit of translation to do sometimes. And it's trans. It's more cultural translation than it is linguistic translation. Let's see. Harry is asking, are there self evaluation techniques that you would recommend for everyone? I don't think I do a lot of self evaluation. I mean, I kind of watch what works and watch what doesn't work and try and learn from that and watch what situations I feel came out well and what did not. And again, try to learn from that. I mean, I'm not, It's not like I'm trying to sort of. Test myself, you know, am I ready for the next math concept type thing? Not been what I try to do. Look, I, I think that the thing that tends to happen in there always as I'm. As I'm kind of thinking about new areas and so on. There's always this time when I don't really have my arms around the area and where I kind of hear this thing, I hear that thing, and they don't really fit together. I don't really understand them. I'm kind of struggling. And then eventually I learn enough that I really feel like I've got my arms around that area. And then each new thing I learn, okay, it fits into something I already knew. Aaron is asking a long question here, actually asking about digesting new content and research papers and so on. In circumstances where time is limited and reading everything is not feasible, how do you determine when it's worth pausing to explore reference citation in depth without fully understanding the citation? Could I share my strategies for efficient and selective reading? I think I'm a pretty bad reader in the following sense. If you just say, read this for the sake of reading it, I don't think I do that very well. In fact, I do it rarely and I don't absorb the content very well. I'm usually reading for a purpose, that is. And the thing that I'm reading has to be something that sort of fits in in some way to something I already know about. Another thing that really helps me in reading things is, is this a big idea that I'm reading about or is it a little idea? And if I don't know, that's a problem. And you know, I know in people reading things I've worked on, sometimes it's really important to signal to people, you know what, this is a big idea may agree with it, you may disagree with it, whatever, but it's a big idea. If you think it's a little idea, you'll never understand it. You'll just crash into the side, so to speak. You don't have a chance. And so for me, kind of normalizing, is this thing that I'm reading about something that's sort of a big idea where I'm having to kind of really think differently about something, or is it something where I'm going to be able to incrementally fit it in? That's a helpful thing to know. I think that in. As I said, I'm always reading for a purpose. So it's. It's like, okay, what do I learn from this? You know, can I find the piece of what's going on here that fits in with something that I already know? And I think, I mean, you know, when I read stuff, okay, I, I sound like a little kid here, but I tend to look at the pictures and that's, you know, I found that a lot of technical material has, you know, you can understand a lot by looking at the pictures. They're often the place where the most is communicated. It's always kind of a downer when I find these papers, and they're very common where, you know, the big conclusion is it's a curve where there's an X axis and a Y axis and there are a bunch of data points and it's wiggling around. It's like, what on earth am I supposed to conclude from this? That's sort of a bad sign. But, you know, if I'm reading a technical thing, I'll also. Actually, more so in the past than now, I'll read the equations. If it's something written with Wolfram language, I will absolutely read the Wolfram language code. If it's well written, that's a really good way for me to understand what's going on. And maybe if I can, I'll, you know, if it's a. If it's something written in a nice modern way of. Things that I try to write, I hope are like this, where you click to copy. You just click the, you know, on some picture, you click, you get a piece of Wolfram language code, you paste it into your own notebook, you run it. I often find myself doing that, even though really all that's going to happen is I'm going to get the same picture that was already in the document. But I still, something about it helps me internalize things when I'm running it on My own computer. I can see kind of, it's, it's. There's no hidden weirdness. I just picked up that code, I pressed shift return, I ran it, it produced this result that gives me a kind of a sense that I've sort of. I've got everything there now I have to just understand it. I would say that, yeah. I mean, usually when I'm reading something, I'll sometimes have to think, you know, it takes effort to make the bridge to something I already know. And sometimes I will, you know, I work with people and I will say to people, you know, look at something. Okay, this is a. This is the kind of, well, perhaps lazy CEO approach is, you know, can you please read this for me? And you know, my ask is usually the same thing. It's read it and tell me how it relates to things I already know. If you just read it and give me a summary of what's there. Well, I can probably get that from the abstract. But the thing is, if the person who's reading it kind of knows the things that I already know or I'm interested in, they're going to tell me what's the connection. I have to say I'm pretty sure that LLMs and things will be able to do some kind of job of that. And it's an interesting thing to set loose something where people send me some paper. It's like, okay, how does this relate to things I care about? If I've got an LLM kind of trained up on things I care about, there's a good chance it's going to be able to point out connections, perhaps including ones that I don't immediately think of. And so that will be really useful. But for me, as I say, it's, reading for a purpose is always the thing that really helps. And it's, gosh, I'm really bad at reading not for a purpose. And that's why when people send me things and I'm just not ready to read them, I'll just say, look, I'm going to put them in a folder. You know, this is a project I'm hoping to do one day. When I do it, I will read what you sent me, but right now I'm not going to read it because, you know, there's not not. I'm just not going to be able to absorb it properly. Okay, maybe one more and then, Let's see. Well, Reyes asks how to go about learning mathematical thinking. The school focuses, they say, on learning formulas and just solving questions. In the age of computers, how to learn mathematical thinking. Well, maybe I'm the right person to ask that. I mean, this is the story of Wolfram Language and Mathematica over the past 35 years. It's kind of just learn to do stuff, learn to take things you wonder about, questions you wonder about, and put them in computational form, then see what results the computer gives. And they often won't be the results you thought you would get. And you often will realize I asked the wrong question here or it gave me something which I really don't understand. I have to dig in and try and ask a bunch of other questions to try and understand that. But I think that this process of try and computationalize everything is a great way to learn kind of mathematical thinking because it's a way, you know, in the past the only way people had to get access to that kind of precision of understanding things was oh you should do the hand calculation, oh you should do the hand proof. That's. But now we have a different way which is let's represent it in computational language where it is precise, where we really have to understand it at some precise level. And then we get the intuition from the feedback of seeing what the computer actually did and we try all these examples and we kind of look at what happened and we try and develop intuition about things. That's to me that's really valuable. And sort of gradually as you kind of computationalize mathematical concepts, they somehow become sharper, more concrete. And people I think find it a lot easier to kind of understand that rather than sort of floating in this world of purely abstract things, now there are people who can successfully float in the world of pure abstract things. Sometimes those people are pure mathematicians who are almost sort of running the code in their heads type thing. I think for most people it's really helpful to anchor things in some sort of more computer, more concrete, computationalized version. Well, let's see. I think maybe. There's questions about business ideas about using chat GPT. I think that's for another time because that's going to be too long a discussion. Well, Liana is asking, suggesting for a self taught programmer on the trader side who wants to get more knowledgeable about computational thinking, books, courses, topics, how to make a personal curriculum. You know, I have to say this is a place where the things I've done over the last many years I think are, I would like to even say probably the best game in town, so to speak, in terms of understanding computational thinking. I would sort of start, I don't know the things like the book I wrote about elementary introduction to Wolfram Language is a reasonable place to start. There's a lot of courses that we provide through Wolfram U, which are again, good places to start. It always really helps when you have your own project where you're trying to say, I want to do this project. Let me see what it takes to do this. I think that sort of our Wolf and language technology stack is, I will claim, really by far the best thing that exists today in terms of sort of broadly understanding how to think about the world computationally. I will also say that in something that is a project of mine is to try to really write a sort of a good kind of course curriculum, whatever you call it, to sort of talk about computational thinking and in the particular in the post LLM world, sort of how one does that and what one needs to conceptualize to Even give the AIs a chance to automate what has to be done. So hopefully in the future I'll be doing more of that. Okay, I should wrap here, but I do want to mention that I mentioned at the beginning changing a little bit logistics of these live streams. I have been doing them on Wednesdays and Fridays. I will continue doing that, but on Fridays I am going to be alternating between what I've been doing for the last three years, which is a science and technology Q A for kids and others that will alternate in in alternate weeks with a future of science and technology Q and A, which I'm looking forward to. And we'll be doing the first one of those this Friday and I encourage people to join me for that and a few of the questions that came up here will be good things to to start off addressing there. All right, I think that's, that's all for today. Thanks very much for joining me and look forward to being with you again on another live stream. Bye for now.