Why AI Can't Replace Your Judgment Yet | JD Margulici (Cruise)

I almost never have to feel incompetent on any topic, right? Because, you know, filling the gap is just one prompt away. At the same time, obviously everyone has access to that, right? So it's interesting because it gives you power, but also invites humility because, you know, that knowledge is not something you own anymore.

Welcome to Vlad on Leadership, interviews with technical leaders and operators about the leadership questions AI makes more difficult. I'm your host, Vlad Moskovski. I coach senior technical leaders and founders who are excellent at the job that's keeping them stuck. First of all, thank you so much for doing this. You know, I really appreciate it. You are officially my first guest.

No way.

Yeah.

Well, that's an honor. Yes.

It's very interesting, the people that I talk with, and there seems to be 2 sides always in people's work or companies to the AI thing. One is internal. And the other is external, the product side. And it seems like you're touching on both. You know, these things are moving so quickly and I'm very interested because you're coming at it from a very technical perspective. Given everything that's shifted with agents, with AI, like how has your work, your responsibilities, the scope of what you're thinking about changed in the last 8 to 12 months?

I guess the most tangible, concrete impact, right, is kind of the day-to-day of which for me, you know, besides the things we all do, like, you know, answering emails and creating, you know, communications documents, is designing and coding or testing. So in both of those areas, right, the AI, I think, yeah, really has migrated in the past year from kind of a assistant, right, that could autocomplete, that could write functions to now, it kind of becomes the go-to, right, for better or worse. Yeah, I think I can probably speak for a lot of engineers out there in saying that one thing I appreciate is that I almost never have to feel incompetent on any topic, right? Because filling the gap is just one prompt away. At the same time, obviously everyone has access to that, right? So it's interesting because it gives you power, but also invites humility because, you know, that knowledge is not something you own anymore.

It's very interesting because in some ways it's become democratized, right? So before it was like your talents and skills as a developer, as a software engineer, and everything that you've done plus your creativity. Is what makes you stand out. But now everyone has access to the same tools. So now is the cutting edge about creativity? Is the cutting edge about how radical your thinking or how creative your thinking is? Is it about how good of a prompt engineer you are? Like, what's the cutting edge now?

Anything can be built, right? So then what differentiates between good and bad is, you know, is sort of your intuition and taste.

Yeah, I don't even know how to define taste. I mean, I mean, what is that? Is that— I know taste in an aesthetic sense. Like, I think that this painting behind me, for example, is nice because of my taste. Someone else might be like, it doesn't do anything for me. But what does that mean in terms of software design or product design? Like, how do you— shouldn't there be something else? Something deeper, maybe.

The jury's still out, right? So we had a brief conversation, right, a few days ago where, you know, I happened to go to the Google Cloud Next annual conference 2 weeks ago in Las Vegas, right? And so the conversations are all about AI agents. But what the current evidence is showing is that people feel that their productivity Improves with AI, but at the organizational level, you know, very few organizations can really assert that their overall productivity has improved because of AI. So when it comes to software engineering, I think that no matter what, right, the you know couple of things are true, right? A, you know, even if AI becomes so good at code generation that no one ever writes a line of code again. That's already a small portion of a job, right? It's less than 20% of software engineering's time is spent actually writing code.

Mm-hmm.

And then 2, you know, the biggest problem in software engineering is interpersonal communication. You have, you have a vision for your product, right? That vision is going to translate into requirements, right? And then I have to interpret those requirements and then, you know, produce an architecture and some artifacts that are gonna be effective at solving those requirements, right? And then I have to translate that to code, right? And every step, right, there's so much potential for misinterpretation, misunderstanding, miscommunication, and we're dealing with extremely abstract concepts, right? Which compounds the problem.

Well, and I'm wondering, you know, 'cause potentially it could be a good communication tool as well as writing code. And it's just a question of how do you use it? Where does it support you? And I know that lots of people have, are trying, have been trying to use AI for personal development, for upskilling their leadership or their communication skills. I don't actually know how well that works. But it's a really interesting point that 20% of your job is actually code, but then the rest is, is really around this kind of passing the ball from person to person, from thing to thing, abstract to concrete, back to abstract, back to concrete. And that's a, that's a really interesting process. Interesting. Yeah, it's very tricky, isn't it, to be a technical leader in a company because you're required to have the technical expertise on the one side and on the other side, you have to have the social and the communication and the influence without authority and all of those things. become more and more critical as you move up. I've noticed this in a lot of people that I support and coach. You know, at a certain point, it's like your technical brilliance is almost overshadowed by how good are you at these other, you know, they call them soft skills, but I actually think they're the hardest skills.

Right. That's a good point. Yes.

It's so funny to me that they're called soft. It's like there's— Yeah.

Yeah. I remember seeing, uh, when I have an MBA from Berkeley, I went to evening program. And Jeff Immelt, who was the CEO of General Electric, gave a talk once, and he was asked about coursework, I think. And, you know, said, definitely, I wish I'd taken even more soft skill courses, right? Like, at the end of the day, when you rise in management, right, it's, it's all about people. It's not about accounting techniques, right? That's—

it's like you need those skills to, to move up, but then There's almost like a glass ceiling that sometimes is impossible to see.

Do you know Marshall Goldsmith?

Sure, yeah.

Yeah, so I think probably his most famous book, right? What Got You Here Won't Get You There.

That's right, yeah.

That sums it up.

Yeah, exactly. And, you know, at the moment we can outsource coding to agents, we can outsource workflows to agents, we can't outsource influence or communication.

Well, and yet we are, right? So just yesterday I was in a technical meeting and we were discussing— actually, we're neither discussing nor debating, to be accurate. We were considering 2 options, right? And so there are 2 engineers who've generated memos, but entirely AI-generated, right? Merits of either option, which, while I may not completely— I may not disagree with the conclusion, but the thinking is clearly flawed. Like, the arguments, you know, for or against just like, just don't add up to a cohesive argument, right? Yeah. And yet, you know, it's there, it's published on our wiki, right? And it's like, it becomes impossible to Ignore, right? And so yesterday I started drafting, you know, comments, but then, you know, I ended up not publishing the comments because I felt like, you know, why, you know, it's gonna, it's gonna come across as negative, you know, and I ended up arguing about stuff that AI wrote and, you know, you know, maybe it's better to just like let it go for now, right?

I mean, the other option is get your AI to argue against their AI.

Right, that seems even worse. Yeah. And I'm liable of this, you know, 3 weeks ago, I spent quite a bit of time in a session over multiple days, right, trying to refine some design concepts. And then I say, okay, you know, I should share this. So I had the AI, you know, create a, you know, relatively brief summary and an analysis with a cost simulation, which were deeply flawed. And so I kind of made a little bit of a fool of myself by sharing that with an engineer who was able to point out all the problems. Sadly, I still think I had a very valid point, right? But the fact that it was made with some flawed assumptions and arguments completely discredits it, right? Yeah.

I mean, we're living in a really strange time, right? Is anything Any document is just a prompt away. And then with one sentence, you can generate 5, 10-page communication brief or document or product specifications. And everyone can do this very easily, right? And so it comes at this moment where, do we do it or do we not? What are the assumptions that we haven't questioned in order to generate what we're generating? And it kind of falls into this category of AI slop, but I think AI slop is too broad of a term for what we're facing. Like for example, kind of similar to your story, I was working with an engineer the other day and coaching him and he was saying, you know, I work with clients and sometimes they post on Slack and they're like, hey, I had this idea and then I came up with this document. And clearly what they did was they had an idea, they sent a prompt, and then it generated a document. And now he's in the point where it's like he's got a job to do, he's got a product to build, and his client had an idea that generated a 5-page document. So it's going to take him half an hour or an hour to really do due diligence to read through that document. And then does he respond? Does he not? Is it a good use of his time? Is it a waste of his time? You know, there's probably all kinds of flaws within the thinking that initially even generated the prompt, not to mention that you see in the document. So it's a little bit similar to your situation where we're kind of being inundated with this stuff that's quickly generated that is not always good, but it has elements of something that's interesting.

Mm-hmm.

And so it makes me wonder in this day and age, I mean, what are the guardrails? What kind of agreements do we make as teams, as individuals, as leaders on how to use this technology so that it's, it's funny, it's like everyone's talking about how efficient I am and I'm wasting more time.

It's, it's well illustrated by the, the software development lifecycle where The ability of AI to generate code, you know, far exceeds our ability to review that code. So that creates the bottleneck, right? Where obviously you can also have AI do code reviews, right? But I think that this is kind of where the conversation is happening, right? In terms of because this technology is, you know, imperfect, non-deterministic, because of the risk of losing sight of what's running under the hood, right? Most organizations for now are saying, okay, we need the accountability needs to remain with humans, with people, right? But then if that's the case, right? Yeah, then we have the problem you described where there's this ability to generate so much content and we can only absorb so much.

So I'm wondering in your role, is that something you're thinking about? And if so, how about guardrails? About agreements on these kinds of situations, like what you mentioned with you generated something, someone else generated something, and both of them, it didn't quite have the outcome that you wanted, it sounds like.

Couple of things, right? So I'm hoping, number one, that, you know, all these issues are sort of transitional, right? In that this technology is evolving and how we use it is evolving. And so hopefully, evolve toward a state where, you know, we kind of normalize those capabilities today.

Yeah, it's very interesting. So our conversation has kind of meandered through like the changes in technology. What does it mean for people's roles? And then it's gotten a little bit into this category of just because you can maybe doesn't mean you always should. generate something. And so I'm curious, what do you see going forward? I mean, you said the technology, obviously we're all kind of sitting around saying, okay, it's gonna get smarter, it's gonna get better, it's gonna hallucinate less. But what direction do you feel like, like if you were the leader of Anthropic or OpenAI, what direction would you be pushing for? That you think would be the benefit to society or the benefit to, let's say, like your world, JD?

You know, I think the— so most of my career, in a way I've positioned my professional profile, I've looked at sort of using information technology to manage physical infrastructure systems, right? Transportation, manufacturing, where the common thread is you have sensors out in the world, right? Where they could be sensors taking physical measurements like temperature on a machine, right? Or it could be location of vehicles, right? And so from all these inputs, you build what some might call a digital twin, right?

Right.

It's basically a digital model of that physical environment that you're interested in. whether it's a transportation system or a factory. So, you know, maybe without answering your full question about benefits to society, you know, I think, yeah, you know, there's a lot that AI can do in that realm that would be very beneficial in that, you know, the things we typically care about, right, in those physical environments are generally primarily safety. Right. You know, whether it's transportation or manufacturing, you have people involved, right? And keeping them safe is usually the highest priority.

In a world where AI gives us more data than ever, the leaders who stand out are the ones who also listen to their body's intelligence. That's why I've created a free somatic intelligence toolkit with practical exercises and use cases. Grab your free toolkit below. Now back to our conversation.

Then safety competes with efficiency, right? Because usually these are industrial systems, you're trying to get something out of them. And then there's environmental stewardship, right? All these activities pollute, right? And we want to avoid that. So now there's, besides the LLMs, which have really kind of been in the spotlight now for 4 or 5 years, there's this branch called physical AI. Where people are trying to develop models that quote unquote understand the physical world. So an LLM does that sort of indirectly through language, right? We describe the world in language and then, you know, you can draw inferences through language. You know, now the LLMs also have access to scientific concepts and mathematics and programming so they can even go further and model things. But the model itself does not have a 2-year-old intuition for, if I drop this ball, it's going to fall to the ground.

It's kind of like a floating brain in the tank without any of the sensory organs.

That's right. That's right. Exactly right.

That's how I think of LLMs. So, okay, so you're talking about these physical AIs and And what's the hope there? What's the idea? How would it impact what you're talking about?

Well, I think the biggest impact is going to be robotics, right? Is that you can now have autonomous agents that are not just, like you say, a floating brain in a data center, right? But can actually be physical entities in the world, both sensing and responding, reacting. Right. So yeah, that's, that's major. But, you know, in parallel to that, it could mean also that, yeah, we get just vastly better capabilities at managing infrastructure, managing supply chains, all things that can mean, you know, better productivity, better outcomes, and also very importantly, can help us mitigate the environmental crisis that we face, you know, not just climate change, but also, you know, all of the other impacts. that we're having. Obviously that's kind of the optimistic version of this, right? Because unfortunately, you know, technology tends to generally create just as many problems as it solves, right?

Generally, yes. And we just keep solving new problems that we create for ourselves.

That's right.

In an endless cycle of ingenuity. Okay, interesting. So there's like this hope for multiple kinds of intelligence. You know, LLM is the floating brain. And then there's, there's potentially like actual physical intelligences that have senses that can operate in the world, that understand the world the way that we do. I'm curious for you, as you see this moving forward, do you yourself see becoming obsolete in terms of your role and your skillset? 'Cause I know that there's, there's a bigger part of JD besides JD the engineer, JD the software architect, right? You have an artistic side as well. So how do you see, how do you see this evolving over time? I mean, what are you excited about? What are you worried about?

So, you know, yeah, so going back to digital twins, going back to software engineering, LLMs, I think that's one thing I'm very interested in is this notion of representations and how we evolve representations to become more efficient and utilize AI and can become both more creative and get more control in that way. So for example, if you think of mathematics, mathematics has notation language. No mathematician would think of using plain English to do mathematics because it's just inefficient. So likewise, if we're programming software systems, code is often the most concise and precise way to express any given concept.

So it's very interesting that in a way our limitations right now with this silicon intelligence is that everything has to be done through language. You know, that's—

Well, so not all, right? So that's just a feature of the LLMs themselves.

Sure.

But there's other AI, like, you know, AI started with, you know, modern AI, you could say, started with image recognition, right, where there is no language involved, right? But yeah, the, obviously, language as an interface is what's made it, you know, so popular and democratized, right? But it comes with all kinds of problems as well.

It does. It does. Absolutely. It kind of makes me think of some, like, sci-fi book or something where, you know, civilization has figured out a way to move past communicating through language into other forms of communication. You know, like I think about, I don't know, we're kind of going off a little bit, but it just makes me think of something like ants, you know, like ants communicate through chemistry, not verbal. And I wonder, do ants have miscommunication? Maybe less.

Of a different kind, probably.

Okay. Interesting. So I really, you know, it's interesting what you're thinking about because in a way it kind of challenges some of the fundamental assumptions of how we write code or how we develop software. You know, right now I think it is very strange, isn't it? Like, let's say I tell an AI, write me a program that can predict when I can exercise during my schedule. Look at my calendar and then tell me when I can exercise. Right? That seems like a very plausible thing. But then I think the challenge is there's so many assumptions. And I think that it's very difficult to surface assumptions. This is probably the most difficult thing of, and I see this a lot actually in communication as well, like going back to some of the initial concepts that we were talking about, about leadership, about communicating to others, about miscommunication, about proving value or getting known in your company. Like a lot of it is really around, and a lot of the work that I do is Helping people break down communication into almost like Lego pieces. You don't want to give someone a whole castle. You want to first take it apart and see all the individual pieces so that when you're giving someone something, it's very conscious, very thoughtful.

One pattern, right, that, that seems to really work well is, you know, going from structure to unstructure back and forth.

Give me an example of that. What's the story?

Right. So, well, so for example, I had a colleague who was researching, you know, prototyping some AI capabilities for our product. That was before I joined the company, but I know of the experiment. And so they started with, you know, a very broad problem, right? And then because obviously the AI could not solve the very broad problem without instructions, they started then chunking up the problem into steps. And that did not work that well either. They realized that actually it was maybe better to let the agent sort of come up with, do the reasoning by itself. But the one thing that really helped is when they started prompting the agent to give structured responses. So for example, formulate a query to fetch a certain dataset, because then the response can be formally analyzed. It naturally sets the guardrails. And in fact, that's how agents now, a lot of them work, is by invoking tools. And to invoke a tool, they have to create a a formal query that follows a format and has some specifications to it, which LLMs were not good at doing that just 3 years ago. That's really when they started becoming better at this, that agents became possible.

So by making that query, it got structured. Now, does it go back to unstructured at some point?

Right, because you can go from unstructured to structured, create a query, get a structured response, back, now you're going to interpret that structured response into an unstructured interpretation, right? And back and forth. But I really, I think that, yeah, this, you know, going from structured to unstructured, back to structured, and, you know, that I think is a powerful pattern compared to, you know, traditional programming is, you know, structured, structured, structured, structured, right? AI slowup is unstructured, unstructured, unstructured, right?

Interesting. I really, I've never thought about that approach before. So I'd be curious to kind of take that and maybe for our listeners, this is kind of like one JD takeaway is experimenting with what does that mean for your work to go structure, unstructured, structured, unstructured. And that process of going back and forth, it's like a translation in a way.

Right. Right.

Yeah. Very interesting. I see that we're, we're getting close to time, JD. Is there any takeaways or ahas that you're, you're, you're coming to from this conversation?

You know, I think the one thing that comes to mind is, you know, you pushed me a little bit on perspective and I feel quite clueless. I think I don't feel bad about feeling clueless because I think that, you know, no one really understands where this is going.

Yeah, yeah. But it's an interesting thing.

As a father, you know, I will say that, yeah, like I don't even know what I should tell my kids to study, or, you know, they worry that, you know, there's the technology and then there's the, you know, capitalistic frame in which it is being developed, right? Which means that, you know, we've got very few people, you know, accruing the profits, the control, right? You know, they've essentially sort of, you know, taken the entire base of human knowledge, right? And now making us You know, pay for it. And that's, that's, you know, that's disturbing. But then, you know, right now, that's the model, right? So, you know, the upside is we're getting this dynamism and these, you know, incredible, you know, improvements in technology, right? But then the downside, it's, you know, still, you know, fairly unregulated, and it's going to, you know, it's creating imbalance.

Thank you, JD, for this interesting conversation.