GA 190 | How to Use Lean in Software Development vs. Operations with Tom and Mary Poppendieck

You're listening to episode 190 with Mary and Tom Poppendick.

Welcome to the Gemba Academy podcast, the show that's focused on helping individuals and companies achieve breakthrough results using the same continuous improvement principles leveraged by companies such as Toyota Del Monte and the U.S. department of Defense. And now, here's your host, Ron Pereira.

Hey, this is Ron Pereira with Gemba Academy, and I'd like to welcome you to another edition of the Gemba Academy podcast. As always, thank you so much. Welcome for taking time out of your day in your week to listen to the show and for watching Arlene and Six Sigma videos over at gembacademy.com, we definitely appreciate each and every one of you. Now, today, I'm excited to welcome Tom and Mary Poppendick to the show. And Tom and Mary work in the lean kind of software development world, and they were recommended by, I think it was a listener shot me an email and said, hey, you should have these folks on. And so we got connected and we were able to make it happen. So during the show, we talk about lean within the software development world and how it differs from the more traditional operational role. And Mary and Tom both have experience in both the software world and also the operational world, so they're able to offer a nice contrast of the two and as well talk about where they're similar. So show notes for this episode can be found over@gembapodcast.com just look for episode 190. 190, and again, gembapodcast.com and look for episode 190. Okay, enough from me. Let's get to the show. All right, Mary and Tom, thank you so much for coming onto the show. Where are you guys calling in from?

From Eden Prairie, Minnesota.

Nice. Okay, we don't have 10 inches of snow or anything yet, do we?

Actually, we had a few flurries a couple days ago, but we're not really expecting snow for a couple more weeks.

Okay.

All right, good.

Stu, where are you calling from?

I'm in Texas. So we don't have snow, but, you know. But, you know, it's been. It's crazy here in Texas. You know, it's 80 degrees one day and then 30 the next. And so it's all over the place, but, you know. Yeah, we deal with it, so. All right, well, hey, thanks for coming on. I really appreciate it. Kind of the topic today we're going to focus on is kind of lean and software development world and whatnot. But Tom and Mary, we'd like to kind of Begin our shows with our guests sharing a quote or two of something that possibly inspires them. So do you guys have one?

I got a quote. It's a quote from Drucker, and it's the purpose of a business is to create a customer. And there's too many people saying the purpose of business is to make money for shareholders or things like that. And I think that kind of attitude drives around behavior. And if you think about business as all about creating stuff that customers like so much, you don't even have to sell it to them and always delighting them, I think you start out on the right foot, especially if you happen to be in the software world, where I'd say a third or 40% of our colleagues sit in IT departments rather than business units. And it's important to think about why a company is in business when you start thinking about what are you trying to do with your work every day.

Good stuff. Good stuff. All right, let's go ahead and tell everybody a little bit about your background, maybe how you first came to learn about all this continuous improvement stuff and then what you guys are up to these days.

I was IT manager in a manufacturing plant in the mid-1980s, a 3M manufacturing plant. We were making magnetic tape that you, you know, that you show movies on. You don't do that anymore. You use digital. But 3M invented magnetic tape. And we were in heavy competition with Japan. And we discovered that they were making tape cheaper than we could sell it for. I mean, they were selling tape cheaper than we could make it for. And we had to do something different. So we started studying what was going on in Japan. And we instituted a very interesting pulse system in our plant following their just in time schemes. Although we couldn't do exactly what they do in automotive because we're in a continuous processing plant. The management team studied what was then the literature from Toyota, from Shigeo Shingo, his first screen book. We read bad translation from Japanese, but really interesting ideas. We instituted those in our plant and we moved to a pull system. And the results were dramatic. Just like we couldn't figure out how to do it in just a little bit of the plant. So we had to do it plant wide one weekend. And overnight we went from a ability to fill orders on a week by week basis at about 60, 63% to about 95% the first week and up from there after that because the pulse system was so much better at moving the material through our plants. That was my first experience with what came to be called lean and I worked at 3M doing that sort of stuff for a while and then I worked on, because I was in software, I worked on putting information systems in manufacturing plants, especially to gather quality information and correlate it with process information. And eventually I got into corporate and I did product development rather than manufacturing. And eventually I did a early retirement from 3M and I started looking at what was going on in the software world because I still needed to figure out something to do with my time and earn a bit more money. And I discovered that there was this thing going on called waterfall, which I said, huh, that's kind of weird. How can that work? And I said, no, these guys don't get it. This is like we were in manufacturing before the lean days. This is like great big batches of stuff. Eventually, after working in a big state government project for about a year and a half, I decided to write a book about how you could apply those principles we applied in our manufacturing plant to the software development world. That book, Lean Software Development, was published in 2003. Tom and I are co authors of it. We subsequently wrote three more books in the area. And we've spent the last, what is it, 15 years traveling around in the software development world talking about how you take Lean concepts and apply them to the software development environment.

Okay, well, good stuff. Let's jump right in. We've had a few folks come on over the last few years kind of talking about software, but not many. So I guess the first question is what's different about Lean and software development versus the more traditional kind of operational environment.

Okay, so let's just talk about what's different about Lean in a development environment, period, rather than software development, because development does not work. The very specific tools that are very common in manufacturing and that even I used in manufacturing don't work in a development environment because your goals are different. The whole goal of minimizing variation, there's nothing about that concept in development. In fact, your goal is to vary things. If you look at how Toyota does set based design, the goal is to look at variations and see how they affect the outcome. Development is more about you're a chef and you're creating a new meal. Operations is more like the meal is created and now you need to reproduce it reliably. Although the fundamental ideas of Lean, the theory of lean applies in product development, the actual practices applied to manufacturing are usually a disaster if you try to apply them to software development. So I've seen that done really badly occasionally. But most of the people in development understand that the manufacturing techniques are not what you're about. It's the theory, it's the problem solving. It's the understanding how to respect people in the environment. It's the way that leaders lead the problem solving exercises. It's the flow, the constant thinking about moving stuff through rapidly because stuff doesn't get any better sitting around. It's that kind of stuff. In software or in any product development, your inventory, the equivalent of inventory, is the mental stuff that you put the thinking you've put into it that's half done.

Yeah. Give me an example or two of a problem that you might approach within that software development world.

Well, so you're at a bank and you want to create a new app so that your customers can do, you know, say, get at their bank account in a different way or something like that, or you're at a company that wants to do something new online. The first thing you have to do is you have to figure out where is the customer, what do they really want? Now what Lane says is you need to be really focused at every step on the customer, and you need to figure out how to take a customer problem and rapidly solve it. And it's that time from when the customer has a problem until the time that you solve it is what we always put into a value stream map. What is the process of going through? The customer has a problem and how do you get it solved? The first thing you want is you want the people to understand what is the real problem. What are the customers doing then? What you tend to do in software development, what you used to do in software development is come up with a great big set of requirements and send them off to a contractor and then have the contractor execute against the requirements and eventually come up with a product and you deploy it. Well, you know, that's not how software is developed in the web world anymore. You come up with a small starter website after really understanding the customer, you solve a piece of the problem, you deploy it. You get a lot of feedback through anything from keeping track of your data analytics to looking how customers make comments on your on your social media. And based on those comments, you do relative very quick adaptions. When we started in this, the typical time between deploys of software in an enterprise, six months was fast. So you would build up a batch of software that was six months or more long, three months. And in that time you'd kind of wait till the end before you tested it because people didn't want to test and then retest as more stuff got added. So you had these massive batches and when you deployed them, everything sort of broke. Ten years into this whole area of agile and lean software development, Jez Humboldt published a book called Continuous Delivery. Today, most companies for most software put software into production every day, sometimes every hour. Oftentimes Amazon, it's every 11 seconds they deploy something new. And what has happened is people build up the tools around that to make sure that that's safe and that it doesn't break anything. So we've gone from really big batches of software to a very rapid flow from understanding the problem to getting it solved. Now in that whole area, you're not trying to reproduce anything, so you're not worrying about variation. What people used to worry about is, did I execute according to plan? Well, that's not the game we're playing here. Instead we're saying, here's our customer and how do we rapidly deploy small batches of software so we can understand what that customer wants and rapidly adapt it in a very safe manner?

Yeah, it's very similar to Eric Ries and the whole lean startup movement with the minimal viable product. Sounds very similar.

Right. That was published in 2010 also. So that came about seven years into the whole area of lean software development.

Yeah, yeah. Good stuff. What about like, so back to your 3M days, I'm sure you came across, you know, kind of one piece flow in a, in an operational manufacturing style environment. And probably that's what we did at our plant.

We went to one piece flow.

Yeah. So obviously it's superior to moving in batches. Well, what about, how does that apply to software development?

Well, the batch used to be a project. We don't do projects anymore. A project is a big batch. If you go down to a feature, you have a small batch. If you go down to deploy every day, you have Continuous flow.

Got it. Now what about the defects? Obviously an advantage of one piece flow is defects come to the.

Let's go back to my manufacturing plant. When I have continuous flow, we can't have defects anymore, right?

Right. Well, you can, but you're going to immediately stop the line, right? Yeah.

As soon as you're done with something, you hand it to the next workstation. If it isn't perfect, it will immediately be detected. Defects don't persist. If you allow defects to accumulate in a batch situation, there's where your problem lies. But actually when you have a continuous flow, you, your quality has to improve. Similarly, if you're going to deploy software all the time, you can't allow defects. We have for Decades in software assumed that defects were necessary and a part of the game and tried to reduce them to a certain level instead of doing what Toyota did and says, well, why do we have to have defects in the first place? Instead, why don't we create, why don't we use this software that we software people are so smart on using for other people, for ourselves? Why don't we figure out how to create an environment in which mistakes are very difficult to make? So what is a defect? Typically, it used to be that defects were described as something where the product doesn't do what the spec says it's supposed to do. But if you could take your software and have an executable spec and execute the spec against the software before you ran it, then you would know for sure that your spec and your software actually do the same thing. So it's like double entry book counting. You have two ways of making sure that it's doing the same thing. So all of that manual testing in a continuous delivery doesn't can't happen. Instead, you describe what you want in a language that's in the domain, something that people can understand if it's a people domain. And then you run, build the software and you run it against the existing spec, which happens to be automated. Now, this kind of stuff was done in aircraft and medical devices for decades before the, you know, the 2000s, but not in enterprise software. And today software doesn't go through. Let's create a big batch and then test it all, the end at the end with manual testers. And of course, if we find defects, we're going to get really as many as we can. Instead, it's from the time I, as a developer, as a software engineer write some code, within three minutes I've pushed a button, that thing has deployed, it's compiled, it's run down a pipeline, and somewhere between three minutes and three hours. I know that it's exactly right, that it's not only that it's right, but it isn't going to hurt anything else. So if you have all the automation to make sure that defects can't happen, it doesn't mean they never happen, but it means that they're not considered normal.

There are several kinds of defects. The defect that Mary's just described is basically, does it do what you want it to do? And the mechanism of having a test that only passes when it does do what you want to do is tremendously effective. A more serious kind of defect is, is the thing that you want it to do, the right thing to do or have you built the wrong thing? And that is another place where the continuous delivery, not just continuous integration, but the continuous delivery to the environment where the software will be used has a tremendous impact. If you deliver a little bit at a time, you can get feedback rapidly about whether this is going to be an effective solution or not. And if it is, you continue working on it. And if it isn't, you adjust to improve how well what you're delivering addresses the problem. So that is a even more important kind of defect than the software has a bug. It's the design has a bug kind of defect.

So I have an interesting question. It's not even related to Lean, but it's software. We were actually talking about it today with my colleagues here. This artificial intelligence and how this. I think I read an article recently where this, these AI engines, if you will, are writing code now and creating software that even like Google engineers can't wrap their minds around. Have you seen anything with this? Where is that going to take the software development world?

Well, you can ask where is it going to take it? But I was just talking with Tom and back when I was doing code, I was a process engineer and I was writing stuff to do automated testing on roll goods lines and I was doing stuff to train robots how to move things and I was actually doing stuff to create 3D visualization of 2D film. So the roots of all of this software that can handle the real environment go back a lot of decades and I've watched as over the years it's gotten more and more sophisticated and more and more things can be done. I mean, you remember when there weren't cell phones and now there are. You don't want to go back to when there wasn't any Google. I surely don't. It's cheating because you could always go to a library and the same thing is going to happen. Once you get into a world where automated cars become common, you don't think back to the days when people used to have to drive them. Just like now TV is ubiquitous. But when I was seven years old, that's when we got our first tv. And when my daughter was seven years old, that's when we got our first microwave. As these tools come of age, we figure out how to fit them into our world. Now I can tell you how much software has been written that people don't really know what it does. It's not just artificial intelligence. How do we know that our banking systems are working right? For heaven's sake. So I mean this has Been a problem in medical devices for a very long time because we use software in medical devices and when you're not got it perfect, it can kill. The very first airplane that was controlled not manually, but fly by wire was the 747, wasn't it? Developed in the 1990s.

The 777.

The 777, that was the first fly by wire airplane. And before that you would never have thought of doing that. When I did process control systems, the engineers told me, you can do whatever you want, but we're going to maintain hardwired emergency stop systems because we're not going to let you touch the safety systems. And as the computers got smarter than the humans were able to do, as they were able to automatically do more, just like now, we can test much better automatically than we can manually because we can put more smarts into it. If you apply the increasing capabilities of computers to solve problems, the world is going to get better. If you do dumb things where maybe the things don't behave exactly the way you want. Well, one of the things we see is there's capability of messing up the financial markets and maybe there's capability of doing other dangerous things. But that's always been true. I mean, software has always been able to do good things and bad things, just like any kind of machine has been able to be properly applied or not properly applied. So yeah, we have to be careful and we have to understand what these things can do. But we surely want to continue to improve our world with the intelligence that smart machines can bring.

Artificial intelligence is extraordinarily powerful in a narrow domain. It is not intelligent in the sense that even crows are intelligent. It's not intelligent in the sense of recognizing and solving problems. Given a good set of data, it can extract a good answer. But it has to have two things to do that. One is it has to be told what the question is. And the other is it has to have training data that will enable it to learn, to recognize the patterns, to address the question that you've given it. What it can't do is identify relevant questions and what it can't do is identify good data to train on. So for quite some time, I believe the domains of artificial intelligence will be fairly narrow.

Yeah. Okay. Well, good stuff. I was just curious. I know. Sorry to get us off on a tangent there, but no problem. Yeah, so we mentioned Google and Amazon. Both curious. I know obviously most folks listening are familiar with Amazon and operationally they definitely have some. I have friends that work there. Very strong culture of Continuous improvement. But I'm not familiar with the software side of the world. So do companies like Amazon and Google practice Lean on that side of the world, the software side?

It depends upon what you mean by practice lean. And I don't like to put quotes or put people in a category because they do something or don't do something because too many people mean lean by use a certain set of tools as opposed to lean by solving problems. Now, one thing that both Google and Amazon are really well known for is they're both engineering driven, which means problem solving driven, and they're data driven, which means they use information to make decisions. Now if you take a look at, you know, back when in the like, late 90s, early 2000s, and Google was trying to scale massively because it had this ambition of gathering together all of the data in the world and answering questions that people presented to it almost instantly. There were incredible technical problems to be solved to make that happen. And the real amazing part about Google is that they had some just incredibly effective engineers that were able to just tackle those problems with one problem at a time and come up again and again with increasingly sophisticated solutions to solve the problems as they came on. And what you'll see with any really seriously good engineering company is really good problem solving skills. And that's what they do all the time. They find problems. They're incredibly good at looking for what we would call root causes, but they might call something else, and then for coming up with ways around those and, and fixing them. And they'll do it based on data with a certain amount of intuition. Those two companies are really, really good at that. Now, is that Lean?

I don't know.

They're very customer focused, incredibly customer focused. Both companies in a slightly different way, but very much they're driven by customers, not by finances. So if you go back to first principles of Lean. Yes. If you go in there and are you going to find a 5s program? Maybe not.

I'm sure you know that Toyota does not use Lean, they use the Toyota way, Right. I'm sure you know that Scania does not use Lean, they use the Scania way. Every company that has applied these principles in a successful way comes up with their own way of doing it. What's common among them is not the tools, it is the problem solving approach, the customer centricity that manifests itself in many different ways.

Yeah. It's interesting as Toyota employees do read Lean thinking, though. How ironic is that? Yeah. So good stuff. All right, well, hey guys, thanks for coming on. Let's go ahead. And I got a few just kind of questions here on the reflection side that I'd love to pick your brains on. This first one we've asked for several years. You know, within the traditional lean world, we talk about kind of those two pillars, respect for people and continuous improvement. Continuous improvements, obviously the things we can wrap our arms around and touch and do. But respect for people can be a little bit harder to understand and grasp. So curious, what does respect for people mean to you?

Okay, let's go into the software development world. When I was doing stuff and I was called a programmer, but actually I was in the engineering department, we were given problems, serious problems, and worked together as a team to solve them. By the time that I left, I got into the manufacturing world and product development world. By the time that I left and came back into the software world and ran into this waterfall thing, people who wrote code were being treated like order takers. They were being told what to do. And specifically, here's the very narrow task that you have. And don't ask why. They were divorced from customers and the job was no longer fun. Which is basically why I got into trying to change it back into a job that was engineering focused and a lot of fun. The order taking mentality does not respect the smart, intelligent software engineers that could make great contributions to ideas of how to solve problems and how to make stuff happen. So in a really good lean software development environment, you have complete teams that focus on solving customer problems. Which is why I said, which is why my favorite quote is from Drucker that says, the purpose of a company is to create a customer and the purpose of a team is to create something that's good for a customer. So instead of software development being this big, big batch order taking environment, we're trying to turn it into something where the smart engineers that do the software engineering are respected and involved in solving the customer problems.

What's the best advice you guys have received?

Received. When I was writing my first book, one of the reviewers who was pretty influential, said to me, don't write and don't say what other people want to hear, because if you really want to have an impact, people want to hear what you think, not what you think they want you to say. And I realized that he was criticizing me because I was doing that. I was listening to. What I thought people might want to hear is sort of like, let's put a marketing hat on instead of saying what do I believe is really true? And once I started thinking much harder about what do I believe is really true. It became a whole lot more fun and also a lot more people cared to listen because I was not saying what everybody else was saying. I was saying something different.

Tom, what are you most proud of as it relates to your journey with continuous improvement?

Back in 2003, we introduced several dozen memes, several dozen ways of thinking about things. We've been talking about them for many years. And today those have become fundamental ways that people in software think about what they're doing. And we don't even get attribution anymore. And that, I think is great, that it has become totally embedded in the way people think about software, rather than just something that they refer to some authority to talk about.

Yeah, well, they say that the best flattery is when someone steals your work. I guess I don't know what the exact quote is. Yeah, good stuff. All right, well, last question. Obviously you guys have written some books. Are there any books that you guys have recently read or in the last year or so that you would suggest folks pick up? It doesn't even have to be lean related or anything. Could be anything.

One of the things that's happening in multiple organizations today is that people are not thinking about software or manufacturing or any given part of the the organization separately. They're thinking about it as just one of the contributors to the purpose, which is to delight their customers. And they're finding that they have to connect all these things together. And in many cases, this is resulting in what is called digitization, whereby the whole customer interaction and the whole effort to support the customer interaction is mediated by digital interactions by software. And this is really, I think, starting to blur the boundaries. It's blurring the boundaries between businesses, between technologies. And the focus is shifting away from lean as a manufacturing or logistics approach or marketing based on the constraints of the manufacturing and logistics strategy and focusing instead on a much broader question.

Now, was there a book that you read that's based on this or is this something you've been thinking about?

Well, because of this, it has migrated from what is under the primary control of lower level people in the organization all the way up to the executive suite. And I think one of the most interesting books that has come out recently that will help organizations at the executive level adjust their thinking is probably Seat at the Table by Mark Schwartz. Schwartz. We've run into a number of executives who have been reading it and starting to act on the ideas in that book. A Seat at the Table.

Seat at the table. Very good. All right, well, hey guys, thanks for coming on why don't we wrap the show up with you guys sharing some final words of wisdom and then be sure to tell folks how they can connect with you.

I have a website called leanessays.com where I post something every few months or my website is popindick.com, but you have to spell our last name to get there. It's www.p o p p e n d I e c k dot com. So that's how to connect with us. And what was the other half of the question?

Yeah, just final words of wisdom.

I think that we have recently encountered Jeff Bezos letter to shareholders in which he had one particularly important bit of advice, and that is avoid proxies. Act and measure on what really matters. Not on proxies for what matters. Process metrics, financial metrics are probably proxies for really delighting the customer, which is the purpose of the enterprise.

He had another thing in that same April letter to shareholders. He has four things that will keep a company, even big companies much more vibrant and much more like startups. And his first one was customer obsession. And we have some people here now in Minnesota that work for Amazon. And I've I know that one of the things if you're a partner, if you're a merchant, no matter who you are, that will be beaten into you no matter what other things are important, is if you don't focus everything starting on customers, then that's not the company to be at. So I think this focus on customers and don't have proxy measurements, go straight to what you're doing to help customers get what they want are two really important things to think about.

All right, well, thanks again for coming on. And as we approach Thanksgiving, this episode will come out right around Thanksgiving and fact. So great. Yeah.

So can you send us an email with a link?

Yeah, we will. For sure. For sure. Yeah. Awesome. All right, well, thanks again, Tom and Mary.

Thanks a lot, Rand.

All right, you guys take care.

Okay, bye. Bye.

Bye. Bye.

Thanks for listening to the Gemba Academy podcast. Now it's time to take a free, no strings attached, fully functional test drive of Gemba Academy's School of Lean and Six Sigma. Over at Gemba Academy, gain immediate access to more than 500 Lean and Six Sigma training videos free of charge at GembaCademy.com.