Seeing is Believing: How to Apply Computer Vision to Transform Manufacturing

Welcome to the Industry 4.0 Club's fireside chat. The club's purpose is to engage the power of diverse worldwide talent to accelerate the global evolution to Industry 4.0. We accomplish this by hosting and supporting respectful, thoughtful, and interdisciplinary open conversations and creating quality content on wide-ranging Industry 4.0 topics to expand the reach of our community. My name is Mike Ungar, and I am the CEO of the Industry 4.0 Club. I have over 40 years of experience, most of that in manufacturing at Michelin. I'm joined today by my co-host, Mike Yost. Mike, would you please introduce yourself? Sure.

Thank you, Mike. Welcome, everybody. As Mike said, my name is Mike Yost. I'm the volunteer chairman of the Industry 4.0 Club, one of the co-founders of the club, and Mike's co-host here on the fireside chats. I've been in industry for about 35 years, with the last 15 on the industrial nonprofit side. And enjoy having these conversations to meet the mission that, that Mike talked about a few moments ago. So today we're really excited to be joined by Brian Marsh, who is the Director of Controls and Automation at Allegiant. Brian is going to be at the 2nd annual MX.O Midwest event in Cleveland, Ohio on September 17th, and he's going to be moderating a panel discussion called Seeing is Believing. Applying Computer Vision to Transform Manufacturing. So Brian is the Director of Automation, as I mentioned, at Allegion. He leads initiatives that integrate digital transformation into advanced manufacturing systems. He specializes in developing and deploying automation strategies that enhance efficiency, reliability, and scalability across operations. Brian collaborates with cross-functional teams to implement smart technologies and data-driven solutions that align with the long-term business goals of the company. His work is focused on delivering measurable impact through innovation and continuous improvement. So we will get to Brian in just a moment. And for those of you who are following along, listening on LinkedIn, if you have questions for Brian, you want to interject something into the conversation, please do so in the comments box. We will get to whatever questions we can. But in the meantime, Mike's going to tell us a little bit about the MXO Midwest event.

Yeah, thanks, Mike. So the MXO Midwest event coming up here in just really a couple of weeks now has become the go-to gathering for manufacturers in the region to connect, collaborate, and learn from real-world examples of successful digital transformation shared straight from the horse's— well, I guess the manufacturer's mouth. So beginning in 2024, the event has empowered local manufacturers to advance their careers and businesses through valuable knowledge exchange and impactful networking opportunities. So hopefully we'll see you all in Cleveland on the 16th and 17th. So with that, welcome, Brian. We're glad that you could take time out to join us.

Thanks for having me. It's great to be here and to talk about a great event.

So, well, that's great. So You know, can you kind of frame what we mean by computer vision to get us started? And is it the same as machine vision, or are those 2 different things?

Oh, that's a great question. And I think it's kind of blurred a lot in the last couple years. You know, computer vision is really just using a technology to replace what's typically done by a human, right? So it's converting that. I think it can use AI and other tools to kind of assist it in a lot of ways. So I think that's more of a generic umbrella term that's used for lots of things. It could be agricultural or equipment. But typically when I start thinking about what machine vision is, I kind of drop down into the factories or the industry kind of definition of that where you're using computer vision-based technology to solve a problem or to assist with manufacturing. in a very given, you know, industrial process.

Now, I know by background, Brian, you spent a good bit of time in the food industry before joining Allegion. Do the technologies apply equally where you are today versus where you come from before?

I think there's foundational elements that are the same. So, you know, I think there's lots of things that can be done with vision that are in the quality and safety space. that are pretty generic and can cross different industries. There's also a lot of individual use cases that are associated with the processes, and there between the two industries I've seen a lot of differences in that.

Yeah, well, that's pretty interesting. Now, you know, having had a background in manufacturing, one of the most important things that we always start with is safety, right? It's a priority. Then we consider that. How does computer vision improve health and safety and effectiveness in your opinion?

Well, for me, you know, historically when I think about all the unsafe things that I've experienced through my manufacturing career, you know, I always hate to bring up that topic because it always scares a professional. You want to stay away from bad effects of that, but, you know, it's really around trying to target what I would call abnormal operations. And what I mean by that is There's a higher degree of having a safety incident when something abnormal is happening in the process. Doing normal daily activities that you normally do as a frontline operator, you know, usually go pretty normal. You don't have high instances of safety risks. When things start abnormally happening, like, give you an example, like if you have a case doctor and now all of a sudden You had a whole bad bundle of cardboard that came in with bad creases or what have you, and you're starting to fight your equipment to make it work. You're clearing jams at a higher routine. You start getting into a higher degree of having an unsafe condition. And I think vision is a really good technology that it can monitor with the assistance of machine learning. It can monitor for those abnormal conditions, which are hard to detect. Sometimes because they're random in nature. Really, you know, you can watch a solution using machine learning and kind of determine what normal is. And then when you start having these abnormal things that pop, you can give alerts either to the operator to remind them of watching their safety behaviors and/or even, you know, having a super— something as simple as a supervisor stop by to check and make sure everything's going. Well for the operator and seeing if they can get any assistance to try and get away from that.

Yeah, that's a, that's a good example. So, you know, I think about cameras on the shop floor watching operators work. Have you seen— I mean, that takes a good bit of savvy to communicate that well to operators, right? You know, because it can come across as Big Brother. Have you seen that? Channeling that well or not?

No, I think that's almost every application of Vision that I've done in any factory that I've gone into. You know, I always emphasize with my team when I'm coaching them around change management, right? You don't want to just drop the technology in there without having some preface of why you're putting it in there, etc. I think that's the most important part of it is just explaining to the individuals that are around it what's going on, what's the intended use of it, etc. I've also seen just recently in the last 6 months, there's a lot of newer technology that can actually put privacy guards in place to protect like the faces and personal elements of the people that are in there, you know, the auto-blurring capability and that kind of stuff. So you can still maintain, you know, the use case that you're trying to do with the vision system and protect the personal aspects of why people are concerned about that.

Mike, let me just finish that and ask another question, but if you don't mind. Brian, have you seen— so have you done this? Have you seen this done in both union and non-union environments?

Yes.

So let's talk about the challenges. Have you had specific challenges in a union environment that were different than non-union?

I think it's about where the voice comes from. You know, in a union environment, it's typically funneled through the leadership of the union to the management. But if you think about it, I think the concern's the same. Like, the people are people, right? So everybody that works on the floor, you know, somebody's watching them. They have that degree of Big Brother looking over their shoulders. So I think the approach should always be the same regardless of whether it's union or not union. And you should give a channel to the operators to voice their concerns and have the conversation as you're doing that. So I— yeah, there's differences, but in general, I think your approach should still be the same.

Yeah, I appreciate that a lot. I think that's something that— and thank you for sharing that. I think we tend to have this perception that there's— that it's somehow it's different. And yes, there are some rules that make it a little different in the application, but people are people. I think it's a really good observation.

Yeah.

And if you're open and clear about what you're doing and why you're doing it and what's the benefit to them, at the end of the day, that should carry the day. Yep.

Yeah, I just, uh, want to remind everybody that, uh, if you're following along, listening on LinkedIn, we're talking with Brian Marsh, who's the Director of Controls and Automation at Allegiant. If you have, uh, comments or questions for, uh, Brian, go ahead and throw them into the, into the chat window or the comments section, and we'll, uh, we'll throw them at him. Um, but I was just wondering, Brian, you talked about some of the advanced features, right? Like like new capabilities within the systems to blur faces and do a better job of, you know, protecting identities and such. Can you talk a bit about how the technologies have evolved over the years or how rapidly they change? I know it used to be a really technical, highly involved, you know, light-dependent kind of scenario to use these types of solutions, but how quickly is the technology changing?

I think really rapidly. I think when I started looking at this 5-ish years ago, it was, you know, you really had to call the smart people in the room in order to figure out how to deploy these solutions. You know, you had to have a pretty in-depth background of the electronics and the equipment, how it all worked together, maybe some programming experience. I think lighting is still a key challenge and you need to have good lighting regardless even today. But I've seen a lot of new solutions that are low-code, no-code now where somebody with just a mindset of a citizen developer can come in pretty quickly and deploy these solutions in a matter of hours. Where it used to be, it was a pretty big effort 5 years ago to deploy generically to deploy a solution. So I think a lot's come in the last 5 years on how quickly you can do these and the ease at which you can do it.

Yeah, shouldn't be a surprise at all with the power of, you know, the speed of technology changes and such. How much of it— oftentimes in, you know, industrial technology settings, you have You know, people that understand the tech, but they don't necessarily understand the environment, the manufacturing process and stuff. Is there a disconnect there or a gap when it comes to these types of solutions, or is that not as big an issue being specifically applied into a manufacturing domain regardless of what type of industry it would be in?

I think that's a really good comment or question to ask. You know, I think For me, there's a lot of partners out there that come from a large variety of different places. So I've seen where I've had some really good vision guys that have— know the technology well but don't really understand how to apply it in the manufacturing space which they're putting it in.

Right.

I've also seen people that are really good at doing both, right? And so I often look for that characteristic of having both before I let a given provider into my my factory, you know, like you have to understand that if you're working like above a food stream, you know, you have to be careful with using things like glass or plastics that are brittle. You can't afford to have that end up in our children's food, right? So that's— there's no, no excuse me in that, in that phrase. So you need to make sure that, that, you know, that's bulletproof and the people that are deploying those solutions really understand the impacts of bad decisions in the space. So. And that's an important thing to consider when you're choosing a provider.

And generally speaking, this is a— is this a type of resource or capability you get externally? Talk about working with third-party providers on this, or is this the type of technology or capabilities you potentially may bring in-house so you can deploy these types of solutions?

So this is just my personal opinion. It's kind of shifted, to be honest, in the last probably about the last year. I used to depend a lot on providers just because the technology was hard, but I'm starting to find that there's a lot of cloud-based solutions that you can go buy, buy this equipment and program it fairly easily with in-house capabilities and deploy it ourselves. So I've been shifting a little bit in terms of my decision space to move more to build it versus buy it, just because it's become a lot easier in the last 12 months.

Nice. We talked earlier about safety. What are some of the other problems that manufacturers might look to computer vision to help them solve?

The other 2 big areas are quality and maintenance. On the maintenance front, as an example, these cameras used to be pretty expensive when you deployed them, and now you can do it with just typical webcams. Things that we would have never done 5 years ago, like put a camera in every electrical cabinet, to monitor the thermography of that cabinet to see if there's something that's overheating or what have you. I've seen where you can now do that fairly economically to avoid— as a form of predictive maintenance on that cabinet real-time all the time. If you're having a thermal overload that's starting to heat up or what have you, you can see that now and you can monitor that real-time.

Well, that's really neat. Yeah, I, you know, that's the one of the age-old OSHA violations is, you know, a cabinet overheating and then having to leave the door open, right? Correct. And, and that's, you know, so now you can be proactive and, and do a PM and take care of the problem before, before having to, to deal with it in a way that's unsafe. That's really neat.

Yeah, I think in the quality arena too, there's just thousands of use cases out there around that. A lot of, a lot of industry depends on our people to make assessments given a standard. And it's really subjective to that person's opinion. And I've seen where there's just a lot of, a lot of use cases now where you can, you know, take the subjectivity out of that and really make it quantitative. Instead of doing a sample every 30 minutes, you can do every piece and, you know, have a larger data pool of that to do analysis against. So, you know, I think like in my current job, you know, we, we like to, to monitor the finish on something like a door handle, right? And that's We have a golden standard that we compare it to, which is another door handle that's done by a person. We're now able to start looking at converting that using a camera so we can actually analyze that on a scale and not have it be so subjective going forward. So, seeing just lots of those.

Maybe talk, Brian, about that, like how the company came to that transition, right? So was that something where My guess is that for many, many years, a person looking and comparing was the process. And then all of a sudden, Brian or somebody else comes in and says, hey, well, we can do this now with the computer vision system and such. And maybe talk a little bit about that process. And was it an easy sell? Did it happen before you got there? You know, did you lead that? Like, how did that process of bringing that up to the company and making such a change?

Well, I would start with that there was no guarantee of success when we started that. Like, we didn't know, we didn't know how hard it was going to be. But I think just— and we haven't, to be honest, we haven't really finalized that particular one. It's just one that we're working on right now. But the— I think some of the things that we worked through is, you know, it used to be just the vision was the human vision space, right? But now there's things with X-ray, ultraviolet, and other spectrums that are also— I would categorize it as an expanded vision. platform where you, if you take those and you blend those together, you can look at something in a completely different way. And if you blend those together, you know, something that's typically not just in the human space or human spectrum, you know, you can sometimes see things that you can't normally see. So we've, I don't know if I've answered your question very well, but we've looked at different, you know, technologies in that to blend them together to actually do the assessment of what the optimal, you know, door handle should look like.

Yeah, no, I just wanted— because sometimes people go looking for a technology to solve a problem, and whether it's going to fit or not, who knows. Some people are very, you know, interested in pushing the boundaries and things like that. And so just always curious with folks about how those sorts of transitions happen.

There are a lot of trial by error. You got to have— be brave and try something. You know, I would always tell my team, like, you might fail, fail fast, because we need to move on to the next thing. But, you know, give it a try. Like, you never know what you can do. And some of them, you know, in the food industry, we— there was a lot of things that people said could never be done. You know, moisture analyzing was typically done offline on most lines, where you would have to take a sample product off the line to measure it and You know, through push, meaning, you know, you go out and try it and fail, push again, try something different, you know, you can, you can find successes. And that's where you kind of make a major advancement in your capabilities is when you are able to take something that's very difficult to do, very slow and methodical, and turn it into, you know, something that you can get real-time data off of and do real analytics around.

I'm hearing a couple things that I really like. One, you have an attitude of, let's try it, let's fail forward fast, right? Let's get on to the next thing. So that experimentation and stuff, and I think that's really important for people to successfully find the right tools and things like that. So I think that's a testament to you and your leadership team that you're encouraging people to do that. I know we talked before we went live about things that you're passionate about, and you said that computer vision is like at the top of your list, that you really are passionate about this. So what is it about the technologies, the solution sets, the whole shooting match here that gets you so excited?

That's a great question. I'm not sure I have a clean answer for it. I think it's that there's so many problems to be solved in industry that can be solved with this technology. I think that's what makes me passionate about it. I like, I like analysis, right? I don't like, you know, just going in subjectively to make, you know, the call on certain things. And, you know, having done that for an entire career, being able to switch that to something that's more logic-based is what I'm all about. So that makes it— makes me more comfortable with the day-to-day decisions we're making, because we're making it around good data, not just somebody's opinion about something.

So it's a versatile tool in your tool belt.

So yeah, good.

That's really neat. No, that's fine. So the, the— you mentioned, we touched on a little bit of this earlier about providers, and there are plenty of technology providers obviously out there. And as, as the— this technology gets used more and more, there'll be more folks in the space selling to manufacturers. How should a manufacturer at the end of the day choose a technology partner? Because You know, we, you know, we often talk about how important it is to not try to be the expert on everything, but to use others who are experts to help you. How do I choose somebody to really partner with in this technology?

So this one, you have to listen really close to the words I'm using. But you know, there's a lot of companies out there that want to sell you a solution. But there are not too many companies that want to come solve a problem. And so I always look for those companies that want to solve a problem. with their solution as the ones I want to work with.

Yeah, it is. It's funny you talk about that. What you said is simple, but not so easy to put into practice all the time. You know, how do you get, you know, if I'm walking around or I'm going to my local MEP and they've got these providers, you know, one of the things we discover a lot of times is small and mid-sized manufacturers especially, have a problem to solve and, you know, don't, don't know necessarily that this technology can help them solve that. And so they'll hear somebody talk about it and, you know, how do they know this person is credible when they go out and talk to them? Any thoughts there?

Um, I like to listen to the use cases that they've given me, you know, even if it's generically of things that they've done for their other customers. I find a lot of value in that. And because I can, in my, my experience, I can convert that into things that I can do in my space. That's, I don't know if I'm answering your question very directly there. But that's one of the key elements that I listen for as I'm listening to sales pitches on different providers and solutions that want to get it, come and work with us.

Well, I think it's— go ahead, Mike.

Go ahead.

So I guess the other thing I think I often discover or is a challenge is I may really find someone who does what you just described, but they only have 3 or 4 good examples to give me because they're relatively small versus somebody who's relatively large. Do you worry about size in that perspective, or how do you deal with that?

No, I don't look at size because I found that there's a lot of small solution providers that have some great people that work for them. that can really deliver technology that can change like even a large organization like the ones that I work for. So I, I can— I found it in both. I'm just saying I don't, I don't look at the size of the company as whether a decision-making component of that, with one exception. Scale, you have to be a little bit conscious of because if you have a very large company with a lot of sites and you have a very small company that only has like 2 or 3 people working for them, Going to scale can be a difficult proposition because you can sometimes overwhelm them unless you're willing to go really slow. But that's the only watch out that I would have to give anybody with a— working with a small company is thinking about long term, what happens if this becomes a very systemic technology and everybody wants it in all of your factories overnight? You can't deliver that because they're just too small to do it.

Yeah, thank you. Makes sense.

Yeah, I was just going to comment, again, taking insights from Brian's comments about come solve a problem. Mike started off by talking about how we like to engage diverse perspectives and draw them into the conversation. So I think that is phenomenal input to folks. Oftentimes we'll get into conversations with people that Or from a solution provider, very passionate about the problems they can solve and the technology and the solution sets that they have, the belief that they can grow and help. But that perspective that Brian's offering, I think it's really valuable for folks to grab onto and to think about the fact that you better come in thinking about how you're going to solve a problem for whoever it is you're engaging with. So I think that's really powerful, Brian. I appreciate that. So I was just wondering, you know, on the— you're moderating the panel, right, at the event here. So what are you hoping that people attending the event and listening to the panel are going to take away from those discussions, take back to their companies?

Well, for me, you know, I think Industry 4.0 doesn't come with any great textbooks yet. We're all kind of still learning what it means, how to implement it, and so my biggest need for myself, and I would encourage anybody that's coming to think about it, like networking is key here, right? Like talking to others, talking about the problems, talking about what's worked, what's not, is really a value add for I think anybody who attends these events. So I would highly encourage, you know, to reach out using some of the tools I think they have associated with this event and try and network because I think That's where you're going to find the most value is listening to others that are all going through the same thing. Having been in 2 different industries now, a lot of the problems are the same. And I think what people have seen and done has helped me immensely. And I like to help others to the degree that I can. And this is a great forum to go do that at.

Yeah, I agree, I agree.

That's excellent. Well, we're getting ready to wrap up. Um, we've been talking for the last, uh, 25 minutes or so with Brian Marsh, who's the Director of Automation for Allegiant, and he's going to be speaking at MXO Midwest here in— as a panel— leading a panel, excuse me, in just a couple of weeks. As we wrap up our conversation today, maybe Mikey, could you share a final thought? And then Brian, we'd like to get maybe your final thoughts before we sign off?

Yeah, yeah, absolutely. Again, I would encourage folks to come to Cleveland, my hometown, and hear Brian and do the networking as he talked about. I think really probably 2 things. One, I already brought up about the notion of, you know, any solution providers come prepared to solve a problem. I think that even applies internally if you're going to go talk to leadership. about, you know, a technology solution or an approach you want to take, you better have a good story as to the problem that you're solving. So I think that's really valuable. And Brian started at the top talking about change management and how you need to make sure that you're bringing your teams along with you. I think that's another really valuable piece of insight. So thank you, Brian.

Thanks, Mike.

Appreciate it.

Brian, your final thoughts as we wrap up?

No, I'm really excited to talk about this topic. Like I said, it's one of my favorite ones. So, you know, You know, I think the other big bogey that's out there is around talent, right? So I'm sure there's gonna be a lot of great discussion about acquisition of talent. So I encourage people to be thinking about that when they come to this, because I'm absolutely sure there's gonna be a lot of discussion on that, that point. So—

Sure.

Buy it versus build it. And, you know, do you want the seasoned person versus somebody that's fresh out of school with a new idea? Those are all some great topics I'm sure we'll have discussion around when we get there.

Yeah, thanks. I appreciate that. Well, um, as we wrap up, I'd like to thank Brian for joining us to share his insights into computer vision. It's been really interesting to hear your thoughts, Brian, and for giving us a preview of your panel that will be at MXO Midwest happening in Cleveland on September 16th and 17th. We hope you're going to decide to join Brian and the Industry 4.0 Club at the event. As Brian mentioned, there'll be a lot of opportunity for networking and a lot of chance really to learn from very different expertise for a lot of different experts like Brian. To register for the event, if you haven't already, here's the website. Please go to this website, register. It's not too late to sign up. We'd love to see you there in Cleveland. The Industry 4.0 Club would like to encourage you to take action. I think you've heard a little bit about that today about solving problems. When you do that, use Industry 4.0 to help you do that. But to get more educated, read an article, attend this event like we've been talking about, develop a smart manufacturing roadmap, or encourage others to do the same thing. We'd like to know what actions you decide to take because of what participating in these conversations or listening to them. So drop us a line at our website at www.industry4oclub.com or on our LinkedIn page and tell us what you're doing. We also hold these broadcasts weekly. And so if you'd like to participate in one of our broadcasts with us, again, go to our website, www.industry4oclub.com, or our LinkedIn page and reach out to us. Let us know you'd like to participate and perhaps we can get you on a future broadcast. With that, thank you, Brian. Thank you, Mike. It's been great. See you guys in Cleveland. Okay.

Thank you. Awesome.

Thanks, Brad.

Thanks.