#131 How the Aviation Industry Leverages Data Science

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey. On datacamp.com.

We don't always come up with a good model. I like to advertise that 30% of my projects fail and I'm not ashamed of it. And what I'll tell people is the reason that they fail is because if not, we're not trying to solve hard enough problems.

Hello, welcome to Data Framed. I'm Adele, Data evangelist and educator at DataCamp and data framed is a weekly podcast in which we explore how individuals and organizations can succeed with data. You may notice the new music and the new introduction, but I want to assure you the commitment we have to share with you. The latest and greatest in data is still the same. Today we're speaking to Derek Sidio, Senior Director of Data Science at GE Aviation. You know, leading data science teams and functions is inherently hard. Data leaders need to consistently think about finding the right use cases for data science, aligning these use cases with business value, managing and building data teams manage change within the organization, and the challenges of managing managing data teams in a highly regulated environment like GE Aviation only adds to that difficulty. So I wanted to speak to Derek about how he approaches leading data science at GE Aviation. Throughout the episode we talk about the type of use cases data unlocks at GE Aviation, how he approaches managing and building data teams, the skillset data scientists need to have to succeed in this space, the frameworks he uses to ensure data science projects are scoped and properly planned for, and a lot more. If you find this episode interesting, make sure to subscribe to the show. Also, make sure to tune in to our annual conference, Radar 2023. This week we have tons of exciting sessions for you with a wide variety of previous data frame guests and maybe future ones. Now let's get started.

Derek, it's great to have you on the show.

Thank you for having me. I very much appreciate it.

I'm excited to speak with you about how you lead data science at GE Aviation, how you approach leading data science programs, how to operate a Data team in a highly regulated environment and much more. But before, maybe give us a bit about your background and how you got into the data space.

Yeah, so I graduated back in the 90s from MIT as a mechanical engineer, and so I didn't have a big mindset for data. But I took a role at ge and within the first couple years I found myself in a particular role that happened to be programming in Fortran 77 and working with data directly from aircraft. And I found just that combination of programming, statistics and my love of aviation all came together perfectly, kind of like chocolate and peanut butter. And I've really made a career of that ever since in a wide variety of ways.

Maybe to start off today's conversation and set the stage, I'd love to know the type of data science projects that you lead at G Aviation and how it provides value for the organization.

I think it could be kind of summed up in our mission statement for our team, which is to provide business outcomes supported by data science intelligence. And I think the intelligence piece of that is really key. What we're looking for is that data science isn't the answer. Machine learning models or AI or anything else in that world does not bring the solution, but you really have to marry it with the expertise of the domain experts. So being able to take that with them and move that forward to find the real outcomes, I think is where it happens the best. And we have analytics all over the place. We do everything from trying to detect the Delta P and a fuel pressure on an aircraft to say, hey, you need an oil filter change, to going and analyzing cost data throughout the marketplace and our MRO services network.

Okay. And there's definitely a lot of value there across the value chain, as you mentioned. What I want to really focus on today's conversation is how have you approached managing complex data science programs, especially in the type of environment that you operate in? I think there's a lot of value for listeners, especially data leaders, listening in today and trying to pick out insights from your experience leading data science and aviation and a large matrix organization. Like geography, creating value with data science is inherently complex regardless of where you are. And I think when it's done in a large organization and also in a highly regulated environment, this complexity is only exacerbated. So maybe to start off, I'd love to know what you think are the key components for successfully managing a data science program.

I think it really starts with solving the right problem all too often and especially in an area of data science where it's such A new emerging technology, someone often is looking to solve a problem that they don't even know they're truly trying to solve. I want lower cost, I want more efficiency. Things of that nature is usually what an executive will come at to say, how can the data science team help? And there's many, many different ways to help them solve that problem. So spending the time upfront to really establish one of our team members likes the term universal understanding. I understand what you're asking of me and you understand what I'm capable of doing. And I think that's really the key to getting the problem started off right. In addition to that, really having high quality data at its source, being able to make sure you can rely on it, garbage in, garbage out, with all models. So that always becomes an issue. And then especially you kind of mentioned the large matrixed organizations. Single sources of the truth is a real challenge for a company of our size. So everyone has their day jobs and they've got to be tactical in what they're trying to solve throughout the business, whether it's running stuff in the supply chain or it's something in a services shop, et cetera. So there's a need for them to have it right tactically every day. And that's very counter to being able to build a complete data thread and have consistent data throughout the entire enterprise so that you can really tie it together, whether you're doing operations, research, data science, forecasting, whatever the case may be. So I think that's a big part of it. And then obviously if you can get that right, then the data governance and stewardship to maintain those single sources of truth. So we've gone through a lot of digital transformation over the years in order to do that. The creation of a data lake is one big step that companies can take to heading in that direction. But keeping it from being polluted is a whole nother situation after you create it.

So let's tease out some of what you outlined above. I love how you categorize the different components of a successful program management, starting off with solving the right problem. I think this is actually a big problem in the industry that a lot of times, especially organizations starting out in their data journey, tend to focus on problems that don't necessarily provide business value. How do you ensure as a data leader that you're always solving problems that provide value for the business?

It's really about engagement and transparency. So oftentimes in this space, again, you'll be challenged by an executive as a, say, a junior data scientist, say, I need you to go do this and be able to have that transparency to say, I can't do that. Your data isn't of the right quality for me to build a model from. Or you want 99.95% accuracy, you know, whatever your accuracy metric is, and I can't provide that, I might be 60 or 70% accurate. And so there's a lot of conversation that has to happen around is that the right problem? Because again, there is a typically a very clear financial metric, for example, that would be stated as, this is the problem. But how you go solve that is very complex. It's no different than designing a jet engine and saying, well, I want to push this airplane so it rises up through the air. There's a clear need to go do that. But how you go do that, the devil's in the details. So making sure you have that upfront conversation, and it's not one conversation, it's a series of conversations. We use a process called the TRLMRO process. And its very first step, the TRL one, is all about what is your business value? What are your metrics for success? What's a rough outline of how you're going to go attack this problem, and who are your team members? And those team members should represent both sides of the stakeholders as well as the data science team.

That's great. And maybe expanding on that process. What are the types of conversations that you have there in that process you mentioned here, maybe, you know, outlining ownership, what is the business value? How do you kind of standardize that across different variety of projects that could have different types of data sources, different type of value drivers? I'd love to learn more about that.

So that's a huge challenge. The key word in the middle of all that is standardize. That is a huge challenge. So when you're trying to solve an analytics or a data science problem on a jet engine is different than the accuracy and the validation you need when you're trying to plan labor hours for a team. So that standardization and how much rigor do you put in the validation is always a factor. So I mentioned a TRL MRL process, and that is a process that we actually adopted from NASA and the Department of Defense a couple years back, which is totally unrelated to data science, but it's talking about bringing new technologies to the market, right? Rocket ships and defense articles and things of that nature. And it's all about the risk reduction of actually getting to the end of making those products work. And so although the MRL in that is called manufacturing process and we hijack it because you're really not manufacturing a data science model per se, but eventually you're going to put it in some kind of production environment. So what we try to do, I mentioned the Tier 01 process up front of being able to establish that what is the need, Are we solving the right problem? But there's also tidbits in there of do we already have something that solves this problem in our catalog and should we be reevaluating that because maybe that's ineffective or there's a lack of awareness before we go try to solve a problem all over again? And then there's other multiple handoffs so that we don't end up building an analytic that we can't put into production. Because again, perhaps what we talked about earlier, about individual sources of the truth, something's not repeatable, etc. That whatever we validated on can only be validated on a desktop and can't run into a runtime production environment.

So you mentioned here at the end, single source of truth. I'd love to kind of dive deep into that as well. You mentioned the importance of, as well of high quality data, good data governance program, easily accessible data. How have you approached that journey of creating that single source of truth that is high quality and easily accessible?

Yeah, it's a challenge for sure. I wish I could say I've attacked it strongly and been able to go lead that. I rely on the rest of our organization to go do that. So, you know, we have our own network chief information officer, for example, that is driving something called the event data thread, which is meant to address each of those kind of touch points of the data from its initial creation at the time that an engine is created, throughout its entire lifecycle, including the servicing, the repair of that engine, etc. Until its retirement. And so that's really not on my team, but we're an advisor to that team, right? So trying to help them to understand where the difficulties will lie when you try to model it, depending on even how you do that. So you can have a good data governance policy or some keys in the data that link everything together. But without a vision of how to tie modeling into it, you may set it up the wrong way. So it's really about just being partners with almost everybody. You know, we gotta be partners with our stakeholders, we gotta be partners with our data teams that actually own the data infrastructure and architectures, and then we have to be partners with our peers that keep that stuff running as well. So it's really about being a team player in order to make that all work.

You mentioned as well, like earlier, beyond the single source, how you approached creating value with data science. The TRL MRL process maybe walk us through as well the project management approach to actually delivering data science projects. Unlike software engineering, you tend to see in the data science space quite a few different various project management approaches in data walk us through. Why you think that is the case and what you think is a great model for managing data science projects end to end?

Yeah, I think it's because this area is so new and we don't know how to apply it. I mean, if you look at the number of universities that offer data science degrees, it's measured in the dozens cleanly, not necessarily in the hundreds or the thousands like mechanical engineering degrees might be. So I think that's part of it hasn't been studied quite as much. Same thing. We've been writing software for a really, really long time. But even the term data science wasn't coined all that long ago if you think about it. So I think that's part of it and we have to be flexible on the way that we do it. So I mentioned the TRL MRL process and that's really checklists and things of that nature to make sure you haven't forgotten to do things and you're going to get across the finish line solving all the right problems and actually being able to put it into production. But there's still an aspect of project management.

Right?

That TRLMO process isn't that different from Crisp DM or the Microsoft Team data science, as far as running a specific project, but you still have to manage a team. We're still teams of people that are doing this work. And so is agile methodologies one of the right ways to project manage your team? There's just so many ways to do it. We're currently in the middle of an Agile transformation, really applying agile software principles to our data science projects. Literally just getting started in it. We've been doing it for about a month. One of our teams did a demo of it, if you will, ran a pilot program and found that we could actually apply the software principles pretty well. The difference that we face, I think, and the biggest challenge in making agile work is especially on a lot of the data science projects, they tend to be run by one or two people, very, very small teams. Compared to software development, which even a software development may work in small pods of, say five or six, a team of one is very hard to go get a Scrum master for and to justify having a scrum master for. So how do you get an individual data scientist to think with an agile mindset and plan their day and run in sprints and do retrospectives all by themselves without other team members is really the challenge in bringing it together. And I think across the industry a lot of data scientists really start off and kind of one off until a project gets really big, more run by single data scientists than they are by large groups. Until it really gets them to be a production process.

That's great. And you mentioned that, you know, skill set component of how do you get that data scientist to be both technical but also have that business acumen of necessarily project management and approach projects. Teases into my next set of questions on how to structure teams around value creation. Right. You know, a big part of providing value with data sciences, structuring teams around effective data science best practices, making sure that they're aligned with the business value. I'd love to focus on how you've approached as well structuring the different data teams you lead. So I would love to start by understanding what do you think is an ideal data team structure?

I think it's not a great answer to start with. It totally depends, but it does totally depend. As I mentioned, I work in a relatively large company and so we have an entire digital technology team that I can turn to them and say, hey, we need this kind of infrastructure. In many cases, data science teams are, you know, Swiss army knives because they are the whole digital technology department for their team almost. So it does depend based on the support structure. But I think the keys of being flexible don't change. So I'm a huge fan of the book by Stanley McChrystal called Team of Teams, the real short version. And I highly recommend go buy his book and read it. So but the super short version is that you want deep experts on your team, but you want everyone to be able to do all of the jobs. So when I look at composing a data science team, that's exactly how I look at it. I look at having strengths spread out throughout the team. So I've got someone really strong in forecasting, someone strong in operations research, someone strong in visualization with bi, and then someone who's a really strong data wrangler. I've got another person, for example, that's really great at image analytics. They're all specialists in that field, but they can all also do a little bit of everyone else's jobs. And therefore we can pivot a lot and we can pivot the Resources based on what the business priorities are. And it also basically creates a continuous learning environment where everyone gets to kind of pick up. And we use things like hackathons within our own teams to create artificial problems. Just so. For example, we just had one on image analytics so that everyone on the team could learn a little bit more about image analytics. So at least they're conversant in case whatever stakeholder they're talking to might happen to have an image analytics problem that would fit in their space. So you mentioned, you know, all the different skills they can have and that's great. Those are the unicorns to go find someone who can project, manage themselves and be a great data scientist and a great data wrangler. Those are really, really hard to find. But the teams of teams approach helps you balance that out in the interim while you continue to find those all stars that can do it all.

This connects to my follow up question because indeed, kind of having that data scientist with the multiple skill sets that we described is a unicorn data scientist very difficult to find. Maybe us, through your teams of teams approach, some of the best practices that you found in developing a pipeline for recruiting data talent. How do you test for these different types of profiles and interview process?

Yeah, again, another huge challenge. I just sound like my job is super challenging and it is, but that's what makes it fun.

It is.

But especially in some specialized industries like the aviation industry, it can be a real challenge to recruit the right kind of talent. Especially I mentioned all the challenges we have with our data structure. Part of it's because of being in a regulated industry and part of it's not. So when you're competing with banks and retail that, you know, have gobs and gobs of data, thanks to barcoding and transactional processing, it becomes difficult to find someone who's more excited about ones and zeros than they are about aircraft engines. And so you have to be really active in the recruiting process. So we partner with universities. University of Cincinnati here in our headquarters is one that we partner with and we help sponsor their analytics program. There's and we'll give graduate students opportunities to work on problems. We have a great educational program of outreach where, you know, for our own team to go ahead and get master's degrees, PhDs, other continuous development opportunities where they network within the university system, then at that point, and then it's just being extremely proactive in seeking talent because especially with such an emerging space, the best data scientists you may find may not have a data science degree. And so finding Someone who, you know, maybe like myself early on, was just passionate about the data and the programming that went with it, but has been classically trained as an engineer, might be someone you keep an eye out for and develop that love that they have, rather than seeking out someone directly from school. Because as I mentioned, there's not that many programs today out there that you're going to get someone with a bonafide data science degree coming out right now.

Okay. And you mentioned here partnering with different universities, creating that, you know, upskilling pathways for your own data scientists as well. Walk me through maybe the type of technical skills that make a data team successful and that you're looking on your data team.

Yeah, I think, you know, bare minimum is just understanding databases.

Right.

There's a couple of key things that you gotta have to get in. One is understanding database and database structures. You gotta know how to query data and be able to pivot it, those kind of things. Python skills are a must. You know, five years ago there was a huge debate over are we Python or R in the data science world and what should we go do? And we had a raging debate within our own office over who should do what. I would say for the large part, Python's kind of won that, even to the point that you've seen RStudio change their name to no longer be RStudio but to be renamed as Posit to get away from feeling on the losing team.

There's Python integration in Posit.

Yes. So that really helps. You know, especially we talked about getting analytics into production. Right. Python is just a more efficient programming language from that perspective, and that's why it kind of won out. Not to mention the fact that it caught up with all of the statistical tools like SciKit and stuff like that would really help folks out, simply being another one. But I think that's really huge. So any kind of program skill, although of course, if someone is strong in R, it's not hard to translate those skills to Python. You just got to spend some time doing it. But a detailed programming language and then I think bi business intelligence visualization is super important. You got to be able to tell your story. Many of our projects start out, like I said, with, hey, I have a problem and I don't even know how to solve it. And the first thing we do through our eda, our exploratory data analysis, is we build a spotfire dashboard. Spotfire is our tool of choice for bi, and we start combing through their data and thinking about the statistical relationships, thinking about what we could work with. And sometimes they're just floored by that and they say like stop, like I've never even been able to see this data before. So being good at exploratory data analysis and being able to do that storytelling piece of is very, very important to one producing a good product. But for two, even going back to our beginning conversation, just making sure you got the right problem and you're solving the right problem with the right data.

You know, we talked about the technical skills here. Maybe walk us through as well, the soft skills. Because you mentioned storytelling, the ability to tell stories with data. I think that's so foundational to be able to succeed with data science. What do you think are important soft skills outside of storytelling that are must have skill sets on a data team.

Yeah. So the storytelling as we mentioned is really huge. But cap or change acceleration process is another one. And like I mentioned, one of the things that you run into is Everybody wants a 99.9% model. Very few models work that way. So being able to convince them of how to use an imperfect model. You know, there's a quote by George Box that all models are wrong and some are useful is something that the data scientist has to fully understand and be able to convince others of. So it's not easy. Change acceleration process, something you need to really work at, but something like that, that helps give you guidelines to think about. How do I move detractors to neutral and neutral folks to supporters of me is a big thing. Transparency is critical. I mentioned we don't always come up with a good model. I like to advertise that 30% of my projects fail and I'm not ashamed of it. And what I'll tell people is the reason that they fail is because if not, we're not trying to solve hard enough problems. So. But that takes a certain amount of transparency for everyone to be open with. Right. Again, that universal understanding. I understand what you want from me, but I can't provide that. And so that recognition of being able to build that reputation within your team. Right. Becomes really huge. Data scientists can change the world. You want to be seen as that go to team. And transparency is a big part of that from a soft skill. And then we did talk about the project management a little bit already and to be able to basically deliver on time. Right. So we're working on our agile transformation to try to even speed up what we do. But again, since many times your stakeholder is trying to find a 99.9% model, they'll ask you to keep going until you get there. You've got to be able to manage both their expectations on your own project to say, look, it gets exponential right in the amount of effort I need to put in to keep moving my accuracy higher. So how do you A communicate that and B program manage to deliver an outcome and hit major on the majors and minor on the minors. So maybe the 80% accuracy answer is right to get a big outcome here and you can go chase the 99, but there's probably another project you could go kick off and even get partially right that has a bigger impact to the business. So that's where the program management kind of meets the transparency in the middle.

I love that, especially on the transparency and the change and acceleration process. You know, a lot of organizations tend to have, especially large organizations tend to have like a data translator, project manager, program manager role within data teams. Maybe walk me through how useful do you find that role, especially within large teams, to be able to translate and be that middleman or middle person between a business function and a data team?

Yeah, and I think that's huge. Sometimes there is. So we have a role called an ML AI program manager which would be similar and that's someone who would lead teams. You know, if we're working a relatively large project that maybe needs five or six people to go work on and they'll help coordinate that. But again, as far as I look at it, I try to train everybody up to have some element of that data translator themselves and we're not. That's where that team of teams comes in. And you know, you ask to send in, hey, I know you've done all the work, but we need this person to present in order to make sure that happens. I mentioned a lot of data science projects tend to be singularly worked, only one person and maybe partnering with somebody else where they run into trouble. And so I think the industry is trying to find what that right balance is. Do you have a translator or program manager that can do that? It really depends on the skills of your team and the breadth of the people working on the projects you mentioned.

Here as well, having a team of teams and like developing that skillset within the data scientist I find is the ability to turn detractors into champions. Also is easier when you have a strong data culture within the organization and you have executive buy in and everyone is really aligned with the data science agenda and the data agenda and the data culture. Maybe walk me through the onus that the data leader has here in developing that data culture and their Non technical counterpart.

Yeah, that's a huge question in and of itself because I can say I've seen it over time. I've been at this company for 27 years, so I've seen us over time going from recognizing digital is important, but having no idea what it means to digital is important. Why can't the teams get it right? To digital is important and forecasting is key. Wow. Our data is a mess and maybe we need to go clean it up a little bit and then taking that to the next step. So there's definitely that high level leadership from the business that you have to get on board to recognize it. And it's a journey. It's a journey for every company. The larger the company, maybe the longer the journey. But I think the key that you hit on there is whose responsibility is it? And I take that on as partially my responsibility. As a data science team, you're often in the middle because you're getting multiple requests from multiple leadership that sometimes they're not even talking with each other. And yet you also are dealing with the day to day of the data underneath. And so you get this unique perspective to influence leadership upwards by saying, I understand what you're asking for, but I can't get it. And this is why maybe you should go fund this other team to improve data quality so I can give you better models. And I think that's a key that kind of comes into play in how we play with and influence the teams into an overall digital strategy so that they can get that data science is not just magic pixie dust that you drop onto things, but rather needs to be an infrastructure that supports its capabilities.

That's really great. Let's shift gears a bit and talk about managing data science in a highly regulated environment. Right. You know, I think an added complexity of managing a data science program in an organization like GE Aviation is the regulatory environment of the airline industry. Walk us through maybe the additional complexity data leaders encounter in such an environment. What that looks like.

Yes, it really gets to be a challenge. And some of it goes back into even what we talked earlier with. How do you get people to accept an imperfect model and understanding that but some information is better than no information. So that in and of itself is a conversation that has to happen with regulatory agencies and even just within the business that even if it's not a particular corner of the business that's regulated, everyone just becomes so ingrained with the regulatory type practices that says it must be this way every time. So one of the things like we face in the data structures is a resistance to change sometimes. And being able to help them address that and recognize that, yes, we always have to do it in a regulatory fashion because this is how we sign things off. And we're used to signing them off with paper, but recognizing that, hey, digital signatures can be accepted and traceable, et cetera. Again, sometimes you'll see a little more resistance, I think, in regulated agencies to change just in general because they're regulated. And so you have to have a little stronger scale at the change acceleration process. I think in those cases.

Okay, that's really great insight. And when developing data products, what is a validation framework that you follow? You mentioned the trl, MRL process. What do these specific checks and balances look like for deployment that are enforced before a data product is deployed?

Yeah. So as we mentioned, that process and some of its roots, taking a product all the way to finish this, again, comes into some of the regulatory stuff. If we're doing dashboarding and we're doing very simple analytics, we kind of omit certain parts of that process from a validation because the outcome, if you're slightly wrong, becomes into something that maybe the negative impact is a phone call.

Right.

An extra phone call that somebody made to contact a customer, et cetera. So we don't want to add all the rigor for the other projects that we're working into something that, you know, is only a few minutes of wasted time. So we still, though, in order to keep people working with standard work and in a lean mindset of let's do it the same way so things don't slip through the cracks, we have kind of exit ramps in our process to say, well, when it doesn't need to follow the full path of full validation, maybe engineering support to validate it at the highest level, then there are kind of off ramps in the process to allow them to have a simpler answer. But you do run the risk of overcomplicating it for the simple projects when you're in a regulatory kind of environment.

Okay, that's really great insight. And you know, one thing that I think is interesting in working in an environment such as the environment that you work in, is that a lot of data teams tend to operate in relatively less regulated environments. Think your food delivery services, your social media applications. Right. And I think there's a lot to learn about the structure of quality assurance and deployment best practices that are present within your industry. So maybe to frame this question in a different way, if you were to move on from your current role into a less Regulated environment. What quality related best practices would you take with you and apply there?

There's another good book out there called the Checklist Manifesto. There's one example that's tied to the aviation industry that says like, I don't have time for a checklist. And it says, well, if an airline pilot in emergency has time for a checklist, then why don't you? And I think that's really important. I think sometimes people bypass even the concept of a simple checklist to just say, look, I trust you to do your job, but we all forget things every day. And so the ability to just bring very simplified checklists, not a thousand line item kind of issue, but 10 or 15s that get to the spirit of what you're trying to do to make sure. Have you closed things off with the Data Structures team? Have you made sure that your production ready, have you had the right conversations? Have you validated that the problem you're solving is the right problem to solve? Right. And those aren't necessarily in the right order, but they're the kind of things that you'd want to make sure that basically are in a checklist format that your team makes sure they hit every time. So that level of standard work, even in a non regulated industry, it might be less standard work. You might have less effort in validation because the outcome of a late pizza is different than accidentally ordering $5 million of extra inventory. But you can still follow the same process to make sure you've got a quality product at the end.

Okay, that's really awesome. Now as we wrap up, Derek, I'd love to be able to learn from you what are trends that you're looking out for in data science that you think will be impactful in 2023? It's definitely shaping up to be a relatively crazy year in the space of research and the operationalization of research. I'd love to learn how you're looking at things this year.

Yeah, I think, you know, kind of Joe, but you know, if I see one more chat GTI recall, I'm going to be a little tired of it. But it goes to show you how exciting it is for everybody, right? I mean, people are looking at this and they don't know what to do about it. They're like, wow, this is fantastic. And again, it goes back to some of that transparency. If you start looking underneath, you start to see how much handholding there might be in some of these projects and stuff like that, that they're not just pure AI or pure machine learning, that there's A lot of preparation that goes into that. And so I'm really interested in seeing us getting better in overall transparency around those things and recognizing the two points of it of, again, not letting it come across as magic pixie dust that you could just drop on anything and automatically it solves. So that first thing an executive does is say, well, let's go put AI on it. It's going to solve all our problems. But that goes with really talking about data structures and being really clear that you need good data to do data science and you need to make that investment almost first. But again, like with the consultation of data scientists, because you want to make sure they understand where they can go with it and you want that feedback into it. So I'd really love to be able to see us recognize and think about the proper data collection, storage and accessibility in order to really unlock what we can do.

Completely agree with your point. In a lot of ways, AI this year is having sort of a Covid moment, if I have to make that comparison. You know, when Covid started happening at the beginning, there was a lot of reluctance to admit that there's a global pandemic, right? And then when people realized, oh, there is a global pandemic, we're going to lock down the entire planet, right? There is this mass realization that things have changed. And I think with AI today, there is a realization that, oh, things have changed and there's a lot of exciting things that are going to happen. But at the same time, I think the messaging problem is that this is AGI, that this is some form of generalized intelligence, and it's not necessarily so. It's going to be interesting to see how a lot of the public is going to react to these types of tools in the future, especially as the robustness conversation is going to become much more interesting and much more full fledged in the future.

Yeah, I think you're right. I listen to other podcasts on data science as well. The Harvard Data Science Review does one and I was listening to that not too long ago and I forget the guest name because I was just walking my dog while doing whatever, but they talked about an AI winter. Right. And it's kind of exactly what you're talking about. Sometimes the expectations get so high, right? Because there's all these tantalizing articles on hedge lines about it that you end up disappointed in the end because you don't realize what's happened. We've seen that within our own industry and company where again, everyone thinks it'll be magically fixed and they Realize like, oh, you have to go invest in data infrastructures and things of that nature and the reality kind of sets in. So those high expectations and it goes back to that transparency point can come with great disappointment. So you've got high expectations that can lead them to great disappointment if you're not transparent upfront and what they can actually achieve and the level of effort required that goes into them.

Definitely. Agreed. We'll see how the expectations will tamper off and will change as the releases of new large language models become more integrated in products that we see every day. Derek, finally, before we wrap up today's episode, do you have any final call to action for the audience?

Yeah, and I think I've hit the word three or four times in this podcast with transparency, and part of that really leads to an ethics discussion. I think it's really an uncharted subject in how we teach it and how we apply it on a day to day basis. I think it's all over popular media from the Terminator to the Matrix and all those kind of shows. There's all these horror stories of how AI and machine learning can go horribly wrong, but we should have more open conversations about it on a day to day basis. And not that I support the horror stories of it, but like I just said about not letting your leadership have higher expectations than what you can deliver. So I think I'd like to see more of that in some of the teachings. You have business, law, ethics classes in the universities and things of that nature. I'd like to see us move more towards like, hey, what are the right responsibilities to do with AI and across.

The board, as you mentioned here, sometimes it's much more useful, not easier to do a logistic regression that has like a 70, 80% accuracy than to try to do some deep learning model that is barely interpretable, that no one can understand and no one has a lot of knowledge and deploying.

That's right. So getting someone to understand a useful model, even if it's not perfect, is certainly one thing that people should focus on as well.

Great. Thank you so much, Derek, for coming on the show.

All right, thank you very much. I appreciate the opportunity. You have a great time.