#4 How Data Science is Revolutionizing the Trucking Industry

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com in this episode of Data.

Framed, a DataCamp podcast, we'll be looking at how data science is being used to revolutionize the trucking industry. I'll be speaking with Ben Skranker, a data scientist at Convoy, a company that provides trucking services for shippers and carriers. Powered by technology to drive reliability, transparency, efficiency and insights, we'll dive into how data science can help to achieve such a trucking revolution and how this will impact all of us, from truckers to businesses and consumers alike. Along the way, we'll delve into Ben's thoughts on best practices in data science, how the field is evolving, and how we can all help to shape the future of this emerging discipline. I'm Hugo Baun Andersen, a Data Scientist at DataCamp, and this is Dataframe. Welcome to Dataframe, a weekly data camp podcast exploring what data science looks like on the ground for working data scientists and what problems data science can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter HugoBown and you can also follow DataCamp at DataCamp. You can find all of our episodes and show notes@datacamp.com Community Podcast Ben welcome to Data Framed.

Thanks Hugh. It's great to be here and I'm looking forward to speaking with you today about Convoy and data Science.

I couldn't agree more. It's great to have you here and in our ongoing exploration on Data Framed of what data science is and what it can be. I'm pretty darn excited to be speaking to you today about your work at Convoy and the role data science can play in revolutionizing the trucking industry, arguably one of the most impactful industries in North America. But first, I want to talk about you.

Okay, thanks.

People always ask me, what do data scientists actually do? Ben, what do your colleagues think that you do?

Yeah, that's a great question. I think that, you know, the term data scientist is really vague and it means different things to different people. And I come to data science from both a natural science and a social science background. I also took a detour through Silicon Valley before I got my PhD in economics. So for me, I kind of spend half my day in meetings talking about the science of how to run our platform or how we're developing our approach to data science. Here, specific questions say experimental design. I spend about half my time mentoring other data scientists. So helping a junior data scientist work on an experimental design for a campaign, or, say, build a survival model to understand customer retention. And then I spend another half of my time working as an individual contributor, thinking about the aspects that drive our platform and just trying to have impact by fixing the thing that's currently on fire. So think about things like auctions or matching or pricing and those kinds of issues. Anyone who can do math will probably note that that adds up to three halves. But I work at a startup, so things don't add up to one.

I couldn't agree more. At Data Camp, I feel like sometimes I'm operating in the form of an infinite sum, actually. And as your background's in physics and you've told me There are only three numbers to physicists. Right?

Right. There's 0, 1, and infinity, and everything else is a matter of scaling. That's the old joke.

Exactly. So how did you get into data science?

That's a great question. And I think I've always been working with software and data and math and statistics and models and. And, you know, after DJ Patel coined the term and it went viral, then someone in the marketing department rebranded me as a data scientist. But even when I was an undergraduate, in college, I was writing software to understand scientific questions. So the first data sciencey thing I did was I wrote a program to understand crater relaxation for Bill McKinnon, who's an amazing planetary scientist at Washington University. And the band Frankie was big there. And since we were dealing with crater relaxation, I called my program Frankie so that may date me. And then more recently, after I did my PhD, I was at University of Chicago and was recruited to work at Amazon. And so in that time, that was like 2012. So at that point, I'd say I was really made the transition to taking stuff from academia and applying it in industry.

So now you're working as a data scientist at Convoy.

Right.

Tell me a bit about what Convoy does, what its mission is.

Yeah, this is a great example of how it's much easier for software to come into an established industry than for an established industry to bring software in itself. And so the freight industry has Been around for a long time in the U.S. it's a super huge market. It's like $800 billion. And there's this problem, which is most shippers are heavily fragmented, like the 95th percentile, sorry, carriers. And carriers are trucking companies. They have one or more trucks, and the carriers are heavily fragmented. And the 95th percentile carrier has like two or three trucks. And so the way the shippers connect with them are through these brokers. And the brokers operate using 1980s technology, so phones and fax machines. And what we're doing at Convoy is we're helping pioneer this new approach called digital freight, where we're using technology and in particular, data science to make this whole process better for everyone. Because it's really important to match the right carrier with the right shipper's load because then everything works better. So we need to price it, match it, and automate the whole process. And that's just going to revolutionize the cost structure of the industry. And for all of you out there, you're all listening and you're also consumers. Most things you buy take eight or 10 truck trips to get to you. And so if we can lower the price of freight or, sorry, the cost of freight, you, prices hopefully should become much more competitive on things you buy.

So what does this look like on the ground for either. So you said carriers, the drivers.

So carriers are. It's a kind of complicated definition because a lot of them are like owner operators. So kind of the American dream. They could be, for instance, here in the Pacific Northwest, it could be like a Russian immigrant who comes over here with nothing and. And then works his way up into becoming a owner operator with one truck. And then as he becomes more successful, he could build out his fleet. And that's something Convoy is trying to help owner operators do. Or it could be a more established midsize trucking company. So typically they have one or more trucks. It's hard for an individual to scale to more than about 20 trucks because then logistics becomes complicated and you have HR and things like that to deal with for sure.

So what does Convoy look like on the ground for carriers? Do they use apps or.

Yeah, that's a great question. So the main way they interact with us is through a. Through the Convoy app. And we have an onboarding process that makes it really simple for them to give us their packet of information, so things like insurance and licensing information. And then we can, you know, activate their account. And at that point, you know, they will see offers that are served to them. Based on the kind of loads they tell us they like. So they might say, hey, I like to operate on the i5 corridor. And then they can accept those loads in app and never have to talk to a human. So it's super efficient. And they can also bid on app on loads as well. So if they don't like our price, you know, or they're competing with other carriers for a load, they, they can bid. The price can go up or down, depending on market conditions. And then the other thing we do that's great for them is say they take a load from San Francisco to Los Angeles. Well, if we see they do that, we then say, hey, we've got a load from Los Angeles coming back to where you were or going somewhere else. And so we're aware of that. To just try to keep them moving and earning money so that they're empty less often. Because it turns out that trucks are empty about 40% of the time, which is bad for the environment and bad for the truckers.

Yeah, absolutely. I imagine there are all types of travel, traveling salesman problems and trying to figure out, you don't want to necessarily have a driver drive an empty truck from Seattle to the Bay Area in order to then transport stuff from. From there elsewhere. You'd like to figure out how they can actually be. Their trip can be optimized with respect to how much, how much they're carrying.

Yeah, absolutely. And so as we get more liquidity on the platform, it becomes easier and easier to be able to solve these kinds of problems and basically keep carriers in a state of almost constant motion.

That's really cool. So this is one of the reasons I find this so interesting is because when we think about the role data science plays in modern industry, we think of tech a lot of the time, which is, you know, an industry that was born with this rise in access to data. Whereas you're talking about revolutionizing an industry that predates all of this technological infrastructure.

Absolutely. And if we're successful in what we're trying to do, we are going to change the cost structure for a major industry that affects many Americans. One of the amazing facts I've learned at convoy is that the most common job in something like 45 ish states is truck driver. So this affects a lot of people in terms of their employment, and it also affects everyone who's a consumer. This can have a big impact on many people's lives in a positive way. In particular for the truck drivers themselves, we're making it easier for them to run their own business. Grow that business and just have a much less friction in how they run that business for sure.

So how does data science then play a pivotal role in Convoy's mission?

So data science is central to what we do. And in fact, from the first day we opened, we've always been automated and so we have to solve a wealth of fascinating problems with lots of economics. So it's very important to be able to predict the price because we need to price correctly to shippers. Often we enter into long term contracts with them and we also have to price correctly for the carriers. Then we need to make sure we solve the matching problem to match the right carrier to the right load, because that is better outcomes for everyone. So for instance, right could mean less deadhead space. So that's the time you drive empty to get to the start of the load. And also the destination endpoints could matter. We worry about auctions and the whole price acceptance mechanism for the carriers themselves, as well as many other things related to the carrier life cycle and making sure that everything's going well once the carrier's picked up a load. So there are a wealth of problems like that.

And it strikes me as a business you might come up against a chicken and egg problem in the sense that to convince shippers to come on board with you, you need to have carriers on board and to convince carriers you need to have shippers.

Yeah, that's something that's really hard for any platform. There's a great post by Simon Rothman, who's one of our investors from the Series A round, and he wrote this great post and he says basically what you have to do is you have to start two companies at the same time. So you have to bootstrap a company on the supply and the demand side at the same time. And it makes it really tricky because we've got to keep this in balance. And so fortunately we have an amazing sales team and they're really good at generating quality demand. And, and then we quickly build supply to try to keep everything in balance. And that's something we track and that's something any platform worries about. They think a lot about liquidity and maintaining balance. It's like if you went to a dating site and let's assume you're heterosexual and if there are no women on there, it's a bad dating experience. And so you need to have equal numbers of men and women.

The reason this is kind of at the front of my mind is previously we've had a similar challenge at Data Camp in the early days, getting, getting Students while getting instructors as well. Because instructors want an audience and. And students want the best instructors, right?

Yeah, absolutely. Yeah. And then the other thing I say is there also are network effects. So once it gets going, these businesses tend to grow exponentially. So it's really exciting. It means that if you do this right, you'll find that you're not sleeping.

Absolutely. So you told me one of the most important things is, is a really strong sales team. How is. Is the data science team integrated into the company with respect to, for example, the sales team?

Yeah. I think one of the big things we can do is help them understand pricing things like how to bid on loads, what loads should you bid on? And those are some ways that we can work very closely with them. Pricing is an incredibly complex thing and there are all kinds of incentives around it. So that would take hours to unpack.

We've been discussing kind of circling around the role of data science. What are specific types of data science questions that you need to answer in your job?

So I think one of the things that's really good about convoy in our culture is we're really data driven. And so we do a huge amount of experimentation, just like everyone else, I would assume, in industry or certainly the top players. And so the way we answer questions is often through experimentation, if that's the only way to solve it. So we do a lot of experimentation. We've invested heavily in building very good experimentation framework that enables us to iterate quickly on experiments so that there are different approaches to running a B tests like Bayesian or Frequentist and, you know, like the Bayesian or sequential analysis will get you an answer much more quickly. They also make it easier to discuss results with product managers.

Let's just step back a bit and maybe you can give us an example of an a B test that you'd perform great.

So the kind of thing that you would typically a B test is, does some new UX flow work better? Let's say one of the things we care about is that when a driver completes a trip that they automatically upload their paperwork. It's something called a bol bill of lading. If we rolled out, say a new process to improve that and make it easier, we could run an experiment with where we put half the carriers, keep them on the old technology and put half in the new technology, and then after some time we can say with some probability whether or not the new process works better.

That's a great example and a great description of AB testing as well. I'm going to probe A bit more. You mentioned that Bayesian methods converge more quickly or give you results more quickly than frequentist methods. I know I'm put you on, putting you on the spot here, but would you mind, for the laypeople out there, just giving a brief description of the difference between frequentus statistics, which people may be more familiar with, and the Bayesian methods that you're discussing in this case?

Certainly. So the first real approach to statistics was the Bayesian method that was developed by Thomas Bayes. He was actually a vicar, and his idea works a lot the way your intuition works, which is you start out with some prior set of beliefs about something like, my new process is better. Say it's going to cause a 10% lift. And then over time, as you observe data, you update those beliefs. And so this Bayesian updating then will converge to what we call the posterior, and that is the distribution that we expect the lift to have. The frequentist view. And so before I go on to frequentist, the Bayesian methods were very hard to compute until, you know, maybe two decades ago. We didn't really have the computational resources. Since then, there have been huge improvements and it's much easier to compute these models. Before then, you could only really solve special cases. So people like R.A. fisher in particular were very critical, I think, of the Bayesian approach. And so they developed the frequentist approach, you know, in the early 1900s, if I'm correct. And there the idea is, there is some true value of the parameter. And if I sample enough data, as I get more and more data, that's going to converge to the truth. And so frequentists tend to talk about P values and confidence intervals. And that's the traditional hypothesis testing you're used to knowing. So in the frequentist world, you'd have to go to a marketing manager and say, conditional on the null hypothesis being true, the probability that I observed an effect as big as the one we saw or bigger is 5.7%. And so then you'd have this argument, because the traditional view on significance is that you would use the significance level of 5% to say that the result was significant. So in this case, if you were a good person, you would not be able to reject the null hypothesis, but your marketing manager is probably going to say 5.7% is really close to 5%. Let's say we just use a 10% significance level and you're in this world of hurt. Whereas with the Bayesian method, you can go to the product manager and say there's a, you know, 94.3% chance or 98% chance that variant A is better than variant B. And so it's A, it's a much easier conversation to have.

Fantastic. And in, in the example you're talking about variant A and variant B, or the parameter we're looking at would be the number of people who successfully upload all their paperwork.

Correct.

As a function of whatever the UX looks like. Right.

So it's whatever you're testing like, you know, does my new workflow have higher click through rate or higher checkout? You know, does it lower, you know, churn whatever you're interested in? So yeah, I think the Bayesian method makes it much easier to have conversations with product managers. In addition to, you know, in our simulation studies, we find it converges much more quickly for our business. You know, your mileage may vary on your business and if you're in Europe, your kilometers may vary.

Yeah, I think I recall Dave Robinson actually wrote a number of posts for Stack Overflow about Bayesian A B testing there and showed that for their experiments it didn't necessarily converge more quickly. I'll need to check that and we can put that in the show notes as well. Yeah.

There also were some great blog post by Evan Miller on the subject.

Let's now dive into a segment called Data Science Buzzwords with Data Camp curriculum lead Spencer Boucher. What's up, Spencer?

Hey, Hugo.

What Data Science buzzword are you going to demystify for us today?

Today, let's talk about big data. We use that term a lot, but a lot of people find themselves hazy on what exactly it is. Big data means different things to different people. And asking how big is big data? It's a bit like asking how long is a piece of string?

So where did the term big data come from?

So a paper published in IEEE by Michael Cox and David Ellsworth back in 1997 is widely considered to be one of the first introductions of the term. In that paper, the authors discussed the problem of visualizing data that doesn't fit into memory. This would actually continue to be a major theme of analytics through the 21st century, all the way up to today. But maybe the biggest milestone in the history of big data came back in 2001 when a white paper by Gartner analyst Doug Laney proposed three defining qualities of big data that have become very well known in the volume, velocity and variety.

Can you begin by telling me about volume, which seems like an intuitive quality.

To think about, Right.

So in general, when we talk about big Data, what we're talking about is data that doesn't fit onto your laptop or even usually onto even one machine. Technologies like Hadoop have been developed primarily to deal with exactly this issue, leveraging large networks of computers to operate on large chunks of data simultaneously for certain algorithms, at least, that allows data scientists to scale to any amount of data simply by adding more hardware to the problem. So, for example, Facebook has a cluster with over 1100 machines and 12 petabytes of raw storage. Just to give you a sense of scale, 50 petabytes would roughly correspond to everything ever written down in all of mankind's recorded history.

Wow. So what about velocity then?

Yeah, so velocity is another big part about what makes big data tricky. In a world where every single click can generate innumerable data points, streaming tools have emerged to crunch numbers in real time while things are still changing minute to minute, or even second to second. Streaming technologies like Spark and Kafka address this tricky problem of analyzing data that arrives and changes extremely quickly, opening the door to really interesting anomaly detection algorithms that operate in real time on a very large scale. For example, these days, streaming engines are capable of handling over 60 million records per second.

So we have volume, velocity, and the final quality was variety.

Yeah, exactly. Variety can refer to the explosion of data storage formats in recent decades. But on a more fundamental level, it means the vast strides that we're making expanding the kinds of data that we're able to cheaply and easily collect. It's not unusual for one company or even one data analysis these days to involve text, click data, audio, video, and all other sorts of unstructured data.

So, Spencer, how would I know if my data is big data?

Well, in the end, Hugo, that's not what really matters. What matters is that you're applying the right tools and techniques to your data, appropriate to whatever its volume, velocity, and variety are.

There you have it, folks. Data science buzzword demystification in action. Thanks, Spencer.

Yep, anytime. Hu.

Now back to our interview with Ben Skanker. So, experimental design, that's a. That's a really interesting approach because I don't think a lot of. A lot of people would expect that experimental design and these types of methods would play such a huge role in reinventing the trucking industry using. Using data science. What other types of techniques and methods are you guys interested in?

Yeah, so just before we go on, I think one of the things I would just to close out a B testing is there's a lot of wisdom that people have about the trucking industry. So our company is half tech startup and half trucking industry veterans. And they have a lot of hard won knowledge and intuition, but it's not always correct or precise. And so by performing experiments we can make things much more concrete.

Interesting. Is there something political about that as well? And social in the sense that a lot of these people have been around for significant amount of time, have certain amounts of power and kind of hard earned knowledge in some ways. And is there a view that tech startups can come and in inverted commas disrupt that and there needs to be kind of social behavior reflecting that?

Yeah, I mean, I can only speak within the confines of the Convoy culture and we have a great, really team oriented culture. It feels like when I played on good hockey teams and you know, I think that both sides are really appreciative for what they bring to the table and you might even have two industry veterans who don't agree about something. And so in convoy we try not to have disagreements that can't be resolved through, you know, Sorry, let me rephrase that. So if a disagreement can't be resolved through some kind of intellectual argument with theory or facts, then we run an experiment instead of sitting around and having an argument that can't be resolved.

So what else, what other types of techniques and methodologies are you guys interested in using?

Yeah, so I mean we're very much a practical, applied data science shop. We're trying to solve concrete business problems in short amounts of time. And so we use kind of the standard toolkit you would expect us to use. So we're very agnostic about tools. So people tend to use the best tool for the job. And so in terms of technologies that could be R or Python, they both have strengths and weaknesses. I think it's good to know both. And then in terms of specific approaches, sometimes machine learning is best. Like, particularly if we need to predict something then, you know, like predict whether or not someone's going to upload a bol or say be a good carrier. Then you might build a logistic regression or you know, some kind of boosted classifier. But there are other times where you need to understand if A cause B, maybe we weren't able to run an experiment and then we would be back in the world of applied statistics and do some kind of regression analysis, kind of. My first win at Convoy was before I started. They had released a feature without an A B test and they wanted to know whether or not this new feature helped. And I was able to use the causal impact package that Google developed using like Bayesian structural time series to show that the new feature had had a beneficial impact.

Fantastic. And that's on data already collected, right?

So we have existing data, and so then you have to go and try to make sure that everything's as good as randomly assigned, hopefully. So one of the key features to be an experiment, right, is you need random assignment to treatment and you also need to satisfy some other things like that your assignment's individualistic and probabilistic and unconfounded. So these are technical terms. And so if any of these fail, then you're back in the world of observational data. And then you're going to use applied statistical methods or econometrics to try to create something that's as good as randomly assigned so that you can then make some kind of causal statement about whether or not A cause B.

Could you remind me what econometrics is?

Oh, Econometrics is the kind of the set of statistical tools that economists have developed for, for dealing with economic problems and for the business world. Those tools are super helpful because most of our problems are economic in nature. And so a classic example would be dealing with something like sample selection and other forms of what are called endogeneity, where you have outcomes that are co determined within the model. So the classic example is if you're trying to understand whether or not increasing the size of your police force will reduce crime. Well, you could think of crime is probably a function of the amount of police, but police itself is also a function of the level of crime. And so there's this, you know, that's an example of simultaneity, sample selection. Once you start to look for it, you see it everywhere. That would be like, you know, I try to run an experiment where to see if small class size improves reading comprehension, but all the parents of kids who are posh insist that their kids are in the small class. And so now you have kids who are in these smaller classes are more clever, and now you've got selection bias.

And so econometrics has created a bunch of tools to deal with these types of challenges.

Right. And in particular, there's one set of tools that are well known to economists, but not to data scientists, outside economists, and that's panel data. And these tools are really good for dealing with what economists would call individual heterogeneity. So if I'm looking at the behavior of carriers or shipments, these have individual quirks that I can't observe. And if I can observe, say a carrier over time, panel data gives me great methods to remove these unobserved Individual effects that could confound my estimates.

That's very interesting and it seems to me, correct me if I'm wrong, that what you're saying is that there's a whole bunch of tools that have been developed by some very smart people in econometrics that could be utilized in data science, but haven't seen the light of day yet in this world of data science. Right.

And hey, I love machine learning and it's great, but there are also problems where it doesn't work. And I think people have become over focused on machine learning to the point of overlooking econometric methods that are often very useful and can solve problems that can't be solved with machine learning. I remember talking with someone at Uber a while before I joined Convoy and he actually said they had encountered problems that they could not solve with machine learning, but they could only solve them by building a structural econometric model, which is a very complicated process. It often takes about a year or more to build one of these models and get it to work well. But you try to model the whole behavior process and utility function, but when you're done, you have a very rich and powerful model where you can make good predictions about counterfactual outcomes.

Cool. And that was at Uber, you said?

Yeah, it actually happened during my interview with them when I was. I interviewed with them before I went to Convoy.

It also sounds as though we've discussed machine learning, econometrics, experimental design, and Bayesian methods for AB testing. It seems like you have a lot of geographical data, geospatial data, and time series of geospatial data. Does this play a role in any of your work?

Yeah, I think that that data is really important and we're just beginning to unlock what it can do. But we use it in the app in a lot of ways to make carriers experience better. So for instance, when a carrier shows up to pick up a load, we can automatically check them in based on geospatial data. Some loading facilities are very poor and, and if they keep the truck waiting too long, they have to pay something called detention. And so we can just start auto paying out detention when the carrier is eligible for it instead of making them go through some laborious documentation process which is not dissimilar from trying to file an insurance claim in the US Something.

That sprung to mind there when you said, you know, we have all this data which perhaps you know, you have an explored in all its potential yet there seems like there could be a potential for all types of social research with Respect to the data you're uncovering as well.

Yeah. I think that there are a lot of really interesting questions we can answer about matching and platforms and auctions that I would love to get into more deeply. I'm sure there are many academic papers that could be written on the subject.

What data science projects in particular at Convoy, have you been involved in that you consider the most impactful on or telling about society?

Oh, that's a great question. So I've primarily, I've been at Convoy a little over a year and we're a bit over two years old and I primarily focused on pricing and experimentation. I think one of the most interesting experiments that we ran was shortly after I started, we ran an experiment where we gave preferential access to loads to higher quality carriers and quality went down and everyone was shocked. Like, wait, we're giving like the high quality people early access to work and quality's going down? What's going on? This is like, this makes no sense. And you know, a bad manager would say, hey, you data scientists, you're stupid. Fortunately, we have like a really good data science manager here, Ziad Ismail, who's built a great data culture and he let us dig into it. And so I started to think about the matching literature and economics. And so my hypothesis was that because we had restricted the pool of carriers to a smaller pool, even though they were higher quality, the match quality on the job went down. And so we were able to verify that and then we did some regression analysis to show that match quality had a causal impact on quality. And so that was a really exciting discovery because I think it showed how important matching is on our platform.

That's incredible. So to parse that just for myself, giving high quality carriers early access to loads meant that the matching them to shippers, the quality of that matching algorithm went down in some sense. And that was. That caused a reduction in quality of carriers of.

Yeah, the quality of, you know, how the work was carried out because, you know, there was a smaller pool of eligible carriers and even though they were better, the fact that there were fewer people to potentially match with the work trumped the fact that they were higher quality.

There's some sort, I think it's a Brazilian ant of some sort that has 30%. I'm not, I'm not saying that this is a direct analog, but I think, let's say they have 30% of ants in each colony that do nothing, right? If you remove that 30% and come back a day later, there's another 30% of those ants that, that do nothing. Now I'm not saying that there are shippers or carriers that do nothing, but there is some sort of stabilizing force happening there. Well, that's super interesting that it actually went the other way. That something that intuitively would you think would result in better quality resulted in worse. Yeah.

And this is a great example why it's so important to experiment. And when we run an experiment here, we've started this nice tradition where people vote on which outcome they think will win. So it gets the whole company involved in experiments and typically the UX team give the winners donuts or cool stickers or something like that. So we've had many experiments like that where what you expect to happen doesn't happen because it's very easy to fall in love with some feature that you think is amazing. And the reality is that it gets ever harder to find something that's going to move the platform forward in a sense.

These types of experiments. You're running some sort of laboratory, right? Yeah.

We are focused on our business questions.

If this is probing too much into company strategy or private material, just let me know. I'm just wondering how you gauge quality of carriers or quality of a delivery or anything like that.

There's this industry wide problem in trucking which is 10% of the time, roughly when a carrier is committed to take a load, they just no show or they tell you at the last minute they're not going to take it. And usually the excuse is my truck broke down and trucks don't break down 10% of the time. What it means is that someone offered them a higher paying load. And there are some carriers who do this all the time. And there's some carriers like, you know, I've looked at their data where they've done 100, 300 trips and they like basically never fall off. And so fall off is one of the key components we use to measure quality because it's super expensive for us when it happens because we're committed to providing a really high quality shipping experience for the shippers. And so we have to then go find another truck who will cover the load at the last minute. And that's really expensive.

That makes sense. Is there any consideration with respect to the advent of self driving cars or self driving trucks within, within your company as a whole?

Yeah. Oh, I should just say there's some other things that we think about in terms of quality like how the, you know, on time, percentage of the driver, like we're trying to get them to use the app is really Important because that allows us to drive cost down and compliance and safety. So all those are really important things. But in the specific experiment I mentioned, fall off was the main thing I worried about. We do find that if we can get carriers to start using the app, then all kinds of good things happen and are possible. And so another thing that we do as data scientists is a very economic thing, which is think about how to structure the incentives on the platform to get the behavior we want. And so like for an example of that would be if a carrier uses the app, they get quick pay and that means we pay them same day. And the standard norm in the industry is carriers get paid about 30 days after they do the work, which means they typically sell the liability to a factoring company and lose another 3%. So we're effectively giving them a 3% raise if they, you know, it's like 2 to 3% for the factor. So we have like a 2 to 3% raise if they use the app.

Yeah, it's a raise. And it also, it means they have more, more liquid assets as well. Right, right.

Like, I don't know about you, but when I do work, I like to get paid. And if I have to wait 30 days to get paid, it's not a pleasant experience. I've got to buy groceries and rent and.

Exactly.

Bike parts.

How about with respect to the advent of, of self driving cars and self driving trucks, is that something that you guys are actively thinking about at Convoy?

Yeah, we are definitely actively thinking about that. And I know the founders have spent a lot of time through their network being very plugged in. And on top of that, at the end of the day, when self driving trucks show up, and it'll probably be in phases where there's different levels of automation, they're still going to need to connect with freight. And we have a platform that does that. And so, you know, our goal is to be able to just integrate that on the supply side of the platform.

We've been talking all about the impact of data science on trucking with respect to the work you do at Convoy. And as we've kind of, we've approached data science from a variety of different directions and it's clear that a lot of things play into this discipline. And you, for one, you've got a background in, well, you're a computational economist, ex physicist, and also previously a research scientist at Amazon. So my question is, how do all of these disciplines and histories play into your role in what you do as a data scientist?

I think you can never have too many tools. And when I was a young physicist, I was really lucky. I worked with John Wheeler. And he used to say, besides channeling Niels Bohr, he would say, never quote anything until you know the answer. And this is a very famous Wheeler ism. But what he means is you should have a sense for what's the right answer for your scientific problem. And there was a famous example with Feynman where Feynman came in and thought he'd proved something. And Wheeler said, you're wrong. And Feynman was really annoyed because how could Wheeler have done this problem? Feynman just did it, he thought for the first time. And there was an error in Feynman's calculation. Wheeler's knowledge of physics was so deep, he just knew that it had to be wrong. It didn't make sense. And so you know things like if you go out and run an experiment to measure lift on your direct mail campaign and you get 10%, like that is probably wrong. You just wouldn't expect that to be true. So I think physics is very helpful in that way in terms of being scrappy and building up math chops, particularly linear algebra. I think linear algebra is super important for success in data science, perhaps more so than calculus. Economics gave me theoretical tools for thinking about business problems as well as econometrics tools for confronting the theory with data. And my time in software engineering gave me the software school skills to turn statistics into code. You know, everywhere you work, hopefully you're gaining new skills in your learning. I think that's super important for data scientists. I think also culturally, Amazon teaches a very adult way about thinking about problems. You know, having a sense of urgency, being focused on impact. It's a lot like being in a PhD program where you learn to ask yourself the question regularly throughout the day is, what I'm working on, going to get my PhD done. And if the answer is no, you're working on the wrong thing.

Now it's time for a segment called Principles of Data Science Education. I'm here with Jonathan Cornelison, co founder and CEO of Data Camp. What's up, Jonathan?

Hey, Hugo, thanks for having me.

For sure. It's always a pleasure to chat. What are you going to tell us about today?

Well, I'd like to chat about the educational and practical principles that underlie how we think about data science pedagogy here at datacamp. You know, Carnegie Mellon has done a great deal of research on learning, and there's one quotation from their work that sticks out for me. What's that to develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they've learned.

That's a great quotation and clearly delineates three aspects to the educational experience.

Exactly. There you have one, learning skills and concepts, two, you have practicing them, and three, you have applying them. And this is precisely how we have approached building our products and content here at datacamp.

So datacamp started out by building courses.

Yeah, that's right. And our interactive courses support our mission to help people learn data science by doing. Over the last four years, we've grown our course library to over 100 courses spread across R, Python, SQL, Git and Bash.

So this covers the first aspect learning. How about the second practice?

So we then developed a practice mode that allows datacamp users to practice the skills they've learned in a course by repeatedly taking short sessions consisting of, say, five to 10 challenges. The secret to deliberate practice is consistency and repetition. And so we designed the system such that it is easy for everyone to get started on top of that. And that's really exciting. We recently launched our mobile app which allows you to keep up practice on the go.

The mobile app is so much fun. And then we come to the third principle. Applying all the skills and knowledge.

Right, right. And to do this, we just launched a new interface called Projects, in which you'll be able to work on tasks data scientists encounter in their daily work.

Tell me a bit more about these projects.

So projects essentially allow learners to take the skills they've learned and apply them to an end to end analysis on a real world task, using real world tools and workflows, and then to showcase their work. And in particular, the project interface combines a jupyter notebook containing the narrative and the code of the project with a sidebar that gives you hints and instructions. And then after completing the project, students can download the notebooks and share them as part of their data science portfolios.

So that's a whirlwind introduction to how we like to think about the educational principles of Learn, practice and apply and how we put them into practice. Thanks, Jonathan, for the chat.

Thanks, Hugo. This was a lot of fun.

Let's get back into our chat with Ben. I remember you gave a great talk at Data Science Pop Up Seattle called Correctness in Data Science. And the reason I liked it is because you gave direction to what types of mistakes are made on what we can do as a field to correct those in terms of building a well defined discipline, which at the moment is, I suppose, a vague conglomerate of techniques. Concepts and applications. So I'm wondering if you could speak to what you think the major mistakes that you see data scientists are making today.

Yeah, no, thanks. I'm really glad you liked that talk.

I loved it. And we'll put it in the show notes as well, so everyone who listens will watch it.

Cool. First of all, I hope the viewers enjoy it. I think correctness of scientific models is super important and a lot of people, particularly when they're starting out, think, oh, my code ran successfully. It produced a number. It must be right. Well, back up. You want to make sure that's the right number. There's an epistemological framework for thinking about that that came out of the nuclear industry called verification validation and uncertainty quantification. I'm indebted to Robert Rosner, who was my postdoc supervisor and more importantly, former director of Argonne, for introducing me to VV and uq. There basically are three parts to VV and uq. The first V is verification. That's making sure your code correctly implements the model, whether or not the model is correct. And so that means you should do things like unit test. You can also generate synthetic data through Monte Carlo methods with known parameters and make sure that you get the expected results and do things like that to make sure your code is correct. Validation is making sure your model has fidelity to reality. So that's doing things like running experiments afterwards to make sure that your model is an accurate representation of reality and uncertainty. Quantification is about thinking about the limits to your model. What assumptions have you made? Do they hold? Could something like a tsunami show up and take out your nuclear power plant? Maybe you should plan for that. I think those are some basic things. I love to ask BI engineers when I interview them how they know if their SQL is correct. And they usually look at me like, this is a super crazy weird question. But SQL is crucial or whatever you're using to pull your data, because if you assemble rubbish data set, nothing you do is going to get better. Even if you do super fancy statistics, you're not going to be able to fix the fact that you didn't assemble your data correctly. So it's important to be very methodical and check as you assemble the data that it is correct. So you should think about join plans. You should test it on subsets of the data and make sure aggregate statistics make sense and distributions are appropriate. Check sensible things like you didn't get 10% lift on your direct mail campaign. And then the other thing that's really important too is models go into deployment. And so that means you may need integration tests or other tests to make sure that what's in production is faithful to what was developed in research.

Yeah. So where does a. Either where you've worked previously or at Convoy now, where does a data scientist sit in terms of putting what they work on into production?

Yeah, that varies a lot by organization and group. So in some organizations or groups, a data scientist will just do pure research and then pass things over the wall and engineering will do something magical. That can be problematic. Things are often lost in translation. Many engineers are not happy if you give them R code. Hopefully they're happy with Python. At Convoy, we're trying to work so that data scientists own their model end to end and that we have a machine learning platform that allows us to deploy models. We're not all the way there yet. We have some more work to do in that regard. But I think that's better for everyone because then the engineers can just call against the data service to get whatever result they need. Also, the way we're organized at Convoy, I think is very conducive for good data science in that we are grouped into product groups that all collaborate closely. But a product group will consist of a product manager, one or more data scientists, and a bunch of engineers. And the thing that's great too about our product managers here is that they're all super technical people. Most of them have masters in computer science or equivalent, a good mba, and they can all write SQL. There's a product manager here. It's in the interview I asked him a question that involved a SQL question that involved using left outer join and he solved it in like 60 seconds. I think he's tired of this example. But like, that's the caliber of PMs we have here. They're technical data oriented, they can write SQL, they can understand undergrad level stats, and that's very helpful because that makes them advocates for doing data science correctly. And then we have a lot of extra social stuff to make sure that the data scientists continue to connect and collaborate horizontally. So for instance, we run like a data science brown bag. We have technical one on ones where I meet with other data scientists and make sure that they're heading in the right direction and answer any technical questions they have so they're not blocked.

And I also think a technical product manager has, as you say, a lot of, a lot of wins. But one of the major ones is that they can really have the conversation with you as well. Right?

Yeah, they, they are very invested in being data driven. They know that if they write a plan for a new feature, they need to work with the data scientist to have a test plan or some other plan, you know, verify and validate their ideas. And they're all advocates for using data. And I think we have a super high bar for PMs. But it's, it's crucial in an organization like this that they can, you know, participate in the data conversation because often they're driving research questions. An example of how data driven they are is Ziad Ismail, who's the chief product officer. He writes SQL. I mean, this guy stays up late at night writing SQL to understand the business and has as deep a knowledge of the data in our data warehouse as anyone.

That's cool. So you yourself though, have, I must say, a very impressive toolbox of statistical econometric chops and data science techniques. I'm wondering what your favorite technique or methodology for data science is.

Yeah, that's a great question. And it's kind of like saying, you know, what's your favorite bird? Or, you know, whatever camera is kind of, you know, the British expression, I guess, is like, how long is a piece of string? So.

And nobody can answer that, right?

So I tend to like, yeah, what.

Are you interested in?

What do you like using the best tool for the job? So, you know, I've certainly, I came from a PhD program that was very strong in panel data methods. You know, like there were people at UCL where I studied, like Richard Blundell and others, who did a lot to drive that forward. So, you know, that's, that's a strength I have is using panel data methods. But I, you know, I also like a lot of the core ML tools. It really depends on the problem. I want to use the best tool. And so what I'm happiest most about is not using the tool, but solving interesting problems. You know, I'm an applied scientist. At the end of the day, I've worked on a range of problems from quantum cosmology to bioinformatics to, you know, trucking and among other things. So, you know, having interesting problems is what matters, and being able to find the right tool to solve it is important. And that also is in addition to tool data and software.

So we've discussed a lot about modern data science. What does, what does future data science look like to you?

Yeah, I think we're in this amazing time. Like, if you've got math and stats skills, there's like so much data just exploding everywhere. And I think we're going to have a very fun and interesting time until Elon Musk figures out how to put us all out of business.

What will happen until then?

Yeah. So, you know, at some point there will be tools that are going to automate away a lot of the simple models. I think you're starting to see companies trying to sell like commodity churn models and things like that, which I think can be problematic because ultimately every company's data is unique and you might as well build your own churn model from scratch. But you'll probably see more commodity models and more tools that automate a lot of the lower hanging data science fruit. And so I think to have a successful and happy career, you want to move further up the value chain where there are things that can't be replaced by automation, like automated feature engineering. Like for instance, at a previous company I worked at, Context Relevant, they made good progress to automating feature engineering for a large class of problems.

And what types of skills would you suggest aspiring and even well seasoned data scientists develop in order to not have their jobs automated?

So the first thing is you can never know too much math. And I think that that's something that's really worth investing. And that starts at a young age. You know, when I taught at Galvanize where they run a data science bootcamp. You know, I can remediate lack of programming with someone in eight weeks. I can't remediate lack of math in eight weeks or 12 weeks. You know, that's years of study. So I think you should continually invest in math. I think, you know, after that you want to master the core algorithms and then you need to keep reading. It's really important to keep reading. A lot of people stop reading when they get into industry. And you know, particularly for the more advanced people, you know, you want to choose some specialization that plays to your interest. So experimentation is something that's particularly interesting to me as are Bayesian methods. And these are both things that I've worked on going, you know, much more deeply in recent years. I mean, I know a lot of people are going into deep learning. That's a very competitive space. So I think there are a lot of other interesting and important areas in data science. And so, yeah, sure, if you want to go into deep learning, go into deep learning, but I think there's benefit in being a bit contrarian.

I think so as well. So with all that having been said, do you have a final call to action for budding and established data sciences alike?

I think the main thing I would Say to someone who's interested in getting into data science is understand that you're setting yourself up for a life of learning and that it's a marathon, it's not a sprint. It's like getting a PhD and you need to pace yourself. This is something that could take you multiple years to pull off. And so you need to keep investing. So maybe you should watch a little less Netflix at night and spend a little more time reading the relevant books, papers, writing code, playing with models. And if you don't have that excitement about data, there may be some other place you're more happy.

So keep learning, keep reading, keep doing.

Yeah. And for me, really, you know, I think and for a lot of us in, in the profession, you know, data is like an Agatha Christie novel. There's like this mystery in there and I want to unlock it and solve it and figure out, you know, if it was Colonel Mustard in the living room with the candlestick.

That's fantastic, Ben. So you're living your data science life as detective fiction?

Yeah, something like that.

That's incredible. I can relate to that a lot because the data keeps, just keeps giving. I've got a colleague who always tells us that you need to listen to your data. It'll speak as long as you're listening.

Right, right. And I think also I've seen when you talked earlier about the mistakes people make, one mistake I've seen a lot of scientists make is they leap into modeling too soon without doing eda. EDA is the famous Tukey term for exploratory data analysis. And it's really worth investing in some time in EDA because you will discover surprising things. So when I joined convoy, I started doing some EDA on the data and I found some data cleaning and outlier problems that had not been addressed and we fixed those and got like a 10% improvement in the pricing model and that was like free performance.

That's very telling. I mean that doing something like eda, exploratory data analysis, because it is tempting to just jump in and try to build models straight away and all of that type of stuff. But I always encourage people to try to visualize their data in 100 different ways and look at their summary statistics and all of that type of stuff before doing anything else. Right.

And I try to teach students a very methodical, standardized approach and that's one of the things I think is great about Crisp dm, which is something we discussed in prep. So Crisp DM is a cross industry standard process for data mining and it's probably the best workflow I've seen for a data science project. And it's really good to go through all the steps to make sure you don't leave anything out. So you start out with just understand the business problem, then understand what data you have, prepare your data model, evaluate and deploy. And at any point you may find some mistake that you need to go back and address in one of the earlier steps. So you start modeling and you realize, oh, I didn't do my feature engineering right. Or I need to add a feature to capture some key behavior that the where the model is failing. It's very good to be systematic like that so you don't duplicate effort or missteps. And ultimately in terms of correctness, which we've also touched on, I'd like to kind of find a way to mate crisp DM and VV and uq. And I think that then you're in a very powerful professional and mature setup.

Fantastic. And that really speaks to kind of a systematic structure for what future data science could could look like or incorporate.

Right. And then the other part of that is, you know, like the modeling is the fun part and getting to the answer, but the stuff that comes before it is super important. And it takes about 80% of your time that cleaning data, preparing it. And then the modeling is like kind of fast and it's like this high and it's like, boom, where'd it go? I'm back to normal. And I think that we will see more tools hopefully that will make that pre modeling period faster because that's where you're going to get your big productivity gains is if you can become faster in that phase. And so that's a good place to invest. So you should learn things like Unix and get really good at using all the command line tools and other technology or platforms that are going to help you get your data ready to model more quickly.

Exactly. Ben, it's been an absolute pleasure having you on the show.

Thanks Hugo. It was a treat to be here and I wish you the best of luck with the show.

Thank you. Thanks for joining our conversation with Ben Skranker about how data science is being used to revolutionize the trucking industry. We found out how the challenges facing an industry that predates the tech industry, such as trucking, can be overcome using modern tech and data science techniques. We need to keep in mind that this will have a huge impact on the ground as being a truck driver is the most common job in a lot of states in the U.S. we also discussed the importance of running experiments of CONVOY and the need for more consistent and rigorous data science practices. Make sure to tune in for the next episode of Data Framed, where I'll be speaking with Mael Saman, a data scientist who has worked in public health, both in infectious disease and environmental epidemiology. We'll be talking about the role of data science in epidemiology, the impact of open source software development in data science and and diversity.