#108 The Hallmarks of Successful Data Training Programs

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey. On datacamp.com. You're listening to Data Framed, a podcast by Data Camp. In this show, you'll hear all the latest trends and insights in data science. Whether you're just getting started in your data career or or you're a data leader looking to scale data driven decisions in your organization. Join us for in depth discussions with data and analytics leaders at the forefront of the data revolution. Let's dive right in.

Hi, this is Richie. Welcome to Data Framed. There's a fairly universal agreement that effectively using data is a huge competitive advantage for any organization. But while almost everyone at least says they are making an effort to get better at working with data, not every organization is doing it that well. A large part of the success of any data transformation or digital transformation program is getting the training part right. Data education is, of course, very close to our hearts at datacamp, and naturally I've developed a lot of opinions on the subject, but I also find it very interesting to hear how other companies think about data training. So in this episode, I'm chatting with Sharon Castillo, who at the time of recording was the vice president of global education at DataRobot. And Sharon's been involved in e learning for two decades and is a true expert in the subject. Hi Sharon. Thank you for taking the time to chat. I'm pretty excited to hear your thoughts on who needs data training and how you go about rolling out a training program for an organization.

Well, thank you for having me.

I guess. To begin with, one really common question we tend to get at datacamp is, well, who? Which My colleagues actually need data training, so perhaps you can shed a little bit of light on that.

Well, the longer I've been in business, the more I realize how everybody needs data training because people have a tendency to use anecdotal evidence and when you start to look into the data, it's astounding all the things that you can learn. And so in my organization, even when we've had interns, the insights that they drive in my own business, they say, hey, look at some of this data and look at what's happening here and point out things that I never knew just by getting an understanding of the data in our business. So the question isn't who needs the training, but what kind of training do they need? You know, some people need more, some people need different kinds of data training. So that's more where I would start.

Okay, so that's a kind of really interesting question about who needs what sort of training. So maybe you could give a few examples of that just for some different roles or different organizations.

So some of it is looking at the benchmarks of what you need in the key performance indicators. Everybody in my team needs to know what's the business we're in and what are we driving at, and then what business do we report up to and what are they driving at? Because I'd like to think we're all about training, but we're really not. We're about customer retention. And they're not really just about customer retention. They're about the larger goals of our business. So you have to really understand all the way up the chain of what your business is about. But beyond that, the more senior people on the team need to dig into sort of narrow pieces of it and really pull it apart and say, gee, if we did things differently, maybe this is a smarter approach. Or gee, if you take something that seems unrelated to something else, maybe there's a pattern there and maybe the offer we have today isn't the smartest way of doing something. So those are all different things to look at.

Okay, so I really like that idea of matching the training needs back to what are your KPIs or what are your business goals. Do you have any sort of specific examples of how getting some data training or getting some data skills might sort of lead back to one of these KPIs or goals.

So we had somebody on our team who started using a particular tool where they looked at what users did at after training inside our products. And so we followed the path of what users did two weeks later, four weeks later, a month later, two months later, to see if we could find patterns. So that person needed to get training on that tool, but they also needed to get training on what does good look like using our product. And they needed training on what customers who were doing well look like. And what's the differentiators between people who are doing well and not doing well. And then that person was able to start to see patterns of people who went to training versus people who didn't go to training and people who went to certain kinds of training. And so we started to pivot some of the training offerings to focus on areas that might have more impact.

Okay, I want to come back to Impact in a moment because that's a really important topic is to try to evaluate what's the actual benefit from training. But if you're saying, well, everyone needs some level of data training, I'm wondering, how do you decide who goes first? So maybe if you don't have the budget to go and give everyone a load of data training, how do you prioritize who needs what most?

So not everybody needs tools training. So some of our training is informal. Every quarter I have a meeting with my team and I say, here are KPIs and here how we did as a team. And somebody who develops training still learns about delivery and somebody who delivers training learns about development metrics and they go to meetings about customers and the data about our customers. So there's informal training and then the people who come to me and say, I think I can dig in here, dig in there, but I don't know how to use whatever kinds of tools they're going to use. BI tools, or in our case, we're lucky we have data robots. So, you know, learn our product and feed all the data in the product and dig into it. So fortunately those people are also trainers, so learn it really well.

I suppose people who have done some training maybe have an advantage in terms of knowing how to learn things. Going back to this idea that sort of different people need different things they have to learn. I'm interested in how you figure out what order people have to learn things in and how you go from just learning individual things to having a whole learning path for people to get to their training goals.

Yeah, so you have to look at the journey and it's more complicated if you're in a more emerging technology area. So I've been in several. When SaaS became a thing, when E Commerce became a thing, when cloud management became a thing. Each time you have to pick apart what do people need to know in different. I hate to use the word roles. I'll get back to that in a second. But what do they need to know to do their job? And what's the basic foundational stuff that everybody needs to know? Like how does this technology work? And then layer the skills that everybody needs to know and then what are the specialized skills? And then the learning path has to include pre assessment so that people can Figure out where are they? So you don't waste their time on the foundational skills and meet them where they're at. Because that's one of the problems of people not wanting to send people to training is, oh, I know that. And then they don't want to go to the whole thing because they think they're wasting time. In fact, they end up missing a good chunk of things that they don't know because maybe they know a third of it or half of it, but then they jump past the half they don't know. And then the other problem is the roles, because there's a lot of overlap or the titles are wrong. Titles don't mean anything. If you work at a tiny company and you're called a data analyst or a business analyst or a data engineer or a data scientist, that means a lot of things in a lot of places. It could encompass a lot of things. It could be a very narrow thing. It could be someone who's fairly junior, it could be somebody very senior. You know, it could be somebody who's just finished a boot camp or somebody who finished a boot camp or master's degree or something 10 years ago and has lots of industry experience. It's a lot of things. So the title doesn't really work. And one of the things we found by using data is that we had roles and some people would take the training for every role and like about a third of the way through the second role course they realize how much overlap there was and then bail out of the second course because they're like, oh my God, I already saw that. So instead we've really pivoted onto skills.

This is really fascinating and I have to say the idea that job titles are sort of nonsense for trying to work out what training needs. Like for a while I had my own consultancy and I was like a one person band. So I called myself CEO, but I don't have CEO skills in general. And so it does really resonate the idea that job titles are nonsense. So I'm interested about this idea of skill mapping that you talk about. So how do you figure out what content is related to particular skills?

So we work cross functionally a lot. I'm also not a fan of just focusing on features. So some software companies get obsessed about features. Features don't make customers. Successful customers don't care about a feature. They care about solving their business problem. And so we look at successful customers and we work with customer success. We work with a lot of data scientists. Our company has hundreds of data scientists. And we talk to them and we say, you know, what are the key things that make people successful? Where are the gaps and where they stumble along the way? And then we map those things out and we don't always get it right. Sometimes you have to iterate back and add those things in.

Okay, can you maybe give some examples of specific data skills? I'm hesitant to use the word data scientist now because I know there's a role and you said you're not keen on roles, but can you talk about maybe some of the skills that you think are applicable for maybe a data scientist role?

The biggest one, I would say, is asking the right questions. So how do you frame a problem? How do you pick the right problem? How do you know the business, the data of that business, what's important data, what's not important data, like what's just sort of tangential and it's interesting, but it's not going to get you to your outcome. And then you really have to dig deep and become a subject matter expert of your data, of your processes, and look at your outcomes. You know, if you work for a public company, read the stuff that they put out on brochures that say every quarter, what's important to the business, and align everything you're working on up to that. And then start small, start to figure out what problems can I attack and how do I prioritize those. That's more important than any cool tool or algorithm. And then you have to start to learn how to see, is your data any good? Is it biased?

Okay, so you mentioned maybe the most important thing or perhaps a good place to start, is understanding how to ask a sensible question about your data. How do you go about learning that? Because that seems like a lot of skills in itself.

That's a lot of skills. I'm not a data scientist. I talk to my data scientists about, you know, where to dig and what's reasonable. I once worked for a financial management company and they were having an issue and I was learning about mutual fund expenses. I know nothing about mutual funds expenses. But like for a certain size mutual fund, is this fee reasonable? And so if you know your business, is this particular thing reasonable? Does it make sense? Is this how you would calculate this thing? And that's where I would start to dig into, is this really how somebody goes about, you know, if you took away the computer and you had to figure something out, what are the steps that the basic level has to happen? And then start digging into each of those gates as to what happens?

You know, okay, and just continuing this sort of theme of trying to figure out what skills people need on an organizational level. So you mentioned it's sometimes quite hard to work out where people need to start training, like what they know already and what's the first course or what's the first bit of training they need to take. So how do you go about sort of auditing what skills people have to begin with?

Yep. So that's a challenge when you work for a software company, because our job is to train people on our software. And sometimes people come without the skills of the things that came before it or the things that sit next to it. So if they want to integrate with somebody else's software and they don't know the other software or they come to us and they want to use our API and they don't know Python, you don't need to know Python to use our product, by the way, but say they wanted to do that, you know, are we going to teach you Python? Are we going to teach you how our API works? Most of the people who come to us don't want us to teach them that, but there are some people who do. There are some people who come to us who don't know what AI is. It's a black box, it's magic. And have no idea what the difference between AI and machine learning is. And so you walk that fine line all the time of where do we start? You know, do you refer them to other things? Do you? Whatever. But you also have to help people assess themselves as to where's their base level and do we need something supplemental to help them, which we can do, or can they start right at what the entry level of the product is?

That's interesting. So this is also a problem we face in a data camp is just trying to make sure that all of our learners are aware of which courses are going to be suitable, which are going to be too hard or too easy. So you mentioned the idea of being able to self assess and data camp has skill assessments. But I'm curious as to what your sort of data robot take on letting users self assess it in order to find out what content is suitable.

Yeah. So we're just starting this. We have little descriptions and our courses are short and we're making them shorter all the time so that there's not a lot of time investment. If you get into a self paced course and it's 30 minutes long and you get to the first bit and you say, gee, that's. I need the basic stuff before it, there's already a little place that you can click out to. I call them on ramps and off ramps. So there's always now an on ramp and an off ramp for every new course, both instructor led and self paced.

Can you tell us a bit more about that? What do you mean by an on ramp or an off ramp?

So an off ramp is. If you make courses shorter, then there's always a place for somebody to go. Instead of saying you need three days of training, if each class is like an hour and a half long, two hours long, the off ramp is. Your next steps could be one of the following, depending on what your interests are. And we always give people a map of what that is going to look like. The on ramp is where they came from. When we design, we assume they came from one of the following places and one of those could be somewhere else. I could assume somebody went to Data Camp. I could assume that somebody came from a sales demo and bypassed the little intro course we have because they said, oh, I saw the sales demo, I don't need the intro course. So then what supplemental things do I need to let them have a sort of sidebar so that they can continue on? We are starting to build in, though, a process for people to take a couple little questions, one for each objective in the course to say we don't have. They're adults, so we're not going to say you can't take this course if you haven't passed this. But here, take this thing. This is what this course is about. If you do well on this assessment and you feel comfortable with this material, feel free to move on to the next thing. If you pass this assessment, still don't feel comfortable, go ahead and take it. And if you don't pass this assessment and you don't feel comfortable, this is what's going to be in here, so you might want to continue on.

So basically, you take an assessment and that's going to help you sort of decide, okay, do you want to do this course or not? That's pretty. It sounds straightforward, at least in theory, doesn't it? I know for a fact there's a lot of technical details involved in this, but that's a good theory. I'm curious if you think about, like, whole teams needing training, what happens if some people have, like, more advanced skills than others? Some people know stuff like everyone's coming from a different opinion, so they need different levels of things. How do you sort of reconcile differences in skill level?

Yep, happens all the time. Some teams like to go through as a Cohort and some teams go through as individuals. So if you go through as a cohort, there's a couple of ways you can do it. You could have tracks in the cohort where there are meetup points and you say each person has their own individual things they work on and then there's either a mentor or a meetup or in our case we have a data scientist who has sort of an enablement session where people can work on what they're working on. But then they come together and they say this is how it applies to your use case. And then everybody sort of is together and thinking together about how to work together as a team because they have to work together as a team team to, to make this thing come to life. But different and different groups do different things. Some of them do hackathons at the end together, but go through the training separate. Some of them have the more advanced people still go through class together in the less advanced class. Works out great in some team. In some teams it's very intimidating to the people who are less experienced. Depends on the culture of the company.

So I can imagine the social dynamics is complicated. So if you've got some people who are very advanced and some people who aren't, if the person who's advanced is, is. Has got a sort of mentoring personality, it's going to work well. But if they're just competitive, then it's going to be a disaster. Maybe it's just. That's just the answer, right? It's just you have to think about the, the human aspect and who's actually involved in this. That's absolutely fascinating. I just want to sort of segue a little bit into talking about like different formats for training. So Data Campus has always been very much focused in the sort of learn at your own pace online training. I know Datarobot originally started out doing more live training than online training and I'd love to hear your thoughts on what you see as the pros and cons between the sort of different formats for training.

Yep. So the big con of live training is Covid. I started a Datarobot three weeks before lockdown and I told my team, you got the week to pivot. But besides that, there's a lot of overhead and lag in doing live training. Live training happens a lot at companies when they're starting up and you want customers who are in an emerging market who have low maturity in the whole technology space and there's a lot of nuance to their questions, sticking them in front of self paced. When you don't really know what their questions are and you don't really know what their journey is, is not going to get them to where they need to go because you don't know where they started. So in those circumstances, typically small companies don't even really have trainers. Typically they use support, they use customer success people. They sometimes engineers like it's not even a formal trainer, they might have one. They don't necessarily believe in customer education as a professional entity. And as they get larger then they start to think about, gee, we really should have professional training and professionals who understand education and all of that. Other companies with more straightforward products start the other direction and start with videos. And you can use video more similarly to sort of a more extensive documentation but more engaging and more designed around education. So it's just a different starting point. And then the virtual training has really changed during COVID I think we all, I've already done it in a lot of places for a long time. But the thing that changed in Covid is that students used to be in the office, just globally and we would teach and they were in their office doing it. Now we see a lot of people who are like on the east coast of the United States but taking a class in APAC and they do it because they want the flexibility. Nobody's bothering them on Slack, nobody's bothering them on email because it's late at night. They can have dinner with their family, they can do their thing, it's quiet, they can focus on the training, there's nothing else going on. And some people really find that to be a more effective time if they're going to do live training.

That's really interesting. Just the idea that people would just want to do training at midnight or whatever. I have to say I'm not sure like how well my brain would work at that time, but I guess it does work for some people. So yeah, I'd love to hear more of your thoughts on scheduling training because this is like a really common problem is people go, well, I've got a full time job, I've got deadlines to hit, when do I find time to learn?

So there's two things about that. One, people are zoomed out. So courses have to be shorter, they have to be more engaging. You can't lecture at people for two hours on zoom. That's not training. People sometimes hand me recordings of meetings that they call training and say, can we turn this into self paced? And I'm like, that's not training, that's a meeting. So you have to be really thoughtful in what training is. A recording of something isn't training. But training isn't an extracurricular activity. You can't layer it on in addition to someone's job. So I wouldn't recommend somebody doing an eight hour a day and then staying up till midnight like you were in college and cramming for the next day. People can't sustain that and you can't get anything out of it. So I would say the two big mistakes that happen with training, and this happened with live in person too, is that the logistics of training make it easy to overload people to the point where the training isn't really effective for them.

In what way would you overload people?

So if you have a trainer who flies somewhere, it's less expensive to train people for five days than to have them come back three times. But what we know is effective is spaced repetition and layering on and practicing in between. And then people have questions and people know what they don't know because they went tried and they're like, you know, you said this the last time but it makes no sense. Or I tried it in my own job and that doesn't work for me. But they don't know that if you're just like giving them the fire hose of information over a five day period because you happen to be there and they forget everything. So that's one piece. The second piece is that just sending somebody to training without any practice is not useful. So years ago I had a student and he used to come to my same class every six months. I'm like, why are you here? And he's like, well, I didn't have time to practice. In between I forgot everything and now I need to use it. And about the third time he took the class, at the end of the class he said, yeah, see, in six months. Because he knew the reality was he wasn't going to get to practice it in between either.

That's, that's kind of a sad story. But yeah, I can certainly understand that. So I want to go back to the. You were talking about space repetition. I know this is like one of the most important theoretical ideas from learning from point of view. Can you just tell everyone what their space reputation is?

So for something to sink in for people, you can't just blast a ton of information at them. You need to take a piece of information and skills and other things that you're going to do in class, go away, let them work on it, come back, recap a little bit of it. But Then dig deeper like peeling the onion and then layer onto that and take sort of the hooks that they know and come back to it and then iterate on that sort of process. These days you could use hybrid learning as well. Where we do in class like virtual and then we have labs that people don't need to do in class that are self based. And maybe you pick your own adventure with those labs. Like maybe one person cares about manufacturing, another person cares about finance. You go off and do a lab on the same topic. You care about regression modeling, but you care about it relative to manufacturing and you care about it relative to finance. But you both go off and do something and then you come back and you learn a new skill, but you're using that basis that you just had and you just keep pushing through that. So the idea that training is just onboarding and then I'm done. It's a fallacy.

So I like this idea of having lots of bits of training that sort of build on top of each other. You also mentioned the idea of hybrid learning. So a mixture of sort of online training and live training. How does that work best? You like online first, then live training or the other way around or what works for people.

We don't choose for people. Some of it's financial too. So it's more expensive to do to hybrid with instructor led. So it depends what people's budget is. Some people solely do self paced with us, but we don't dictate what you do first. I think it's cultural. Some companies do only self paced. Some people, my kids only look at video. I don't know that they would not. I'm wondering what happens when they're in the corporate world or they they're gonna Want to watch 3 minute TikTok videos or YouTube video.

Having to read reports, how terrible. But that would actually be really interesting if there's like a generation's time all like corporate reporting goes away and is replaced by I don't know, 30 second videos or something. That'd be a very interesting world. All right, Talked a bit about scheduling time to learn. You were talking about how some people will just do their learning at midnight and maybe it's like a terrible idea for people to have like full time job and then have to do learning afterwards. This is a point that I would really love like all the managers listening to this sort of aware that don't just like overload people's brains and make them learn in their own time. So when you've got these competing priorities of okay People have to hit deadlines, but also they need time to build their skills. What's the sort of message you'd give to people who are managing or involved in, like scheduling things to say, well, this is how you justify doing Some.

Training should be their okrs. Just like everything else. The only things that work are, you know, money talks. So if you don't compensate for people, you don't build it into their metrics, you don't build it into the company culture. Nobody says, oh, nobody should be trained. Everybody nods and says, oh, training is a good thing. We should train our people. But like you said, competing interests. I have to get a B and C done or I don't get my bonus. This person has to get a B and C done. Well, it should go all the way up the train that there's an OKR or KPI or whatever it is you have to hit to get your bonus. One of them should be that your workforce is trained and they should be trained on the things that are valuable. Don't put things in to just check a box. Put things in there that add value to your business.

Wonderful. And maybe you can sort of elaborate on that. Like, if you're trying to prioritize and go, okay, these things are valuable to our business. What's the sort of strategy for deciding, okay, these are the important trainings we need. And like, is there any sort of quantitative way to justify this?

So take it back to what you asked me in the beginning. The data. If people are trained on the data and that data helps you improve your business, that data helps you find insights about your business, that data helps you benchmark and improve or find insights you never knew about before or do things more productively that you could take off the list so you could be more effective. Isn't that worth the time and the money you just invested in that training?

Absolutely. I've got this horrible feeling you could end up in a catch 22 situation where the manager doesn't have the data training to be able to analyze the data to figure out what people needed, what people need to be trained on. But okay, so maybe that those managers need the data training first, but that's pretty interesting. So do you see any common sort of success patterns or any common, like, disasters that organizations make when they're trying to figure out these learning programs and trying to figure out when to schedule things?

I don't see disasters. I just see more when it's ineffective. They really should be looking at high quality training. Is it the right training? Don't Just pick a program just because it's a program that has the topics they want, does it really address the needs that they have? Does it address it in the way that their company works and have a trial the same way you would with Software? Don't send 500 people to trending. Do a couple of people through it and say did you get anything out of it? And if you did, then continue on and if you didn't, then find something else. I would say that would be both a success way of doing things and the disaster if you don't.

I like that idea of sort of like trialing things with a few people, then gradually sort of building up how many people are going to take the training so you don't have a big mistake in the wrong direction direction to begin with. That seems really sensible. So going back to the idea of having like several, like groups of people taking training at the same time. This idea of cohort learning where several people do the same training at the same time and then they can socialize, this seems to be becoming very popular. Have you seen any examples of like cohort learning? A datarobot?

Yes, but we do it a little differently. So we could do the same people in the same class at the same time. It doesn't always work for everybody from a scheduling perspective because if you have global teams, that's painful. It's painful to schedule. There's always somebody who's unavailable, who missed it. So the better way we've been working with, and this is new, so I'm saying it's better, but I don't have the data is having a certain period of time that everybody has to take certain courses. And so we have a schedule of take it and anybody in any time zone pick the ones you want to take. But you have to take this course in this two week period of time, say and everybody has to take the next course and next period of time and then have reporting to report back to the coordinator of the cohort to say how you do it, are you progressing through and so forth. Now some of this depends on the size of cohort you have. And again you need accountability and all that other good stuff. But it helps make it more scalable because if you just say I'm going to do a cohort, it works great if you have a lot of resources and you have works in certain size organizations, but it's very difficult to scale.

That's really interesting. And you mentioned the idea of time zones being a problem. And so this is something that doesn't really exist with on demand learning because people just take it whenever they're awake. But I can imagine with cohorts it seems like everyone in the cohort should be in a similar time zone then, is that correct?

If you do a self paced cohort. So we've done that too, where people take the self paced training and then you have a mentor in different time zones and they have a Q and A session and they kind of tee up and preview and you could have one for the Americas, one for EMEA or a couple time zones in emea, a couple time zones in APAC kind of thing. And at least then you're accommodating different zones. But people still are doing the self paced and there's still a schedule that people have to adhere to. And you can still, through the learning management system, get reporting back on, oh, these people completed. And if you take a test at the end of each, these ones are doing well. These people seem to be falling behind or having problems. And then if you use things like community or chat or whatever method you want to use to say, hey, you seem to be confused on topics or office hours or whatever it is you want to use to make sure that people stay on track.

Okay. And one of the sort of, the big sort of selling points behind this idea of the cohort learning is the idea that because people have got some sort of social interaction direction, they're going to be more engaged. So what sort of techniques have you seen for organizations to have to encourage social interaction between people doing training?

The cohort idea works well. And again, using whatever kind of communities you already have. So whatever form of social media or whatever your company's culture is to engage with people, whatever kind of meetups you have. So if you have community meetups around, if it's a message board, if you have postings on an internal wiki with, with information for people, more advanced companies have centers of excellence that run these things and have best practice sessions and they're the ones who kick it off in the first place. And I had mentioned hackathons and other kinds of more creative fun prizes where you would be amazed how much prizes. Even inexpensive fun T shirts, pricey.

I definitely like that work. Yes, always happy to receive prizes. So yes, I agree, that's like a really simple way of getting people encouraged and engaged in this sort of thing. That's brilliant. So one thing that sort of, we seem to have talked about sort of intermittently throughout this is just the idea of management sort of role in data. So I'd like to Sort of get into this a bit more detail. So in terms of like running a sort of successful training program, like how important is getting like management buy into it?

Extremely important. I think early in my career I read some study that said the manager before training is probably the most important thing because if the manager says to you, go to training, I cleared off your calendar, ignore your emails and your slack or whatever it is, focus on class and when you come back, worry about it getting caught up. But right now this is the most important thing you can do. That message is really different than, oh my God, we can't do without you next week and yeah, I'm going to let you go, but we really can't do without you and I'm going to blast you 15 times while you're in class and I'm going to pull you out of class. What does that say to people? So what they say beforehand is important. I think another thing that would be really helpful for a manager beforehand is to set expectations of what they hope someone gets out of the training. I'm sending you to class for XYZ reason. I really hope you find out about A, B and C, because I'm really curious about that and I'm hoping that that'll apply to this project in this way. So keep an eye out for that while you're in class. I bet people pay more attention and then I ask them when they get out of class and I bet the next time they go to class, they know somebody's going to ask them about it when they come out of class. And if you have a project that uses it afterwards and you get people engaged in those projects and oh, by the way, I also went to that training before too. Maybe you guys could work together on it because I know that they worked on it and they're probably about six months ahead of you on it. And again, that perpetuates a culture of learning.

See, I really love that trick of just asking people, like, what did you learn in that training? They're like, oh, well, I wasn't concentrating, I was checking my email. Yeah, they'll only do that once. So certainly having those sort of tricks to make sure that people concentrate seems, seems really useful. I also like the thing you talked about, how you should just sort of clear your mind, get rid of any distractions beforehand. Sounds a lot like the sort of thing you get at the start of a yoga class where it's just, I don't think about anything else. You just hear in the zone for an hour and just concentrate so that seems like a really good trick for learning. I've now got ideas for productizing this. I'm going to speak to a product manager afterwards. Okay. Maybe we can talk about the flip sides, like what happens if there's sort of no real management interest in training. Now I know we have Data Camp has some customers where there's employees do their own thing and just get training on an ad hoc basis. Do you have any sense of whether that sort of technique works? Just letting employees go at their own, like figure out what they want by themselves?

It depends on the employee. If the employee is super motivated and probably has their own reasons for wanting to go, their own career development, something they're working on, and they're really stuck on it and they're like, wow, I could get further with it. Yeah, they'll be successful. But beyond that, probably, if you go back and look at your students, you could probably pick out which ones went because there was real push behind them to go and support for them going. And you could probably pick out the ones who begrudgingly were like, okay, well we have a budget and you're allowed to go, but I don't want you to go.

I guess we're going back to like human dynamics here. So it just very much depend on the individual. Like some people need a bit of hand holding and guidance and some people like unmanageable and they have to do their own thing. And I guess most people are somewhere in between. So sort of last thing I want to talk about is about how you go about measuring the success of a training program. So do you know has this training been useful or not?

So I talk about big C's and little C's meaning the customer. So a little C is a user and the big C is an organization. So the little C a customer you really do look at. What's the outcome for that user? Did they use the product and the aspects of the product that they took in training within two weeks of class, they took it six months later. It probably had nothing to do with doing class. If they did it within two weeks, probably what you taught them stuck and they're using it and it's effective. If you have enough data, you can see what they did. Did they do it correctly? If you can really trace their path through the product, if they get certified, they take an exam and they do all that kind of stuff. Did they go deeper? Did the user get engaged? Did they come back and take more training? Did they follow the path and go to community? Did they do other Things. Now, some of that is a little bit circular. Were they motivated to start with? Were they coming to lots of things? So it's a little bit hard to tease that out. But in general, the people who go to training, if you take a trend across all of your users, the trained group tends to do better in these areas by a lot. And then you look at the big Cs and you say, gee, overall, how are those customers doing? And generally, they hit all the targets of onboarding. Better engagement, better churn is lower, all those other things. So if you're talking about a training program for customer education at a software company, those are the types of things we would look at.

Interesting. And maybe can you talk a bit more about engagement? Because I know at Data Camp we have lots of different metrics for how engaged users are, but I'm curious as to how you think about this.

Yeah, there's so many different ways. And my colleagues and I sit around and my colleagues in customer education across software companies talk about all the different ways you could measure engagement. And it's difficult because we don't have access to the data. It's hard to see inside your product. We don't always know what the journey should look like, especially if you have nascent products that are newer or changing all the time. And it's hard to know because everything's changing all the time. So we change. Our training offers a lot to adapt to everything that's going on. And so somebody will bring me data and I can say to them, yeah, the data doesn't work for me before this date. And they'll say, why that date? And I'll say, because that's the date we changed all of our curriculum around this, and that's the date that we did this other thing, and that's the day we did. So we make some major changes. It's really hard then to tease apart what happens around that, because everything sort of shifted. It's very, very big challenge.

All right. And maybe you can talk a bit about what you find are the kind of most requested things that people want to learn about.

So a couple of things people ask about when they're earlier on. They ask a lot about things that have nothing to do with our product. They really need change. Management, project management, basic skills around the. The industry itself. What is AI? What is. What is data? What is all of that stuff? Then they want the basics of the product, tons and tons of things about the basics of the product and how can they be successful with the product? And then as they move up the maturity curve. They really want best practices and deeper and deeper knowledge about how to solve more and more complex problems that are very, very specific.

All right. And so just to bring this back to something you said at the start. You talk about how everyone kind of needs some sort of training and some sort of data training, perhaps. So what would you say to people who are hesitant about learning some data skills?

I would relate it back to something that they do every day and think about it in terms of data that they know. So think about something you're really passionate about, whether it's sports or music or something that you really, really get into and are passionate about. And think about the information that goes with that. And how could you use that and use that to frame it? Instead of worrying about algorithms and math and statistics. And don't get too far ahead of yourself, just think in terms of really small, basic things that you could do with that data and take small steps.

And last question. Do you have any final advice for any organizations wanting to start a data training program?

Figure out what your goals are. It's the same thing. It's small steps. So what are the most important goals? Break it down. What's achievable? What are quick wins?

All right, super. Thank you very much, Sharon. Lots of really exciting things we've talked about today. I think lots of great advice in there. Thank you very much for your time.

Thank you. You've been listening to Data Framed, a podcast by Data Camp. Keep connected with us by subscribing to the show in your favorite podcast player. Please give us a rating, leave a comment and share episodes you love. That helps us keep delivering insights into all things data. Thanks for listening. Until next time.