MSP Matchmaking: How I Would Vet Your MSP: An Insider’s View

Alright, my friends, what's going on? Uh, hope you guys are all doing awesome today. I'm joined by my good friend Jason Slagle. Jason, what's going on?

Not much. I was digging that music, right? Kind of got the little stream thing hiding back there. It goes—

there it goes. Well, good deal, man. How's, uh, how's things been going for you?

Good. How about yourself?

I cannot complain. We just did a cyber insurance boot camp, um, last week, and it was really good. And, uh, topic of what we're talking about today kind of fits right into it. Um, and, uh, really had a lot of fun just diving deep into, uh, cyber insurance, how to build that into your process.

Eric, stop giving me the FOMO. I was supposed to be at that, but due to a handful of reasons, I couldn't actually make it out.

So you had good reasons, my friends. It's all good. Um, what we're gonna do, I think we're gonna redo one here in Tampa. So maybe here in a few months, we'll, uh, we'll recycle it and do it all over again. Um, it was definitely a whole lot of fun. Um, all right, so we're gonna be talking— by the way, look at my awesome mug.

Sweet.

I got that from you, just actually from your kind mother, actually.

Yeah, I'm rocking the—

there you go, that works.

Hunter's pen.

So if you guys are joining us today, what we're going to be talking about is how to vet an MSP. And this is something that comes across— it doesn't matter whether it's an MSP. Aaron Peters, what's up? Thanks for joining us today. It doesn't matter if you're talking about an MSP, you're going in a vendor hall at a conference and trying to vet vendors, you're trying to vet a like a car mechanic, like we always have this question, don't we, of like, are they any good and how do I know? Uh, what do you think about that?

Yeah, that's, uh, it's definitely a thing. We all have processes that I think we use to vet them. So I think it's actually a good idea to share, you know, some of the things that— I'll lean, obviously I'm not vetting MSPs, I'm not purchasing MSP services because I am an MSP, but, uh, I think some of the same things apply when we pick vendors.

Yeah, I agree. And I think we always want to know, you know, whether it's like Consumer Reports or whatever, we just want to know from people that have been in the, in the business, like, here's what I know. I do this for a living so I can tell you what is good, what is not good, what to look for, what to watch out for. And so that's what we're going to do. I think pretty much everyone knows Jason, but in case they don't, tell us a little bit about who you are and you'll quickly find out why he's qualified to be talking about this.

My name is Jason Slagle, president of CNWR. We are actually an MSP in the Toledo, Ohio area. Been doing this for a bit. I try to stay relatively security-focused, and I don't know, somehow I've got myself invited to these various things, so I'll take it.

Yeah. Yes, indeed. Now I'll take it as well. Okay, cool. So here's kind of what got me thinking. What I wanted to do, Jason, is just share my screen out and talk a little bit about a report that came out. You and I had— we We were talking about this quite a bit last week or week ago, and it's not that this report is like the most incredible, like awesome report. We got to dive in every bit of it, but it made some comments and some things I thought were pretty good. So let's take a peek at this. So many of you have heard of Beazley. They are a cyber insurance carrier and a really good one. I really, really like Beazley. They've always done a very good job at being accurate in what they do, and they typically are a leader in terms of the archaic processes of PDF-based testing. But here's what I wanted to take a peek at. So this is their areas of vulnerability. They just came out with this report just a couple of weeks ago, and they're talking about some of the normal things that we would expect, right? Watch out for social engineering and spear phishing, bypassing MFA. But look at this, Jason, targeting MSPs. So there it is right at the beginning, areas of vulnerability, area of concern. Do any of these that they put up here, any of them stand out to you or strike you as interesting?

Yeah, I mean, uh, as an MSP, we are a force multiplier in exploits, right? So you, you know, if you can pop 20 small businesses, 30, 50 small businesses with one exploit and one kill chain, you know, why would you not do that? So it doesn't surprise me, uh, that we are targeted, right, at all. Uh, in the, in the ways they're targeting us, yeah, MFA bypass, MFA fatigue, right? Like, all of those things are a huge uptick. Uh, I've actually, in the past, uh, 2 weeks, we've seen a pretty big uptake in phishing attempts and, uh, attempts to do business email compromise, right? So this is definitely a thing that we're seeing actively right now against our clients. So I can only presume it's also happening against MSPs. I mean, I get several of them a week to myself.

And can you explain a little bit, you mentioned, um, there's so much to unpack here, but you mentioned MFA fatigue. Can you kind of talk about that a little bit? Because I think that's pretty interesting.

Yeah, it's, uh, basically humans are lazy, right? So, you know, at first we had TOTP, right? You know, you get that like 6-digit code in your Google Authenticator and you had to type it in, but people are lazy, right? So everyone kind of moved to this world of push notifications, right? So instead of having to enter the 6-digit code, uh, you just send a notification to my phone and I hit approve, right? Because that's still something I know, my password, and something I have my phone. The problem happens when I send 300 of those requests to my phone in an hour, right? Now I'm getting bombed, and the easiest way to make the bombing go away is to hit approve, right? So, uh, rather than, you know, controls or something in place, uh, these threat actors, if they get the credentials, will sit there and just bomb the MFA. And at some point or another, somebody slips up or hits approve, or in some cases they're super egregious and they'll actually text and say, hey, this is your IT department, something's wrong with the MFA provider. You're going to get these notices until you hit approve. Right. And this is like a backhanded way to get somebody to hit approve. And once they do, you're done, right? Like any benefit you get of MFA is gone at that point.

And I think this just points to like we had Mackenzie Brown from Microsoft DART, their detection automation research team, on Cyber Call, and she said just freaking enable MFA, people. 99%, it stops, right? And it's That's powerful coming from Microsoft. But I think we know at some point, and we're starting to see the beginnings of this, that bypassing MFA has always been a thing. We just haven't seen as much of it because bad guys haven't needed to. But we're going to see that envelope being pushed even more. Another area, and I'd love for you to talk about this too, before we kind of get into the thesis of what we want to talk about as well, is what about things like OAuth? What about things like SAML? Can you kind of explain what those things are and how those could be bypassed or used as avenues of attack as well?

Well, I mean, it's the same thing, right? So in the end, the underlying technology used by most of these authentication providers are OAuth or SAML or OpenID, or there, you know, there's a handful of them. And, uh, if you can get into the middle of that flow, you don't even have to bypass MFA, right? You could legitimately let the MFA pass through, you know, if you could proxy the connection to Microsoft, and you could just capture, capture that session and capture that token. And once you have that token, you could just replay it and you can act as that user as if you were them. There There are some protections against it, but that's an attack avenue that I'm seeing increasingly commonly.

Yeah. Okay. So shout out to you guys in chat. There's a whole bunch of you guys in chat right now. I want to know from you— I'm going to pop the chat overlay open in just a second. I want to know from you, MFA as a whole and MFA bypass, is this something that you're thinking about? If you're an MSP today, are you talking about MFA bypass? Are you like educating your users on things like the weaknesses that could exist and what Jason's talking about, or even like the fatigue kinds of attacks? Do your people know about it? Do you see it as a threat? A lot of opening questions, but I'd love to get your feedback on just kind of where you stand with all of it. Jason, what about you guys? Are you teaching your users about some of these things and getting them prepared?

Yeah, I mean, as a matter of fact, I think in April, Microsoft is going to turn on something called number matching for everybody, and we already have it on for our own internal tenant, and we're starting to roll it out to users because we'd like to get ahead of it and do it before April for for a number of reasons. So we're actively educating users on MFA. I think at this point, pretty much all of our users have MFA, although we did discover one recently where the CA policy was set to enabled and not enforced. So we had a user turn off MFA. We have since corrected that. But yeah, in every case, we're educating users about what you need to do to do MFA. And then, along your question, I have another question. If you are an MSP, I know that you tend to draw a lot of them, and you're using, you're in the Microsoft 365 world, are you using conditional access for you and your clients? And if you are, are you using it to set things like refresh token length and stuff like that to protect against some of these more sophisticated attacks?

Yeah, yeah, well said. I'm curious to know, as well. So we'll be watching chat to see that. Scott checking in, saying he talks about, MFA fatigue and forcing an app for MFA. It's great. Carrie checking in. Carrie, good to see you as well. MFA absolute must. Matt Collier, we've enabled Duo's verified push and require for all our engineers. Good. Aaron Peters looking at potential options to avoid MFA fatigue like YubiKeys, Trayware. So YubiKeys and the whole like FIDO2 thing that— so Aaron brings up a good comment there. I'm glad to have you in here because I can just ask all the hard questions to you, Jason. Talk about what FIDO2 is. Talk about why it's a better solution, but also give me your opinions on it, like good or bad. And where the adoption curve needs to be at.

I want more things to support it. I own 2 YubiKey, actually 3 YubiKeys at this point. I have a main one, a backup one, and then another one. It kills most of these attacks dead because you physically have to have the key, right? So again, it goes back to something you know, your password or passphrase, and then something you have or are, right? Biometric counts too. But the key, you physically have to have the key. If FIDO is enabled, short of some sort of attack in that ecosystem, there is no way for them to bypass that, right? So especially in cases, we have a good amount of manufacturing clients and they just won't allow phones on their shop floor, right? So we get around the MFA phone thing by just issuing YubiKeys and they use the YubiKey for their second form of authentication. I love it. It's great.

Why are we not seeing the level of adoption we need to with YubiKeys or anything FIDO2?

I don't think there's enough support yet. I mean, even things like— I use— I'm a Mac user, right? So I got a MacBook here. Like, even things like logging, uh, to those devices, it's really kludgy to kind of set it up. Uh, I, I really wish we'd see a greater adoption of it. Even I looked at it yet. Uh, we use it for Duo and we use it for LastPass, which we're migrating away from. Uh, but as far as like Microsoft 365 and stuff like that, it, it doesn't look like— at least last time I looked, and it's probably changed since then because it's been a little bit— it wasn't like a first-party like thing in that ecosystem. You had to do some things to actually enable it. Uh, so I think support by the vendors for it is largely what's kept some of the adoption away.

Okay. Yeah, I think, I think that makes a lot of sense. So let's get to this question here from Greg. Really, really good one. So he says, can't you just— or can you talk about a reason for MFA bypass? I hear a lot of people talk about bypassing MFA for service accounts. Let me hit hide. So funny how it cuts all this stuff off for service accounts. Where was it? Because it was hindering their automation. What's the best way to handle that scenario? Is there a legitimate scenario where MFA bypass, uh, makes sense?

Uh, I kind of— let me look at the question again.

Yeah, I know, it's— I wish, uh, Restream would quote it better. Does this cut off?

Yeah, so I think the way to protect against that, especially with service accounts, because that's an unsolved problem, right? Like, MFA on service accounts is something that is really difficult and really hard to get Right. I think the key there is password rotation, right? Like, if you're not going to enable MFA on that account, then don't let that password live more than a couple days, right? Limit the amount of time that compromise could take place on that system. Like, if the password gets leaked or somehow they get it or get the hash and then crack it, like, the goal is to have the password rotate before they could reasonably crack the hash.

Yeah. Okay. Okay. That, that makes sense. Okay. So this good, good conversation here. And, you know, all of this, if you're just joining us, what we're talking about is leaning on an expert like Jason. How would you actually vet an MSP? And what got us into this discussion was this right here from Beazley. Let me pop this back open. I just wanted to segue a little bit on all of this because MFA is just so important and MFA bypass is going to become more of an issue the more we implement just standard MFA, right? So one of the things they also talked about here, Jason, was this is from Beazley. If you guys don't know Beazley, they are one of the top cyber insurance carriers in the US. They're great. And they produced just a couple of weeks ago some documentation, just sort of like they call it a cyber snapshot of what they're seeing from their incident data. With 45% of incidents now cloud-based, according to some estimates, estimates will also see more compromise of cloud environments in 2023. Organizations can't simply assume their cloud services provider is handling their cloud environment securely. Minimum security requirements should be established when used when vetting MSPs and included in contracts. So we definitely want to zoom into this part in a, in a minute. But first, just this whole paragraph, Jason, comments, thoughts?

Yeah, it's— I am somewhat— it is somewhat interesting to me here that they seem to be commingling, you know, the thoughts of MSPs and cloud service providers together. I thought, I thought that was really interesting for sure. Uh, I don't disagree with what they say on the, the face of it, right? These things do need to be, uh, they do need to be considered and they do need to be contractually protected. But one of the things that I think that we need to talk about, which is the elephant in the room, is the shadow IT role in this, because me as an MSP that is relatively secure, I really limit the ability of the user to go do dumb things— I'm just gonna say— without my knowledge, right? But what it's really hard for me to protect against is them going and buying cloud dumb things, right? So I secure and protect the things I know about, but I don't necessarily secure and protect the things that some guy in engineering went and bought because he couldn't, he couldn't be bothered to wait for me to install the proper solution. Yeah.

Okay. Okay. Yeah. And we talked a little bit too about this whole idea of shared responsibility matrix and client knee-jerks of, well, if I shove it to the cloud, the cloud is responsible for everything. We've done a pretty good job over the past couple of years of like, not so fast, my friend. Understand what their responsibilities are and what yours are. And if you have any doubts, go look at their MSA, right? Because they're going to very well protect themselves in every single way, at least with the default standard MSA. So that, I completely agree. And then when we get here to this comment, you know, this used when vetting MSPs, that really is what kicked me off into, wow, interesting that Beazley would talk about this. Interesting that they would finally get to the stage where they know who MSPs are because they've insured them before, but they're also now going to really grill and jump deeper into how do you vet MSP, but that they give no guidance, no advice on to here's how you should vet an MSP, here are the 5 questions you should ask, here are the things you look at. And that just is frustrating because I agree you should vet an MSP, but it brings up the question of how and what. And so that's really what we wanted to talk about today is that whole question of how do we vet an MSP. And then also another one I wanted to share and just show as well is also comes from Beazley on their snapshot fine-tuning default configurations. And they talked about MSPs in this one as well. Businesses that are working with an MSP must also talk to their provider to make sure they're implementing the right controls and to make that part of vetting and managing their MSP relationship. How many clients are actually doing that, do you think, Jason, across the board? Maybe not your clients, but across the board.

I think it's, it's relatively low, I would guess. Uh, I— this, this whole section of this document, it frustrates me because it clearly is aiming the responsibility in all of that risk at the MSP if they have one. And I think that's unfair, right? Like, in the end, the business owns the risk, whether or not the business is the insured entity. In the case of Beazley here, right, the MSP carries its own, uh, insurance. And, and by, by all means, if the MSP is, is responsible for that technology, they should be helping you guide that, right? But you can't— I don't think you can carte blanche punt that responsibility and that risk straight to the Agree.

Do you think that they're writing it that way and they have that intention because they have been— got caught in the middle of incorrect configurations on the MSP side, and it's the client that has the claim and they end up having to pay out? Do you think that's where that mindset comes from, from them?

Yeah, I mean, some of that comes from MSP marketing, right? So if the MSP— I mean, there are still MSPs in this, in my metro in the Toledo, Ohio area that are preaching 100% protection, right? They are preaching that they have your back and that if you use them, you're not possibly gonna be breached. And that is a really dangerous and really naive mindset from my perspective, right? And so they are reacting to their clients getting breached and then coming to them and saying, well, my MSP said I was secure and had all this stuff taken care of, right? So I understand where where they're coming from. But as somebody that's on the other side of that, that feels like we do a relatively good job— like, it'd always be better, uh, but it feels like we do a relatively good job, uh, it really hurts to see that.

Uh, yeah, exactly right. You get the BS call from Aaron who agrees with you of that BS, uh, for sure. Um, okay, so that, that kicks off then, I think, what we want to talk about a little bit more today is it's— I think it's good that the carriers are setting in stone some amount of, are you vetting your MSP? I think that only will turn into, you know, we talked about this last week. Someone asked, well, will you ever see like an addendum or some kind of expanded thing when you're going through your questionnaire as a client and you say, do I use an MSP? You know, are they going to give me like another follow-up one or two pager that goes to the MSP for them to answer some questions? Right. And I think you may eventually see some of those things because the carriers clearly have woken up to supply chain risk. And, you know, they are tied to the hip, the client to the MSP, in many ways for good and bad. So that kicks off, I think, this whole question of like, how do we vet an MSP? And so I guess we're going to start into this. There's a big difference between a good MSP and a poor one. There's a huge gamut. There's a huge spectrum here between the two. So as a client, how can you give me— just start here at a high level, Jason. How can you tell the difference if you're a client? How do you know the difference between a really good one and a really not so good one?

Uh, I mean, we— I think there's several areas that we can start at, right? And I think that at the very minimum, you should start with, uh, what's their incident response plan, right? Like, uh, I know we have some show notes here, and this is definitely not the order that we wrote them in the show notes, but I'm gonna give what I think the most important one.

You do you, my friends.

Uh, which is, you know, have an incident response response plan, right? It— ask them what it looks like, how they plan to protect you. Have they been through an incident? Have they practiced their incident response plan? Will they share it with you, right? Like, it— there are things I— in mine, actually, I think other than redacting the contact information, I don't see any reason I wouldn't share it if asked, right? Like, obviously I'm not going to share the phone tree and the call list, right? But I'm— but I will definitely share the fact that there is a phone tree call list and a chain of custody and note-taking and documentation procedures and things like that. I think along that, right, like, ask them about what their insurance looks like. Does the MSP carry proper insurance, right? Because if they don't, then if they don't understand their own risk, then they're probably not going to understand your risk either.

Agree. And I still run into MSPs today that I'm like, do you have tech E&O? And they're like, tech E&O what? Like, okay, we gotta take a big step back. And not only do you— Tim, thanks for joining, my friend. Glad, glad you're on with us for a little bit. Um, not only is that important, it's also difficult to get it in this day and age. It's really expensive, and the minimums are going up. One of my favorites that offers good tech E&O for MSPs is TechRug. If you've never talked to TechRug before, do yourself a favor, give those folks a call because they do a serious amount of vetting for MSPs before they're willing to take the risk with you. And so yeah, I think that's a great point and certainly something I would ask as a client is, you know, you ask my roofer, are you like bonded and insured? I should ask my MSP, do you actually have tech E&O and who's it with and what are your coverages? And that I think could tell you a lot, just that question alone.

Do we see a world where there are bonds and there is some sort of system like you see with some of the skilled trades in that to, you know, help share in some of that protection. I don't know.

Well, let me ask you this: should we? Is, is there a good use case for that?

I, I think there might be. I, I don't— I think that in the end, uh, what you're gonna start seeing, and my prediction is that you'll start to see the, uh, computer consulting and IT world— if we can avoid the regulation and start self-regulating before the government steps in, uh, you'll see it go down the road of some of the skilled trades thing, right? And if that happens, then by all means, I could see some sort of like bond pool they buy into essentially to help share some of that risk.

Yep, I agree. So that's something to think about. I wish I knew more about all that. Maybe we can one day get someone on that understands bonds and the technicalities behind them. But I think that makes— it's, it's interesting for sure. Okay, so that's the first one is we're talking about if you're joining us, how I would vet an MSP, right? Like the things I would ask. So the first thing Jason jumped into is incident response. You actually have an IR plan? Have you tested it? Is it documented? If I were to ask for it, would you share it with me? Do you even have something that's out there? Is it just a copy pasta you got from some online quick Google search, you know, because— and not testing it? And I guess before we move on to the next one— and Matt Collier, you're reading my mind, my friend, we're going to go into frameworks for sure in just a minute, so good, good call on that— before we go to that, why is IR so important? Is it purely just because if they don't have it planned and when something hits the fan, it's just going to get a lot worse? Is that the essence of it, Jason, or is there something more?

Yeah, there is more, uh, because an immature MSP with, with regards to incident response can put the client in a really sticky situation with regards to their own insurance because it's really easy to destroy evidence in an attempt to get the client back up, right? So, you know, realizing what the requirements are there and what you can and can't do, like, you know, going in and wiping and reloading all the PCs is probably not the right call. And I think that's the default reaction to a lot of these smaller, uh, smaller MSPs would be to do that, right? So having a, having a document that says you can't do that, you need to preserve evidence in all cases possible, you need to document things, there needs to be some amount of chain of custody, right? Like, you know, uh, that, that becomes important. Okay.

Yep. Um, That makes sense. Okay, so that's number 1 is incident response, left to boom and right to boom. Uh, so we got a couple others. Um, which one you want to— I'm gonna let you pick.

What do you think?

Next one.

I think number 1— so the reason I changed the order here is I think number 1 goes into number 2. I think you need to look at the financials, right? Because there's a lot of, uh, one-man shops that their incident response plan and their thing, if they have an insurance claim against them, is to close. Right? Do they have enough capital to sustain some sort of systemic attack without closing? Uh, it's, you know, annual, annual revenue, right? Do they have enough profit? Are they, are they financially healthy to be able to weather the storm of some sort of cyber attack?

So if I'm a young, new, small MSP, what does that mean?

Uh, it means following best practices, right? Because everyone has to start somewhere, right? But if you, if you start upside down, right, like I— Paul Dippel, I think he says, uh, something like half of MSPs lose money, right? So if you're already strapped and you're already losing money and you're already doing it, uh, right on the edge of shoestring, then, uh, your financials aren't going to support, you know, any sort of added pressure there, right? So asking for some amount of financial statements I, I think it's a good practice. I— we've never been asked for them. Uh, no, that's not true. We have a couple of enterprisey clients that have definitely asked for some amount of guarantees, basically, that we're financially solvent, uh, and we're not gonna just nope out of a mid-contract. Uh, but I, I think that's a good thing. I think if you're a 10-person shop, it's going to be hard-pressed to get an MSP to put in the work to cough that up., but, uh, I think you should definitely ask.

I agree. And there's a lot of regulations that even require asking that in vendor and third-party risk management. Take banking. I always talk about banking, but if you go look in the FFIEC guidelines on vendor risk management, one of the things they say is if they're publicly traded, go pull out, pull out their K-1s and understand exactly where they're at. How solvent are they? What does their cash flow position look like? Because and I'll get to private in a minute, but for public companies, even if it's not so hot and they're burning cash like crazy, they have no method to get out of it. Well, what's the outcome for them? An outcome is absolutely some kind of acquisition. They're going to get bought up. And if they're going to get bought up, are you okay with that? What does that look like in a situation where your MSP all of a sudden got bought up as a client? That's a question you should ask. Now, private companies are difficult because I don't have public data I can go pull. And so I have to ask them. And so then you have to make this decision as a client. Am I going to ask for financials and am I going to require it or am I not? Because a client, a private client, is under no compulsion to share it with you unless they're forced to in the agreement. And there's certainly nothing public out there that you're going to get. So I agree. I think you should ask for it. And I think as a small MSP, if I'm getting started and you're like, well, it's easy for CNWR to say because you guys have been around forever and you're big.

Yeah.

What do I do if I'm small? Well, I think you have— when you start small, I think what we're saying is is it's not that you're saying, Jason, you're not allowed to get into this because you're small. It's that you're saying start small and grow ethically, grow conservatively, grow carefully, and don't bite off more than you can chew. And you'll get to that stage where all of a sudden, before you know it, you have 15, 20, 30 clients. And when you look at your financials, all of a sudden you're $1 million to $2 million a year and you're like, wait, okay, I've done this and I can command larger clients now. You shouldn't just jump in and say, well, my first client's going to be this you know, 800-employee hospital and everything will go fine. We call that concentration risk. And if things go south, they go south quickly. And so I think asking for financials is a way to protect you as the client and the MSP as well, to make sure we're both biting off what we can chew.

Yeah. And I think that, you know, if you are new, right, like, it is not to discourage, your target customer profile needs to grow with you, right? Like what you just said, you can't You can't open a one-man shop and your target customer profile is companies with 1,000+ employees, right? You need to start small. You need to build up that risk and put the good practices in place. And then as you grow, if you want to move into upmarket and target bigger customers, you have the ability to do so.

Bingo. Exactly. So a few more questions you could ask are, what's the— what's your staff size? How many people do you have at your MSP? That's important because if something does hit the fan, how many resources could be emergency devoted to this incident. Other questions you might want to ask: what's the average revenue size of your other clients? I want to know if I'm, you know, if I'm, you know, $15 million in revenue and all your other clients you work with are $1 million or $400 million, there's definitely a difference there in the expectations and your volume and your scale that, that's there. You know, how many other clients in my industry do you work with? That's always an important question. There's nothing wrong with an MSP picking a one-off industry that they want to grow into. That's great. Great. But if you're just a one-off weird one, uh, they may not know how to handle you correctly. They may not understand your regulations. They may not understand your complexity. So yeah, when we talk about financial vetting, these are a lot of the questions I think we want to ask. Um, any other thing? Anything else stand out to you there on that one?

Uh, I, I don't know how to word this correctly, but to a certain extent, financially, you get what you pay for, right? So in, in many cases, if you're shopping only on price, the lowest price guy probably— if, if all— either he's getting everything cheaper than all of his competitors, which if he's small is almost certainly not the case, or he's not being financially responsible, right? Because essentially all of our costs are roughly the same, and I would actually argue that my costs are probably higher than a lot of my clients locally because I do a lot more security stuff, right? So Uh, if, if they're coming in, you know, $35, $40, $50, $60, under $100, under $150 in many cases, dollars a seat, right? Like, the— are they financially stable?

Yeah, yeah, well said. This, uh, just steal a page out of Gary Pica's book, you know, the whole like, well, I can tell you they can't afford to secure you at that price, right? Yeah, for sure. And I think when you're a new young scrappy startup MSP, you want to be like, well, I can do this for $50, $75, $100 a seat and, you know, I'll undercut the market. But as a client, you should understand, well, you know, cheap and good do not usually go together, especially not in the MSP space. And so certainly a question to ask, right? And so at a high level, Jason, I'll let you give me a big range. But if I'm a client and I'm trying to understand like per seat state pricing, what should be reasonable for an MSP that's doing things right to charge? You don't have to give me exactly your price, but give me a range.

Yeah, I mean, it really depends on how much labor is included, but I think much less than, uh, and it depends on the number because there are some economies of scale that come across, but I think anything below about $150 a seat these days, uh, at I'm going to say a 20-person company that you're servicing, not like employees of the MSP, but a 20-employee SMB that you're servicing, anything less than 150 is probably hard from a security standpoint.

Yep, yep, exactly right. Um, some comments coming in I thought were really good. Um, we'll, we'll sync back up into, um, I like Greg said here, you can grow too quickly and that growth can sneak up on you. Everyone's aware of the need to make money, but the nuance is making sure you have the bandwidth to handle the work. And so yeah, we call that concentration risk, this idea of like, well, I grew too fast, I had these juicy customer, I I couldn't say no to. I mean, he's going to change the game of my MSP. And then here's what happens in the aftermath of that. Let's say you have to go and hire 15 people to— that's too many. Let's say you have 5 or 6 people to support that client. Things don't go well because you're not scaled to handle it. You misinterpreted the resources required. They're now unhappy with you and they pull out 8 months later. And now you're left with all this bloat that you can't scale and sustain. Like, this is called concentration risk. We have too much concentrated into one client or two clients and things can go south quick.

Yeah, I mean, we're 17 employees as of today right now and we are presently onboarding two. Neither of them are super technical roles. I hired a web developer and a bookkeeper. Even onboarding two employees at once at my size is hard, right? I can't imagine onboarding three or four resources at a time at this size. It's— you're just not going to do a good job. Yeah.

Yep. Yep. Well said.

Yeah.

Well said. So that's, that's right, Greg. And growing in quality is just as important as quantity. And Scott checking in. Agree. Pick 2: cheap, fast, and good.

Yep.

I remember someone telling me that years ago, and it's one of those things that's always held true. Okay, so that is number 2, is financials. We've talked about the first one. Question you'd use to vet your MSP would be instant response. 2 would be financials. Let's pick number 3. What is that?

Yeah, so I have a single question I can ask most MSPs that will determine Uh, it— what I think of them, and that is, if you are in the Microsoft 365 environment, which I think most MSPs are— there are some weird Google stragglers— but if you're in there, what license level are you recommending in selling? Because it is— if it is not Business Premium, then I have some very serious questions to ask you.

Okay, let's say I'm an MSP today and I say it's not Business Premium. Roleplay with me.

Yeah, so the, I mean, real thing there is that I don't actually really care about Business Premium. What I care about is Azure AD P1, right, Plan 1. And Azure AD P1 comes with conditional access policies, right? It comes with the ability to centrally manage MFA, and it doesn't come with the cool P2 like our internal users. We actually run Azure AD P2 so we can get get the risk stuff, the risk-based conditional access. But the— for end clients, like, you really need those conditional access policies to effectively secure Office 365. So if you're not going to run Business Premium, if you're going to run Business Standard or Business Basic, you really need to layer ADP 1 on top of it. And there are some other licenses, like like, uh, we sell F3 quite a bit, which is like the Frontline Worker 3 plan, and that comes with some of those same policies. But the big thing there is really whatever you're selling has to have Azure AD P1 in it.

Okay, got it. That, that makes sense. Um, anything else inside of cloud management that is important to you if you are vetting an MSP and how they handle— I mean, even the Beazley report mentioned a lot of like cloud configuration and are you doing best practice? What else would you point to?

I mean, the "Do you require MFA?" I mean, at this point everyone should be answering yes.

It's—

the pushback is going away when I tell clients that, so maybe it's finally getting through to people. And then, "What other cloud applications can you manage?" Can you manage, like, I use Dropbox. Can you manage Dropbox, right? Do you manage I don't know, Salesforce, right? Like, do you— like, what does that look like? Like, what other of my line of business software are you willing to— maybe you don't manage it day to day. Like, I'm not in Salesforce writing Apex, but from a security standpoint, can I essentially manage the security aspects of that?

And I might even go so far as to even add in, what does your MSA look like? And have you delineated the lines of responsibility between me as client and you as MSP in terms of what we're doing? I think a lot of I don't think— I know a lot of MSPs have very archaic MSAs that have not been brought up to speed here with the shared responsibility world. And there's probably a lot of, a lot of risk for that MSP. You talk to really great lawyers that are out there that do— that are very experienced in cyber law and MSAs, and there's a lot of egregious stuff that they see in there. Anything you could talk to about that in particular?

I mean, we have contracted away a lot of our risk there. However, you know, even just this week, I noted in our leadership team chat that I really want to put a shared responsibility matrix in our MSA. I think that's a great— or at least in the statement of work for each of the services that we offer. We're not doing it now, but it's definitely after listening— was it you guys earlier this week? Was it yesterday? It could have been yesterday. It's really something that I think is important so that the end user fully understands this is what I support and this is what I don't support. This is what I secure. This is what I don't secure, right? So that the business needs to understand what their risk is and, you know, where that line is drawn.

Yeah, really great point because play it out. Let's say you're now in deposition and there's a lawsuit against you in the aftermath of the breach and the client says something like, well, you never told me this nerd stuff is my— there was responsibility there for me. Imagine you going, having to say back, yeah, no, we told you that, we documented, here's an email or whatever, versus no, this was so important that we put it in the MSA and you signed that MSA because we knew it was complex. Therefore, we put it in the MSA to be properly disclosed, to be properly communicated. And there it is. I don't think a lot of us are probably doing that in the right way.

Yeah, it's really complicated, right? I'm going to give an example here, right? Like, so say you service dentists— I'm just gonna pick on dentists— and they're HIPAA encumbered and they use a cloud-based EHR, EMR, whatever you want to call it, right? Like, you do all the proper things to ensure that they're HIPAA compliant on-prem, right? But if that cloud provider doesn't have proper controls around things like IP access restrictions and that, like, you get doctors going home on their home PCs and they're downloading all sorts of stuff, right? So you have these these, you know, HIPAA issues that aren't even on devices that you necessarily manage. And, you know, as an MSP, I can't, I can't own that risk. I can help you secure it so that they can't do that, but I can only own the risk for things that I directly manage for you.

Yeah, okay, well said. Um, I want to get to a comment Scott made. It wasn't fully following, so Scott, maybe you can add, elaborate a little bit more. Sometimes because of the chat delay, sometimes a question comes in and we missed the stream of what we were talking about. So that wasn't it. Where was it, Scott? You had it somewhere for us. Um, there was a question, I swear there was. Uh, oh, I was way too far up. Let me go way back down. Here it is. Um, how would you— how would that be answered if it were a solution like SaaS Alert? So I'm not sure what the that is. Do you know what he was talking about? If not, um, Scott, maybe you could clarify a little.

I'm guessing he's talking about, uh, shadow IT and, uh, tools like SAS Alerts that, well, I know, I think he's talking about tools that help you find, you know, shadow IT things. Scott, can you clarify?

Yeah, maybe you can clear it. We'll be on standby for that question, Scott. And then Carrie checking in again. We don't let our customers use Dropbox. They must use our cloud file services. Standard is built in and provides ransomware protection. They need advanced file stuff, they can go to Ignite. And then that brings up even what Ray said is like, how do we enforce that? And no, because, you know, they're always just a quick sign-in to something else that we didn't know, right?. So just to finish deploying SaaS Leo to see that, or Augment. I think Carrie said they're using Augment for the same thing. This, this world of like sprawl and cloud is becoming a pervasive issue, isn't it?

Yeah, it is. And, and you know, there are some various tools, uh, like SaaS Leo and a couple others that will help you find some of those shadow IT pieces. But you know, Carrie says there, like in the most recent, if you want to highlight it, uh, doctor's home PC must use our security stack or they cannot connect to protected data. That assumes that you can properly secure that EMR and EHR with things like IP restrictions, which I can tell you firsthand isn't always the case. So what do you do when they choose a line of business app that is specific and very tailored to their business, that makes their business run, you know, very, very streamlined, but it doesn't support some of those things, right? Like, it's, it's— you have to you have to know about it and protect against it in some way. And with regards to finding, securing cloud access, things like SAS Alerts are great for tools that they support, right? But there's still a significant number of cloud tools that don't let me properly get data out into something like SAS Alerts or into, you know, a SIEM. You know, we run a SIEM and we pull the logs from everything we can into it, but there are still tools, including tools in our RMM stack or our MSP stack, that don't properly support me pulling some of those activity logs and stuff into any arbitrary solution like SAS Alerts or a SIEM. They only work either in that vendor's walled ecosystem or not at all.

Yeah, and that's a real problem. Yeah, so, and Scott, you're right, there is a big delay. That's just the way all these platforms work. It's like a 10-second delay, which is frustrating. I know. Yeah, that's, that's exactly right. And it is a problem for sure because we have tools that can ingest and accept things. And if the vendor just doesn't support it, is archaic, you're up the creek without— you know, if I could pick on RMMs for a minute, they're notoriously bad at this, of not giving us the insights that we need.

And that's a pretty scary thing given the power of those toolsets, remote access tools, RMMs. Uh, even some AV. Like, they're— yeah, I pretty much want to essentially collect the logs of anything running agents on client systems if I, if I can. Yeah.

Um, so Matt Collier checking in with some bravery here. Uh, commend you for this. We fired a few clients for refusing to either update or upgrade their LOB to something we can properly secure. I mean, that's a big deal. That's a pretty, um, that's a pretty bold thing to do, but sometimes you have to do these, these hard things to protect yourself. Comments and thoughts on that, Jason?

We are going through that now with, uh, we've cut a couple recently. Uh, we've, we've had a couple that we basically put, uh, we've had a couple go away, churn away, because we put costs to work around that or, you know, secure them in, in other ways in front of them and they won't accept it. Uh, and, you know, when we service, we, we target like 20 $20,000 to $150,000, you know, end user. So at the smaller $20,000 end, uh, sometimes they just can't— literally can't afford it. And we've had to let a couple go because they can't afford— like, their own margins are so small they can't afford to properly secure things. And at that point, we just walk away.

Yep, it's hard to do, uh, very hard to do. But, you know, you're leaving money on the table, right? That, that's, that's the difficult piece. And sometimes Yeah, that's always a topic. What are the thin red lines for which you will fire a client? Right. And it's, it's difficult. Okay. So I think that leaves us back to the fourth thing. So just as a quick recap, incident response, Jason would vet an MSP through their IR processes. We'd vet them through their financials. We'd vet them through their cloud management. Number 4, what is that?

How do you secure your own house? Right? Like, as I think— is it Andrew Morgan? Somebody on the Cyber Call says, you know, you are customer zero, right? Like, we are a force multiplier as far as bad guys and threat actors go. So if we don't secure our own houses, then at some point or another, bad things will happen. What kind of framework do you align to internally? There is no right answer. We use CIS internally. NIST is fine. If you want a CSF or 800-171, there are a stack of options here, but you should have something. Do you use least privilege access for your technicians, right? Like, do all technicians need access to all data? When you're small, maybe, right? Like, maybe you don't have enough technicians to, you know, have proper real help desk people. We're just now getting to the point where we're starting to pull access away from our help desk, help desk staff, because we have enough above help desk staff now to keep up with that. Uh, again, there aren't necessarily right answers for that either, right? You just have to understand what that looks like. Uh, and then, you know, what steps is the MSP taking to make sure that— to use Bryson's favorite term, there's not going to be a buffalo jump, right? Like, or, uh, we're not going to be a jumping off point where we get all of our clients, uh, right? And that goes back to number 1, right? Instant response. Do I have a plan in place if that does indeed happen?

Yep, well said. I, I was at, at the insurance boot camp we did last week, uh, I asked everyone there, I'm like, how many of you guys are following Framework? And a few folks raised their hands. And then I asked, how many of you guys are aware, could name me some frameworks? And, you know, a few gave some examples of like NIST and all that. And a few said CIS. And then I asked the question, how many of you guys are aware of CIS? And maybe half the room went up, their hands raised. I was surprised by that, Jason, is that of course the makeup of a lot of the folks, and they're not all like technical people that joined us in the bootcamp, but I think we still have a ways to go. And sometimes I get lost in my confirmation bias. Because I spend all this time on Cyber Call. We do the Cybercast. Phyllis Lee is a personal friend of mine. Like, so I'm just like, I'm very well aware of CIS, but in my confirmation bias, I get shocked sometimes in those, like, how many of you know CIS? And only half the room goes up. So I think we still have a long way to go in framework alignment, even though we've been talking about this for years.

Yeah, I think CIS has done a relatively poor job of engaging and marketing to MSPs. Like their whole pricing model isn't super well suited, uh, for MSPs. I mean, it costs us a small truckload of money. Like, I'm spending more on CIS every year than I think I am on PSA at this point, right? Uh, so I think that plays into it some. But that confirmation bias is real. I mean, I, I brought up in the last, uh, section the conditional access and the AAD P2 or P1 at ChannelCon, we pulled the room with about 25 MSPs in it, like how many of them use 365 Business Premium? And of the 25, I think like 6 of them raised their hand. And then how many of them have a conditional access policy that either protects against token theft or lowers the default refresh token time? Down away from the default in 90 days, and 3 out of 25 raised their hands, right? So it's easy for us to look at what I would consider a relatively mature audience that is the people attending CyberCall. It is the people going to write a boom next week, right? And assume that the entire population is that way. But those are the outliers for sure. Yep.

Okay. So we have a long way to go in that. And you said it, Henry, awareness education piece is huge. So, you know, and this is where the more those of us continue to carry that torch, the more those of us that continue to talk about it, to make it an industry standard. I want to see everyone else pick this back up and say, okay, I guess I got to do this to compete. If you're at least— if that's the reason you're picking up a security framework is because Jason ate your lunch and took your client because they care about frameworks and can explain why they care and you don't. That's a win in my book. Book that forces you to go do the same thing, right? Because it's amazing how this continues to just, um, it could— it should be more viral than it is, this, this need to follow a framework.

I mean, it's hard. The other side of that is, is that like, you know, we go into and we follow best practices and we lose sales because of it. And it's really hard to, uh, show the end user that value and really have to drive it in. Because as immature as the MSP industry is, in some cases the SMB industry is infinitely more immature, right? And I come across this, uh, spoke— I spoke at Gherkon about MSPs, and I, I'm gonna title— I think I'm gonna submit a talk for this year entitled like, I get you all hate MSPs, or I understand you all hate MSPs, the world without them is worse, I promise, right? And just talk about how even a bad MSP is probably better than trying to get the, the guy in finance that kind of understands how a computer work, like having that guy take care of it. I—

that sounds like an intriguing talk. I'd like to attend that. Where is Gherkon? Gherkon is not in Toledo, right?

It's in Grand Rapids, Michigan.

Grand Rapids. Okay, so it's nearby. It's nearby. Okay, so these are 4 good things, and I don't know— I don't know what we do with them, but they're, you know, I think it'd be great if someone ever wanted to create a whole, like, this is how I would vet an MSP. This is the It's not the Consumer Reports for MSPs, but I think clients need better guidance in how to assess. And I don't think we talk about this enough. What are the things you should be asking if you're going to be looking at a new MSP, if you're looking at one for the first time? Because it can be confusing stuff. So this has been really good. I think we will have to call this— someone said, I can't remember who it was. Let me go pull that back up. We need to call this the JSON queries. This is like episode 1. It was Greg. Where I just interview you. I'm down with that, my friend. Always appreciate you sharing your wisdom and thoughts here. As we kind of close up, any other comments, thoughts you want to say about vetting MSPs?

No, but I will say I'm excited to hang out with you next week. I want to have fun more next week.

Yes, I'm looking forward to that as well. The thing I'm not looking forward to— let me see if I can find this really quick. So I got challenged by Keith Barthold on— let me show this to you. I don't know if I I've shown this to you or not. Uh, he— how do I bring this up? Let me get this into a new tab. He is doing— so do you— did you know he's a pilot?

Oh, I did. I, I saw this and I said— I commented on it that I am in. If I ever get the opportunity to do that, I am totally down to do it.

Oh yeah, it's gonna be, uh, barftastic. I have not known to hold my, uh, my, my anything when it comes to this. So yeah, this is gonna be a lot of fun. He, uh, has a whole video that he created, and I don't know if he's like literally is like pulling like, uh, what do you call them, barrel rolls and stuff while he's challenging me to the flight. Um, so yeah, here it is, and let me find this little section here. The guy's just nuts. So we're gonna do this the day before I get in. Yeah, this— would you do this, Jason?

Oh yeah, for sure. I mean, you and I did the helicopter thing. Are we— hold on, I haven't looked to see if there's—

we didn't flip upside down in the helicopter.

That is true. I don't think helicopters support upside-down travel.

Oh, I'm not looking forward to this. He did say there's an eject button, so I don't know if it's real, but I will probably be taking advantage of it.

Uh, I think that the eject will be way worse than anything he puts you through.

Oh, it's gonna be a lot of fun. Yes. So if you guys are going to be at Ride of Boom, I can't wait to see you guys. Same with Mr. Slagle. It's going to be great to connect and sync up. Um, but Jason, thank you for jumping on with me today. Thank you for sharing wisdom. Really appreciate all you do for the community. You make the world a better place.

No worries, man. Until next time.

All right, my friend, thank you so much. Thanks, guys. Take care, everyone.

Yep, bye.