How to Build Secure Data Pipelines with LogDNA and McAfee

Analytics and Data Science at McAfee join us for this session. They will be discussing how to build reliable data pipelines. There will be time for Q and A, so please make sure to drop your questions throughout the session into the chat below and unmute yourself and turn on your video to ask Tucker and STASIA live during the Q and A portion. Tucker, stasia, take it away.

Thanks Brian. So just as Brian mentioned, my name is Tucker Calloway. I'm the CEO of LogDNA. I've been with the company for about 18 months now and prior to that I've spent the last, I'd say probably about eight or nine years in DevOps. So just quite just a little bit before DevOps was really a big thing. So then focusing on DevOps has been a big part of my career and I'll let Stacia introduce herself.

Hi Mystacia Zamishlaeva, I'm VP Data Engineering, analytics and Data Science at McAfee. This is very interesting role where the goal of my team is to empower employees to excel at work every day through the data. Also we are focusing on the consumers where we provide the better protection better service for our customers by using MLA. Prior to McAfee I had huge experience in building data management platforms. In 2011 together with Manish Sooda, I co founded a company called Reltio and helped to grow to an award winning SaaS company. And there we were driving digital transformation for enterprises by providing single source of truth for the data. I'm very excited to be here and talk about my experience with data.

Great. So just a quick overview of the agenda for today. We have five main topics we want to give a quick history of data, talk about some of the trends we're seeing today and how to build reliable data products and then we'll focus on the leader's role and how trust plays in that as well. So thanks for the introduction. Stasia certainly have a long history there. Can you maybe describe for the group? Have you seen the volume and complexity of data that companies deal with change over the last five years?

Yeah Tucker, the volume of data is growing exponentially but I'd like to deep dive and discuss the reasons behind it. First, with democratization of costs for the storage and for compute through the cloud, now we can pay much less for more and organizations started to instrument collection of more and more data sets so that potentially will have chance to be used in future. The second driver for that increase is that look at the systems they're having increased requirements for availability for performance Velocity releases. This requires a new level of monitoring and logging that allows to identify by different signals that hey, there is going something wrong with a system and trying to predict some of the failures and address them in the shortest time frame. But I really like the last reason. This is the value. I see that a lot of enterprises start getting the first major wins under the belt and this is really exciting. The biggest contribution to this I believe is decreased expenses. So when we are making our employees being more productive by reducing by different tools and data, we are also reducing the cost of acquisition or retention of customers cost of goods. But even further through the data, companies often identify new avenues for innovation or for disruption. So we are discovering new segments of customers that we were not penetrating well enough before because we didn't have good understanding about them. Especially this is true for consumers or even we identifying unmet need where we can launch new products. And as we are launching new products or existing products, there is very fast launch with customer obsession when you're defining the metrics where we want to trend and you are using this build measure, learn about customers to make sure that we are building what customer is actually looking for and that is that is addressing their needs. So to summarize the top sources for contribution of the growth, this is applications with telemetry, cloud computing, especially after all the story with microservices and a development and adoption of more technologies around AI and IoT.

Thanks Dasia. I mean we obviously see that in our business at LogDNA and hear that from our customers. I think one of the key themes that we hear often is that so much has changed in the last five years. Even that the systems that were put in place five, 10, 15, 20 years ago, they can't scale to both the volume as well as the application of value that you mentioned in data today. So it's an opportunity for a lot of exciting opportunities. So with those opportunities, what trends do you see? How do you see things evolving with data today?

I'd like to represent the trends of the data through the lens of 5vs. There are different variations of Vs for the big data. But I really like this approach when we talk about volume, velocity, variety, veracity and the value. Volume is growing for sure. But what is becoming even more critical is the velocity. Real time is a new norm for big data. We are using speed now as competitive advantage to make fast decisions. We are getting more different data sources and more variety of the structures that we have in our data lakes. And think about all the organizations that now are adopting graph structures to Derive insights. This is really impressive and adds more to the variety. Veracity equals to quality. Garbage in, garbage out. You may be wasting your money by not getting the relevant insights because of the poor data quality. And the top of this pyramid is the value. This is the biggest fee for me. If you're not getting the value from the data, then you just can scratch all the previous ones. You're just wasting money. You really need to focus on how can all of these bring the business outcomes. And as we are going to that kind of world, this requires changes not only in the technologies, in the tools, but this is new processes, new ways, how do you build organizations, new types of roles for the team structures. I feel that with my past experience in applications, I feel that data is going through the same path as seven years ago. We were going with microservices. We are breaking the monolithic data lakes into the data domains and we started to treat those business data domains as products. We have the owners of the data who understand not just some piece, they understand this whole domain, how it is being used in and out. Those data domains are focusing on the customers, how we can drive the adoption of those data domains and how we can bring value to the customers they'll be using those. This approach is changing how organizations are switching from what my gut feeling is to what data tells us. This requires very different velocity of the changes even in the data world, not only to the pipelines, but also to the outcomes and what we are doing in applications. In this case with microservices, we were adding a lot of automation testing in the release, management through CI cd, testing in production, be able to roll back if something goes wrong. The whole new area of data observability tools appear in the data world. That again is very similar to what was happening in microservices. I went through the path in applications and I'm really excited about the opportunities that you have in big data by focusing on the value and the business domains.

Yeah, I think it's exciting. I really love how you take a construct or a premise that's been that we've been through before and a transformation we've been through before and apply that to data. I think that informs us a lot on the opportunities with data, but also some of the challenges that we'll all face organizationally, technologically to go to do this breakup of these kind of monolithic data structures into the more domain specific. So it's an exciting opportunity. It'll also be an interesting journey for all of us to make or focus on this domain. So as you think about that, what tips do you have for us? What are the key considerations we should all be thinking about when it comes to building reliable data products? Because as we're focusing more on trusting those, and I'll talk a little bit more later about my emotional decisions versus my data decisions, but as we trust it more and as we build more reliable products, what are the things we need to be considering.

When we talk about reliability? Data products and company can fail on the business and on technology side in order to remediate the business side. And this is something that I would suggest to address first is we can use some kind of strong structure and thought process. And by the way, I can't express how many data initiatives are failing because of not enough to the business and bring the value. So this is why when you think about the data initiatives, you start with company top initiatives. What are the KPIs that we are trying to address? What are the driving initiatives and how data can help after that? You are building data roadmap with focus on users, on end customers, how you can help them. You define requirements around your customers and you're defining what success will look like that you constantly keep measuring. What are the KPIs? Are we moving with our data products in the right direction? What is the number of active users, business impact, revenue, cost reduction, customer satisfaction and all other measures that are relevant for you. You're assigning owners and making sure that you're using data management platform to accelerate this whole journ and the governance model to make sure that all data products can play together and not be silos. And when we talk about reliability for the technology, I would use the same approach as we have in applications with SRE model focusing on the reducing time for mean time to detect mean time to resolve meantime between failures by having the automation for testing for release management by easy rollback. So again I would like to highlight that for reliable pipelines you need to focus both on business and technology. And Tucker, you being CEO and being a leader of the organization, seeing so many implementations of the data, what do you think is the role of the leaders building this reliable data products?

Yeah, well, I think you started the conversation and saying that it's key that we buy into it and we believe in it. As I referred to earlier, I have a tendency to make more emotional decisions than I do data driven decisions. I've kind of learned over time that I need to balance those two. In doing so, what I find is if my emotional or my gut opinion as stasia referred to earlier matches with what the data is telling me, then I have a really strong feeling that we're making good decisions. But then what you have to do, especially a person kind of has my orientation that way is when the data is not telling you what your gut's telling you, you have to have the courage to not make the decision even though your gut's telling you something or your emotions are telling you something. So relying on the data is very important. But that of course that really comes back to common definition making sure we're looking at common sources of data that we all believe and trust in the data we have. And so trust more than anything I think is what it comes back to, what it comes down to. Like if you're going to really make decisions, if you're going to really standardize and also not overcomplicate, you just want to have a very simple set of data measures to drive your decision making process. But if you're going to do that, you have to have a super high degree of trust.

Yeah, and trust is such an essential thing for the data. So when you really need to focus on the trust of the data starting with having the common vocabulary and common single source of truth that is aligned across different departments. So the meetings about decisions on the company level will be not about hey my, my data, my numbers are better than yours numbers. It will be about the making decisions based on the data. Also for the trust we need to consider and put as essential part security and make sure that all the sensitive data is protected from the start. It's impossible to gain trust and become data driven organizations if you don't show the value. Value is critical part for having the trust and constantly deliver certain milestones and celebrate wins on the company level to gain that trust for the data. Through this you'll be having more and more of adoption of the data. People would like to use it more frequent basis. So this is when reliability comes and become a bigger part. Making sure that you're defining service level agree for your products, for your data products that are driven by what is the criticality and what's the business value. Making sure that you have your data availability, freshness, level of quality and defined and you're keeping, measuring and keeping make sure that you're achieving those goals.

Great, thanks. Yeah. So if I look at our Susan's top takeaways here, it's, it's on the topic of creating reliable, I'm sorry reliable data pipelines. It's really that you have to think of it as a Product. I think that resonates with probably most people on the call. Most of us are focused on products in general. And so applying that thinking to data, and not thinking data as an output of a product, but actual product on its own is super important. Using data to drive business outcomes and having the courage to not just go with emotion, but to go with what the data tells you is extremely important. And finally, focus on trust. And one of the things I love about everything Stacia said today is really taking, as I mentioned earlier, really taking just general concepts like what she laid out for us in terms of creating trust is, I think, a way you would apply the creation of trust in almost any situation in your life. But applying that specifically to data is super helpful and such a foundational thing to creating reliable data pipelines. Thank you. I think we'll take questions now. Brian, is that right?

Yep. We can go right into questions. If you have questions, by all means, please go ahead and put them into the chat. And Tucker, I am starting to see some come in.

Okay, great. So the first question is, how do you address or prevent confirmation bias in data analysis? So, Stasi, do you want to take a pass at that and I can take one, or do you.

So typically, you start with defining the what's the common vocabulary across different stakeholders and make sure that you're using this common vocabulary across. Across the organization, and then. Making sure that this vocabulary is being used across organization. Sorry, a little bit. I'm looking right now.

Yeah.

Okay.

I could go, too. So mine's actually related to Ricky's. I mean, I have a lot of personal examples where we have different data sets telling us different things. And so what happens? Oftentimes there I am. I think I came back. Sorry, I got hung up. What we often have is, like, we might have competing data sets, and one tells you one thing, another tells you another thing. And so, like, how do you avoid the confirmation bias of a when one tells you one thing, tells you the other, or you have this temptation to go out and find another thing that will tell you what you want to hear, too. And so I think what stage is saying is if you go back, go back to the vocabulary, you go back to the trust, you go back to standardizing on what that KPI is. But you have to commit to it and believe in it, and you have to adjust it. Of course, if there's a defect in a product, you fix it. Same with the data. But I think you have to really standardize and commit to it.

Yeah. In our case, what we are doing. There could be different data data groups making sure that you are aligning, making sure that you're sitting together and there could be different definitions and you're coming together and getting to what it is. What is the single source of truth for the data, what is the equality level when what the definitions that we are using. At some point you'll just need to cut that knot.

Yes, we have another question that came in too that said or a statement. To have success in a data driven project, the leader will have to believe in the data and understand it. What do you think? I would clearly agree with that. And I think again, going back to treating it like a product and testing it and deploying changes and things like that, I think that's where that thinking really comes in. You have to have that discipline. And so many times we create a report and then we just leave the report as a report or something like that. We create some data set and we just look at that data set. But the ability to test it, to watch it change over time, to treat it like something that's important to be observed and apply observability principles to it is super important.

What I find usually is that data culture comes from top and so you show that picture where we are all heading through the data and then it need to be delivered when all of this data culture is being accepted and being promoted from the bottom to up.

Great. So great.

I think we are at time. But that isn't to say that questions can't be answered. They can be answered. And if you can all head over the logDNA will have a room together, the lounge together in their booth so that you can ask those questions there. But thank you Tucker. Thank you Stasia for that great and insightful session. We're going to have a lot more great content still to come today, so stick around and see you all in our next.