Introduction to Microsoft Fabric

Welcome to the webinar. This is Richie. And one of the biggest problems for data analysts and data scientists is finding the right data set. I mean, you often waste more time hunting for the data than you do analyzing it because you aren't sure which tool it's been stored in. And one solution to this is the data fabric, where you have a single platform to access your data regardless of where it's stored. And Microsoft's implementation of this is called Microsoft Fabric. So it's a unified set of tools for accessing and working with data. And that's what we're learning about today. So you're going to find out what the use cases are and how to get started. And our first guest is Angie Ruddock. So Angie is a senior content developer for data and AI at Microsoft. Sorry. Bring you a full size there. And Angie creates Microsoft's courses, interactive labs and other training content content for data analytics. She got 10 years of experience in data and analytics and she's an expert in SQL, Power bi, and of course, Fabric. Previously, she was a senior customer engineer at Microsoft and she's also been a SQL Server consultant at Brent Ozar Unlimited. And second up, we have Shannon Lindsay. Let's see, we've got, we've got everyone the right size. Just about Shannon, my. Can I see it? There we go. Shannon is a senior program manager for Power BI and also at Microsoft. And Shannon enables Microsoft's customers do more with Power BI and Fabric. And previously she was an analytics instructor at Data Society, a business intelligence leader Blue Raster, and a senior informatics officer at Elizabeth Glaser Pediatrics AIDS Foundation. And Shannon also writes about data at data data witches.com so great to have you both here. I'm very interested to find out more about Fabric. So, Angie, I believe you're taking it away.

I am taking it away. Thank you so much, Richie. We're really excited to be here with Data Camp. I am, I guess a Data Camp veteran because I've been here before. A little bit about myself. I am based in Orlando, Florida, just down the street from Disney World. As Richie said, I have been a senior content developer at Microsoft for two and a half years, working specifically on our Fabric Analytics Engineer and Power BI data Analyst portfolio. I will go ahead and hand off to Shannon.

Hang on, Shannon. We're just having a little connectivity issues there, so it's difficult to hear you. Okay. You know, this all work? It's all working five minutes ago. Should we try with camera off and see if that works?

Yep.

And I'll Go ahead and just hand it over to Angie to take it away with the presentation.

Awesome. Well, Shannon, as much as I love seeing your lovely face, I think we might have to do off camera a little bit. I think that's okay because we're going to get into the presentation. Okay, so our agenda topics are talking about what analytics platform really is focusing around the lake centric and open architecture Office integration is, you know, really valuable about fabric here and then security and governance. It wouldn't be an important talk if it wasn't talking about security and governance as well. So what's the big thing about fabric? The point here is that fabric is integrating power BI data factory and the next generation generation of SYNAPSE into one single platform. It's also a software as a service platform. If I say SaaS, please remember that's what I mean. You know, it's providing you all of your data in one place, accessible to Richie's comment for data scientists, data analysts, and really giving everybody the control to access the data as they so need in today's current world. Before fabric implementing an analytics solution, especially one that can scale, some people may only need, you know, a super simple solution. But for really big companies, enterprises focused on analytics, you end up with something looking like this or this, or this. Nobody likes working on that. As a past bba, trying to implement solutions like this is incredibly difficult, complex, hard to do, and it takes away resources from doing the actual analytics. Every data officer out there apparently does not want to be an integration officer. So how do we really focus on the data Again? You have to bring the data in, of course, you have to transform it and load it. But if that's all you're spending your time doing, you miss out on the opportunity to do the analytics or development for AI now training models. So this idea here is that generally when you create an analytics solution, it has these basic pillars here. You need data integration to bring your data in. You're processing and storing the data somewhere. You are leveraging streaming data for real time intelligence. You're developing predictive models using this data for data science. Then you are of course creating power BI reports for your business intelligence. This all sits on top of a data lake or wherever your storage is. And of course governance and administration is required across all of these different workloads. These solutions all have existing opportunities. They all already exist at Microsoft using Azure. But to configure all of these services, you're going to have to have expertise in so many products. The experience is different depending on what your Role in the overall architecture solution will be there is a variance between platform as a service and software as a service. So while you may have an easier time for some, you'll have to work harder for the others. It's going to be different for each business model and you have to scale up on everything here, which is great to make you an expert. Can be really hard barrier to entry to actually get you to implement an analytics solution that'll work for you. So of course, here comes Microsoft Fabric. It gives you all of these same workloads in one place in an easy to use friendly UI platform. And you can alternate between each of the different workloads as you need. You keep it all in one place. It really provides everything you need to get started, whether you have a tiny bit of data or petabytes of data. Again, here, just this idea is how everybody signs in one place, everybody navigates the same way. Everybody can use workspaces for their individual business needs and can collaborate. It's all stored in data lake, the same storage format, security, continuous integration and deployment. There's a monitoring hub, a data hub, and then there's built in opportunities for governance and compliance. With all that said, I'm sure everybody wants to see some of this, so we're going to hand off to Shannon.

If we could do a pop quiz, if anyone could recite all of the things that you just mentioned.

Yes, I can.

Perfect. Okay, well, let's jump into the demo. Angie, if you could just let me know if you can see my screen.

Yeah, your screen. Your screen is visible.

Okay, fantastic. All right, so this. What are we looking at here? We are looking at Microsoft fabric and as it reads right on the screen, you can see all your data in one location. The backbone behind this technology is something called OneLake and we're going to take a look at that. But before we just go through a bunch of different slides, Angie and I wanted to really show you what this thing looks like and how exactly you can use it. It's really user friendly. As Angie mentioned, there's a low barrier to entry. There's a lot of point and click, which in the Power BI world we call clicky, clacky, draggy, droppy. A lot of point and click experiences. For those of you who are familiar with Power BI as a product, this should look really familiar to you. We are on the landing page for Microsoft Fabric and you can see here all of the different experiences that you are able to use inside of Fabric. This includes things like Power Bi, which is where your users are going to consume the data that you're working with. Then we have all of these other workloads where you can transform your data, you can orchestrate things, you can act on real time data. We have data warehousing and data science. Let's jump into some of these and just take a look at exactly what's under the hood here. The first thing I'm going to do is I'm just going to navigate to the Power BI experience. Actually, let me back up. Let's start with what I talked about is the backbone behind this product, which is our OneLake Data Hub. The really, really cool thing about this OneLake Data Hub is that you can see all of your data that you have stored in fabric. And we'll talk a little bit later about what the format of this data is and how the storage works. But what's really great is you can see all of the data that you have stored in fabric. You can filter by the different type of asset that you're looking at. And my favorite thing about the OneLake Data Hub is it has a desktop client, a OneLake Data Hub Explorer that looks exactly like the Sync client for OneDrive. So data that is stored here in OneLake you can access from a local machine or which really drives home the point that we like to talk about, which is that our OneLake data hub is really now acting as our OneDrive for data. It is fully integrated with all of your other office products and tools and it really just makes working with your data seamless, which is something that I know we've all been looking for. I'd also love for someone in the audience to keep track of how many fabric or sewing related puns that we make here today. Our seamless data storage here. One thing that's really nice is you can see all of your data as it is stored across the fabric landscape. So what you'll see over here, these are the workspaces that I have access to. And this is a demo tenant. So these are just for fun. These are the workspaces I have access to. And then of course, this is all of the data that either I have access to or the data that is endorsed in my organization. Now, I mentioned earlier that if you are familiar with Power BI fabric will feel very comfortable. And I'm sure you're seeing that already. But one thing that I wanted to drive home here is that all of your data and workloads and workflows throughout Microsoft fabric are organized similar to Power Bi or the same actually, where things are organized Via workspaces. So workspaces are where you have a shared set of assets. And for example, you can see in this workspace there are only two of us that have access. This is just another one of our layers of security that we have within fabric and similar to how it was in the past world. If you want to distribute reports, so thinking about the end of your data pipeline, usually we end in a report that users can access. You're still going to distribute your reports in something that is called an app. So this is where you have your reports organized for people to see. That's how you're going to distribute. So similar to Power bi, fabric is organized via workspaces and apps. Apps are where people are going to consume information. Workspaces is where you are going to build your data solutions. So let's take a look at one of these data solutions under the hood here. And I am in a workspace that is the public workspace. And I actually like to just take a look at what's in here and what's related to what. So I'm going to click on this lineage view to see what's going on with my data and reports that are stored in this workspace. And I, you can see here I have a lake house. Let's do a little bit of zooming in here. I have data that is stored in a lakehouse and I am doing a number of different things with this data. I am using a notebook, which is a part of what perhaps a data scientist would use. Analysts are using it more and more. I am accessing the data and managing it using SQL. If that's a tool that I'm familiar with. I can write SQL directly inside of fabric that is then populating a semantic model, which is then populating a report. So let's take a look at each of these assets just so that we can see what they are, what they look like and how we interact with them. So I'm going to start with the lakehouse. Now just real quick, there are different ways you can store data here in fabric. You can store data in a lakehouse, you can store data in a data warehouse. And then we also have huge real time data analytics capabilities. For now, we're going to start by looking at the lakehouse. This should look really familiar to folks who are conversant with SQL. One thing that I do want to point out here is if you see these little triangles next to the tables here, these actually have a significance. They're not just a funny little shape. These are your delta Tables as they are stored here in fabric. You can actually take a look at those delta logs to see how the data has changed over over time. You also have the ability here in your lakehouse to use your SQL analytics endpoint and to go ahead and query some of your data. So here we've just got to select our top 100 from our customers table. I'm going to go ahead and I have not typed a single code so far and we are at the mercy of the demo gods and goddesses. Here we go. That took about 10 seconds and we got back our top 100 based on the schema that we specified here. This is what we're seeing in our lakehouse. We are querying our lakehouse using the SQL analytics endpoint. That is one way to work with data in fabric. Now jumping back to our workspace, let's just take a look at some other things that we can do within fabric. So the next thing that I think is really, really fun and has really opened my eyes. I come from an analyst background, but working with fabric I have become really conversant with a bunch of different tools that I never thought I would use. But my most favorite by far is the notebook. So within notebooks in fabric you can use many different languages, you can use Spike, Spike, that's Spark. And combined you can use Spark, you can use Python. There is a copilot inside of our notebooks so that if we hide that we need with our code, we do just that it contains different kinds of. So just like any other notebook we can do create markdown cells, we can create cells from code and then from here we can do large scale data transformation that perhaps your data science team needs and then your analysts need to access the data to report on it. So what I love about this platform is that we're really bringing everything under one roof. Your data science team isn't siloed over here, your data warehousing team isn't siloed over there. We're all working in one environment. Okay, I'm going to jump back to the workspace and I am just going to go finish going through this flow. So from our, we've got some semantic models that have been created. These semantic models are populating reports and these are the end product of what we're trying to do with our data. Right. This is the piece de resistance. We want to make sure that people this information. So using tools that exist in fabric, like using our notebooks, orchestrating things with a data pipeline, transforming our data using SQL, we can do all of that right here under one single pane of glass. Alrighty. So I think for our demo for today, I'm going to pause. I do want to actually just navigate back to the homepage to remind us all we now have all of these experiences under one roof. And the real key here to my analyst heart is that I do not need to get any separate Azure resources spun up or have access to Azure subscriptions. Nothing is managed outside side of fabric as far as end users are concerned. So that was just a little sneak peek of everything that you can do and see inside of fabric. Angie, was there anything I missed that you wanted me to highlight?

No, I think you did a great job, Shannon. I think. I love how you reminded us all that notebooks are becoming more and more popular around and with analysts. Python has been around for a long time. I know people on how do I get Python with Power BI and now with fabric in notebooks? You get both, I think. Should we swap back to the deck?

Yeah, let's jump back to the presentation.

Awesome. Here we are leaving you off where you just finished demoing for us. Perfect segue. And let me move you along. Oh, sorry. We are swapping over to Shannon to finish this bit out.

Can you hear me?

Yes, Back.

Okay, fantastic. If we could jump back to the slide deck, that would be great. Richie. Perfect. So you see here in Fabric, we've got our Persona centric experiences. Yes. And all of the Personas are able to work under one roof. And this is the really nice thing, in addition to folks being able to work under one roof. And this is something that I failed to show in the demo, but that I think is really important. Fabric does have Git integration. For those of you who have been wondering, when are professional CICD tools coming to my analytics workflow, the time is now. There is git integration and it enables folks to collaborate and work together on these analytics solutions. Here we are looking at the data integration workload or experience and this is something that we did not show in the demo. But just a quick screenshot here to show you that the data integration experience is Data factory. And this is where you're going to do your ETL or elt. You're going to transform your data, you're going to orchestrate things. You're going to create pipelines with this nice branch point and click experience. It is easy to use. You can either create pipelines here or you can leverage your experience using Power BI and create a dataflow which in this experience is now a dataflow gen 2. So new and improved, the data integration workload provides data engineers, and potentially analysts, maybe analytics engineers with the ability to unify your hybrid and multi cloud data estates in a single experience that combines the ease of use of power query with the scale and power of Data Factory. This really is leveling up what you're used to seeing in Power BI and bringing a more familiar experience that you know with Data Factory. Let's go ahead and jump to the next slide and we're going to talk a little bit more about our data engineers and what they're able to work with. So we did look at this briefly in our demo and this is just showing that we have data engineers have the ability to do everything under one roof. Here in this view we're looking at the Lakehouse Explorer and you can see all of these different delta tables that are stored in here. The Lakehouse integration here democratizes data access so that everyone who knows how to use the data lake can access the data in here. And it provides advanced tools for efficient data engineering right inside of fabric. Again, there is no need to work across resources or subscription groups or anything within Azure. Everything, as Angie mentioned, is software as a service. It's right here under the fabric roof. So the data engineering workload empowers your data engineers with an Apache Spark based platform that can help you transform data at scale through this common lakehouse approach. And we'll talk a little bit more about how this runs on Spark when we get into some slides further down the road. Let's jump to the next slide. Again, we are focused here on our data engineers who are likely the ones using this data warehouse. Perhaps it's your analytics engineers as well, but we've got this lake centric design and a unified experience. I think the key here for our data warehousing is that this supports querying across multiple data sources without duplication. So what we're talking about here, when we talk about OneLake, the real idea is that we are eliminating the need to copy your data into a million different places to do different things things with it. So we are talking about having one copy of your data for all of your data needs. You can then take it using either the lakehouse or the data warehouse and you can build out a medallion architecture if you want different folks to have access to different layers of the data. If you want only data engineers to have access to the raw data, but you want data scientists to have access to that sort of cleaned up layer, the silver layer, and then you want analysts only to have access to to that gold layer, your layer that's going to be used for reporting, you can manage that using either the lakehouse architecture or a data warehouse, depending on what tools your team is comfortable working with. And that here is the real nice bit about fabric. There is a lot going on here. As you can see. We've already talked about a bunch of different workloads, but the key is that you get to use what you are comfortable with. If you have a team that is new to jumping into the engineering space, but they're really strong analysts, they can dabble in these tools, familiarize themselves, they can take some of Angie's learning modules on Learn, and then they can jump right in and work with the data warehouse. Let's go ahead and jump to the next slide. This is something that we have not talked about much yet, but that I would assume a lot of people here on the call are excited about. And that is our focus on real time intelligence. And this is something that is servicing our data engineers, our analytics engineers, by enabling transformation of data and data into action, of course, on huge volumes of streaming data. This is something that we've been able to do in the past using different tools that we've stitched together. But right now it is again all located under the hood. We've got event driven solutions so we can analyze data in real time. You can get instant insights out of this that you can then act on and set up a learning. And we also have versatile applications so we can support IoT, we can support fraud detection, business operations, large volumes of real time data that we may not have been able to do in the past. Let's go ahead and jump to the next next slide. This is the one that I personally am the most excited about and I really hope that other folks get a chance to play around with this and we're going to show you at the end how to sign up for a trial. But the data science capabilities under within fabric are something that I think we are kind of doing this new blending of worlds, right? So we have our data scientists working on the same data in the same place as. Wondered why data science.

Sorry, you were just breaking up there. Can you just repeat the last bit about data scientists in the same place?

Yes, of course, yes. Thanks, Richie. Yeah. So I always wondered as an analyst why data scientists had to work in a completely separate environment and transformation steps, as I was doing as an analyst, but using different tools with Microsoft fabric. Really incredible how we're able to now do this all in one place, under one roof. And I know I keep saying that, but I have been truly blown away that these different Personas on a data team can work within the same tool. So we are empowering data scientists here. We are building machine learning models and building AI with science speed and with scale. This is all integrated with OneLake, right? This is all integrated with our lakehouse. So we have our next generation of data science right here in Microsoft fabric. And then of course, the collaboration piece can't be understated enough. We are able to train, deploy and manage machine learning models right here in fabric. This is another big piece of the puzzle that I think is has the potential to really move your organization forward or a step ahead in the analytics game because we now are incorporating machine learning and AI all in the same place as the rest of our analytics. This one is ground shaking to me. Let's go ahead and jump to our last workload here that we're going to talk about. And this is Power bi. For those of you who are not familiar with Power bi, welcome. We're so glad that you're here. This is the end product of those apps. Hours and hours and hours of work that you and your team have done on this data. This is often the only thing that your end users want to see. This is here for your business decision makers and it becomes your single source of truth. The real key here to success is that we are no longer working on. Okay, Mary in finance has an Excel spreadsheet that has some sort of numbers someone else on the data engineering team is looking at. It is querying a data warehouse that has a completely different set of numbers, while your analyst is working on a semantic model that has a slightly different set of numbers. The upstream data manipulation in fabric enables everyone who is then accessing data, getting data out to be accessing the same set of data. It is rather a miracle. This one is easier said than done in, in theory than in practice. It does require some strong, you know, government governance and process and of course the people within the process to make it work. But Power BI here is bridging the gap between your engineers, your, your data analysts, your folks that are using the data, so your end users, your business decision makers and it's putting everything you know as one source of TR truth in one place. And the idea here is to build trust for our end users. We want to make sure that people know when they're looking at this number. We don't have to guess is Mary and finances Excel spreadsheet. The right version is the data warehouse. The right version what they See is what they get and it's going to be right here connected upstream to all of the super powerful tools here in Microsoft fabric to organize your data. And Angie, let's just jump to this very last slide here. The AI that is woven into fabric. There was another pun if anyone is keeping track. The AI that's woven here into fabric is really incredible. I may be a little biased because I am working on this myself, but the fabric platform includes built in Azure OpenAI workloads. So this is in a couple of different forms. We have this in the form of what we call our copilot and I'm not sure if anyone has had a chance to play around with copilot in fabric, but it is really incredible. I personally love copilot in notebooks, but we also have Copilot in our other experience. Well, we have Copilot in our data warehousing. We have CoPilot In Power BI, of course, the copilot in the data integration. So in the data factory workload is really incredible. I think that this has the potential to be a game changer and to move different folks around in different disciplines. If you are able to accomplish something by asking Copilot and doing it yourself without having to wait on a data engineer further upstream, it's going to enable us to do more with our data and to get more insights out. So we've got other AI workload, other AI capabilities as well within fabric, but we're going to just stick with these for today. So if anyone has played around with the AI assisted creation in fabric, I'd love if you could just put an example in the chat. If you have not worked with it, that is. Okay, there is time and I am going to pause there and hand it back over to Angie to talk us through some more of the kind of fundamentals under the hood. Oh, you know what, I'm going to talk about this. We're going to talk about some of the fundamentals under the hood of Microsoft fabrics. So we're going to talk about this kind of Lake Centric SaaS solution, the software as a service solution and this may demystify some of the questions that folks have in the chat about how the kind of lake centric solution works. So Angie, if you wouldn't mind jumping to the next slide and I believe we've got some animation going on here, so if you. Perfect, thank you. All right, so I mentioned in the demo that we are really thinking of OneLake as our OneDrive for data. This helps kind of demystify Things for folks that are not as back end tech savvy, but it really just is centralized data storage and access. So we have like I mentioned before and you know this is the most important thing in data. We have our single source of truth for insights. One other thing that you may have noticed when I was demoing is we have an organized namespace so so there is an intuitive hierarchy and indexing for all of the data that is stored within OneLake. We also have integration with Microsoft Purview and this is going to enable you to really organize your data estate and keep track of everything that's going on. We're not going to go into that today. It's a separate product but we do have built in organization or I'm sorry built an integration with Purview. One other just quick note about the way that the data is organized here in Fabric. It is organized in a hierarchical namespace which makes it really easy to see what you're working with. And then we also have indexing for data discovery using the MIP labels, so the Microsoft Information protection labels that you may be familiar with data lineage and then PII scans. So personally identifiable information scans to enforce the rules that you at your organ have set for governance and compliance on data stored in your lake. Let's go ahead and jump to the next slide and I believe this one. Oh perfect. Thanks Angie. So here we are talking about our serverless compute, our OneLake data storage and I want to talk a little bit here about what I have mentioned with the delta format and this is really important and I think critical to understand why it's so important here in fabric. So we do again as I've mentioned possibly 20 times now, we we have one single data copy. There's a common querying method for all OneLake data and it is one copy of the data. The you will hear us say here with the advent of Microsoft fabric, something that is we refer to as the delta parquet format and this is that you know the data is optimized for the compute engines to access the data directly. And this delta format is a key component of fabric's data architecture, particularly within OneLake. So the delta tables are the storage unit that supports the ACID guarantees. So your atomicity, consistency, isolation and durability. This is an open source format, plays well with others and is really fast and compressible. So the data is stored in delta tables. The file format itself, delta tables use parquet as the underlying file format. So for those of you that are familiar with CSVs, Parquet is in my mind, kind of the next level of CSVs. It's a columnar storage file format optimized for large scale data processing. So it's really optimized for performance. There's something called V order optimization. We're not going to get into all of that, but I think what's really important here is that we are using a single copy of data for all of our compute jobs. No matter what we're querying, all of the data is stored in the same place and in the same same way. So the delta delta parquet format, what you hear us talking about, is a key component within the data architecture of fabric. Alrighty, so let's jump to the next slide here and we are going to talk a little bit about what, what we mean when we say this one copy. And this is something that is really next level when it comes to sharing data within fabric. So there is a concept known as shortcuts, or a feature rather, where you can link to data on other cloud providers without duplication. So this is something that I think is going to be a huge game changer for a lot of us. OneLake supports data in Azure, AWS and Google Cloud at the current moment. I'm sure this is going to continue to grow as the product matures. And the idea behind shortcuts is that you link to data stored on other cloud providers, but you're not duplicating it, you're creating a link which makes OneLake a kind of a multi cloud data lake. So no matter where your data is stored, you can make your Data available in OneLake and you can query those using any of the query mechanisms that we showed you in that demo. So you can query those using the common query mechanisms that exist within Microsoft fabric. So I'm hearing the audible oohs and ahs from everyone who's listening today. Shortcuts. And one copy of your data. Again, this is just really taking your data to the next level. All right, and then finally, this is one of my favorite parts because we as data folks, we love our data and we want to work with it as best as we can. But the whole reason we're working with our data and the reason we're working with it in Microsoft Fabric is to empower every Office user. So we want to show how data that has been, you know, transformed and stored using Microsoft fabric is integrated with familiar Office products and services. So you all probably recognize some of these names that you see here today. If you are a Microsoft shop, you probably already collaborate in teams. You may already be using OneDrive and SharePoint sites. So Microsoft Fabric has native integration within each of these products. One that I'd love to point out here is how many folks are still doing fairly massive or fairly important, rather, maybe that's a better word, fairly important. Data transformation and analysis using Excel. I think if we were in a room and I were to do a show of hands, the number of hands would be quite high. And that's okay. We know that a lot of businesses run on Excel. Well, the good news is you can get to your data that is stored in OneLake and is shaped into semantic models. You can access that right inside of Excel. We're staying all within the same ecosystem and I think that this integration is incredibly powerful. And then we're going back to the fact of making sure that we are working on one copy of the data. I'm going to take one more moment here and highlight the integration with PowerPoint, because this is something that I find really, dare I say, powerful. But we now have the ability to take those reports that we've created in the Power BI experience here in Fabric and we can look at live reports within PowerPoint. We no longer have to copy and paste off of a Power BI dashboard or report. We can just insert a live report into PowerPoint and use that as the basis for our meetings, use that as the basis for the decisions that we are trying to make. So I'm super excited about all of this collaboration and the way that we can really use this kind of integrated ecosystem in a way that makes sense and in a way that is not scary because it's all tools that we are familiar with working with. I could talk about this for days, but I'm going to pause and hand over to Angie, who is going to talk a little bit more about some of the more technical questions that I'm sure some of you may have. Angie, take it away.

Thanks, Shannon. I love to be able to be the one that gets to talk about the hard stuff, everybody. I'm already seeing some of the questions in chat, asking about security, asking to make sure is your copilot data being used beyond your different tenants and things like that. So another big important thing is regulation, right? We all know about gdpr, which acronym I do not remember, but in Europe it's very important that you are taking care of your data. The US has several things too. I suspect that across the globe more and more regulatory compliance is being enforced. Fabric is available across all of our regions and it is being encrypted both at rest and in transit for all of your data. It also has control over where your storage is being, where your storage is being stored, where your data movement is across regions with multiple, with multi geo command migration. That's just the really important thing. Know that your data is being stored in the appropriate places. Right. For the big scary slide here, I don't want to go too too in depth about each one of these levels but as you can see here with our concentric circles is that there is security at every layer. Right? Your data is in fabric. We are also securing the access to that data. Shannon demoed a little bit on how you are managing at the workspace level. You can also manage down to the items. So if you don't want somebody seeing access in your lakehouse, you have various permissions even from that level where maybe you could just share read through the SQL endpoint to your lakehouse. Beyond that back to the workspace and item and then the network control and security. I don't want to miss out that within the data stores. So your lakehouse, your warehouse, you can also have role based access control such as there is row level security, column level security, object level security. So again you are really keeping all of that data safe. Shannon mentioned a little bit earlier about MIP sensitivity labels. Obviously keeping with that same governance, preventing data from being distributed or used in the wrong way, even unintentionally. Right. The idea of the grant the least privilege, you know, the less you grant the better your data will be secure. You can see here on this very busy slide there are all sorts of different ways that your data is being being kept safe. Again, we could teach an entire webinar plus on just admin and governance. So we won't go too much into detail here. But I think what's really important is to make time for questions because there are a lot of really great questions in the chat. So we will just show you our amazing resources. As Shannon said, we're going to show you how to try fabric for free. Correct me if I'm wrong Shannon, but it is a 60 day trial still. And there's lots of information here on this slide where you can go learn more. You can go find my learn training modules. There is an ebook, there's a fabric community and if you go to our career page, our career hub, you will find Shannon there as well. So we have really been working with fabric since before it was announced and so we really enjoy talking about this. And just one quick note from something I saw in the chat, we are doers we are really working with the product. We are not the marketing licensing people. So we're going to give you the real answers on how to answer, how to use this tool for yourself, your platform for yourself. We won't be able to go into specific licensing and cost questions. So I think. Richie, what do you want to do for questions?

All right, yeah, thank you both of you. That was really informative. Very cool stuff. We've got tons of great questions from the audience. Before I get to those, Alex, Data Camp's head of Cloud curriculum which has been messaging me frantically saying tell everyone about the course. So there is going to be a fabric course coming out on Data Camp at the end of September. Watch out for that. It's going to cover some of the stuff that we've covered in this webinar, but also you're going to get hands on and learn all about technology. So very excited about that. I actually have a ton of questions for you, but I'm just getting those because we have so many audience questions. So well done audience for coming up with all these questions. The first question is from Christian. So Christian says your low code, no code solutions for doing working with data. We're going to see a rise of citizen developers in more organizations. And is this something that fabric can help with, specifically self service reporting? I think self service analytics is a very hot topic. So yeah, talk us through it.

I love it. Yeah, yeah. So enabling self service is something that genuinely every organization wants and with Fabric you can see that it is going to become easier to enable self service and the big and here is that self service depends on, on governance and process being put in place for you to ensure that the things you're passing off to your end users become true Self service. Any analytics tool, whether it's Excel, whether it's Power bi, whether it's a competitor tool, those are not intuitive to be able to use. So yes, absolutely. Particularly with the integration of Copilot, I see, I see the group of people who are end users but then end up becoming analysts or doing the skills that analysts would do. Absolutely. I think that's going to explode through the roof. And you know, having worked in the real world, I will. And as you all do every day, I think the kind of the pieces around the process and governance that are not in any product, this is something you have to do with the people that you work with to enable self service. That's going to be key. You can expose different resources to different people in different workspaces and make sure that you're doing the best job you can to enable self service. You can promote and certify data sets. You can give different people different levels of access to different pieces of the medallion architecture. There are all sorts of things you can do and yes, I see this. Absolutely. Enabling self service with a little help from you all on this call.

Nice. So it's sort of self service but you need a little bit of analytics expertise probably. If you're going to create a good dashboard, then you need. Yeah, true, true. All right, so next question from an anonymous LinkedIn user. So I think you're coming a bit. Bit of this talking about parquet, but yeah, what if you need files instead of tables?

I think the beauty there is that you can. Yeah, yeah, exactly. The beauty with the lakehouse. Yeah, just the beauty with the lakehouse. The onelake is the data is there right in the onelake hub. You see the data, whatever you've been given access to, you get to access and you can work with it as however you want. You don't have to make them a delta table. You bring your file in and you leave it as a file. It's. That's the thing what the. The multi threaded to use Shannon's yet another sewing fabric reference is. It is almost like fabric, right. Thinking about an actual piece of fabric where there's many different ways it's woven in and you can. You pick your own path. I always think it's like a. Choose your own adventure with fabric.

I like that idea. Choose your own adventure. Okay, yeah. So files, no problem. Excellent. Next question is from Jonathan. So how do you control costs and workloads at a user and data source level? I think this is like the big sort of worry is like once things get a few layers of abstraction away, you don't necessarily see the cost of your queries and then at some point your chief financial officer is going to complain at you.

Absolutely. First of all, kudos to Jonathan for the pun. Well done. This one is interesting and it's something that I actually really advise that everyone play around with to understand what your costs are going to be. And we have some tools within fabric that you can use to understand what kind of compute the different workloads are consuming. We have the Fabric Capacity Metrics app, which is an app that you can download and install within fabric. We also have something that is new called the admin monitoring workspace. And within the admin monitoring workspace you can see what people are doing, where and when. You can see what is spiking. Your compute fabric does have something under the hood where we bursting and smoothing so we smooth out what your, your compute is doing over time essentially and you, you purchase the amount of capacity that you need bit by bit. So we are licensing this. You can purchase smaller capacity, you can purchase larger capacity. But the way that I advise that folks try to sort this out is go ahead and sign up for and start a trial immediately. Start a trial asap. Put some of your assets in there, some of your bigger workloads, maybe other transformations that you've already been doing using ADLS Gen 2 Azure Data Lake Storage, Gen 2 combined with Azure Data Factory. Try to set those up within fabric and then monitor them, see what's going on with them, see how much compute they're using. The calculating the compute is very complicated. So I think the best way to do it, sign up for a trial, start using it and monitor.

Okay, nice. So it's good that the admin tools are there and hopefully you're going to figure out whether you're spending too much money before you get shouted at. That's good to know.

You get 60 days. So I mean that's pretty great. And it's real. You're getting full access. You're not being limited in what you can do.

And it's also not one key piece to this puzzle here. It's not pay as you go. So you are not going to be racking up thousands of dollars in cost. You're buying a subscription and you are using capacity that is within your subscription. So it's licensed very differently from other similar types of products.

Ah, okay. Yeah, because a lot of these things like your paper query and.

Exactly that's.

Then yeah, you have that long running query and suddenly your bills are higher than you thought.

Yes.

All right, okay, so subscriptions slightly more predictable cost then. All right, next question comes from Igway. So Igway asks can use VS code. So you mentioned notebooks, you mentioned Power bi. What if you want a fully fledged ide?

Yeah. The answer is still yes. Primarily you're using VS code for access accessing the notebooks. But there's also a preview extension so you can looking at it. The extension gives you an opportunity to do local and remote development so you can run and debug your code. Yeah, that's pretty much it. Yes is the answer.

It's becoming obvious to me with some of these questions that an intro to what this platform is could be like an eight hour course.

Yeah. Because to your point it's a platform. Right. What did we cover? Six different workload experiences. So there's going to be a little bit of something for everyone.

Okay, I guess flip that around then because I know like on Azure there's what, 200 and something services now? Are there any bits that just aren't included in fabric?

Yes, yes. Fabric is not all of Azure at this current moment. It is a very specific set of tools that you are building end to end analytics solutions with. You probably have seen since fabric has been released. Bits and pieces continue to be added, which is very cool. That's what I love about these Microsoft products that continue to change. It also means that we all have jobs to continue teaching people about them because they continue to change. So if there are kind of pieces of the analytics puzzle that you need to run your business and they're not in fabric today, I would actually recommend you check out the Fabric Ideas forum. It's ideas.Microsoft.fabric.com I believe Angie can check me on that and that you will see there that folks are writing in with basically feature requests or things they need to be able to be a fabric customer that don't exist yet. The product team then takes those, looks them through and incorporates them into their roadmap. So if you are thinking, oh this product might work for me, but there's this one thing that we rely on and it doesn't exist here. I really encourage you to check out that ideas forum and upvote something or add your own idea.

Yeah, I'd add to that. That just the key part. Again we talk about it being an analytics platform. It's not where you're running your operational databases and your front end websites necessarily right now. Many people make a business out of analytics which is why the real time intelligence is giving you that direct immediate response. But I think if you're expecting oh I'm going to go put my CSV files and run a web app, maybe I wouldn't recommend that. And yes, Shannon ideas fabric.Microsoft.com Go submit your ideas, get what you need.

Nice. We've got a couple of minutes left for a few more questions. Very long question from awa. But I think the question is, is there a cost difference between running your queries in fabric versus just native Azure SQL?

This is a good question. I think that I would need to get a little bit more information from, from you to understand exactly what you're doing and how, how much it is costing you. I would imagine that the different pricing levels for fabric are actually quite flexible. We go from capacities as small as I believe an F2 is. The smallest capacity all the way up to capacities that are in the hundreds. They're all at different price points. So this question is a bit complex. I think if you have just stood something up using SYNAPSE and Azure SQL, it's working for you. That is great. You're going to find those exact same products within fabric. So if you determine over time that you would rather have a SaaS type solution and that's what your data team really needs to collaborate, perhaps you end up switching. I will say Microsoft fabric is still very new. So if you have productionized workloads that you are relying on data and they work for you, I don't see any need to switch unless you're doing a cost analysis and that may be the piece. Angie, do you have any response to that one?

Yeah, I think the key part here, just reading the whole question is, and to your point is connect to the data first. Right. Kind of see how it's working for you in fabric. You could kind of see the cost of those queries because just to the same point, it's not a per query cost, it's not pay as you go. So you would be able to use that capacity and scale it as you need. So I think, yeah, again we're going to keep saying try it.

All right, excellent. So we are, we're coming to time. I want to go over a little bit because we got some great questions. Before I do that, I just want to say we've got some upcoming webinars. I'm just going to put the, the QR code up there. So if you want to register for our future events, please do make use of that and then you can then I'll be able to see you in, in future sessions. All right, so this question from comes from Nicholas saying, will Data Camps Ms. Fabric course be connected with an official Ms. Microsoft certification exam? As in Power BI case?

Yeah.

So it's early days for the curriculum. We're starting with one course and I think there's a second course in early stage planning at the moment. At the end, actually, I don't know whether there is an official fabric certification yet. We do. All right, in that case we will almost certainly follow the Microsoft curriculum. Yeah, we like to align with, with Microsoft for their certification. So yeah, I imagine the, the fabric curriculum, once it's fully fledged, will align with that. Do you want to tell us a bit more about what's involved in the fabric certification?

One that's what makes you a great partner datacamp, is that you align with our certifications and really help us help certify people in real world roles. For there's the Power BI Data Analyst which we've covered before. I know you have a course for the DP600. Fabric analytics engineer is kind of the next level up. You already know how to do data analysis, modeling and everything using Power bi. Now your company has fabric or you're working with fabric and how do you go to the next level? All of this stuff we're demoing. How are you using notebooks? How are you bringing data into a lakehouse or warehouse? Using semantic models off of those data stores. Creating new custom semantic models. It's really this self serve data ingestion to make curated semantic models. To Shannon's point, it's that gold layer. Really the analytics engineer is the one that takes the data that the data engineer has brought in. Somebody asked about data engineers, they still have a completely valid job. There's lots of work for a data engineer in fabric, but you need that in between. You need somebody with the business context to make such a great curated gold layer data store that downstream users can report off of. So that's really the goal objective of the Fabric analytics engineer.

Okay, nice. So sort of several layers of certification from beginner and then a bit more advanced once you like tying it to the the business cases. All right, let me see. Time for one or two more questions. So LinkedIn users asks, does it work with other forms such as Iceberg? So you mentioned that you can use the delta open table format. Icebergs, another one of those. I think the other ones Apache Hudi as well. There's a few of these floating about. Can you use all of them or is it limited to the delta format?

Great question, because the good news is your answer was in the BUILD 2024 keynote. I think where Arun demonstrated that coming will be full support for Iceberg and some data bricks. So more file types coming. So absolutely. Check out that Build 2024 keynote. We don't have a link for that because we were not expecting the Iceberg question. But great information out there on what's more to come to Shannon's point, lots of new features are being added. They know you need these extra features.

And I believe that Iceberg support exists today using something called mirroring in fabric. So I believe that it has been released. If if not it's definitely preview. So coming soon.

All right, something to look forward to then. And one last question. So you mentioned that bonnlake is a multi cloud setup. So yeah. What happens if you've got some data in aws, some in gcp, some in Azure. Like is there some sort of transfer cost there?

Or.

Okay, possibly a tough question.

I don't know the answer to that question because I have not actually used those features. And I would mention our documentation. Microsoft Docs, which is@learn.Microsoft.com is certain to have some of these types of questions.

Okay.

I have to say Microsoft documentation is generally very high quality. So, yeah, that's probably your place to go for that. All right, brilliant. We are well past time now. Thank you once again, Shannon. Thank you once again, Angie. Very, very cool stuff. Thank you to everyone in the audience who asked a question. So many good questions today. I'm very pleased with the audience. You've done well there. And do make use of the QR code to register for future events. We've got a very nice session tomorrow on bringing Generative AI to the enterprise. There's a cool session on LLAMA Fine tuning LLAMA 3 next week. A lot of fun to be had. All right, I will see you all again soon. In future webinars then. All right, bye.

Thanks, Richie.

Thanks.