#61 Creating Smart Cities with Data Science
3 May 2021 · 44 min
Amin Rashauriki, former Chief Analytics Officer for New York City, explains how cities use data to both react to crises and predict problems ahead. He built a system to catch tenant-harassment landlords and a tool that helps small businesses find ideal locations using city data. Government data teams face legacy systems, privacy laws, and data decay, but embedding data scientists in agencies and running data-drills helps overcome them.
Chapters
Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.
Hello, this is Adele Neme from datacamp.
And welcome to Data Framed, a podcast.
Covering all things data and its impact.
On organizations across the world. Throughout the years, we've definitely seen a rise in data science and analytics in a city and government agency context. Whether that's governments investing in smart cities, hiring chief data officers, opening data up to citizens, or improving operational efficiency within the government itself. However, that does not come without its unique opportunities, challenges and concerns. This is why I'm excited to have Amin Rashauriki for today's episode. Amin is a Principal Scientist at Nvidia and is the Global Director of the Data Lab of the World Resource Institute. Previously, Amin was the Head of Machine Learning at URBINT and served as adjunct faculty at NYU's center for Urban Science and Progress. Amin also served as a Fellow at the Harvard Ashe center for Democratic Governance and Innovation. He was also the Head of Urban analytics at ESRI and was Chief Analytics Officer for the City of New York and the Director of the Mayor's Office of Data Analytics. In 2012, Amin was one of the 11 individuals appointed by President Obama to the 20122013 class of white House Fellows. Immediately after the fellowship, Amin he was appointed the Chief Technology Officer for the Office of Personnel Management. Amin earned a doctorate in Engineering from Morgan State University, as well as a Master and Bachelor of Science degree in Computer Science from Howard and Lincoln University, respectively. In this episode, we talk about the distinction between Chief Data Officers and Chief Analytics Officers, how he defines smart cities, and discuss some of the use cases he worked on to make the City of New York more data driven. We also talk about the unique challenges in ethics and privacy when working on data projects in a government setting, and discuss the state and importance of data literacy and government to drive efficiency and the responsible use of data. If you want to check out previous episodes of the podcast in Show Notes, make sure to go to www.datacamp.com community podcast.
Amin it's great to have you on the show.
Adele, it's my pleasure. Thank you so much for inviting me to be a part of this conversation. Really looking forward to this discussion.
I am really excited to talk to you about data science and government agencies, the importance of data drills, how to leverage data for better disaster response. But before, can you tell me a bit about your background and how you got into the data space?
Yeah, I think the way I would talk about how I got into the data science space is to sort of give you a sense of my journey as a computer scientist. My undergraduate, master's and doctorate degrees were in computer science and engineering. And so I really started off, you know, as a computer scientist, and I was an engineer at Motorola for a long period of time. And then I moved into academia, got my doctorate, moved into academia, and I really focused on sort of the bioinformatics domain at Johns Hopkins. And then from there I moved into the federal government where I focused on tech and policy. And really it was the larger concept of tech technology. All facets, but because I was a political appointee in the Obama administration, and the Obama administration really focused on this relatively new concepts, specifically for the federal government, of big data and big data concepts and big data technology. And that's where I really began to cut my teeth in the data science space, really thinking about how big data can be used in the federal government space.
Right.
And so I spent a lot of time working with agencies on big data, big data concepts, big data technologies, investing in cloud computing, and so on and so forth. And then I got the opportunity of a lifetime to become the chief analytics officer for the city of New York. And that's where I really jumped into the deep end with both feet and grew and perfected sort of my ability to use data science to solve really complex problems, specifically in the urban context. So for me, data science was never conceptual. I didn't learn it in academia. It was always understanding it from the applied perspective.
That is awesome. And I would love to zero in on your experiences in the public sector especially. So we often hear cities employing a lot of chief data officers, even organizations as well. Can you define the difference between a chief analytics officer and a chief data officer, specifically in a government context?
Yeah.
Yeah. I'll start by saying that one, I think chief data officers themselves are different across each other. So in the United States and globally, municipalities are adopting this concept of chief data officers because city leaders, urban leaders want to be data driven, and so on and so forth. And so they're really adopting this concept of hiring chief Data officers. But I think there isn't one clear definition of a Chief Data Officer. So what you'll find is that depending on the city, their role, that Chief Data Officer role will be different depending on the background of the person hired into the role, depending on where that role actually sits. In some cities, the Chief Data Officer reports into the cio. The Chief Information Officer in some cities reports into the mayor, or under the mayor's office structure. In some cities, it can report into the Office of Management and Budget. Right. And so it just differs the. Even the concept of Chief Data Officer itself is different depending on where you go. What you won't find is Chief Analytics Officer. I think the Chief Analytics Officer in New York City is the only title across, definitely across the United States. And I can't speak intelligently for international, but I know within the United States there's no other Chief Analytics Officer in the city. And so one, I differentiated this role a couple of ways. One is I reported into a deputy mayor. And so when this role was created by Mayor Bloomberg Back in 2013, I took the reins in 2014. The executive order that was written to create this role was written to say that this person needs to have an executive leadership stance within city government. We have to put this person as high as possible such that they can maximize their impact. So that's one differentiator. And the other is that we worked on analytics projects that helped to drive operational efficiencies within the city. So we didn't do performance management, we didn't do sort of data services, we didn't do data management. I didn't have to think about databases and cloud infrastructure. All I thought about was a problem and how to take the data scientists that reported into me and use that to solve that problem. I would differentiate Chief Analytics Officer and Chief Data Officer that way.
So in summary, if I'm capturing this correctly, it would be a Chief Data Officer would often be sometimes someone who is tasked with managing the infrastructure, scaling the data science practice in some sense that granting data access. Whereas the Chief Analytics Officer is someone who's just looking at data science problems and trying to operationalize them within a government setting. Would that be correct?
I could not have said it better myself.
Adele, so then, given your experiences and expertise creating smarter cities with data, can you share your thoughts on the importance of smart cities and how that enables government agencies to be more efficient and some of the work you've done to make New York City a smarter city?
Yeah. So in general, you know, this term smart cities has gone the way of the term big data. Right. It's used a lot, and the meaning sort of is malleable. And in many instances, it doesn't mean much. It has very. It rings hollow. What I would say is when I think about a quote, unquote, smart city, I think about a city that can be reactive to things that it hasn't been able to be reactive to before.
Right.
So let me pull back. When you think about cities, if you think simply about cities, there's two things that city leadership does. The first one is strategize, you know, in terms of urban planning, in terms of strategic initiatives. And so you put these strategic initiatives into play. When you think about strategic initiatives, you want to think about, oh, how do we lower greenhouse gas emissions within our city over the next 30 years? Right. And so what's the strategic mechanisms and levers that the city can pull in order to meet a goal of reducing greenhouse gas emissions by 50% over the next 30 years? That's a strategic effort. Then cities respond to things that happen to them. Right. No case is better than Covid. No one expected this to happen. It happened. Cities have to do things. City leadership, citizens residences, residents have to do things that they've never done before. And especially city workers have to react in a way. And so the way you want to think about how I think about a smart city is one is a smart city that has the ability to react in such an efficient, timely, and precise way that it minimizes any damage to infrastructure and. And any damage to the residents of that city. That's one reaction. Two is being proactive. A city that is proactive around its infrastructure and understanding how to get out in front of any possible challenges. So the way I think about it is there are known knowns, known unknowns, and unknown unknowns. A smart city is the one that has its arms wrapped around the known knowns.
Right?
So we know what we know, and we can solve those problems quickly and efficiently. The known unknowns, hey, there are things that we know that we don't have our arms wrapped around, but we can be thoughtful around ways to invest, to get our arms wrapped around these challenges. And then for the unknown unknowns, we say, look, there are things that we just don't know. In 2019, no one was preparing for a pandemic. So we didn't know that was coming down the pipe, but that when it hits, we have things in place to be able to proactively get out in front of big problems coming down the pipe. Right. And so that's the way I think of a smart city. A smart city is one that can be reactive where it hasn't been reactive before, and one that can be proactive, all in the pursuit of protecting residences and citizens and growing their quality of life. So one example I give you one reactive, and I give you one quick proactive. Reactive was tenant harassment. So New York City City Council found that landlords were harassing tenants in order to get them to leave their apartments, such that the landlords could raise the rent rate. And to harass tenants in order to force them to leave is illegal. It goes against human rights policy and civil rights policy that was set up in New York City and legislation. And so City Council and the mayor said, hey. They pulled together a task force and said, hey, we need to get out in front of this. Too many people are complaining about being harassed and ultimately being kicked out of their apartments illegally. And so then they came to my office and he said, can you help us use data to find landlords who are participating in illegal tenant harassment? And what we did was we looked at data and what I refer to as the timeline of harassment. So we looked at many different things that gave us early indicators of where tenant harassment is likely happening, right? So we weren't able to put together a list and say, these are seven landlords who are conducting harassment. What we were able to do in data is to identify a number of names based on occurrences to identify where tenant harassment was likely happening. And so we took that list and we would give it to inspectors, and inspectors would go knocking on doors. And so the way I think about New York City is finding things in New York City is like looking for a needle in a haystack. My office's job was to use data to burn down the haystack to make it easier to find the needles. So our job wasn't to find the needle. Instead of having inspectors go to 400,000 homes, which would take forever and which would be fairly cumbersome, you say, here's a list of 843 landlords that you should reach out to. And so that was reactive. We were being reactive to the need to stem the tide of tenant harassment. And so a more proactive solution that we put together was we worked with a small business agency to build this tool called Business Atlas. Now, Business Atlas was we tied together data across the city to provide sort of free market research information on the city such that small business owners could actually use this tool in order to identify places where they should likely open up businesses. So, for instance, we took data on crime data, park data, how many parks are in the neighborhood, we took education data, how many schools, and so on and so forth. We took data from the census, we took data from where businesses were opening up, we took data around where businesses and what types of businesses were closing down to give you and the users of the business Analyst insight onto what was happening in certain neighborhoods to be proactive around. Oh, I would like to. So if you put in an address into this tool, you got all of this information that we connected through fuzzy logic to basically say if you're opening up a toy store, you've got a lot of grandmothers, older people who live in this community, you've got a lot of green space, a lot of parks, you've got a lot of open space. And so, yeah, there's going to be a lot of kids around. So it's best to open a toy store in this neighborhood. And so we built this tool called Business Atlas where we connected all of this data from the city to give insight. So that's an example of a reactive solution in tenant harassment and then an example of a proactive solution in terms of helping the city drive its strategic goal of growing small businesses.
That's an awesome definition of a smart city. And taking that definition of a smart city as a city that is both reactive and proactive about solving problems with data and especially taking into account these use cases that you've led operationalizing of and help operationalize. What do you think are the unique challenges data teams face when working in a government agency setting?
Yeah, yeah, the challenges are unique for these data entities that work within cities. One is people often assume cities, there's just tons of data just sitting around in cities waiting for some smart data scientists to just use it and find insights about that city that has never been surfaced before. And that's just not true. Oftentimes. A lot of the systems that cities have invested in were not brought online for the express purpose of, you know, 15, 20, sometimes 30 years down the line for data scientists to use.
Right.
So one, a lot of this data is just locked away and unable to be used in any thoughtful way unless you invest in, you know, modernizing and transforming some of these legacy systems. That's one. So there's tons of data that we just don't have access to just by virtue of it wasn't built for us to have access to. That's one. Two, is the quality of that data degrades over time and how it's managed, who manages it, as you know, you know, city governments, people work in positions for three, four years and then they leave. They may not leave city government, they may move to another agency, but then they've left behind this, you know, system. And then next thing you know, 10 years later, it's degraded over time. The data in it is of low quality. There's oftentimes I've exercised projects and, you know, thought you hit the jackpot with a data set, and then when you dig into the data set, you realize you really just can't use it because it's just degraded. So, so there's limited data there. Also, there's so many regulatory and legal issues to getting access to data. Yes, there's some really rich data about residences, but there's all sorts of thoughtful and purposeful regulations around using that data. FERPA with education data, HIPAA with health data, so on and so forth. There's all sorts of challenges with using a lot of the data. So oftentimes you actually find yourself as a data entity in a city with access to very little data that's usable. And then now you have to build relationships with private sector, build relationships with agencies. And so oftentimes it's not just a data science problem, it's actually a relationship and data sleuthing problem. Where can you find data sets that can substitute for a data set that you think should exist? Right. And so you spend your, a lot of your, you spend a lot of your wills just digging for data in cities.
Yeah. So there's definitely an infrastructure data access problem and a data quality problem that are unique to government agencies and to a government use case. Now, when given the diverse nature of the data being used in a lot of data projects and government setting, there's a lot of social, economic, political dimensions of the data and the use cases you've worked on, what do you think are some of the best practices you've developed to ensure the successful and responsible delivery of these use cases?
That's a great, that's a great question, Adele. One best practice is I always exercise the ability to embed data scientists within the executing agency.
Right.
So what that means is if Department of Buildings in New York City reaches out and says, hey, we want, we would really like to use your team's expertise to help us identify illegally convert. This is a true problem. Illegally converted buildings. So illegally converted buildings in New York City is a building that was zoned and licensed and permitted for eight units. But the landlord says, hey, I can make more money if I can turn this into a building for 12 units. But I'm permitted and licensed for eight units. But I'm going to do some illegal things and convert it into a 12 unit building.
Right?
And those illegal things, you know, splicing, you know, wires to get electricity and gas pipes and so on and so forth, often cause fires. So a high predictor of where fires would likely happen, right. And so Department of Building says, hey, we want to minimize fires in the city. We want to identify buildings that are illegally converted. So they came to my office and they said, hey, can you help us build a data science model to identify buildings? Because there's over a million buildings in New York City, right? You can't just send inspectors to every building. Can you help us? So we said, yeah, but we did not do the project. I took a person or two from the data team and embedded them within the Department of Buildings team. Because the insight and the knowledge that is needed, the types of data sources that could be used, that should be used, where are they? So all of that sits within the Department of Buildings agency. So instead of sort of trying to be this all encompassing group of data scientists, hey, we're going to be the domain experts, we're going to be the data science experts, we're going to solve this problem and we're going to be the superheroes of this. Actually, the superheroes sit with the agencies. And so really your job is just to add value, data driven value to their work. So embedding data scientists with these agencies is the best way that I saw to solve really complex operational challenges.
It's ultimately about combining the subject matter expertise with the data driven expertise of your team in that sense. So then going beyond, a step beyond that. And you've mentioned as well, regulation, data quality as being unique challenges here. How do you view the friction between using data to, for example, help disenfranchise communities while also maintaining data ethics, privacy and the responsible use of data?
It's non trivial. It's non trivial and I think we need to see it as such. So this challenge of juxtaposing solving really big hairy problems in cities, but also respecting ethical concerns and privacy concerns of citizens, there's no easy way around that. And so we need to tackle these non trivial challenges with really complex solutions and really thoughtful solutions. So one that's never going to go away, right? You're never going to have citizens or residents who say, hey, listen, do whatever you want with my data, it doesn't matter. Actually that's growing, right? The concern of citizens and the use is growing. So there's always going to be friction there. There's technical ways that you can begin to move the ball in terms of ameliorating any challenges there or removing some friction. But I think a lot of it has to sit on two things. One is success in your execution. When citizens and residents understand that there are going to be times when you need to use their data to successfully execute a problem and that you show a record of successfully executing these problems, then one that goes a long way. The second is be responsive to their concerns. So whenever they bring up any type of concerns around privacy, don't dismiss it. Engage them. You know, don't sort of fall back and get defensive and dismissive. Actually lean in, actually turn into that challenge and engage them. And to make folks feel more comfortable. There's no way you get around solving problems without using what could be considered sensitive data. I haven't seen that as a possibility in cities. But what you can do is deploy some technologies like obfuscating data and so on and so forth. There's some technical solutions, but then there's also the non technical side and that's the engagement side.
Yeah, that's definitely spot on. And one thing pivoting away slightly from the operationalization of use cases in government and just thinking about data maturity and government agencies as well. One thing I've seen you speak about in the context of data and government agencies is data drills and how it helps cities better prepare and better be reactive to emergencies and how it helps them understand their own data capacities as well. Can you define what a data drill is and some of the best practices you've learned organizing them?
Yeah, absolutely. Data drills were a great. They were very interesting, very helpful and really fun in executing them, I must say. So data drills came about when I became the Chief Analytics Officer for the city of New York. This was right after Sandy and Mayor Bloomberg did an after Action report. And in that Sandy After Action Report, that After Action report for any of your listeners, it's still available online. If you do Sandy AAR or Sandy After Action Report New York City, it'll come up. And in that After Action report, it mentioned multiple places where the Mayor's Office of Data analytics needed to play a role in galvanizing agencies around data, using data, getting access to data, all of these things the city did not have the capacity to do during Sandy. So the After Action report said, moving forward you need to kind of get out in front of us. And so my office got pulled into a couple of emergencies and what I noticed was agencies are fine in terms of managing their own data when emergencies happen, where agencies need to share data. And let's keep in mind, sort of Citi's Covid response as I'm talking about this. Whenever there's a need to share data, everyone sort of freezes and goes to lowest common denominator, which is Excel spreadsheets. Let's just dump data in Excel spreadsheets. Let's fat finger data into Excel spreadsheets and let's just send them via email and thumb drives all over the place. And you can only imagine how big of a mess that can be and how imprecise your data response would be if you're just emailing Excel spreadsheets all over the city and so on and so forth. So I realized we just do not have the capacity as a city to respond to catastrophes and challenges from the data perspective. You know, NYPD knows how to work well with fdny, with the fire department in New York City, which knows how to work well, with the sanitation department, who knows how to work well, the Department of Buildings, and so on and so forth. These agencies have been doing this for decades upon decades. They know how to work with each other. They know who to call, what the protocol has to be, where to call, where to go. But when it came to sharing data, we just didn't know anything. And so what data drills were essentially an effort to say, let's start practicing how to share and use data during emergencies before an emergency happens, such that when an emergency does happen, we've actually practiced this and we understand what we need to do. So it's not only practicing process and protocol, but it's also practical, practicing the use of tools. Because one of the things we ran into was even if agencies wanted to use tools and software that they had purchased to share data and integrate data, what we found out was that, hey, some agencies invested in this company's tools, other agencies invested in another company's tools, and they just didn't talk. They weren't interoperable. And so one of the things about data drills was understanding what tools do we need to use and invest in in order to make sure that at the time of emergencies, our data systems are interoperable if they need to be. Right? And so it was all about training, doing drills on how to share and use data. So literally, we would create a scenario, a real life scenario in which something bad happened in a city and we would get data leaders and data members. You that could be data scientists, it could be data analysts, it could be database Managers from across all of these agencies and literally sit at a table and present them with this emergency, and then everyone would have to start sharing. How do we think about working with each other at the table in order to share data, share insights in order to solve the problem? And that really went a long way in getting the city to be more thoughtful about responding to data emergencies.
Now, one other thing that I'd like to pivot away from here and to talk about is really discuss the state of data skills in government. So love to deep dive into the skills you think are needed to become more successful while working with data in a government setting so broadly. There are two components to this. The first being data experts honing specific skill sets that, that will help them navigate the government landscape per se. And the second is government employees developing data literacy to use data on a daily basis in the work. If we want to expand on the first component, what do you think are the skills data experts need to hone specifically in government to be able to drive more impact and accountability in their projects and work?
Yeah, I think insight on. I think data experts need to really understand the definitions of data quality and where data quality is useful. The concept of data quality is useful and really when to identify and how to identify when a data set is appropriate to be used or not. And so understanding data quality is going to be key for a data expert understanding data provenance. One of the things I used to say when I was in New York City was that data are like waves. You never know where they began, right? And so it's like, you know, you get this data set and that data set could have been around for 15, 20, 30 years and changed hands and changed owners and been manipulated and modified over time to the point where, you know, the definition of the data set, the description of the data set makes it seem like it's going to be helpful. But really, if you actually understood the provenance of that data set, you would say, like, whoa, there's no way I could use this here. It has just been, just went through the ringer and the quality of this is probably going to be very low. So you need to understand, sort of have data experts who build out systems and mechanisms to understand data provenance. And then I think there's a term, you know, it's a little tongue in cheek and it's a little kitschy. But the term that we used to use was to people who were sort of data sleuths or data curious. So you really had to have people who had the ability to track down data sets in the city. We were working on one project, actually the tenant harassment project. And to know which units are rent stabilized under law is managed by the state. And so the state does not share this list of rent stabilized units with the city. And so what my team found out was that during the Bloomberg administration, I was in the de Blasio at the beginning of the de Blasio administration, during the Bloomberg Bloomberg administration, some smart bureaucrat in the Bloomberg administration figured out a way to get a proxy list of every building that has a rent stabilized unit. Where they did that was charge the landlord of a rent stabilized building a nominal tax. And this is important that they call it a tax, A nominal tax, you yearly tax of 10 bucks. So every year they had to just pay $10. If you own a building, you're like, I'll pay that $10. But because you pay that $10, New York City's Department of Finance tracks that tax payment. And so that tax payment now gets triggered with the building address in the Department of Finance building list. So now Department of Finance has an authoritative list of every rent stabilized building in New York City because they are required to pay taxes. Now when you transfer from the Bloomberg administration to the de Blasio administration, when things like that were done, that doesn't really get shared, you know, over at the, you know, water cooler, right? These things get lost. They, they just stay with one administration. They never get transferred over to another administration. Why would they? Who would remember to share this? Who would know to ask, who would remember to share?
Right.
And so you needed to have people who were focused on finding data sets and understanding those data sets and where they came from in the history of those data sets and, and understanding the context so that you can understand how useful they would be. I mean, it was a big find for us to dig in and ask enough questions to find that particular data set at the Department of Finance. Right. So you need people who are very curious about data and who ask tons of questions.
So definitely that ability to operate and navigate in a government setting that is so unique to government agencies there. So diving into the second component, how do you view the importance then of building data literacy for non technical stakeholders across agencies?
Well, I think data literacy is so incredibly important because at the core of it all, most people within government use and manage data, right? Whether that's Excel spreadsheets, whether that's access database, whether that's, you know, punching something into some information, into a content management system, into a case management system, Everybody engages with data in some Capacity. And so one democratizing or streamlining or ensuring that everyone or the majority of people within city government have these core basic data literacy skills maximizes your ability for data driven government to be executed in a really efficient and precise way. And so I certainly have seen an experienced government in a way to be able to forthrightly say that you cannot have a data driven government if only a small, and it seems, you know, intuitive and logical, but you cannot have a truly have a data driven government if only a small privileged sect of your government employees are data experts or data literate. You have to ensure all are data literate, but not only data literate just in general, but are consistent across that data literacy in terms of shared knowledge, shared understanding. And so people getting taught on the same things to be at the same level, to have the same understanding of vocabulary, verbiage and concepts and processes. Right. So data literacy is going to be key. We think that we can hire a team of four or five people, then have the leader and give them a fancy title like Chief Data Officer and say, all right, now the city is data driven. No, the whole group of government employees need to have some minimum data literacy capability. I'll even go one step further and then say that community members who often engage with the government need to have mechanisms in which the government can invest in their data literacy as well.
What do you think are then the main data skills needed for government agencies to get started, especially on the data literacy side.
So I think a key skill that could be taught across the government forward data literacy is exploratory data analysis. Just from a pure standpoint of understanding, well, what I've been collecting data for decades. What does this data say? What is this data representative of? What that does is two things. One is we're always so excited about insights and we always want to, we always believe like there's this data set that's sitting here and if I probe it, then it's going to tell me me the world's secrets, right? Or the city's in this context, the city's secrets and all these things we didn't know about the city. That can happen. It's very rare. But having the ability to do exploratory data analysis gives you the ability to baseline and understand the data set that you're working with. And here's why that's important. It gives you the sense of, well, this is not high quality data set, right? So you can immediately tag your data set. If you as a government employee understand exploratory data analysis. Oftentimes, you know, hey, this is not a high quality. Let me give you an example. I'll leave out the name of the agency to protect, you know, people. But one agency asked my office to do, help work with them, to do an analysis for them. A really, really, really time sensitive and very important analysis. As a matter of fact, Federal Department of Justice was looking to sue New York City around a number of things. And so this agency said, hey, we've got two weeks to respond. Can you help us do this data analysis? So we said, sure, give us all of your data. They gave us all of this data and my team dug into it. Just exploratory data analysis, just simple bar graphs and line plots and so on and so forth, violin plots. And they said, this is interesting. The data that this agency gave us seems to be orders of magnitude different than the national data set. So we went to the federal government, downloaded the federal government's national view of this sort of local data set. And when you looked at the two data sets, it was like, New York City just seems to be way off from the national average on block, right? And sure enough, you go back to this agency and you say, hey, there's something up with this data. It's just we don't think that it's good data through exploratory data analysis. And they were like, oh yeah, you know what our bad, we forgot to give you this other data set and we forgot to give you the more complete data set, right? And so just having simple exploratory data analysis skills allowed for us to understand if we were looking at quality data. What it also does is allows, and this is a big conversation and you began to allude to this a little bit, Adele, in terms of services, in terms of equity, in terms of engaging underserved communities, is understanding the biases that exist in that data set that you have. So many people want to take a data set and train that data set and build a model and start identifying insights on that data set without even beginning to understand whether or not and what the biases of this data set is.
Right?
And having exploratory data analysis skills lets every imagine every person at every agency has the ability to look at a data set that they're working with and better understand the biases that are being introduced by that data set. So I think exploratory, I think EDA is a good, good sort of core capability for government employees to have then.
Given the importance of democratizing some of the data skills, especially exploratory data Analysis in the government setting. What do you think is the data team's responsibility when equipping non technical stakeholders with these skills?
Well, I think identifying quality standards. So what ultimately should happen for a city data team is that the city data team should get out of the role of doing all of the work, but get into the space of setting standards, setting best practices, setting processes and workflows that can be used by everyone who is looking to do some core data work within their agencies. So training can be a part of that, but training with respect to standards, best practices, strategies, tactics, so on and so forth.
So ultimately it's about creating the infrastructure for scalable data science throughout agencies. Now, Amin, before we wrap up, I've also seen that you recently joined Nvidia as a principal scientist. Can you walk us through the work that you've been doing there?
Yeah, you know, I was excited to recently join Nvidia as a principal scientist, primarily working with the AI nations and helping to build our capacity to use AI to help cities solve some of the more complex challenges. I think my key role, when I think about it, is if you look at a world class AI company like Nvidia that has high quality, best of class AI infrastructure and technology, and then you look at cities and there's all of these use cases, these, these complex, challenging use cases. How do we bring these two things together in order to help solve these city problems? Because there's a lot of use cases in cities that should not and don't require AI to solve them. But then there are those use cases that just quite frankly, if we don't bring accelerated computing and AI to the table, we probably will never get to solve some of these really sticky city problems. Right. And so my job is really bringing these two things together, the use cases and the world class AI compute and see where we can really accelerate solutions for cities.
That is very exciting. And finally, Aman, before we let you go, do you have any call to actions for listeners?
A couple of things. The first one is when I grew up, I grew up with this term each one teach one and I love YouTube and going on YouTube and you see a lot of data scientists teaching on YouTube and sharing insights and best practices and tactics on how to do data science. I would say we need more of that. We need more of that. In terms of what we talked about, Adele, in terms of training up city personnel and public sector personnel so where you can share best practices and capabilities that are specifically focused on growing the capability within city government, that would be amazing. And I think the other is Share solutions folks who are listening, who are students undergrad or graduate, or who are just students of life who are growing their GitHub platforms with projects and initiatives. Share those and make those more public so that we can reuse those where needed as well.
All right, thank you so much for the insight, Amin. It was a pleasure to have you on the show.
It was my pleasure. This was fun. I expected it to be fun. It was absolutely fun. I really appreciate it. Adele. Thank you for having me.
That's it for today's episode of Data Framed. Thanks for being with us. I really enjoyed this conversation with Amin and how he thinks about the use of data science and government. It will be fascinating to follow how data science can help governments become more reactive to crises and how it helps them become more proactive to achieve their strategic goals. If you enjoyed this episode, make sure to leave a review on itunes. Our next episode will be with Dan Becker, CEO of Decision AI. In it, we will talk about the intersection of decision sciences and AI and how to align machine learning to business value. I hope it will be useful for you and we hope to catch you next time on Dataframed.
More from Richie
#359 My Best Friend is AI with Valerie Tiberius, Professor of Philosophy at University of Minnesota
#358 How AI Agents Will Work While You Sleep | Ruslan Salakhutdinov, Professor at Carnegie Mellon
#356 The Forecast for Time Series Forecasts with Rami Krispin, Senior Manager of Data Science at Apple
#355 AI's Impact on Databases with Shireesh Thota, CVP of Databases at Microsoft
#354 Beyond BI: Decision Intelligence with Graphs with Jamie Hutton, CTO at Quantexa
#352 AI Agents at Work: What Actually Breaks (and How to Fix It) with Danielle Crop, EVP Digital Strategy & Alliances at WNS
#351 Will World Models Bring us AGI? with Eric Xing, President & Professor at MBZUAI
#350 How to Make Hard Choices in AI with Atay Kozlovski, Researcher at the University of Zurich
#349 From AI Governance to AI Enablement with Stijn Christiaens, CEO at Collibra