How Microsoft is using AI to help the Earth

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@robar.com and we're hosted on Linode servers. Head to linode.com changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning, data, science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're PracticalAIFM. And now onto the show.

Welcome to the Practical AI Podcast. This is Chris Benson, your co host as well as the Chief AI Strategist at Lockheed Martin RMS APA Innovations. This week you're going to hear one of a series of episodes recorded in late January 2019 at the Applied Machine Learning Days conference in Lausanne, Switzerland. My co host, Daniel Whitenack was going to join me, but had to cancel for personal reasons shortly before the conference. Please forgive the noise of the conference in the background. I recorded right in the midst of the flurry of conference activities. Separately from the podcast. Daniel successfully managed the AI for Good track at Applied Machine Learning Days from America, and I was one of his speakers. Now, without further delay, I hope you enjoy the interview. I have Jennifer Marsman, who is principal engineer on the AI for Earth Team at Microsoft, with me today. Welcome to the show, Jennifer.

Thank you for having me, Chris.

So I was fascinated to learn that Microsoft had an AI for Earth team. I think that's super cool and I would like to know more about it. Could you tell me a little bit about yourself and kind of how you got on the team and a bit about the team itself? Give us kind of a broad intro to it.

Absolutely. So I'm very excited about the AI for Earth team as well. I actually heard about it before I joined the team as one of the kind of new initiatives Microsoft was going forward with. And I wrote an essay to the hiring manager on why they needed to hire me because I was so excited to be a part of it.

I love that.

Oh yeah, oh yeah. I went hardcore for this one. But here's the idea behind it. AI for Earth started by the guy who is now my manager, a dude by the name of Lucas Joppa. And Lucas is actually Microsoft's first chief Environmental officer.

Okay.

Which I did not even know that we had prior to taking this job. But apparently, apparently that is a thing. And what's really cool is Lucas worked for this guy Eric Horvitz. For a very long time. And Eric Horvitz is of course a big name in the machine learning world. He's been at Microsoft Research for a long time. And Lucas's background was actually in biodiversity and conservation, and that's where he got his PhD. And so coming from that kind of environmentalist background and then working with like for Eric for a time really got him thinking all the time about the intersection between AI and these hard environmental issues. And when you think about the progress we've made with artificial intelligence just in the last 15 years, I mean, it's astronomical. It's so much like things that were. So I'll give away my age here a little bit, but when I finished grad school, I did my grad school in. Actually my degree was in artificial intelligence, which gives you an idea of how long ago I did my thing. That that was still a degree then. Now that's way too broad a field to have as a degree. Now it's NLP or computer vision or whatever. But back in the day, this was maybe 16 years ago at least when I got my degree in AI. We've made such progress since then with deep learning. Just in the last 10 years or so, we've been able to solve really hard problems like automatic machine translation and speech recognition and a lot of these other things. And so the idea was since machine learning is having this exponential effect, could we apply it on hard things like climate change or like being able to conserve animals that are going extinct at a rate that's like thousands of times the natural rate currently, or how to build or how to grow more food on land in a sustainable way. So you're not, you know, burning out the land by doing, you know, you can get the short term benefits, but then it's not good for the land long term. But to be able to increase the yield while still doing it in a sustainable way and then provide clean water to people. So all of these really hard challenges, like what would happen if we applied machine learning to them. And that's what was keeping Lucas up at night. And so he wrote a white paper and that white paper became the basis of essentially the AI for Earth program. So at its heart, or at the beginning, it is a grant program. So Microsoft publicly committed $50 million over a five year time frame. And that is money that is available as grant money to anyone who is doing machine learning or data science work in the areas of agriculture, water, climate change and biodiversity. So any one of those four is fine and it can be a startup, it can Be a nonprofit. Large corporations, independents, anyone at all is welcome to submit a grant. And we accept grants on a quarterly basis. I think the next deadline is in April 2019. And so folks are welcome to apply for a grant. And so that's kind of the one, the start of it. But our kind of aspirations are even grander than that. As you said in the intro, I am an engineer, so I want to be building models. I want to be doing the fun data science work myself. And so we're actually doing a great combination of that. So we partner very closely with Microsoft Research. So there's several Microsoft Research projects that we're funding and working very closely with. We're actually, our engineering team is kind of building some of our own APIs where we take models that may be useful to a number of people who are doing sustainability efforts and then exposing those models as APIs so that people could call them and utilize them. And then the third bucket of engineering work is in the area of our AI for Earth grant recipients. So being able to support them wherever they're at in their machine learning journey. So some folks that we work with are these brilliant environmentalists, but maybe they don't have as much of a machine learning background. And so they have these fantastic data sets and so we help them get started with how they can apply machine learning techniques to that data. And then Some people are PhDs in machine learning, academics and such. And those people may be just absolutely brilliant at machine learning, but maybe they would want to know how to use Azure to be able to scale out their work or get results faster. Any of those things. Yeah. So my job, like I try to, I actually own the three E's for AI, for Earth engineering, evangelism and education. So I do work.

Sounds like a fun job, right there.

It is. Oh, it is. So just kind of trying to work across those. It's a lot of. It's a lot of work, a lot of stuff. But I'm really, really exciting to be able to be a part of it.

So that sounds super cool with it. So it sounds like you both, you know, fund these other projects that are out there through the grants that you talked about. But you also mentioned that you have your own engineering team, so what kinds of things would you focus on helping other people from a grant perspective, what kinds of things would your own engineering team and how do those interact?

Yeah, great question. So essentially the grant proposals are whatever people submit. So those are not something that I have control over. It's whatever people want to submit. We accept Things that seem feasible, that seem like they're doable, we want to make sure they have a data set that will actually work and something that seems like it's possible, but as long as it's a feasible, a data set that would support what they're trying to do and it seems feasible, and they seem like they have the knowledge and background that they could do it, it's probably worth at least a small investment of Azure. Give them at least some Azure credits to let them try it, that sort of thing. And then our own engineering team, what we're trying to focus on is are there things that we could build that are things that would work for a large variety of people? Right. So almost trying to do, like, the way that I think about it personally, and this, this is kind of just Jennifer Marsman speaking, not an official AI for Earth Day.

No worries.

But for me, I try to think of what are things that would be useful to a variety of people. Right. Because a lot of times the way a grant recipient should work. So let me take a quick tangent. Quick tangent. So for a long time, there's a guy who actually does a lot of speaking now about AI in China, Kai Fu Lee, who has worked at Microsoft and Google and such. And so he's also a big venture capitalist and funds a lot of startups. And one of his words of wisdom about startups is that startups should never build a platform. They should build a product. Right. They should solve one specific problem and solve it well and then expand out to a platform. Right? Okay. And so I, I think about the work that these nonprofits and academics and such are doing the exact same way. I think that they, a lot of times they are trying to solve such a specific problem. Like, I need to be able to predict how well, you know, flooding for this particular river in Africa and this one place in Africa. So it's a very specific problem. And they shouldn't focus on the high back. They should solve the problem that they need to solve. Because if it's a nonprofit trying to solve a thing, like, they need to focus on their mission. Right. And so what I think our team should be focused on is taking that step back and building real APIs. Like, okay, is there something, is there a way that we can then build things that would work for a wide variety of people, kind of in the spirit of that rising water lifts all boats so that we can enable the entire sustainability community to do great work. So We've started with two specific APIs that our engineering team has produced. One of them is around land cover mapping. So we did. And essentially if you're not familiar with land cover mapping, what that means is that given some aerial imagery as input. So typically what we're using is four band imagery. So it's the NAEP imagery, if you're familiar with that. But it's essentially rgb, so natural color as well as a near infrared channel. So those are the four bands. And then take that as input and then be able to say, okay, what, what pixel by pixel level. And this is, we're actually doing it at one pixel resolution. Is this water or tree or barren? So essentially it's a road or a house, something you can't build on. And we have this enumeration of different classes and so pixel by pixel level, we can enumerate those things. And so that's great for use if you are disaster relief kind of work, trying to look at or flood levels, like realizing, okay, where are the water lines? Now you can use that. It's great for urban planning, it's great for a lot of different scenarios. That's one great general purpose thing that we're trying that we have an API for that's available already. If people would like to email, I think it's AI for Earth. Apisicrosoft.com I can give you a key.

Otherwise we can add that to the show notes as well later on. So we can check that and make sure that's good.

That would be perfect. Thank you, Chris.

Sure.

And then another one we're working on is Inaturalist, if you're familiar with them, has released a large public data set. They do really amazing work and we've partnered with them as, and given them some funding as well. And what they have, if you haven't downloaded it, oh my gosh, you should because it's really cool. They have a really great mobile app that's available for iPhone and Android. And essentially what you can do is they have a classifier that allows people to, you can take their app and then wherever you are, any animal, any plant or any fungi, you can actually take a picture and then record an observation of where that is in nature. So it's then biologists and such can use that data to get a good sense of the, you know, the ecological makeup of any given area. And it's just citizen scientists, everyday people like you and me who like, might care about the environment but are not necessarily a biologist, can take pictures. And then that way scientists know that, okay, this particular species of plant is located here and this animal was Seen here. That's fantastic. Or that sort of thing. It's so cool. And so what's really nice is that they release this data set then and make it available for folks. So we actually trained a classifier that will distinguish between. It's remarkably accurate for plants and animals, giving you the species. So given an image of something, here's the species, it will return the exact species name. And so you can do this across animals, you can do it with plants. And I believe here at this conference, Grant Van Horn is actually speaking on Inaturalist. He did a lot of the machine learning work. Okay.

I can't wait to hear them speak that.

Yeah, it's really cool. It's really amazing stuff. And so they're one of our grant recipients as well.

Awesome. So a little follow up question to that. Just so how is it normally when most people are thinking about Microsoft, they're thinking about technology, they're thinking about cloud, obviously thinking about Windows and Office and other things that Microsoft is doing. How did they get into this in particular? What was the motivation for the company to back this?

Yeah, great question. So as Microsoft's Chief Environmental Officer, Lucas owns not only the AI for Earth program, but he also owns our environmental sustainability work. And in terms of environmental sustainability, Microsoft has won numerous awards for that. We've been carbon neutral since 2012. We actually have different departments inside Microsoft pay a carbon tax to offset usage and flights and stuff like that. We pay a carbon tax on that. Something that was not very popular when first introduced at the company, but has been great. But the way we look at it is even if Microsoft was absolutely perfect. So let's say our data centers were all completely like underwater data centers and everything was just fabulous. That's only so much, there's only so much impact Microsoft as a company is having just in our own operations. So how could we scale out even more? And so AI for Earth was really our answer to that question. By dedicating this $50 million over five years to people that enables everyone to be able to partake. And what I love about that is that we are asking oftentimes the people with the least resources to solve the world's hardest problems. Right. It's nonprofits, it is these sustainability groups, it's academics who were asking to solve these hard problems of like climate change and, you know, reducing the rate that these species are going to extinct and a lot of other really, really hard challenges. And so this is a way that Microsoft can kind of dedicate what we're good at. Namely we can provide machine learning knowledge. And we do that through the education arm of the AI for Earth program as well as our data centers. So here's Azure. Here's like all this infinite compute power with using the cloud. Great. Use this. And then that gives people both the knowledge and. And the access to cloud that they need to help them be successful in their areas of expertise. So these folks who are grantees tend to be good at the stuff that I'm not good at, which is the environmental end of it and that strong domain knowledge in the areas of agriculture and water and such. So put those things together and you can do something really powerful.

That sounds great. So do me a favor and in your mind, think of maybe one of your favorites that come through and take us through what it looks like from the point where they decide to apply and how you help them. What are the different both from what you might call, I don't know, the business side of it as well as the machine learning side. And just give us a picture because there may be people out there and nonprofits listening to this right now that suddenly jump at this opportunity and tell them what this project will look like when they engage you.

Absolutely, Absolutely. Okay, so, Chris, I'm gonna have a really hard time choosing a favorite because I love. There's so many good stories. Like our grant recipients are doing such amazing things. Okay, so I'll try. Let me tell you a few stories. Sound. And then we'll. And then we'll kind of go back to the. You know, if you're a startup, how do we get started here? Or if you're a nonprofit, how would we get started? All right, so let me. Let me do one in particular that is a. A something that's kind of inside of Microsoft Research. And then one grant recipient. So let me do a Microsoft Research one first.

Sounds good.

Okay, so there is a amazing gentleman by the name of Ranveer Chandra who is part of Microsoft Research. Fabulous guy. Very, very smart. He got his start doing networking and battery technology, which is kind of cool. And he essentially, there is a very, very hard problem where by the year 2050, it is predicted that we are. The pace of the amount of food that we can grow is just not going to be sufficient. In no way.

I actually heard that separately outside the.

Yep. This is. I think it was spoken at the meeting of the United nations in 2009. Someone put forward something that it was. We would need to essentially double our current rate of food production if we're going to be able to feed everyone with the world's growing population rate. And so like he started out like really trying to think about that problem, like how could we solve that? Like what, what kind of things could we do? And so he hit on the idea of precision agriculture. And I wasn't familiar with this prior to joining the team, so let me explain it for the folks who aren't familiar. So the idea behind precision agriculture is instead of, for instance, like homogenously watering a field where every part of the field gets the exact same amount of water like we traditionally do with the large sprinklers and such, precision agriculture or precision irrigation for example, is let's only water the parts that actually need water. Because if there's little dips in the field or things like that, water runs down, some of the field may get more water than others. And so obviously that seems pretty easy to do, right? Because you know, there's, if anyone's been, does any Iot work and has ever played with these things, you can go to adafruit.com or whatever and get little moisture sensors that you can put in the ground. So pretty, pretty easy to do. But. So I was talking with Ranveer and I'm like, okay, this seems doable. And then he told me it actually worked out to about $1,000 a sensor to do this. And I was like, you must be kidding me. Here, look at this website. It's much cheaper than.

So what was driving that cost up?

Great question. So the actual sensor of course is only like tens of dollars. My next guess was maybe power, right? But it turns out we use solar panels to drive in power and those, the little chips don't take that much anyway. A little board doesn't take that much power anyway. And a solar panel is like 50 to $100 US so still nowhere in the neighborhood of 1000. Turns out where that thousand dollars is coming from is actually connectivity getting data from the actual farm into the cloud. And so there's actually a couple different ways that we can handle that. A lot of times when you think about it in this farm, if you've ever driven through a more rural area and your cell phone coverage kind of goes out, right? There's a. The cell phone coverage tends to not be quite as good. Cause there's not the population to support it. And then there's also when you think about third world countries and stuff like that, where the wireless access may not be as good, or wi fi is good, but the problem with WI fi is that it only stretches so far. Like I know when I'M going to get my kids from the bus stop across the street. I'm literally right across the street from my house and can't reach the home of wifi. You lose it. Right?

Totally get.

So you. If you're dealing with like hundreds of acres of farmland, like WI fi is just not. You have need to repeat a million repeaters. So it's just not feasible. So there's actually two different ways we've solved this problem. One of them use machine learning and one of them uses cutting edge networking technology. So the first one using machine learning is you can actually reduce the number of sensors that you need by putting sensors in fewer places and then augmenting that with either drones flying overhead or other ways of collecting aerial imagery. And then you can feed a machine learning algorithm both those aerial images as well as the data from the sensors that you do have and from that be able to extrapolate the values of the entire field. So it uses essentially conceptually what the machine learning model is using is visual smoothness as well as spatial smoothness. So the idea that two things that are close together are likely going to have similar values and then things that have similar colors. So think of if we, if we have a patch of farmland that's darker because it's been freshly watered and another patch that's darker. Those two, both dark patches might indicate that they have similar amounts of water applied to them. So that sort of thing.

It makes me think of topographical maps, contours on them, maybe different colors.

Yep. That also. So having that information, feeding both of those in, we have a machine learning model that's based on Gaussian processes that can extrapolate and then give you essentially here's how much water in the field. So that's one example. The other part of it that's really cool, not as machine learning related, is that he also brought in his old networking background to do the concept of television white spaces. So this is really cool. Chris. So he's actually using.

And for the audience, she's leaning in and she's so excited about that.

So this is the idea of using unused television channels to be able to send data packets over there, like your own Internet. Okay, sure. So think about it. So a television station can broadcast very, very far. Cause it' lower spectrum. Right. So those frequencies are going to stretch further. And so you can actually use one television right spaces router and it can, you know, stretch miles and miles and miles. Right. Tens of miles. Right. So that's a really, really cool thing and a lot of times when you get to these, either third world countries or rural areas where we're setting these things up, there's a lot of unused television stations, so a lot of empty bandwidth there. So you do have to work with the local government. You can't just start broadcasting things on television stations. Right. It is very regulated. But if you work with the government, you can actually send data over these television channels. And so that's the other part of what they're doing, and that's also driving down the cost as well.

That sounds great.

It's such cool work. So it's amazing stuff. So that is called Farmbeats is the name of the project. And I can provide a link to that in the show notes as well. Absolutely. Okay. And then let's talk about one of our grant recipients, our fabulous AI For Earth grant recipients. So I love all of them, and they're all doing such cool work, but one in particular that would be fun to talk about is a company called Wild Me, and their platform is something called Wild Book. And so the problem they're trying to solve is to be able to recognize individual animals. So not just, you know, zebra, not a zebra, but rather this is ziggy the zebra versus Zoe the zebra, like Zebra 5715, you know, kind of thing. So. And this can be really, really helpful because when you think about it, there was actually an amazing article, well, not an amazing. A very sad article in National Geographic a while back where some folks were actually trying to tag this rare whale in the Pacific Northwest, and they actually botched the tagging job and ended up killing the animal, which was the exact opposite of what they wanted to accomplish by tagging it, that it was such a rare animal, they wanted to track it and all that sort of thing. And so the idea is, can we use computer vision to do that instead of. And that can lead to a whole. When you understand, like, the animal population, that can drive so many benefits. You can estimate population densities, you can track migration patterns, all kinds of cool things. And so.

So, and let me, as you answer this, how are they going to use computer vision to accomplish this particular task?

Great question. So they are taking images and training, essentially a wild book per animal. So when you think about it, I don't know if you've ever heard this, but you know how humans have very individual fingerprints where each of us have a unique fingerprint? Well, with zebras, their stripe patterns are unique.

I've heard that as well.

And then same thing with giraffes, actually, the spot Patterns on their neck, their long necks have uniqueness and.

Really, I didn't know that.

Yeah, that. And then like the shape of an elephant's ear, the spot patterns on cats. So they've actually created a wild book for a lot of different large mammals and they train it with this kind of data and then they're able to recognize individual animals. So it's a very, very cool process. But here's where the story goes from good to great.

Get this part, I'm waiting.

They actually are then augmenting their data using social media. So the idea is, let's say. So one of their wild books is whaleshark.org that is the wild book for whale sharks. And let's imagine that some random person goes on a whale watching trip and they see whale sharks and they're taking videos of them and then they're posting them to YouTube. Well, wild book has an intelligent agent that wakes up every night at 10pm and it searches the Internet and it looks on these social media sites and then it will find instances of people. Using just like natural language processing, it can find, or just regular search, you can find people who are posting about whale sharks. And then they're extracting frames from those videos and then running their audio object detector to find the whale shark in the object and then classifying those things and recognizing that individual animal. So they're finding both new whale sharks that way their researchers haven't been exposed to before. Isn't that great? Isn't that so much better than using Instagram to show your food?

Yes, right.

Like what an innovative use of social media.

I could be on a vacation with my family, we're on a boat, we see a whale shark, we take a picture, we, we tweet it and then later on that evening it comes. And not only is it able to detect it, but it's also able to know exactly which individual in the population.

Is that correct? That is correct. That is correct. And they actually give you a little comment, which is kind of cool. The agent will, it's looking not only for the animal, but it also wants to know the, the when and the where. So when did they see that and where did they see it? And so they're actually using some NLP actually on the, on the, like the YouTube page itself, for example, if it's posted, it said, last month we saw this and you get the post date there and you can do some. Their NLP is good enough to like do the math and figure out, okay, it's approximately this time. And then where? And they'll again try to extract that from the video. But if they can't, it will, this is all automated. It'll post a comment and say, oh, where did you see that? Or they'll take whatever information is missing, post the information and then get that data back. And then, and then they actually post a little comment saying hey, you know, your picture was able to help science and help us help this conservation effort. And then they'll point to the whale, the page of that particular whale that they've contributed images to, which is really, really nice. So YouTube is a great one because that's that all is public domain by default. Any public things are shared based on their licensing. So anything like that, where it's a public image, they can track those kind of things and then be able to utilize that for data. But it's really cool because then you can like go click on it and be like, oh, there's my whale shark that I saw. And they, they have some neat stuff where they allow people to nickname them so you can like give your whale shark a name and stuff like that. And then you can see all images, other images that people have taken of them. There's like an adopt a whale feature type thing and then it actually shows. Oh, and this is why they call it Wild Book. They call it like the Facebook for animals because it actually shows co occurrence charts. So the social graph of this whale has been seen with these other whales. So you can see exactly kind of the some of the parent child type relationships.

So you get the whole graph of the relationship that the whales have with each other.

Exactly, exactly.

That's pretty amazing.

So isn't that neat? And then you can actually see using the when or the where sorry information you can see migration patterns of individual whales. It's phenomenal stuff. It just, oh, it gets me so excited.

So all this sounds incredibly fascinating. I'm an outdoors person so I'm particularly interested in covering this. And frankly this may be the first practical AI episode that my 6 year old daughter wants to listen to because she's really into animals and stuff. So I suddenly have a hook for the family on this. So like if you are out there and you are interested in engaging you guys, I guess there's a whole lot. One of the big challenges obviously is data sets. Do you have any recommendations on where people can find data sets that will help him? Do y' all have a repository? Are there good. Go to sources that you recommend.

That is an awesome question. 10 points to Gryffindor. So we are actually in the middle of that process right now. So currently we are collaborating with several other organizations and we established something. If you go to HTTP Wack Wack LILA Science. Yes. That is a repository of camera trap data, which one of my esteemed colleagues, Dan Morris, is very passionate about camera trap imagery. And so he started putting together this data set on LILA Science. And the. The LILA is actually a reference to the Library of Alexandria, which is kind of cool. So that is available today. So you can go. And there are several data sets that contain lots and lots of different animals and imagery there. So that's one great start that we have right now. And then we're trying to do even more work. I would love to be able to. We're currently in the process of trying to host more and bigger and better data sets. So that is work in progress. So hopefully this time next year I will have even more great data sets to share with you. But in the meantime, Kaggle is also a wonderful resource.

It always is.

Oh my God, I love Kaggle. Isn't it the boss?

It is.

So Kaggle is another great resource for. They always have AI for good type challenges on there. I mean currently right now there's some great earthquake predicting data that's there. There's an identify a whale type data which is very similar to like the wild. Me work is available right now. There's a data set on that. So there's so many cool sources for data sets. But we're trying to compile even more of that as part of our AI for Earth mission as well. So we have a start, but we want to go even further with that.

Gotcha. So what types of deep learning algorithms do you find that are the most common that you use with people? Are there areas in particular that are on the rise or that you're most interested in? You know, whether they be CNNs or GANs, or even if we move over into reinforcement learning and such, what are some of the things that you're seeing out there that are in use?

Great question. So in particular on the AI for Earth space, a lot of the problems revolve around computer vision. It's the idea a lot of things just seem to reflect, okay, given like this data, maybe there's cameras in a farm or something and we want to recognize pests on leaves. Like that sort of thing is one example. Or we're trying to recognize like in camera trap. I should mention I haven't defined camera trap. So what that means for those who aren't familiar is mounted cameras that Are usually enclosed in waterproof casing that may be mounted in trees or other remote places. We've worked with the snow leopards, for example, and those guys are so elusive. They're so quiet, they're so hard to spot. There's one researcher that devoted his life to snow leopard research and he was up in the mountains, like living there in the area. Only saw a snow leopard twice in 11 years.

Really.

It's insane. So you really need camera traps to be able to do that. Where.

So they move in.

Exactly. They move exactly. Only the problem with camera trap, the thing that's historically challenging about camera trap data, is that camera or animals don't line up nicely for their selfies. Right? They don't. They're not in the middle. So you get somewhere, it's like an animal sniffing and you'll get like a close up of a nose that's kind of checking out the. The mounted camera itself. Or you'll get them very far away. Or if it's at night, which is when a lot of animals are most active, you'll see like two eyes squinting out of the darkness. But it's hard to tell what they are. So there's all kinds of fun challenges with camera trap data. And so that in particular is one thing. But that's very much a computer vision problem. So of course, computer vision CNNs are a big convolutional. Neural nets are a good way to go there. So there's a lot of work in object detection and computer vision also with climate. So given satellite imagery, for example, spot the little swirl that means like a hurricane is gonna form or something like that. Or in the water space, there's like monitoring plastic flow into the oceans, which another group, ocean cleanup, is doing that we've worked with. And they have like mounted cameras on bridges. And then as water flows underneath, they're tracking plastic flow going into the ocean. So. And being able to tell the difference between a water bottle and a fish can actually be. Or pieces of plastic is actually harder than you think, because they're both kind of clear or silverish in the water and that sort of thing. So it's just an interesting problem.

As a side thing, before you go on, do you actually have a project associated with that this time or is that.

Yes, there is. So the ocean cleanup has done a lot of great work with them. And they actually came to Microsoft for one of our hackathons and have partnered with some of the folks here as well. But they are doing amazing work in terms of tracking plastic flow into the ocean.

Yeah. I mean everyone is hearing in the news these days about the giant amount of plastics, particularly in the Pacific where that is and what's being done to help that. So it's really interesting to hear about something.

Yep. Ocean Cleanup is a great organization doing exactly that.

I love hearing about how these technologies that we love so much are being used to solve these real world problems that we actually hear about in the News or watch TV on, I guess. Other than CNNs, is there much room for things like natural language processing or capsule nets or gans or anything?

Great question. So I've seen, I'm actually would love a good RNN problem. So if anyone has any, please let us know. We have a few of them. There's one where people, one of our AI for Earth grant recipients is doing some work using, like working through text to be able to find what makes a difference to people the most. So for example, like let's pretend there's all these like flyers or communications that go out to people. What actually inspires them to act to prevent climate change, that sort of thing.

Sure.

And so they're doing some NLP work around that, trying to find the most motivating things for people. So I know that's one particular project and there's a couple like chatbot type scenarios, but there's not as many RNN things as you would think. And then gans, I am just in love with as I think everyone. Yeah, they're, they're, they're the hotness expanding.

So many more use cases.

Oh my gosh.

Month by month.

Oh my gosh, yes. So if anyone has a good GAN project, let me add it. Like I'd love to see a few more GAN projects here, but I don't think there's a few. There's a few GAN projects, but again, it's primarily dominated by convolutional neural networks and computer vision problems. I will say on reinforcement learning, I think that is another really big area for AI, for Earth in particular. Because if you can do things like modeling climate change and then tweaking different variables to see, okay, running, you know, using reinforcement learning to run simulations and see, you know, as I toggle these different things and I try these different actions, what is going to make the most impact and help us fight climate change the quickest. And so that's a great area for reinforcement learning as well.

That's fantastic. You know, the thing that I love about all these stories you're telling me is that, you know, historically we've gotten into Data science. And then we might do environmental things on the weekend or as a hobby or donating our time to charity, but it's for most people, kind of a separate activity. I love the way your team has brought this together to where you can both have a career in this field and also do tremendous good for the world. And I think that is as people realize that Microsoft is there supporting organizations and people that want to do this work. I think it's a fantastic whole new way of using artificial intelligence and machine learning technologies to do this.

Absolutely, absolutely. You had asked earlier about how people could get started. So if you go to go for it. Yes. If you go to the AI for Earth webpage, please come join us. Come help us save the world. Because my background is in machine learning and AI, but. And so I'm. There's some folks on the team who are stronger at the Earth side, some people are stronger at the AI side. I'm definitely one of the AI people and I'm just in awe of what people are doing. These amazing environmentalists and conservationists who are working so hard on these huge problems that face our Earth. And so if there's anything Microsoft can do to help, I would love to support folks. So people are welcome to apply for grants again. We evaluate all the applications every quarter. So I think it's January, April, was it October, and I'm forgetting. But every four months we evaluate proposals and then give things out or apply or award grants. We have a couple different grant types. One of them is just standard Azure compute hours. So just getting some cloud computing time. We actually have a data labeling grant as well now where if you have a great data set, but you need to pay someone to get it labeled and sometimes it may be a specialist, it's not something that a Mechanical Turk type thing can solve because you need very specialized knowledge of climate, for example, to be able to read these satellite things to label it properly. And so we actually are providing funding for data labeling as well.

So that before you even go in, that is fantastic because I know in real life at the companies that I have worked for, that has been one of the biggest problems about getting a project off the ground is not only getting the data set, but then after you have it, getting it labeled in a useful way so you can use it. And at times that can slow down a project by weeks or months to get that done.

Completely agree with that, Chris. I completely agree with you machine learning folks. I mean everyone listening to this can probably relate. Data is like, that's the hard Part, right. The algorithms are the easy part. It's getting the data and getting it in right format. All that stuff is usually 85% of the work. So I completely agree. We're on the same page there. It's how can we help people? If you kind of think of it as like a pipeline of, okay, we start with some data and then we need to get to the stage and the stage we're trying to find things to help at each stage of that pipeline. And so the data labeling grant is essentially, how do I get from just having some data to actually having data that is useful that a machine learning algorithm can consume, or supervised machine learning algorithm, I should say. Yes. And then we provide education as well. We have some online resources that we've compiled for our grant recipients. I hold office hours monthly so that people can just show up and ask questions. And then we also twice a year have an AI for Earth Summit where we bring some of our grant recipients to Redmond, Washington, where Microsoft's main campus is, and we give them training. So it's like a day of training. There's networking opportunities, which is great, because if someone is working on hard agricultural problems in India and someone else is doing it in North Carolina, great. We see people coming together and collaborating and sometimes sharing data sets and stuff for getting even better results. So it's really amazing to see the, you know, the power of what we can do collectively like that.

I would love to be able to see that and actually talk to these people with this passion firsthand somehow.

Oh, you should maybe come do a podcast there.

I may come do that. Be careful what you offer there. So I guess, you know, as had been mentioned before, we are at Applied Machine Learning Days in Switzerland at this conference, and you, in just a few minutes are about to go up and give your keynote. Can you tell us quickly a little bit? By the time this airs, it'll. It'll be past that. But could you tell us a little bit about what you're gonna talk about?

Yes. So what I'm going to do is I'm going to try to cover kind of the basics of the AI for Earth program, basically, not spend too much time on that, but just let people know that it's a resource that's out there. Because the worst thing is when there's this free pot of money that people don't know about. Right? Just spread awareness of the grant program that it exists. And then I'm just gonna tell a couple stories of AI for Earth grant recipients and what they've Done. So we'll talk about Wild Me, we'll talk about Farm Beats, which we've already spoken. And then the final thing I'm going to talk about is another project, Project Premonition, which also started with Microsoft Research. This one is very cool as well. It's focused on how can we predict outbreaks of disease before they happen. So think about, like, Zika virus and West Nile and mad cow disease and some of these things that we've seen in recent years. So the idea here is, is there a way we could get out in front of that and be able to predict these things before they happen? And the way this is so cool. So the principal researcher on this one is a guy by the name of Ethan Jackson. And the idea behind this one is, what if we take advantage of little data collectors who are out in the environment already collecting random blood samples? And those are, of course, mosquitoes.

Oh, yes.

If you can collect data, basically, mosquitoes use them as data collectors. They're collecting all these random blood samples, and they bite humans, they bite animals. They, you know, they're collecting.

Essentially, they discriminate.

Exactly. And so they're getting a great random sampling of the environment and feeding off of these various hosts. And so what we do is we actually have. There's two kind of big contributions of this work. Number one is a smart mosquito trap that can selectively just trap mosquitoes. Other insects can go in and go out of the trap. Like, they'll head in, check out the lure, and then fly away. And then there's little trap doors, and they only close if it's a mosquito. And we can actually differentiate between species of mosquitoes as well, because I did not know this prior to this work, but it's actually like the Aedes aegypti species of mosquito is the one that's responsible for Zika. And then there's a different one, the culex is responsible for West Nile and all these other things. It's fascinating. And so we can actually differentiate between those two species of mosquito if we need to focus on one disease or the other, which is very cool. And 75% of these large diseases actually originate with animals and then come to us. And I know a lot of them have these cutesy names of like mad cow disease or avian flu or some of these other things. But a lot of them, outside of the ones with the cutesy names, do actually start with animals and then spread to us. So if you can catch them while they're still at the animal stage, that's Great. So the first thing is the smart mosquito trap, which uses machine learning to be able to differentiate mosquitoes from other insects. The second big contribution of this work is our metagenomics pipeline. And so what we've done there is now that we have those mosquitoes, we can actually take the blood meal they've consumed, reverse engineer it, and be able to tell what host animal it came from and what diseases they carry. And then from that, you can actually get a sense of what diseases are there and then recognize them before the outbreak happens.

That's amazing.

So cool. Such amazing work.

I am looking forward. I'm gonna come watch your talk. I can't wait to see it. And I wanted to finish up by saying, how can people listening reach out to you or to the organization and start that conversation with you?

Great question. I am on Twitter so you can find me enfermarsman on Twitter. I have a blog as well, which is blogs.msdn.Microsoft.com Jennifer and then the AI for Earth team has a great website and there's information about the grant program and such on there. So I and some of our amazing grantees is all available on the AI for Earth website, which is Microsoft.com AI fourearth and then you can also contact us. I think there's another Twitter account, Microsoft Green, which deals not just with the AI for Earth, but other environmental initiatives around Microsoft.

All right, well, Jennifer, this has been a fantastic conversation. Thank you very much for taking some time out of your day. I know you're rushing around doing a lot of stuff, but it was a fantastic conversation. Thank you so much.

Thank you.

All right, thank you for tuning in to this episode of Practically I. If you enjoyed this show, do us a favor. Go on itunes, give us a rating. Go in your podcast app and favorite it. If you are on Twitter or a social network, share a link with a friend. Whatever you got to do. Share the show with a friend if you enjoyed it. And bandwidth for changelog is provided. Provided by fastly. Learn more@fastly.com and we catch our airs before our users do here at Changelog because of robar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers. Head to linode.com changelog check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this@change law.com com when you go there, pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.