Artificial Intelligence in Business Keynote at The University of Queensland

Yo, what's happening team? In this video, I recorded a talk that Athan and I did at the University of Queensland, and it was on AI and business. So someone from the university, Margot, thanks Margot for organizing it, reached out and asked us, do we want to come along and present to a group of Chinese academics, a group of professors from a university in China who are touring Australia because they came around to visit different cities in Australia to see different technology innovations, particularly in machine learning and artificial intelligence. And that's a lot of what we work on at Max Kelsen. So Athan's the machine learning and research engineer lead. So he's my team lead there, absolute wizard. I've learned so much of him. And so in this talk, we go through a bunch of case studies that we work, things that we work on at Max Kelsen, AI in healthcare, which is what I took care of. AI in business, and a bunch of other cool things. And now some of the questions in the video are a little bit inaudible. I tried to record with my iPhone, so you can't really hear some of the things that people ask. But if you have any questions, leave a comment below. I'll answer them as best as I can. Otherwise, I've left a link in the description for the slides so you can check them out along the way.

But I hope you enjoy.

Yo, Athan, what are we about to do?

Deliver a talk to 28 Chinese traveling academics.

Yeah, we're on. Machine learning. Let's do it.

All right, enjoy your session this afternoon. I'll see you just before you leave. Okay, thank you. See you soon. Bye. Well, it's a great pleasure to meet you guys, and we're thrilled to kind of share some of the work we've been doing with Max Kelsen. So I'm Athan Mullane. I'm the machine learning lead at Max Kelsen. And this is Daniel Bourke, resident machine learning engineer. So my background is electrical and computer engineering from Queensland's best university, UQ. And Daniel's background is in health sciences.

Yeah, health science, health and nutrition, food science at UQ as well. But then I decided to get into Tech, so self-taught machine learning engineer on top of the health science.

Yeah, so Daniel's been designing his own master's degree through all of the amazing online resources that are available, and yeah, and has been doing applied work with Max Kelson for the last few months and is really killing it. So yeah, I just would like to say for you guys, Please stop us at any point if there's anything that you're not understanding or wanting to ask questions about. We'd like to keep this conversational to the degree that it's useful to you guys. So yeah, but yeah, a little bit more of a background on MaxKelson. So MaxKelson's been around since 2015. We are a Brisbane-based analytics and software consultancy. So we aim to utilize cutting-edge technologies to provide powerful insights, strategies, solutions across government, enterprise, and startup sectors. So you can see along the bottom there, I'm not sure how familiar you are with the Australian government and industry, but Westpac is one of Australia's largest banks, they're a client. Shine Lawyers, a big law firm. EB Games is the biggest gaming company or gaming retailer in, I believe, the Southern Hemisphere and at least Australasia. And we've got state, both state and local government that we work with. So, We have a strong partnership with IBM, and we've now grown to a team of 27 data scientists, computer scientists, data engineers, and machine learning engineers. We have delivered cognitive and big data solutions across all the industries described. So our agenda, we definitely want to share our recent exciting news in the healthcare space, but I also thought we'd give a broader overview of applying machine learning in the business space and the challenges that we've encountered as an up-and-coming startup in Australia. So what we do is a I'll give a brief introduction. Dan's going to discuss AI in healthcare and the genomics work we're doing. We'll do a deep dive into 3 case studies, some discussion about moving from a minimum viable product into production, and the summary of some of our most recent projects from MK Research, as we've coined that term. So Yeah, that's the gist of the talk. I'm not sure if that'll play out for the full 2 hours. As I said, I'd like to keep it conversational to the degree that you guys want it to be, and we'll save some time for questions at the end. So, machine learning in business. So this comes from the Reuters report. The purpose of AI is not to replace you, but to augment you, to scale you, and help you to focus on more interesting tasks. We have, through our applied work, we have come to realize this more and more. AI is nowhere near the development at its present state to to be replacing people in the roles that were, at least in the problems that we've encountered. For the most part, what we're finding more and more is the best solutions are those that are designed to scale and assist others in solving the existing solutions we're encountering. In terms of the business problems we face, the first one would always be finding a training set. So we've got the examples there that you would find in, I guess, any classical— any deep learning article from the last 3 to 5 years. But the one in the bottom right bottom left corner is the one we're actually encountering the most of, and this is the traditional large datasets, large groups of features, usually categorical features, highly cardinal features, features that aren't easily represented as a number. So you've got the MNIST dataset there. You've got potentially a dataset that would be applied for face recognition. In either case, for image problems, now that we've solved convolutional neural networks, this— these can actually be less of a challenge than the kind of structured data that you're going to see in organizations. So a lot of what we come across is in that bottom left corner. So our principles for machine learning in business. As I've already suggested, we see that the best solutions And artificial intelligence, as it's seen today, is most successful when it's the combination of humans and machines working together to achieve a task, where machine learning is used as a catalyst to the work that people are doing, to either speed their work— speed up their workflows or inform their process, either in the way of descriptive analytics, as in describing very large datasets that cannot really be observed, or the kind of analysis that can't be performed by a single person or even a group of people, distillation of ideas, predictive models, and even from there prescriptive models, which are actually assisting in the decision-making process. A lot of the kind of data that we encounter is not big data per se. It's not terabytes, it's not petabytes. If we're lucky, it's gigabytes. So what we need to do is design intelligent, tailored, custom solutions for the exact business problems that these people face, because the majority of modern machine learning as we see it is the ImageNet dataset, massive numbers of images, huge models trained for months on end on very large amounts of natural continuous data. So that's another challenge that we face every day. And the final one I guess is ethics and explainability. The ethics of decision-making are more important in business than anywhere else. Building for fairness and explainability should be core tenets of any AI program. So the case in point of that would be the legal work that we've done. So I'll elaborate on that case study a little later on down the track. How to identify an AI problem. Is it a problem that can't be solved easily by deterministic computational decision-making? So what we've found, and what we're seeing a lot of, is that there'll be a large number of people out there who are marketing AI solutions and really what they're doing is sophisticated if-else statements, and I guess this kind of thing has been seen for a long time. We are finally at the point where machine learning solutions are becoming common enough that they can actually take over and have more power in the space. Is the problem integral to the overall strategy of the organization? So that's the other one. We'll come in and have initial sessions with organizations, and they'll realize that they want the AI, but they're not quite sure what it is or which problems best apply to it. And we'll often end up giving recommendations to them that are quite distinct from where they've come from, from the outset. So A large part of our role is actually also helping organizations recognize where the most value is going to lie in that data. And not to say that deterministic decision-making is a bad thing. A lot of our solutions will incorporate both. And so we try to have some breadth there. The difficulty of the solution as opposed to how complex that would be for a human to perform. So there's always going to be a way off there. And that would be how many full-time equivalent salaries are going to be either doubling the throughput or, you know, automated through this assistive process. So, Availability of data. So I already touched on that. You can see this plot. So either you will have high volumes of data where model complexity will be much simpler and closer to the kind of things that you'll see provided in public models trained on large corpuses of data, and it allows for model, model generability. Generality because you can learn all of those things. When we have very limited examples of data and you can't feed a model 10,000 examples of the exact same thing so that it can see every, every possible angle, you need to increase model complexity and incorporate as much there as you can. And I've already touched on privacy, ethical, explainability concerns. We'll go into that a bit further later on down the track. So as I've said, a big part of what we do with the clients that we work with is assist them in designing an AI problem that's best going to work for their organization. Based on the framework just described, we want to find a program that complements the business strategy and offers the right effort to reward ratio. We want to provide a proof of concept which can demonstrate the ability of AI to solve the problem. So we're engineers, we aren't gonna go out off the bat and try and reinvent the wheel. The beauty of working in this space in 2018 is that there's a very large number of of, um, offerings available from software vendors so that we can rapidly establish viability of the solution. Um, further to that point, once we've established viability, um, that's where the work just begins, and we, um, have our teams hone and tailor the models to work exactly with the data that's available at hand from the client. 3 is production. So that's kind of what sits in between proof and production. In addition to developing custom algorithms, we want to be developing custom pipeline solutions and the potential for online learning and auditability or explainability within the decision-making process. Again, this is really important. It comes back to AI ethics in decision-making. And scaling our infrastructure, inference, information security, and sovereignty planning. So I'll pass over to Dan now, and he's going to have a— give a bit of an insight into the work we're doing on genomics.

Hello, nice to meet you all. My name is Daniel Bourke. Before I get started, does anyone have any questions they'd like answered from Athan's section? Otherwise, I'll just jump into what we're working on in AI and healthcare. And if you at any time have some questions about what I'm talking about, just shout out. Alright. So first of all, State of play. So what's, what's AI in healthcare right now? So at the moment, there's a gap in the information. Okay, so AI in healthcare— biology is a 24/7 process, and it's not like our bodies shut down at any time until right to the end, you know. So 24/7, 365 days a year, we're producing information in the body, but At the moment, there's a gap to how we track that information. And we'll go through some graphs in a second. But what the goal of AI in healthcare is to close the information gap. To move from an episodic model to a continuous model. So the current system, episodic and reactive. So if something happens, you go to the doctor, the doctor diagnoses you, you get a treatment, and then you move on with your life. What the goal is of AI in healthcare is to have a continuous and proactive model. So what does that look like? On the left, reactive. So say this is an instance here. Doctor collects some information from you at a certain time, and the red dot, and then gives a diagnosis. Happens again, green dot, loses some information here because it's a different appointment. You get cured. This one was a pretty bad one, so there's lots of information there. Cure happens, etc., etc. What we'd like to happen is a proactive model. So as an event happens, information gets kept over time, so we can keep this diagnosis happening as a straight line rather than up and down episodic and losing information between, between sessions. So the goal is continuous and proactive. How do we do that?

Well, with this.

It's a GPU. Not in the sense of the hardware. The hardware is important, right? Because it allows us to do all these advanced computations and information and process it. Stuff we didn't have 5 to 10 years ago. We can get a laptop like this, jump on the cloud, have a stack of computers ready to go. Just from, from right here if we wanted to. So the hardware is important, but I've made a little acronym using GPU. So the first step is to gather. So what does that involve? Well, it means collecting data in any, in an ethical way. So to do good AI work, to do good data science work, to do good machine learning work, you might have heard the saying garbage in, garbage out. Well, it's the role of a data science— the role of a good machine learning engineer to take the garbage in and turn it into gorgeous out. But it's even easier to do that if you have good data to begin with. So that's what's really important, is collecting that quality data. And that's the information you get, right? It can be anything from genomics, which we'll get into in a second, to appointment times you visited your local GP, to fitness devices that we're all starting to have now on our wrists. And in the future, more advanced sensors that will be continuously monitoring our every movement. So, you've got the data. Now, it doesn't help just having numbers sitting in a database. Alright, so you need to process that information. But what's also important is to not overwhelm people. So, a lot of people, if they see a big, large table of numbers, it's not going to mean much to them. Right, so the important part is in the processing is to get that information into a way both humans and machines can understand. Alright, so it's one thing for a machine learning algorithm to run over a database, but it doesn't help if all you get at the end is going an accuracy number of 98%. What does that number actually mean? Which that brings into utilize. So once you've gathered the data, processed it, It's in a, it's in a useful format. How can you utilize that? So that's where you involve all stakeholders. So it's not just a bunch of engineers like ourselves sitting in a room getting really good results on a set of numbers. We need to bring that into the clinic space to patients, into the hands of doctors, into the hands of patients, because that's what it's about at the end of the day, right? It's improving, improving healthcare for everyone. So you have to make that information useful, and that might look in the form of a better treatment, a better diagnosis, whatever, whatever comes to play. It has to have some use point, because having a state-of-the-art model sit on the cloud doesn't help anyone. So this is where our partnership comes in. So we've got Max Kelsen— that's us here— but the work we're doing would not be possible without other people from industry. So QIMR, they're a big medical research facility that's based in Brisbane, world-leading in things like immunotherapy, which we'll go into in a second. Genomica, which do some, some world-class precision analytics on whole genome data, and which comes from BGI, which is an international genome, next-generation genome sequencing company. And then Metro North Hospital, of course, which is our getting it into practice, getting it into clinics. So that's, that's the local big hospital that we have here in Brisbane. So what's the problem? Cancer treatments are expensive and damaging, right? So some treatments, you'll see in a second, can be up to $100,000+ per year, which is— of course it'd be great if treatments could could be delivered to everyone. But at the end of the day, reality is that these treatments have a cost associated with them. And not only are they expensive, they're damaging to the health of the people who receive them. So most cancer treatments at the moment, the three big pillars are surgery, radiation, and chemotherapy. So what, what those three are, are all very invasive. So surgery, you have to literally get cut open. Radiation and chemotherapy are both poisons for the body to not only kill cancer cells, but they also damage your own healthy cells. And at the moment, delivery of those treatments is mostly trial and error. There are some ways that you can accurately pick, but at the moment it's sort of like, if you're a patient and you have some kind of cancer, it's very hard to tell which one is going to work best on you. Person X might respond really well to radiation, but terribly to chemotherapy, and it might be the opposite for person Y. So, what's our goal? In 4 words, what we want to do through the partnership, Max Kelsen Genomica, BGI, QIMR, and Brisbane Metro North Hospital, we want to improve cancer treatment predictability. And so, what, what does that look like? Well, with the knowledge from Genomica, as in the genome processing, BGI, sequencing genomes, the hospital practice, clinical know-how, QIMR, research on immunotherapy, and Max Kelson with AI. The collaborative, collaborative effort is to somehow work out what a better way to predict which treatment will work best on which patient. So say for example you were to try immunotherapy. At the moment the best success rate is about 40%. We'd like to improve that so we could go, if you would like to try immunotherapy, it's going to cost this much, but because of your— because of the information we've gathered, we can predict that it's going to be 78% accurate. Now, those numbers are arbitrary, but that's essentially where we want to end up. So, first step is gather. So, I'm going to talk about the 3 main ways to gather information: genome sequencing, clinical information, and sensors. But today we're going to stick with genome sequencing because that's the one that we're going to be working on in the first phase of the 3-year project that we have coming ahead. So we're going to be spending the first 12 months, first little period of that before we get into the clinical information and sensors. So, GATA. How is data collected? Genome sequencing. So genome sequencing involves— is a part of genomics. And genomics is, as it says, the study of a whole— an organism's whole genome. Now what's a genome? Now each one of your cells in your body has, including cancer cells for that fact, has a sequence: A, T, G, T, A. It doesn't matter. Adenine, thymine, guanine, and cytosine. There we go. 3 billion of those 3 billion base pairs. And between you and I, or anyone in this room actually, if you paired up 2 people, there's about 1 in 1,000 difference. So about 3 million differences. And so what we plan to do is essentially treat this as a language, treat this as something that, that we can derive patterns out of. And now if we look inside these differences and inside— what this means here, sorry, inside is that each one of your cells has 3 billion base pairs. Your cancer— say, for example, you had a cancer cell, that will also have 3 billion base pairs. However, that cell will also have some differences. So what cancer does is it's a mutation of your own cells, and what will change is some of these. Now what changes, it's hard to tell, but that's where we're going to come in and start using some of the techniques we've been using elsewhere to try and see, okay, so there's differences between Sally and differences between John. Sally has this type of cancer and John also has the same cancer. How can we match those two in terms of their differences? And why doesn't the treatment of immunotherapy work so well on Sally, but it works so well on John? So this is, this is the overall goal, is to figure out Why the differences between people matter, why the difference between your cells and cancer cells matter. So that's the crossover. And now, why is this important? Well, we look at whole genome sequencing. Who's heard of the Human Genome Project? Anyone heard of the project that was going from 1990 to 2003? So anyway, to sequence the first human genome cost $3 billion US and was from 1990 to 2003, required 54 laboratories across the world, hundreds of scientists. It was a global effort. Now, 2018, it's about $5,000 to sequence a whole genome. But in the next 5 years, that number you can expect to drop as technology improves. At the moment, as I said before, Cost for some treatments, $100,000+ per year. So you can see immediately the value add here if there was some way to predict how likely a treatment is to work. And of course, we— treatment cost is something you'd like to be able to negate. Ideally, everyone would be able to get cancer treatment free of charge, but at the end of the day, cost is an important issue. So what does this lead on to? Well, if we can— the ideal goal: sequence someone's whole genome for $5,000 and then find a way to compare, figure out, okay, now we know your whole genome sequence, we can now target which treatment is going to be best for you, so we don't have to spend hundreds of thousands of dollars potentially going for the wrong treatment. And then what's the spillover effects? Okay, so we figure out that for John, treatment immunotherapy might not work, but that's still a valid— beneficial to John knowing that a treatment won't work because John can try another treatment. So rather than going down the road of damaging treatments such as radiotherapy, chemotherapy, we might be able to say, hey John, We figured out you're more likely to respond to this, so we think we'll proceed in this one. In terms of saving John time, saving the government, saving medical facilities money, and ultimately potentially saving lives, extending the lives of, and the quality of lives of people with cancer. What is immunotherapy? So I think this is important to take into consideration because the whole The AI solution, it's not just all about the tech. It's not about running some good numbers and getting a really great accuracy model on living on the cloud, right? It's a whole encompassing— where the value comes in is the encompassing environment between us, healthcare providers, and businesses. So that's where the real value is, not just in us being able to accurately model patterns in genome sequences. So immunotherapy, why is it different to— how is it different to radiation or chemotherapy? First of all, it tries to harness the power of the body's immune system. So within each of us, we have an immune system, an existing one that does a very good job day in, day out, kills numerous cancer cells every day, but sometimes some slip through. So what immunotherapy tries to do is create the best defense by making a stronger offense. What does that mean? Well, it means really gathering up the strengths of your immune system, so when a cell, one of your cells decides to turn into a cancer cell, decides to mutate, your immune system is already ready to take it on. What happens if you've already got cancer? Well, what it does different types of immunotherapy, such as adoptive T-cell. What that will do is look at your white blood cells, specifically T-cells, which signal which cells for your immune system to kill, aka cancer cells. The adoptive T-cell type of immunotherapy will take the best T-cells, which ones are reducing tumor size the most, and amplify them. Thus in turn further reducing tumor size. And that's just one example of immunotherapy. There's a few more, such as cancer vaccines, which would work similar to how you would get a vaccine for any other, any other disease. You might get a vaccine for the flu or something else. So essentially what that does is it prepares your immune system so that in the future, if you were going to get some sort of cancerous cell, your immune system's already ready. And that's one of the spillover effects for whole genome sequencing. Is if once we've sequenced the genome of Sally, we can go, okay, Sally, you're likely in the next 3 to 5 years, based on what we've seen in other people, develop this type of cancer. We can say, why don't we start with some immunotherapy now to prevent you from getting cancer later down the track. That isn't an— that isn't our initial goal to do. But it's certainly something that, that may come out of a project like what we're working on down the track. Why immunotherapy? Well, first of all, as I said at the start, to do good AI work, you need good data. And QIMR, who is one of our partners— it's a big research firm in Brisbane— is a world leader in immunotherapy research. So, goes hand in hand. They have access to some of the best quality data on immunotherapy, and if we want to do quality AI solutions, we have to take what they've got. And of course, as the project goes on, we'll start to gather insights on what data works best. Maybe we need some more of this, and then in the future, design trials around that and see if we can further improve the data collection. Because at the end of the day, it's the collaboration factor that matters most. Immunotherapy, also one of the major things about it, it's starting to become the fourth pillar of cancer treatment alongside surgery, radiation, chemotherapy. Why is that really good? Because it has a potential for a universal long-term treatment. Now, what does that mean? So cancer, you can imagine, there's several different types— lung, it all over your body, there can be different types of cancer cells. and radiation, chemotherapy, surgery work great on some but not others. However, immunotherapy, because it works strengthening the body's existing immune system, that means that for any type that comes along, the immune system's going to try and fight it regardless. So no matter what type of cancer you get in the future, immunotherapy may be able to, to fight it because it's working off your already existing immune system. And now what this is called is it essentially teaches your immune cells how to defend from future cancer cells, and that's known as immunomemory. So, we've gathered the data. How do we prepare it? Now this is a really good question, because you've heard of big data. Well, a whole genome might as well be considered as huge data. Because the 3.2 billion base pairs inside of us, the A's, C's, G's, T's, if you line them all up together, put them in memory, computer memory that is, that's about 300 gigabytes for one genome. And we're talking, if we want to scale this out, we're talking thousands of people per year diagnosed with cancer in Australia alone, not to mention, and we're a small country, as we said before, compared to compared to China's population, where we, we have population of one city, even less. So it's a very big computational problem as well. And sequence multiple times— what does this mean? Well, if we sequence your one genome to really get an accurate measure of the patterns and of the A's, C's, G's, T's, what combination it is, and if you have a cancer cell has its own combination of A's, C's, G's, and T's, we might have to sequence that multiple times to get an accurate reading. And so that can, that can lead to almost a terabyte of data per one person. So if you have 1,000 people, that's 100 petabytes. It's crazy amounts of data. And then the next question is, how do you interpret meaning from it? And at the moment, we'd love to have an answer in depth, but as we said, the This project is new to us, and so this is something we'll be definitely exploring in depth in the next few weeks, next few months, and sharing what we learn as much as possible online on MaxKelson.com. And then once you do interpret meaning from it, how do you explain this meaning to others? So that comes back to making it useful. How do people understand it? Because as As I said before, if we figure out, okay, 300 gigabytes, we found some patterns, and we can get it really accurate to matching certain patterns in certain people, how do we explain that to someone else? Because it doesn't mean— just because us as engineers, as AI engineers, we've developed a model that lives on the cloud or something like that, and it can get X amount of accuracy, 89% accuracy matching cancer differences to your individual genome, that doesn't mean anything to a patient who's sitting in hospital with the disease. Doesn't mean anything to a doctor who's diagnosing someone, who's having to plan a treatment plan for someone with cancer. So that's a really important point, is not only processing and preparing it, getting it in an explainable way to all the parties involved. And then of course utilizing So, the real value comes from the crossover of data sources. So, not just taking genomics data, because that's, that'll be one source of information. But the best AI solution is taking multiple sources and combine them and figure out some sort of solution. So, that's where it'll come in as genomics data, which is what we'll be working on first, and then clinical information. Such as when you go to a doctor, you get diagnosed, the first diagnosis, how often are you receiving treatments, what are the dosages of treatments. An interesting article the other day, actually got shared the other day, from MIT, using reinforcement learning to figure out the ideal dosage for radiation or chemotherapy for cancer patients as well. So, rather than just giving the patient as much radiation as possible, the algorithm was able to figure out, okay, maybe we shouldn't treat them once a week, we should treat them once a month and give them this amount of dosage, which resulted in the same amount of tumor, tumor reduction in tumor growth. And because these treatments are damaging, radiation and chemotherapy, it's not, it's not nice to have. So Stepping up the treatment and just making it as high as possible wasn't a valid solution either because it would dramatically deteriorate the person's quality of life. So the balance is what's the maximum you can reduce the size of the tumor while also maintaining a person's quality of life as much as you can. So that's the real— that's where we want to head in the future. Why does immunotherapy work on some patients and not others? That's the key question we want to solve. Getting to market. So, as I said before, the solution, if it's not in clinics, if it's not being used by doctors, it's no value to anyone. Just because we've built a great model and it lives online and we can see some awesome results and build some awesome graphs around it and say, "Hey, look at this arrow, it goes up like that," it doesn't mean anything. Unless it's bringing real value to people. And at the end of the day, that's the goal, right? It's tool versus treatment. What we'll be building is a tool rather than a treatment. So to go along, the treatment is immunotherapy, the treatment is radiation, the treatment is chemotherapy. All we're doing is building a tool, not to replace the treatment, not to replace the doctor, but to augment the doctor's workflow so they have some idea of what they can use to make their treatments better. Why us? Well, this is where the most value comes in, as I said before, as I've been preaching throughout the talk. It's in the crossover. So any one entity are all doing great work. So Max Kettelsen, QIMR, BGI and Genomica working with the whole genomes and sequencing, but where the real value comes from is this circle in here. So the collaboration. So that's what it's going to require in the future, not just for our project, but for any AI project, any project in business, any project in healthcare. It's a collaboration between the research here, the health in terms of knowledge and already existing know-how, and then the tech. So the tech and business world. Because at the end of the day, any one of these could produce awesome work, but when the real value comes from the collaboration. And of course, with anything, there's going to be some roadblocks. Data quality is a big one, as I mentioned before. If you don't have good quality data at the start, it's going to be really hard to get good quality outputs. Privacy— genome sequencing, someone's genome is about as private as you can get because that's That's someone's instruction manual, right? You know someone from the inside out. There's— that's going to be a big discussion to how these things play out. Time, of course. Ideally, these solutions, these diagnostic tools would be— there'd be no time involved. It'd be, yep, we found a good solution, we'll get it to the market. But in Australia, there's of course regulatory systems that you have to go through, as I imagine in any other country. So that's a big roadblock, is just literally time and gathering up evidence of how good such a treatment can be, or diagnostic tool, sorry. The technical challenge, which is what we're probably going to spend the most time with in this first phase, is working out how do we process such large files. Do we take the whole thing and not miss out a single piece of data, or there are other smaller versions of it But then again, work on smaller versions has been done before. What's most exciting to us is taking in the whole genome sequence, which is still a relatively new field. Well, actually really new. And then the final one is getting to market. If we're able to, if we're successful in building a diagnostic tool, how do we bring that into clinics? How do we get that into patients' hands? How do we help out someone like John who doesn't know whose doctors don't know which treatment will work best for him and figure out, okay, John, immunotherapy is probably not the best option for you, but we can take this. Similarly for Sally, immunotherapy might be the best way instead of what she's already doing with radiation and chemotherapy. I'll throw back to Athan to— oh, actually, does anyone have any questions about what I just went through? Genomics? Immunotherapy, healthcare, AI.

I have a question. Actually, for noise data, usually they are quite high-dimensional with quite small sample size. So how to overcome it?

How to what?

Sorry.

I mean, usually the noise data are high-resolution data with quite small sample size.

Yeah.

So how to overcome it?

That's a good question.

So I can touch on some of the strategies we're looking at. So we have a strong applied experience in character-level language models. And so what we are proposing to do, at least from the outset, is to use recurrent character-level language models to learn sequences of A, C, T, and G. We are considering just using that 4-letter alphabet, but also exploring different encoding strategies to take into account various mutations like single nucleotide polymorphisms and flips in genomic data. So, but that's one of the strategies we'll be looking at is using recurrent models. And that allows us to design a— well, if you're at all familiar with recurrent neural networks, they take in an input. They have a hidden state that is constantly constantly evolving throughout time, and they're constantly outputting a best guess of what might be coming next. And in the process of updating the difference between their best guess and the actual result, you get a semi-supervised model, which is actually basically unsupervised because you can always provide the next character, and you get a hidden model that is evolving across time. That's the initial experiments that we're going to be doing. Yeah. Sorry, so that is actually a dimensionality reduction strategy. So what you'll have is, for instance, you might have 1,024 hidden neurons in that, in that hidden state of the recurrent neural network. And those are actually the way that you can encode any particular sequence that would be coming in, and you'll end up with a 1024-dimensional input sequence, which is significantly smaller than obviously an entire sequence.

How do you combine the genomic data and the clinical information?

So actually, the clinical information, at least from the outset, will mostly be our supervising signal. So it'll more be a matter of of information about a person and looking at how their genome is actually reflective of that and which elements of the genome have the strongest contribution to that particular piece of clinical information. So, we may explore integrating the clinical information as a training set, but I think from the outset, what we're most interested in is explaining whole genome contribution for a number of these clinical factors. Thank you.

Awesome. So, I'll throw it back to Athan now, who's going to go over a few case studies of projects we've worked on, and as always, feel free to ask any questions.

Do you want—? Yep.

Yeah.

Actually, I'm going to jump back behind here because I haven't got my glasses, so— Cool. So I think we've got 3 case studies that I am going to run over with you guys. These are works we've done over the last year and a bit with our clients in Brisbane. So first one we've got Brisbane City Council. So this is the regional council for Brisbane. They are interested in tracking developments across Brisbane and Queensland. More broadly. So what we developed was an image classification pipeline to predict and classify which type of land a particular lot might be. So we were provided with a very large list of lots across Queensland. So basically a series of boundaries on pieces of land. And our goal is to map out the development of that land across time. And as a process, obviously, we need a system that can be classifying that land at any point. So, um, We combine satellite imagery with our image recognition models to detect changes across time. It can also provide pre-construction indicators, such as land being cleared or the foundations being laid on site. So this model is currently in use by Brisbane City Council and also by property insight startup CityShape. So, some of the challenges we encountered in developing this model. When we were extracting imagery from our mapping provider, we found that we were only able to designate a particular bounding box, but that bounding box is oriented in a latitude-longitudinal plane. And so there's no rotation there. So the first thing we found was that either we would be providing a bounding box that would chop off a portion of the lot, which is happening— oh, sorry. Yeah, a bounding box that would chop off a section of the lot, which you can see in this picture, or a bounding box that also is happening in this image is capturing the neighbour's property. So this was quite an interesting engineering challenge. So what we ended up doing is designing an automated image rotation pipeline which uses computer vision libraries through Python to recognize the orientation of the plot— oh, sorry, the orientation of the lot automatically will rotate the images in process and crop them as needed for training or inference in machine learning models. So we use this in our training pipeline to pull down the original queries and train our model. And also it's used in inference. So our model is frequently updated, but at present our pipeline is an inference pipeline. We can pull down as many properties as needed and get a recommendation then. —so a little more about the design there. So it's actually quite an interesting geometry problem. We couldn't find any— I should get you to update that every time I update that. We couldn't actually find any previous work on this, and so what you see there is the designated lot boundaries are in orange, which is the middle rectangles. It's probably better designated when you've got more complex lots. So the lot boundaries are designated in orange, and you've then got— I think any computer scientists in the room can probably tell me what the blue polygon is. Basically, if you put an elastic band around a polygon, it's what you would see. So, and from there, we allowed 5% margin on, on that polygon, and that would have a certain rotation in the latitudinal-longitudinal plane. We've got the bounding box, which you would typically design for that property. And then you've got the necessary bounding box, which is the one we have to make a query with, which is the bounding box around the entire rotated frame so that we could then have our pipeline detecting rotation, rotating the image, and then on to training in our pipeline. So I guess that's a strong example of the fact that systems aren't just ready to go for applying some, you know, image recognition problem. There's quite a lot of thought needs to go into developing a robust pipeline because from the outset we tried this and realized that we were pretty quickly detecting neighbors' properties and things like that. So, yeah, that's a custom solution. Second one I'll probably go into a bit more depth on. So this was with a technical group in Brisbane called Nerve Solutions, who are a subsidiary of Shine Lawyers, which are a large law firm in Australia that deal with personal injury disputes. So, business case. We partnered with this law firm to implement solutions to aid in determination of whether a firm should take on a case. We used a number of metrics to assess this without going into— there are still certain privacy concerns around this because it's still in action, but things like the estimated return for a given client and the length of the case would be the kind of metrics that you could imagine. This included processing and understanding supporting documentation within a case. A large part of this— I mean, extending back to beyond 5 to 7 years ago, we had scans of documents. So a part of our pipeline was a character recognition processing system, as well as natural language understanding. So we work through medical reports, case notes, a large amount of supporting documentation, and used entity— named entity extraction to build a knowledge domain and use that to create our structured knowledge corpus. But in addition, we had over 100 other known variables about the case in a much more structured database kind of format. And the goals were, as I've described, making a prediction of the likelihood of success, the overall quantum, which is the amount received for a given case, and the time to resolution. The challenges encountered. So the structured data in the CRM was not of a sufficient quality or quantity. So we have a large number of features and not a huge number of examples. So I'll go into deep dive in a minute, but there was— across the entire structured dataset, we were seeing 10,000 examples, but what we were actually finding was a great deal of sparsity in that data. And so there are a lot of interesting kind of techniques we had to get into there, and I'll get into the nitty-gritty of that in a sec. We also had to find a way to improve— yeah, improve the quality of the data frame and also how we could extract more robust features using the unstructured documents and using the language techniques I described. So I'm just going to go into a bunch of the challenges we encountered and some of the lesser hyped, lesser known machine learning strategies that we have to employ very regularly. And again, this is not something you're going to see with your classic image recognition or even natural language problem. This is more like the data science techniques you're going to see on rows and columns data with a certain number of features and a certain number of provided examples and just trying to do your best with what's available there. So one is selection, data selection. Two is imputation, which is filling the gaps in the data that we have at hand, encoding strategies on features, namely categorical features, because at some point if you're using any kind of machine learning model, it has to— those features have to enter the model as numbers. And obviously our learning strategies there, which we came up with some of our own interesting custom solutions there as well. So, selection, making the most of the difficult data that we had available. So, like I say, in the structured data, we had over 200 features. Says 15,000 rows there, but really at least 5,000 of it was beyond any use, and we actually ended up trimming down to a great deal more. A great deal less, sorry, with varying levels of quality across time. So one of our initial experiments— one of our initial experiments was a sparsity analysis on the data. So across the x-axis, we've got the features broken out and basically sorted by how populated those features were in the DataFrame. And you can see that overall, more than half of the acquired data is actually missing. And, um, I guess our goal, um, as data engineers is to try and find the situation where the most possible available information can be used in that data without introducing so much bias that we can't be getting value from the system or just confusing the model because there's not enough available there. So I guess that comes down to cleaning and consolidating, and there's— at least at this phase, there's a large human element there. So based on conversations with our subject matter experts, lawyers, and others involved, we removed the features that were misaligned with our prediction goals and those that weren't well populated enough. The other thing to do is to consolidate categorical features. Which ended up being a very big job. So if you have a doctor that is involved in a case and the case is across Queensland, it would be reasonable to suggest that if the doctor's name is Thomas Harold or whatever— I've made that one up for privacy reasons, sure— but there's only going to be one or two of those, ideally probably just going to be one in Queensland. So if you have 1,000 doctors across Queensland and they're spread across 5,000 cases, that's going to be a useful thing. But if that doctor's name is spelled 17 different ways because the data recording pipeline at the source was not designed well, which is something we encounter every day, right, is that people have data recording systems, but they've not been taught the value of data and the value of defining things and having autofills and things like that so that if you're filling out someone's name, it's always the same way. So that's quite a challenge, right? So we had to— so I guess going back to the doctor's name example, you might have, yeah, Dr. John Smith or lowercase dr or just Smith, John or even just John if they're feeling exceptionally lazy, in which case there's not a whole lot you can do with that and we probably want to throw it out. So we used Levenshtein distance and MetaPhone, which are kind of old-school fuzzy matching algorithms that work well on text, and a whole lot of regex, which I imagine you guys would be familiar with. So, yeah, so there's a case in point, right? In the first case, you might— that's probably the ideal situation and we saw a lot of those. So you might use a simple regex filter to identify all the names that fit that pattern. For the third one from the top as well, regex would be another one where it would make sense. For mis-capitalizations or misspellings, the Levenshtein distance is a very good metric because it just measures the overall difference from— between different examples and can very rapidly help in consolidating features, categorical features. And I guess The second last one is potentially ambiguous. So this is what further complicates the issue is there can be people whose name is spelled just J-O-N. And so you kind of need to check, are there other instances of that doctor? If there's— if that's the only one found on its own, then perhaps there is just a misspelling there and meta-phone might be a better fit. and we would want to add that to the instances of Dr. John Smith, as spelled at the top there. And I guess the last one is there's really not a lot you can do with that, because in Australia there's a lot of people with the surname Smith. So the next one there is imputation. So as I described, what we're doing is filling the gaps in the data. So you could take the food analogy. I think when I was making this, I was quite hungry, so there's a few food analogies throughout this, the slides. But we, we want to start with the Swiss cheese on one side and finish with the nice Gouda on on the right. The thing about imputation though is in almost any circumstance you're going to be introducing some degree of bias. In our reading on trying to resolve this, we came across a few different strategies. The simplest strategy was to use some kind of statistical measure purely to fill the gaps. So you look at— for a given feature, you look at all of the populated instances of that feature, and for all of the instances that are not populated, you simply use the mean. The hope there is that for down the track when you're doing machine learning, those are effectively going to disappear in terms of the way they contribute to the space. It's not 100% the way things work and it's highly dependent on which kind of models you're using, especially if you were going to be using still highly favored techniques like random forests or XGBoost or decision trees. You can get all kinds of different behavior there. One— another metric— another method which we ended up favoring was to use a decision tree regression to impute the unpopulated values there. So, what that did was learn how to bucketize features in order to minimize prediction error, impute every value for every feature as the mean value of the bucket they fall into. Now, we could tune these buckets, so it's a lot more sophisticated than a very simple mean imputation across the entire dataset. We actually— we chose this method because we could start at a larger number— at fewer buckets and larger buckets and hone that down to basically a model that would calculate the most reasonable value to impute on for any given example of any given feature. The other interesting thing is that in the process of imputing features, as those are becoming more populated, they're informing the imputation on the remainder of features, which is a useful thing. So that's a more intelligent strategy than using mean, and it learns directly from the data. So now that we have, to the best of our ability, populated the sparsity in our data, we want to encode all of the categorical information we have into numbers which can be understood by a machine learning model. So we've got another food analogy here. It's pretty clear to see that things like fruits may be ascribed any number of metrics. They may have a certain weight which can have a numerical encoding. They may have a volume, or even a color could be an RGB piece of numerical information. But for a large amount of the information that we work with, we have just distinct sets of categories, and all we really are allowed to assume about them in the model is that they're different and that they're fundamentally different. And how different? Well, that's up to us to figure out, right? So this is what we call categorical features. All data points have to be eventually represented as a number. So mapping categories that have no inherent numerical value is challenging. And what we want to do is minimize the introduced bias in this process. So one fairly simple method would be to use a numeric encoding. So you might just have a group of people, different in many, many ways, and so according to the categorical model, we want to say that they are distinct categories. So within this group, you might just decide to assign each of them a number. Why is that a problem? Well, that's a problem when you're doing arithmetic or linear algebra on these numbers. I don't think you can fairly say it's true that Adam plus Alice equals Jian and Louis minus Gabriel equals Raphael. There's no ordinality, as you would say, to these features. It is useful when the character— when the categories have some ordinality. So for education level, if you were happy to say that there is an order there and that potentially they're equally separated in whatever space we're ascribing to them, then you could have high school is 1, undergrad is 2, postgrad is 3. There's still somewhat of an issue because You're then saying that high school plus undergrad equals postgrad, and postgrad equals 3 times high school. But there is at least some ordinality, so there's potential there for a numeric encoding. The strategy that is actually the safest bet in terms of encoding from the outset is one-hot encoding. So you ascribe every category within a feature. So that feature may be this group of people. You assign every one of them their own binary feature, and you then break that out so that each one of them is— what's the word? There's no intersection in the feature space. They are— they all have their own feature. So in this case, they can be categories with multiple values can easily be encoded. So the value there is if you have seen 3 or 4 different doctors, you can have interesting cross-sections where, I mean, it should be saying that these are potentially doctors' names applying to the analogy of this case. You're not just going to have 1, 0, 0, 0, 0, 0. You could have a number of 1s across the top there lit up. So, it reduces obviously the sparsity if you have that kind of information and definitely lends itself to that kind of application. And we found it very useful, yeah, when we were having across a group of 1,000 doctors, maybe you'd have across someone's case they might actually have seen, you know, half a dozen or a dozen different doctors. So there's a lot more information that's actually encoded in there, in the interactions. What we actually ended up going with was called impact encoding or likelihood encoding. So the pseudocode for this strategy is as follows. For every categorical feature in the dataset, we randomize the rows. We split rows into a number of equally sized folds. Calculate mean of the desired prediction feature. And for each unique category, calculate the grand mean of each fold. This allows for much more compact representation of the data and allows for one column per categorical feature and learns directly. Now, a common question that we would receive is why not to use a— something like a word vectorization strategy and learn a feature embedding. That way. What we found— this is another kind of engineering paradigm— is that in practice this statistical method actually for the most part outperforms the embedding. And we had a number of attempts with embedding encoding, but with the limited amount of data that was available and the particular dataset at hand, that's what worked best for in our application. The other one there is normalization. It's kind of the— yeah, the final step before learning is optionally you may want to normalize the spread of every given feature. So now that you've got your features in a fully imputed numerical array, you might like to normalize the spread. Normally you might set the mean to 0 or 1 and the standard deviation to 1. This is most beneficial in deep neural networks and linear models. For deep neural networks, it can drastically increase— or sorry, reduce training time, because otherwise you'll be spending a huge amount of time just trying to normalize those features. And yes, I can— it can— I've seen it increase training speeds by up to 90%. In tree models, you won't encounter this issue quite so much. So we used a number of different models, but there's no— generally we find there's no harm in normalization in practice. And going into some of the models and learning that we've worked with. So exploration, exploitation, and evolution. So in the end, what we have is a numeric array with a very limited number of examples and many features. So we need a model that is general enough that it can explore the feature space, yet focused enough to find the strongest approaches and exploit their capability as far as possible. So this is what we call the exploration versus exploitation problem that is seen very commonly in reinforcement learning. I'll touch on that near the end with our reinforcement learning work. One of the first libraries that we decided to work with was H2O.ai. They have a really good open source implementation. Also, it's very powerful and accessible AI designed for enterprise. So the architectures that they had available were DNNs, distributed random forests, generalized linear models, gradient boosting, stacked ensembles, and naive Bayes. And one cool thing that we discovered early in the year was their implementation of AutoML, which basically takes all of these architectures and— thanks a lot, I actually appreciate that. So, cheers. Takes all of these architectures in parallel and allows us to do a full hyperparameter search on different combinations of those architectures and their hyperparameters. So that's automatic machine learning. Performs a grid search on model architectures and hyperparameters. Now we were kind of shocked to find that it was merely a grid search, and I guess a little disappointed because it was the same offering as Google's AutoML. So I did a Cloud ML course last year, and we found out about AutoML then. And at the time it was purely a paid service, and to see it coming out on H2O was quite exciting. But that is all it is. It's a brute force search of every combination of hyperparameters and model architectures. And if you have a large set of hyperparameters that you want to explore, that's, sure, it's exhaustive, but also very time-consuming. So time-consuming and expensive, obviously, in compute. So we were running our models on the cloud and leaving them running over the weekend and often would be spending days or weeks training and searching for the optimal model. So once we'd done that initial proof of concept, we wanted to explore more efficient techniques. So we decided to look into XGBoost, which has performed very well in applied— in application, and is often the top-performing architecture in Kaggle. So we looked— so that's based on a paper called Greedy Function Approximation: A Gradient Boosting Machine from 2016, I think. And he also designed this library. So it's an ensemble of parallel gradient boosted decision trees. So I guess is building on top of the forest architecture. And yeah, it's highly, highly favored in the traditional rows and columns kind of machine learning problems. So we took what was already a very powerful architecture and decided to introduce our own strategy, which was a custom genetic algorithm to explore the hyperparameter space. So our evolutionary algorithm worked as follows. It's initialized randomly or with a heuristic. Every generation, we select a new genetic pool. Which takes the best-performing 25% of models, 25 randomly generated new models, and 50% is crossbred models, which allows interaction within the gene pool. In addition, there was some random variance introduced, which is not listed off here, but I think it was around 5 to 10% in this crossbreeding, which I guess is a— you could think of analogous to this— to simulating the random mutations that we have in our genetics. It says here convergence visible after 7 to 10 generations. There's no metric of absolute convergence, but this I guess is the difference between 10 generations and maybe 50 generations. We at least reached a local minima— or a local maxima, sorry— in performance. And we were seeing performance that was outweighing every best attempt we'd made with stacked models from H2O. So yeah, I guess obviously the perks of this is much more efficient than a grid search, and the space is explored intelligently via artificial natural selection. Now, one that's come out very recently and actually since we concluded our initial proof of concept on this project is the Neural Architecture Search. So we are quite excited to give this a run and I thought I'd drop it into our most recent slides and just share that we are working with this. So one open source implementation is called AutoKeras, and that's using— it sits on top of the TensorFlow library and implements an API fairly similar to Keras. And there's a PyTorch equivalent, which is ENAS, the Efficient Neural Architecture Search. And given that we work a lot in PyTorch lately, that's probably the one that I will be using most. The exciting thing about this neural architecture search is it doesn't just explore hyperparameter spaces for an architecture. It actually designs and explores the components of the architecture. So for a convolutional neural network or for an image model, that might be the number of layers, how many dropout layers sit in between, things like that. And here we have an example of an RNN cell, which is pretty exciting because we haven't seen a huge amount of change in recurrent models. And even looking forward to our work in genomics, we would be very excited to actually come up with a recurrent architecture that outperforms state-of-the-art for our given application. And it's already been shown that for both image and language models, new architectures, new alien architectures have been discovered that outperform these existing architectures. So I don't know if you— what, how are we going? 3:26. So that was quite a long presentation on our two case studies. Do you guys have any questions after those two?

What's your criteria for your search? For the search algorithm, do you search for different architectures or different pieces? What's your criteria to guide your search? Yep.

So in either the hyperparameter search or the neural architecture search, I guess the end criteria is going to be the performance of the model. So in our example, that was keeping everything else the same. So we've already performed feature engineering, so all of our features remain the same. So on the, on the exact same subset of data, how is a given model architecture performing in terms of its predictive performance. Now, the metric that we actually used was called MAPE, or Mean Absolute Percent Error, and you can think of that, I guess, just like a bullseye. The percent of the absolute value amount that it was within. So, for instance, in predicting the length of a case, if the case actually went for 100 days and you predicted that it went for either 90 days or 110, that would put the mean absolute percent error at 10%. So that was our metric, and we found— we actually had some really exciting results on that, which I've more included in this one. It's more just been about, I guess, business application. But our findings from our initial case study, we had the lawyers' predictions from the outset of the case, which was also at least an estimate of the length of the case, an estimate of how much the case was going to cost the firm, and an estimate of the quantum or the results from that legal case. What we found was that the mean absolute percent error of the lawyers' predictions at the start of the case through to the actual results was outperformed by the model at a rate of between 150% and 200%. So looking at that the other way, the mean absolute percent error was reduced or close to halved on a number of those metrics. And I think Our best performing one was about 2.25 times reduction in absolute percent error. So even in this proof of concept, we've built an automated model that is far outperforming a lawyer in their initial assessment of how a case will go.

The potential architecture and the parameters are limited. What's your philosophy behind your solution? I mean, the potential architecture and the parameters are limited, right?

That's right, yeah. What's your philosophy behind your solution? So in terms of the neural architecture search? Yeah, yeah, that's interesting. And because this— the way this is actually implemented uses reinforcement learning. So I don't know what your experience is, if you've read much about reinforcement learning, but one of the most— no, so I'll give a little demo on what reinforcement learning is. There's a bit at the end of the slides there. But I guess the general idea is to model the reward system of the brain. And you've got an agent that makes decisions within an environment. So it receives information about this simulated environment and has some internal decision-making process, and then makes actions, and then receives a feedback, which is like a reward. And there's a lot of evidence that shows that that feedback loop is actually paralleled a lot in nature and actually in like lab rats wanting to get sugar water or whatever. So the brain learns through reinforcement or an artificial model learns through reinforcement. But one of the biggest challenges, which I'll also go into in slides, is exploration— sorry. Exploration versus exploitation. Yeah. So it's the metaphor is the multi-armed bandit, where you might have a— in Australia, we have like a gambling machine, and you call it a one-armed bandit. So it's like a slot machine where you put money in, and you pull this thing, and you might win or you might lose. Or you might win or you might just not get any money out. But say that you had a slot machine with 1,000 different arms on it. You put your $100 in. You may do okay on one particular arm, but after you've done that a few times, maybe you want to explore the different arms, but— or maybe you want to exploit the current arm that you've got. And so that's a problem that humans face actually quite a lot. If you're at a restaurant, do you want to get the same meal that you always get that you know is really good, or do you want to try something new because there's 50 other things on the menu? So that's actually a really difficult problem in reinforcement learning. And yeah, it's— I kind of lean towards as much exploration as possible. It's something that's basically factored into every reinforcement learning algorithm today is taking this into consideration and getting the right balance between the two. In the era that we're in at the moment, companies like OpenAI and DeepMind, a large part of their amazing work is just huge amounts of computation and huge amounts of exploration. The Dota 2, the recent results in Dota 2 and on whatever, DeepMind StarCraft. So these are like huge, complex, continuous spaces where agents are fighting against each other. The only way they can do that is through simulating hundreds of years of real-time gameplay on thousands of parallel compute cores. So I don't know if that quite answers the question, but I guess our philosophy is to follow what appears to be working, and that is as much exploration as possible but also designing algorithms that exploit the local minima when they're found. Yeah. Do you try the things or follow Bourke or is it just— Yeah, so— For the legal work? Yeah, for the legal classifier or for this one? So AutoKeras is an API that sits on top of TensorFlow, so the backend is TensorFlow. Yeah, so AutoKeras, as far as I know, actually uses TensorFlow. A lot of our work we do in TensorFlow. Lately we've been favoring PyTorch. Python, which one do you like? I prefer PyTorch just as an overall— I mean, usually we either work with one or another on a project. We try, you know, if we've got some large complex project, we don't want to chop and change too much because the models can be quite hard to pull in and out. But yeah, we've been favoring PyTorch lately because the model architecture can be built dynamically, whereas TensorFlow you've got to kind of initialize everything from the outset, including the backpropagation. And it can be very tedious when debugging, and if you're designing any kind of custom architectures, TensorFlow can be annoying. If you are trying to scale the project and you're doing work on large numbers of GPUs or across a large number of cores, or even, for instance, wanting to work with— there's Facebook's very large machine learning scaling platform for multiple CPUs. Anyway, for deployment, we still would probably defer to TensorFlow because there's— I guess the Google team has done a huge amount of work in optimizing for scaling and for GPU performance. So TensorFlow in the end is— a large part of it is built in C++ and there's a lot more work into the optimization there. But yeah, obviously for R&D, PyTorch is a lot faster. Another question, please go to the Q&A. Yeah, for the NLP process, uh, did you use supervised method or unsupervised method? Yeah, so I didn't go into this one too much because we actually, for this particular example, we used the Watson Natural Language Understanding API. So as far as I know, the way Watson's Natural Language Understanding works is, yeah, through named entity recognition. Sorry? So, sorry, yes. So we built, we made data dictionaries. So it's more like going through with a highlighter and yes, so labeling doctor or injury or certain things like that. So what you kind of get is is a dictionary of terms and the ones that would map to them. But the interesting thing is once you've provided a few examples, the whole value of the NLU is it can then extrapolate and learn a whole lot more from that. And if you use custom word vectors— so if you vectorize a word, right, any word, you can view it in a high-dimensional space in terms of where it sits as compared to other words. And so, as far as I know, Watson is actually sitting on a very complex word vectorization model which they have, you know, put a lot of R&D into. And that is proprietary and that's the way that they can then— if we're providing a dictionary, they can have very high performance on predicting words that are out of corpus in terms of what we've provided. So if there's new things, it can quite easily recognize where they must sit in the feature space as compared to the ones we've provided. I'm not also quite understanding this. Case. What does it mean to aid the determination of whether the firm should take on a case? Is it to help the judge to invite the firm to a case? So these are civil cases. I'm not sure. Do you collect previous law firms and learn something from other people and help the judge to do something? Yeah, so it's not a binary result. It's not just the judge, go to jail, or you're good. These are civil cases. So these are— they carry out over a long period of time, and this is an insurance company, right? So it's the progress of the claim, and that's going to be a continuous slider of possible outcomes. So everything from your case is not found to be strong enough and you have to pay all damages, and the worst possible case scenario, but it's usually going to be some kind of financial value and a time series value. And based on those two metrics, those are the two examples I've given. We actually have four. So there was the work in progress, which is the, the cost to the law firm. So what this is coming from the perspective of the law firm, right? So we had those for the previous cases. So the data I described, we have those supervised signals. So at the end of the day, it's a traditional supervised learning problem. But I guess the whole point of all of that stuff I went through was to show that it's not as simple as it might look from the outset in engineering. Involved in forming the semantic analysis? Sorry?

Semantic analysis involved in this case study?

Yeah, yeah, so all of the— so most of what I just went through was the structured part, but another big part of it was the semantic analysis in the unstructured reports and case notes. And we've actually got another big project going on at the moment with a state organization. So this was the first time— oh, sorry, this— in this case we use Watson. This time around we need a more custom solution, and we're actually using more custom libraries such as spaCy and Natural Language Toolkit from Microsoft. As well as custom word vectors and LDA as well, Latent Dirichlet Allocation, which is a clustering method for finding topics within data. So you can learn an unsupervised set of topics with— across a corpus of of documents, which is also really handy. Case space may be come out for this project. Sorry? Case space. Case space? Yeah, case space may come out for this project. What do you mean by case space? Case space. Because I think you may— inference based on prior case study? I'm not sure what the question is. I think let's just start with maybe have many prior cases written by text. Yeah. So you may Do you have a test base for— Oh, yeah, yeah, yeah. So a lot of the value that we found was through the natural language understanding part. So that was a large portion. Another part of explainability in this was the model that we built, we did a full feature contribution analysis. At the end of the day, once we've incorporated all our unsupervised structure that we've found through semantic analysis, we get features for that as well, right? So in the example of the doctors' names, those were scraped directly from case notes, right? Those are from medical notes. We're identifying those and running them through that pipeline. So Yeah, in a large amount of the value there has come from the cases, or as you say, yeah, the documentation that comes with it. Yeah, I guess we just didn't go into too much depth because we had used the Watson solution in this example, but we're definitely trying to pull those custom parts into our pipeline as well. Yeah, and this is a problem that we see across a lot of the examples that we work with. Could you give us some examples of the variables you add to the system? Yep, see if I've got anything handy. Just an example. Alright, so demographics. Would be one. So information about who the defendant's lawyer was and who the complainant's lawyer was, information about where the event had taken place. We did all kinds of interesting encodings there. So we did a geographic encoding and also just a categorical encoding separating them. Information about— well, because this is personal liabilities, informations about injuries that occurred, severity of injuries. So we did our own mapping for both injury location, which is one way to represent the severity of an injury. So if the injury is to the head, it's probably worse than if the injury is to the wrist. And then also the injury description. So we scraped and parsed all of these things out of the medical reports. And did a lot of our own mapping to build these categorical features, which we then put through the deep learning model. Well, in this case, XGBoost, not technically a deep learning model, but it's a very highly performing one. Is that sufficient? Yeah. I also don't understand, very quite understand the Auto-CARES workflow. Yeah, it's pretty interesting. So the system means that the search, the neural network architecture space, find optimization architecture for the input data. Yeah, so I guess you could think of every one of these as building blocks of a recurrent neural network. I think this example is a recurrent neural network. You've got the input feature at a given time t and the hidden feature at the time beforehand, and based on those, you're trying to predict what the next hidden state is going to look like. But all this stuff in the middle is a chosen architecture. So the idea behind neural architecture search is to actually explore that space using the most efficient way possible. And currently the best implementation for these kind of decision-making systems is reinforcement learning. Okay. Yeah. First step, have an initial architecture. Yep. And do some evaluation. For such architecture? And use reinforcement learning to adjust the architecture? Yeah, maybe I'll jump into reinforcement learning because we probably don't have time. We've only got about 12 minutes, I think. So maybe I'll jump right down to the end with reinforcement learning. I think that was probably the one slide which is— well, this is introducing what we're calling MK Research. So this is for the slightly more exploratory and new ideas. What we've found is that in a lot of the challenges we're finding with working with clients, actually the best way to go is from looking at things from first principles and saying, well, this paper came out 2 weeks ago. It's 10— it performs 10% better than any other implementation we've found. There'll be some code available soon, or we can start working on this code. Let's contribute to the open source community and we might as well work on research-level stuff, given that we're already operating at that level anyway. So we are— we have different levels of partnerships with each of these organizations, and we are really trying to find a strong grounding in research and are looking forward to collaborating and releasing work and papers in the coming months. So yeah, I guess our biggest areas of interest— I've already gone on about reinforcement learning, and I'll give a bit more of a description of that. Along with that is autoencoder architectures, natural language processing, healthcare, obviously, with the genomics work that we're doing. And I guess we've dropped quantum computing in there. We do have a postgrad in physics who's very interested in quantum computing, and it'd be pretty cool to do a proof of concept deep neural network with a quantum computer. And given that they're actually available to use now, you might see that from us in the future. So, there's just a bit more insight into the genomics project. One thing that Dan didn't mention is we've— the entire value of the project is $9.1 million, and we've secured $2.6 million of government funding through the CRCP grant. And I think— I think that's the majority of new information there. So yeah, quick refresher on reinforcement learning. So like I said, this borrows a lot from what kind of intuition you might have about the way we operate, and or even other animals operate in the well, you've got an agent that is taking in observations through sensory perception within some defined environment, is choosing from a set of possible actions, whether continuous or discrete, in that environment, which influence what occurs in the environment and hence influence the perceived reward that comes back to that agent. So this is a very exciting field of research. This has been around for— I'm sure you guys would be familiar— probably 15, 20 years. But I guess like the rest of machine learning, it's really coming back into the fold with the large amount of compute and interest that's going on lately. And is at the heart of the really exciting results we've been seeing from OpenAI and DeepMind and other organizations like that. So in terms of business applications, we've not actually seen a whole lot of reinforcement learning applied in business just yet, but I think it's it's right on the edge where it's— we're about to see a whole lot more. So this kind of breaks down problems in analytics as descriptive analytics, where you are describing what events that might have occurred in hindsight. Diagnostic or predictive analytics, which is giving some foresight and is probably closest to the case studies that I just described. And the, the next step there, which happens to be the most challenging but also the most valuable, is prescriptive analytics, where you can actually prescribe what should be done in the future based on everything that's happened in the past. You can learn the space, and you can choose the actions to take that will maximize the possible reward. So I'll break that out a bit further in the next slide, which— so those are the 3 questions: what happened and why, so reflecting, what might happen in future, and what should we do about what might happen. Broken out a few different models there. Descriptive analytics kind of can be really important in either of those other two phases as well. That's the whole thing is you don't want systems that are black boxes that are just making decisions without any description of why they might be having that thought process. So explainable models are really important, and two very interesting libraries there are LIME and SHAP. Both of those are only, I think, well, the release for SHAP, which we've been using in production in the last few weeks actually, is only a couple of months old at best. And LIME is a paper from, I think, 2017, which is Locally Interpretable Model-Agnostic Explanations, which which means that it can be— it can apply— it can explain the reasoning of a model even for nonlinear deep learning based models. It's model agnostic. But obviously, yeah, reinforcement learning is what we're talking about. It's identifying the optimal policies within a simulated environment. So I've already, I guess, touched on the reward hypothesis. This has a lot of grounding in neuroscience and psychology. Reward systems as a feedback loop in animal brains have been shown as at least one layer of abstraction on how things are working. And have been shown to be a very powerful and general kind of system in making decisions in an environment. So again, I've— I think I went over the exploration versus exploitation problem. That's the— pretty much the exact analogy I gave. So it's one that you want to consider. And I don't know if you guys have seen this one, but it's only a couple of weeks old. Very, very cool. So that's a hand with 24 degrees of freedom, which I think that's about the same as the human hand, give or take. It's obviously actuators versus tendons and muscles. But the really cool thing about this is you can apply simulations in the real world if you make them robust enough. And that's another thing that we're finding a lot with applied business solutions. You see, you might see this in finance, or even in the case of potentially our legal predictor. If we wanted to go from predictive analytics to prescriptive and actually making decisions in the real world, what we want is a model that is robust to the real world. Now the way that OpenAI did that was through modeling thousands of parallel universes that aren't quite the same. So the physical constants in these universes might be a little bit different. Gravity might not be 9.90 meters per second squared, they might add 10% variance on either side. Friction coefficients, much, much the same. There might be variance introduced there. And that's inclusive in the vision model as well. And on top of that, using different shapes. So by simulating all of these possible things, They were actually very ready to go when you get all of the noise and chaos of real-world systems, noisy sensor readings or different shapes and things like that. So that's a pretty good example of the need for breadth of exploration, and the models that— the architectures we're using at the moment are actually pretty good at that. But you can see there's a huge amount of training required there. So just to get a reasonable understanding of the space and the ability to manipulate the block, there's over 100 years of training experience there for a single system. So we're coming up on 4 o'clock. There's a little bit more here on world models, but I could probably just direct you guys to this work. So we tried to take on this competition. This was a competition out of OpenAI to build a model that would play Sonic. Now there's a benchmark called PPO, which is the same algorithm that OpenAI just used for this hand articulation system, and that's been found to work really well and broadly across a lot of systems. One that we wanted to explore was quite different, and you see our results weren't all that thrilling. The agent's kind of just jumping up and down, so we encountered a lot of challenges there, but we kind of take the philosophy of for this kind of research-level stuff. We just want to butt our heads against really interesting new problems. So we tried to apply some more very new research, which is a world models paper, which I'd highly recommend looking up if you're interested in that kind of stuff. And that's by David Ha, who is currently with, I think, the Google Brain team. And also Dan's written a great blog post on our work. On attempting to play Sonic with this system. So, yeah, I mean, I'm probably happy to— so that's the other one is the Pommelman, and that's one that we're working on at the moment, and that's a multi-agent reinforcement learning where agents can either compete against each other and learn in the process or cooperate with each other and also learn in in the process. And a lot of these examples are seen in games, but there's also a huge amount of value. It's just a really great way to hone algorithms and general learning systems that can then be applied to much broader use cases. So we'll be at NIPS in Canada at the end of the year presenting our work. On Homer Man. And I think we probably— yeah, Dan's added a great summary there, and that is doing good AI work in industry is hard, and that's why it's worth doing. So we'll probably conclude there, and any more questions we're happy to answer. Interesting question. You did not answer— that's a question. For the network architecture searching, what are the actions and what are the rewards? So the action might be to— You mentioned you used reinforcement learning to learn network architecture, right? Yep. Then for this problem, what are the actions and what are the rewards?

So for autoencoders, an action might be to try one variant. Of a model architecture, so it might have 4 layers. Add a layer or more layers? Yep, yep. And then— Actions is unlimited?

No, they're not unlimited. For a given timestep, there'd only be a set of discrete possible choices. You can add one layer, you can add two layers, you can add— Yes, yeah, but for one timestep. So this is generally the way these models are done. Sorry, yeah, so I understand now the basis of of the question. The way that these reinforcement learning problems are described is usually a Markov chain. So in a given state, without any knowledge of what happens previously, you can make one step and end up in a new state. And that's the Markov decision process, and that's what this whole thing builds on top of. And on that basis, you have a discrete set of possible actions which you can choose from. And that could be 10 or it could be 100. In the case of that hand, it was 26 times a continuous space. So you can actually work in a continuous action space as well. So that's probably the best I can do. I mean, I can refer you obviously to the academic work We come from problem— we attack problems from a top-down approach of what's the best thing that's going to solve this in the quickest time possible that's going to do the job we're interested in. And so in an applied context, that's, I guess, where we're coming from. But more than happy to share the work there. I'm sure there's plenty of— plenty more information on implementation. Yeah. About the case study one, the city shape, how do you get the neighboring data? Oh, sorry? How do you get the neighbor data? Oh, yeah, yeah. Because you can find some construction area or some areas. Um, yeah, yeah, we actually started with a manually labeled dataset. So it's a supervised problem. We started with a manual dataset of only a couple of thousand, but by having the system learning in an online way— How do you get the ground truth? Yeah, manually. Manually? Yeah. The people who work in your company or some public or what? So yeah, this was through the council, so they had, I think, I think a couple of thousand manually labeled images. And we started with those and incrementally strengthened the system by finding the outliers, sending them back to them. And as we went, the system was getting stronger and stronger and making less and less mistakes until we got to a level of confidence where we were happy. We also get a confidence confidence score for each one of these. So part of our pipeline is to drop anything that falls below that confidence score because we'd rather have a section where we say, well, we're not going to make any confident decision on this than having false responses. So that's generally in application, that's what we find is we want to use some kind of confidence cutoff.

Sorry? Will you tell us about the difference between this model and CNN? I think it's just a stack of several— just a few kinds of activation functions, right? And it needs the pooling process?

The difference between this and a CNN? Convolutional neural network. Right, so this is a method for designing architectures of networks. A convolutional neural network has an overarching description of the way it works, so using convolutions, but then there's a whole bunch of different ways obviously that you can design a CNN, like how many layers, the size of the layers. You have— you probably have the inputs the input dimension would be constrained. You might want to constrain the size of the convolutional layer, but beyond that there's a whole bunch of things in the middle that you can play with. Like the ImageNet model has like 70 or something different layers. So I think this is originally a paper out of Google, probably from about a year and a half or 2 years ago. But the idea is, well, if we are manually through trial and error trying to design these really hugely complex architectures, how can we automate a search in that space? So it's not quite the same as the hyperparameter space. I guess you could call them hyperparameters, but this is actually not just the hyperparameters but the shape of that particular CNN, for instance.

How do you add the reinforcement learning to it?

Sorry?

How do you add reinforcement learning to it?

Yeah, so like I said in response to that question, we don't have PhDs. We are engineers who are applying these problems. But my general understanding is that according to the reinforcement learning paradigm, the agent is choosing from a set of possible actions. So in this example, this is an RNN cell, so recurrent network, was discovered through the neural architecture search, and it's decided all of these things. It might potentially decide to have another ReLU here. And so the set of possible actions, as far as I know, I think it actually progresses through from maybe from start to finish. But the full architecture, we can, if you want the paper, I can look it up for you now, but it'll be all described in a paper. Yeah. So the Neural Architecture Search is a single paper, and I think it came from Google. Yeah, it's about a year and a bit old. Yeah. Cheers. This is on the website, you can find in the code, you mean that open source implementation? So this one here? Yeah. Yeah. Yeah. AutoKeras and ENAS. It's available. This is not our research. This is just the state of the art.

So yeah. Something that we're looking at using.

Yeah. Yeah. Yeah.

Very interesting work. I've seen your work involving different aspects of data science and machine learning, but and different applications. So I just wonder how to choose the model with this different problem. How to choose your model with this different problem?

Yeah, so I think a lot of it just comes from experience. From the past experience, or what do you mean by experience? Through experience in applying solutions. So I guess whatever the client's problem is will inform some set of possible logical solutions. So you might say, well, it's an image model, so We want to just get the best image classifier we can, but then you might also recognize that you want to select— it might not just be a classifier. You might want to select a certain space within an image and do semantic segmentation or something like that.

So, I mean, generally the problem is going to inform the solution, but I guess Now I only just realized that right at the end the camera, my iPhone, died as it was recording. It went for 2 hours and 15, so the battery literally ran out. But that was virtually where the talk cut off just now. That was virtually the end of the video. If you've made it all the way through, I hope you enjoyed it. If you have any questions, feel free to leave a comment below and I'll answer them for you. Otherwise, I'll see you at the next talk.

Thought I'd let you get away, did you? Keep learning. Peace out, team.