AI Beats Radiologists at Pneumonia Detection | Two Minute Papers #214

Dear fellow scholars, this is two minute papers with. In this work, a 121 layer convolutional neural network is trained to recognize pneumonia and 13 different diseases. Pneumonia is an inflammatory lung condition that is responsible for a million hospitalizations and 50,000 deaths per year in the US alone. Such an algorithm requires a training set of formidable size to work properly. This means a bunch of input output pairs. In this case, one training sample is an input frontal X ray image of the chest and the outputs are annotations by experts who mark which of the 14 SOT diseases are present in this sample. So they say, like this image contains pneumonia here and this doesn't. This is not just a binary yes or no answer, but a more detailed heat map of possible regions that fit the diagnosis. The the training set used for this algorithm contained over 100,000 images of 30,000 patients. This is then given to the neural network and its task is to learn the properties of these diseases by itself. Then, after the learning process took place, previously unseen images are given to the algorithm and a set of radiologists. This is called a test set. And of course it is crucial that both the training and the test sets are reliable. If the training and test set is created by one expert radiologist, and then we again benchmark a neural network against a different randomly picked radiologist, that's not a very reliable process because each of the humans may be wrong in more than a few cases. Instead, the training and test set annotation data is created by asking multiple radiologists and taking a majority vote on their decisions. So now that the training and test data is reliable, we can properly benchmark a human with versus a neural network. And here's the result. This learning algorithm outperforms the average human radiologist. The performance was measured in a 2D space where sensitivity and specificity were the two interesting metrics. Sensitivity means the proportion of positive samples that were classified as positive, and specificity means the portion of negative samples that were classified as negative. The crosses mean the human doctors and as you can see, whichever radiologist we look at, even though they have very different forms, false positive and negative ratios, they are all located below the blue curve which denotes the results of the learning algorithm. This is a simple diagram, but if you think about what it actually means, this is an incredible application of machine intelligence. And now a word on limitations. It is noted that this was an isolated test. For instance, the radiologists were only given one image and usually when diagnosing someone, they know more about the history of the patient. That may further help their decisions. For instance, a history of a strong cough and high fever is highly useful supplementary information for humans when diagnosing someone who may have pneumonia beyond only the frontal view of the chest, it is also standard practice to use the lateral views as well if the results are inconclusive. These views are not available in this dataset and it is conjectured that it may sway the comparison towards humans. However, I'll note that this information may also benefit the AI just as much as the radiologists, and this seems like a suitable direction for future work. Finally, this is not the only algorithm for pneumonia detection and it has been compared to the state of the art for all 14 diseases and this new technique came out on top on all of them. Also have a look at the paper for details. Because training a 121 layer neural network requires some clever shenanigans as this was the case here too. It is really delightful to see that these learning algorithms can help diagnosing serious illnesses and provide higher quality healthcare to more and more people around the world, especially in places where access to expert radiologists is limited. Everyone needs to hear about this. If you wish to help us spreading the word and telling these incredible stories to even more people, please consider supporting us on Patreon. We also know that many of you are crazy for Bitcoin, so we also set up a Bitcoin address as well. Details are available in the video. Description thanks for watching and for your generous support and I see you next time.