Neural networks trained to estimate matrix rank from images, a tongue-in-cheek paper that humorously suggests deep learning could replace basic linear algebra. The researchers visualize matrices as color maps and test predictions on rank, products, and inverses.
Dear fellow scholars, this is two minute papers with this piece of work is not meant to be a highly useful application, only a tongue in cheek jab at the rising trend of trying to solve simple problems using deep learning without carefully examining the problem at hand. As always, we note that all intuitive explanations are wrong, but some are helpful, and the most precise way to express these thoughts can be done by using mathematics. However, we shall leave that to the textbooks, and we'll try to understand these concepts by floating about on the wings of intuition in mathematics, a matrix is a rectangular array in which we can store numbers and symbols. Matrices can be interpreted in many ways. For instance, we can think of them as transformations. Multiplying a matrix with a vector means applying this transform to the vector, such as scaling, rotation or shearing. The rank of a matrix can be intuitively explained in many ways. My favorite intuition is that the rank encodes the information content of the matrix. For instance, in an earlier work on separable subsurface scattering, we recognize that many of these matrices that encode light scattering inside translucent materials are of relatively low rank. This means that the information within is highly structured and it is not random noise. And from this low rank property follows that we can compress and represent this phenomenon using simpler data structures, leading to an extremely efficient algorithm to simulate light scattering within our skin. However, the main point is that finding out the rank of a large matrix is an expensive operation. It is also important to note that we can also visualize these matrices by mapping the numbers within to different colors. As a fun side note, the paper finds that the uglier the color scheme is, the better suited it is for learning this way. After computing the ranks of many matrices, we can create a lot of input images and output ranks for the neural network to learn on. After that, the goal is that we feed in an unknown matrix in the form of an image, and the network would have to guess what the rank is. It is almost like having an expert scientist unleash his intuition on such a matrix, much like a fun guessing game for intoxicated mathematicians. And the ultimate question, as always, is how does this knowledge learned by the neural network generalize? The results are decent, but not spectacular, but they also offer some insights as to which matrices have surprising ranks. We can also try computing the products of matrices, which intuitively translates to guessing the result after we have done one transformation after the other, like the output of scaling after a rotation operation. They also tried to compute the inverse of matrices, for which the intuition can be undoing the transformation if it is a rotation to a given direction, the inverse would be rotating back the exact same amount. Or if we scaled something up, then scaling it back down would be its inverse. Of course, these are not the only operations that we can do with matrices. We only use these for the sake of demonstration. The lead author states on his website this paper shows that linear algebra can be replaced with machine learning. End quote. Talk about being funny and tongue in cheek. Also, I have linked the website of David in the description box. He has a lot of great works and I am surely not doing him justice by of all those great works covering this one. Rufus von Wuffels, graduate of the prestigious Maddie Paws University, was the author of the paper, overlooking the entirety of the work and making sure that the quality of the results is impeccable. As future work, I would propose replacing the basic mathematical operators such as addition and multiplication by machine learning, except that it is already done and is hilariously fun and it even supports division by zero. Talk about the almighty powers of deep learning. Thanks for watching and for your generous support and I'll see you next time.
