Gilbert Strang explains singular value decomposition as the key tool in linear algebra: any matrix factors as rotation times diagonal stretch times rotation. SVD generalizes eigenvalues to rectangular matrices and captures the most important components in data.
So what concept or theorem in linear algebra or in math you find most beautiful? That gives you pause, that leaves you in awe.
Well, I'll stick with linear algebra here. I hope the viewer knows that really, mathematics is amazing. Amazing subject and deep, deep connections between ideas that didn't look connected. They turned out they were. But if we stick with linear algebra. So we have a matrix, that's like the basic thing, a rectangle of numbers and might be a rectangle of data. You're probably going to ask me later about data science, where often data comes in a matrix. You have, you know, maybe every column corresponds to a, to a drug and every row corresponds to a patient. And, and if the patient reacted favorably to the drug, then you put up some positive number in there anyway. Rectangle of, of numbers. A matrix is basic. So the big problem is to understand all those numbers. You got a big, big set of numbers, and what are the patterns, what's going on? And so one of the ways to break down that matrix into simple pieces is, uses something called singular values. And that's come on as fundamental in the last and certainly in my lifetime, eigenvalues. If you have viewers who've done engineering math or, or, or basic linear algebra, eigenvalues were in there, but those are restricted to square matrices, and data comes in rectangular matrices. So you got to take that, you got to take that next step. I'm, I'm always pushing math faculty. Get on, do, do, do it, do it. Singular values. So those are a way to break, to make, to find the important pieces of the matrix which add up to the whole matrix. So you're breaking a matrix into simple pieces, and the first piece is the most important part of the data. The second piece is the second most important part. And then often. So a data scientist will, if, if a data scientist can find those first and second pieces. Stop there. The rest of, of the data is probably round off, you know, experimental error maybe. So you're looking for the important part. Yeah.
So what do you find beautiful about singular values?
Well, yeah, I didn't give the theorem. So here's the, here's the idea. Singular values. Every matrix, every matrix, rectangular square, whatever, can be written as a product of three very simple special matrices. So that's the theorem. Every matrix can be written as a rotation times a stretch, which is just a matrix, a diagonal matrix, otherwise all zeros, except on the one diagonal. And then the third factor is another rotation. So rotation, stretch, rotation is the breakup of any matrix.
The structure of that, the ability that you can do that. What do you find appealing? What do you find beautiful about it?
Well, geometrically, as I freely admit, the action of a matrix is not so easy to visualize. But everybody can visualize a rotation. Take two dimensional space and just turn it around the. Around the center. Take three dimensional space. So a pilot has to know about, well, what are the three things? Yaw is one of them. I've forgotten all the three turns that a pilot makes. Up to 10 dimensions, you've got 10 ways to turn. But you can visualize a rotation, take the space and turn it, and you can visualize a stretch. So to break a matrix with all those numbers in it into something you can visualize. Rotate, Stretch. Rotate is pretty neat. Pretty neat.
That's pretty powerful.
Sa.
