Everything Arrays

Today I'm not going to be talking about, you know, with a few exceptions, new features in the Wolfram language. I'm going to be talking about a fundamental part of, of the Wolfram language and that is, that is the ideas of arrays. Arrays come in all types and say is in that previous picture is actually the Very Large Array in Socorro, New Mexico which is my, my home state. And that's not the array I'm going to be talking about today. So, so the arrays, and as referred to them in in Mathematica or the Wolfram language is is basically a regular arrangement of data that has a depth D with dimensions of length D and such that each element of the data can be accessed by a part indices. And these are the, this is sort of the fundamental contract that we have for an array. This includes vectors, matrices and higher ranked tensors. So in, in Wolfram for some array A we have the array depth gives the depth D, the dimensions gives the dimensions and the part, as long as you have B parts will give an element of the array. So I mean that's pretty basic. I think probably all of you know that, but it doesn't go to talk on without talking about definition. So let's take some examples and I'm gonna start as everybody who works with me know, I like to do really simple examples and then eventually get slightly a bit more complicated. So basic tests for an array and this is, this is gonna be useful in a lot of functions you write is whether it's array Q and it's either going to be, you know, true or false, a Q function will be true or false. And then we can also do the dimensions of those. So the dimensions of this first expression are just two because the last level is not consistent and here the level is consistent in length. And so dimensions of x2, I get that 3 level. And that's sort of the, that's kind of the idea of a difference between something that's an array and something that's not an array. Um, so in that second expert, everything, everything's accessible by a part with two indices. So I can, I can do that. And those parts just have to be bound by the dimensions and, and the part will always work. Um, you can also test arrayq is a little more useful and you can test the elements, you know, an element at a time. So you can test whether they're all integers. The answer is true, you can test the depth. The second argument here, which I had as a pattern can be the depth. And so this is testing basically It's a matrix. And vector queue and matrix Q are actually just, they're the same as array Q, but they just have specific depths, you know, coded in explicitly. So they're sort of syntactic sugar really. So matrix Q. Now here's the tricky thing about what you consider array, because here I'm considering elements as like, you know, fundamental types, namely, you know, integers or reals or you know, what are typically considered scalar. But you can actually trick array queue and say, okay, I want, I want to test whether that expression one is an array of depth one with anything could be an element, okay, and the answer is true. Um, or another test I could do is that I could say, okay, I want it to be of depth one, but I want the elements to be lists of integers which are positive, okay, of any length. And that's another test you can read. So you can actually have list type or array type elements of arrays themselves as long as you put in the test correctly. So sometimes that's. So if you have an array, you know, which is mostly an array, but the last level has different lengths, well, you can use this sort of thing to actually work with it quite effectively. I think that's a little known fact in, in the Wolfram language. Okay, so how do you construct arrays? Well, I'm gonna show a bunch of different ways during this talk. Here are a few very obvious ways. The obvious way is with the command array, okay, that creates an array and one I'll use one I use a lot for testing things or understanding how commands work and cause I always forget like the way transpose goes and things like that. That's just, just the mental block I have. So I, I create a symbolic array with indexed elements and then you can kind of see how, how things work. Another super useful array for, for constructing things and putting things together is a zero. So this is actually a, this is a zero, but it's a zero array. And, and types of zeros probably in a lot of stuff you've done don't seem very important. But as soon as you start using compilation and trying to get things to run faster, the shape and structure of a set of zeros becomes very important. So, so that's a really useful one. I can also make an array of arrays, okay? And so this is an array of that constant array. And if I do, you know, dimensions of that, then I get something which is depth five. Okay? So it's the two, two or the first two dimensions and then the two, three, four, the dimensions of this symbolic array. I put together before. And so you can kind of repeat. You can have a repeated array which in many cases can be, can. Can be useful for constructing code and doing things efficiently. Another thing I use all the time, especially since I wrote a lot of the random integer functions and stuff, so, so there's some of my favorite functions is just random integer, random real, random complex allows you to generate arrays of different, of different types for testing code. And so random. The random functions are a really good constructor of arrays. Now, there's lots of other constructors of ARR, but I'm not gonna go to it. But if you wanna construct an array which has different sorts of elements, sort of a very common one is table. Okay, so I can do. So this is kind of like that symbolic array I did before, but this is doing table. And I always get confused myself. This may be my little bit of dyslexia as to which index goes where and this table. So, so it's actually the outer index goes first and then the inner index. And another way of expressing this is as a nested table. So I make a table of list three and then that. And there can be in some cases some efficiency of using this table with the multiple indices because it's automatically compiled, but pretty rarely is the case that it makes a huge difference. The story which I'll tell you eventually in this talk as I get farther will be that, you know, if you can, if you can avoid using table to construct things or do operations, you're probably gonna be better off. So here's an example of a real table. Okay? This is the Chebyshev nodes, okay? So, and I've just done a. The fifth degree Chebyshev polynomial. And let's check. So we can check that these actually are nodes by doing the following. We could do another table that iterates over the elements and a table syntax that iterates over the elements of a vector. And it'll always treat this as a vector in table, even if it's actually a deeper array. So you'll get list elements is I just give the, the variable and then instead of giving limits, I just give the. The table. It's the. The list itself, okay? And sure enough, they come out to be exactly zero. Another way of making a, A an array which is often super useful is to use outer products. And in this case I'm, I'm doing a, you know, just the same thing and I can do a list plot of out. Now the trick is here that the dimensions of this are so the dimensions Of Chevy Nev. Nodes were six. Cause there's, you know, six. Six nodes to the. Actually I think I have an extra one that I didn't really need there that's repeated. So it's six, six. And then the, the inner dimension is this function while I'm creating a list. Okay. And so that makes an extra depth in the array. Um, and I really won't go into many, many uses to outer here. Now I wanna talk about where do. Where do arrays appear? Okay. And it turns out that arrays are basically. That's sort of the title of the talk. Arrays everywhere. Okay. And they're all over the place. So let's take this very large array image, okay. And I can do array queue of image data of image. So that the, the image data itself. So the fundamental data behind it is. And you can actually get some more information by looking at the image data. And it turns out. So it's. So the image itself is 577 by 1024 pixels. And for each pixel we have three color channels and an alpha channel and that's that four dimensions. The inner dimension is that alpha channel. And you can actually change how those dimensions affect by using the interleaving option of image. So let's look at another example. So here's our favorite spiky. And this doesn't look like an array at all. And it's true, it's not an array. But the actual data which is used to plot that and stored in the polyhedron data is indeed an array. So the coordinates themselves, which are the, the vertex coordinates you see are an array. Okay. And I can do, I can actually play with this a little bit. So I can get the, the maximum of the, the value of the coordinates. So this goes up to 2.73. Okay. And then the cells themselves are an array. Actually the first element of the cells and it turns out that, you know, head of cells is going to be polygon. So these. So it's, it's showing the actual cells in this graphics thing are, are polygons which give the faces. So what I'm going to do is I'm going to. I'm going to play a little trick on spicy with the spiky with these arrays by doing a, a transformation. So I'm gonna get the. So I made a mesh region of that which is gonna be mesh region spicy. I'm gonna get the cells. Okay, Now I'm gonna. Now what I'm gonna do. And here's another way of affecting a transformation on an array. So the coordinates in an array, what I'm saying is I'm gonna map each element of the coordinates. So this works on the first level. So each coordinate is gonna be mapped by the coordinate times its norm minus three. Okay? And so that's gonna, that's gonna change the shape of the coordinates. I'm gonna basically remake the mesh region with new coordinates and I get sort of a morphed a morph spiky by this, this operation. So this is the first example which I'm really doing an operation on an array and map is a really useful one to do operations on array. There are more and I'm gonna show you ways to do operations on array as I get farther in the talk, much, much faster. Okay, another example, this is a really simple one. I do an ND solve. So all of these things are scalar. The variable is scalar. But the actual way we store the data for this is as an array. And you can see each element, every third. Every other element of the interpolating function is an array. If you look at its input form, you can kind of see, well, the, this is the dimensions or the limits, that's an array, these are the coordinates. And these, this is actually a special way of storing the actual data. And so that it says packed and compact as possible. So again, if you look inside of data that's returned to you, typically there will be arrays and often you can transform those arrays to change the original object if you want. In some cases it's recommended, in some cases it's not super recommended, but. Okay, so now let's talk about how arrays are represented. And this is, this is probably the key topic that you'll get something out of this talk. Okay? So the fundamental representation is a list of lists of possibly lists of lists, et cetera, et cetera. So let's just take a matrix. This, the full form is list of list, list of list, okay. And if you go deeper, it's just nested that much deeper in list. Okay? And these are, these are each expressions, they're references to the elements. Okay, now there are other implicit and explicit representation. There's a packed array representation which is implicit. Ok? So that's sort of like, you know, the analogy for that is I've got a friend with, with a, with a, a VW Vanagon, okay? And it looks like a Vanagon, it looks like one of the original ones, but, but somebody put a Ford engine in it. Okay? So it actually goes 60 on the highway instead of like 45. So under the hood it's different, but it actually looks the same externally. And that's what a packed array is. Okay, now sparse arrays are explicitly different and we also have structure representations that I'll get into. And an example of that is quantity array. Other examples which we're starting to add in development are structured matrices such as permutation matrix, block diagonal matrix. And then we have a final type, which is numeric array. And this allows you to work with different types in the compiler or say for a GPU which might only support 32 bit floats instead of 64 bit floats. And I'll have an example that uses numeric array with the compiler eventually. Um, so let's look at packed array. So let's do range of 37. This is a constructor of only a vector, so it's yet another array construction. So this is, this is a packed array queue. So let's look at the byte count. How much memory does that take? Okay, now I'm gonna take another way of doing this. I'm gonna take table, okay, and the bike. If I just do that table, even though the data itself is the same. Okay. And if the byte count is, is almost three times more, and I can check that they're the same. Okay, so they're, they're just like. The look is the same. So to the system they're the same, but the actual storage underlying is. And you might ask, why doesn't table produce a packed array? Because everything's integer. Well, here's sort of a cryptic answer that I'll show as a plot. Okay, well, eventually the byte counter table goes way down. And what happens is above a certain size and that limit actually happens to be 250, we auto compile the table when possible. And so we use the compiler to generate table, and it not only generates it faster, but it generates a packed array out of it. And so there are a number of compile limits you can look at. If you look at compile options, you'll see a bunch of blah, blah, blah, compile length, including table compile length that give these limits. And those, those can be quite useful. We've tried to set defaults that are appropriate, but sometimes you may want to change those. Okay, so let's talk about packed array conversions. So this is developer two packed array converts to a packed array. So if I have a mix type, okay, then I. So, so if I take this UA and so it, it has a real element and an integer element. Well, it doesn't pack automatically, but you can specify coercion. So if I say developer Two packed array. I want to coerce to real. It will go ahead and coerce to real and generate a packed array. Ok, and that's next. And yeah, if I convert from packed array, then it's not a packed array anymore, it's ordinary representation. Okay, so let's take a bigger, bigger packed array and let's compare. So we have a packed array and an unpacked array and let's compare the timing. And I almost always use repeated timing to compare timings because absolute timing usually doesn't get enough information within the granularity. And so repeated timing is a much more effective way to test timing. Okay, so here I did an outer product and if I do it with the packed arrays, it takes 10 times less time than doing it with the unpacked arrays, just because of the way the operations work. And so if you can use packed arrays in operations like outer or other things, then you're gonna have a much more, much faster code and it gets even more extreme. So there are messages you can turn on, on and off to see if an operation has to unpack actually works. So let's see. So I've got. So unpacking gives this message because it's telling, it's unpacking. And also say if I take map of some arbitrary function F over that packed array. Well, there's no way. The packed arrays only work for fundamental machine types. Now F of 1 is not a fundamental machine type because it's sort of an arbitrary function. So it has to unpack. The messages are super useful, but they're gonna be irritating. So I'm gonna turn them off for now, going a little too slowly. So I'm gonna get to the next thing. So here's a sparse array and you can always. So so here we're talking about representations. Now this is explicit and it shows it as a sparse array. Okay, so the dense array, the dense version of this is the normal norm of the sparse arrays. This is huge. It's kind of unwieldy to look at. So I'm going to delete that. Okay, so I can do an operation. Now, the contract basically for representation, we say that, you know the, that the operation. So that the sparse array, it represents the array. So if you convert to normal, it would be the same, but the actual objects are not the same. So they're not same. Q. So what's the advantage of the sparse arrays? Well, again, it takes a lot less memory to use and there are many tasks that are super effective. Here's a linear algebra example. Again I use repeated timing, so the sparse array is very slightly faster. But if I do a bigger size, okay, with the same thing. Now this is a particularly simple case, so it's very advantageous to the sparse arithmetic. Now I'm getting, you know, like almost several hundred times faster with a sparse array. So, so 200 and 250 times faster using the sparse array. So for linear algebra tasks like, like Oliver wouldn't be able to do is find an element stuff if we didn't have sparse arrays and, and efficient solvers that work on those. Okay, so there's a, there's a representation contract and I'm explaining this for sparse, but this is general for these things that we call array representations. The idea is that normal of some operation of the sparse array is equal to the operation of normal of sparse arrays. Okay. Now occasionally I use equal versus the same because sometimes like with numerical stuff, the error, the algorithms are different, so the error may be slightly different, but typically this will be the case. Okay, and here's two examples of that. And so that's an important thing to keep in mind. So basically the operations are effectively the same. They may be done differently, they may be done more efficiently. Okay, so now structured representations, this is a relatively new thing. They're different representations based on structures. So, so again the structure, operation array, Q array, depth, dimensions all work. And again this same contract the operation normal. And when there's a special algorithm for operation, it's used. Otherwise the representation is typically converted to a different representation for which there is an algorithm. So let's look at an example. So let's construct a quantity array, ok? And this is just kind of an ordinary array underneath, but the quantities are separate so it's more efficient. Here's the normal representation. So it just looks like a list of meters and then seconds. So the big advantage of quant one, big advantage of quantity array, it's big memory savings. So it's basically a factor of more than 10 in memory savings. And then if there's a special algorithm, there's a huge time savings. Okay, so here's another example. So the quantity array takes, is 22 times faster. Now notice the quantity array thing comes out as a quantity array. So it has this more efficient storage versus if I do the operation on the ordinary array, I get the, the lists. So again it depends what you want, but if you look at these times, I'm gonna skip the permutation matrix since I'm, I'm running low on time. Um, now numeric array is, is another type so here I can use integer, 8 bit integers type. If I do normal, I get a packed array. Um, so this again gives you a memory savings. Now the trick about numeric arrays is they're not considered array representations. They're special and sort of interface to the compiler and things like GPUs. So we don't map and we actually give a, give a slightly useful message saying we don't do traversal here. Ok, so this is, this is a big part of, of, of working with arrays. So I'm gonna give a bunch of examples. So how does array arithmetic work? Okay, so if I take two matrices I made symbolic. If I add those, it does it element wise, okay? Here. And if I add a vector to that, it adds the elements of the vectors by row element wise, okay? Now this doesn't work, this case doesn't work with at length 4 because the way that the default arithmetic works is from the top down. So the dimensions have to match from the top down. Now we've added recently a new thing and I suggest you look up the documentation for this. There's some nice examples. So threaded. And what this does is this says, okay, I wanna work from the bottom level of the array up. And so it adds a V to each of the columns versus adding it to the rows here. So this threaded wrapper allows you to do operations which before were somewhat inconvenient. Okay, so the, the way this all works is through this attribute called listable. Okay. And I can set a function to have the listable attribute and it does that thing. It, it applies F to the elements. Okay. And again I can do, you know, threaded v4. It, it applies to the elements. Now there's a huge number of commands that are listable. Basically all the arithmetic, arg, you know, blah, blah, blah, blah, blah, and, and, and that's a useful list to look at if you want to get an idea of what. So let's look at this Chebyshev nodes example again. Okay, Now I've made them numerical so that this, there's even a fair comparison. Okay, this takes a little bit. So here I've done the Chebyshev nodes by using a table here. I've used. Yes, Oliver. No, this is it. I'm sorry to interrupt, but we're going. Oh yeah, I've got to, I've got to stop quickly. Okay, so let's, let's look at the. That's okay. I always, I always have too much. So the listable speed is tremendous. So you're getting factors of like 30 to 30 by using the listable operation versus the the other operation. And I'm sorry, I didn't have time to get to the rest of my talk. I started a couple of minutes late. But I, I do if you look at the notebook later, you can see that I've shown how the you can use a compiler to do operations that are not listable. So, so the summary is that arrays are about everywhere. There are many different representations of arrays which you know can be very useful. And using listable arithmetic on arrays is super efficient. So if you can write your operations so that you use arithmetic operations over whole arrays, your code's gonna run a whole lot faster. It's not always possible, and when it's not possible, often you can compile functions to make them much, much faster. And so, so thank you. And sorry I didn't get to the get to the whole thing.