Live CEOing Ep 700: Language Design Review of Video, Image, and Audio continued

Okay. Hello everyone. We are continuing our series of design reviews for the upcoming version 13.3 of Wolfram language. And this is a continuation of our review about sound and vision features. And I think we'd already covered quite a bit of this. We're going to come back to some of that. Where should I start here?

Speech computation in that audio section you're on.

Okay, okay. Do I care about this? I mean, who have we had requests for the. These Google services?

It's not just Google, it's Google and other ones. But the quality of synthesis and recognition is higher compared to what we're using now. And eventually we can see having our better version of it as well. But for the time being, allowing people to connect to those services seems reasonable.

Okay, but aren't we getting our own networks for these things?

It has just been approved by legal.

Okay, fine, so. So then we'll get our own network, right? You're. You're in process to produce that?

Yes, well, yes, it has been approved, is in the pipeline and we'll have to test it once it is imported. But the plan is that and we'll see if it works out. No reason to suspect they won't.

But.

I guess it might be nice to discuss do we, do we want to have these services now and continue supporting them? There is, I think four that we have identified and have some connection to.

If we have them, then why don't we have them? I mean, you know, if we've already built them, it's okay. I mean, we can always tell people nobody's going to be upset if we have our own built in service that works just fine. And we say, well, you don't now have to do this authentication. Nobody's going to be upset about that.

Okay.

Okay. These are multilingual. Yes.

Yes.

Depends on the service. Google, I think is.

Okay, okay, okay. Would we finally have SVG import? I thought we finally had that a few years ago.

We had that last version import as image. As a raster image we had. Now we're importing it in this version as vector graphics. There are some limitations. Part of it depends on front end. It's not huge though. A lot of SVGs come in just fine.

Okay. I mean, years ago we had issues with this, dealing with Flag images. Johan team was working on that. What's that? Remember that?

I remember that now that you mentioned.

Why don't you, why don't you point out to that team that we now have SVG vector import and see if they can, you know, I don't know, maybe they'll get more flags that way.

Okay, will do.

Okay, what is ranged data?

That's when a column can have a range of values. It's a new element that is being added. Now we consider anything out of that range that can be specified as an option. Out of range.

I don't understand. So you mean it can only go from one to a thousand or something?

Right. And then clip beyond that or with some option allow different behaviors.

Why do we want this?

It's a part of Excel specification and we upgraded our Excel import in the previous versions. This is one of one small leftover from that update.

Okay, question. I mean the whole issue of recognizing what part of the spreadsheet actually contains meaningful data, how are we doing on that?

Haven't. Well, I haven't really started on that. We see it as a collaboration between import, export and machine learning to understand what part is data and then move on with upgrade to semantic import. Neither of the two teams had the time to work on that, but I think it's a completely reasonable thing to tackle soonish.

Okay. Talking of sound, whatever has happened to the music stuff?

Also no bandwidth for that, but. But we really love it as much as you do. It's just a priority. Do we continue on video? Video has been what we had momentum on and we have stuck on that for the past few releases.

Yeah, I mean, I don't think you know this chess stuff. I'm not a big fan of.

It wasn't a whole time thing.

The disjoint resources.

You.

You can't.

Right.

Just because someone is good at chess doesn't mean they're good at music.

Yes, Roger. Then. What is this? I don't know. Okay, you want to demo this here.

You can run Chess Player. More of those other top sections I would say are brainstorming. The four symbols that Roger listed on the top are the four that we are suggesting going to 13, three.

Oh God. Okay, where do I start here?

Just a top. You really shouldn't necessarily look at any of those subsections unless we want to.

Think what's in the future. I want to see what it does.

Chess square, bracket, bracket and start playing.

Or go to the source page if you want to sort of see and then you can pick an example and edit from there.

I mean this is like. Like. I mean, I understand that chess is a thing, but it just seems, you know, what are we going to do? Play Dungeons and Dragons next?

Doom is the plan. Clearly.

Yeah, I think Dungeons. Dungeons and Dragons was next in our big plan.

This is weird. This is weird. Why can't I? Why can't I?

You're. You're white. You can.

You know, white starts always. That's one of the rules.

You should drag. And do something weird. Don't necessarily play well. I think this is. This is a rather good tool. Now, you see evaluation down there, you can undo if you don't like that move of yours.

It's magnificent. But these are not. I mean, I just don't understand the priority of this.

As I said. Yeah, go ahead, Roger.

It's not been a priority. This is completely passion project. Outside, kind of like seller automata or something. It's a passion project.

Wow.

A lot of it was also developed before they became projects with us.

Yeah. No, no, I mean, this is. Outside of. It hasn't been a priority. Stephen. Really?

Rafa was the one who started importing the chess formats in the previous or I have to say he had the development for importing chess formats. And then as soon as we committed, Jay from the docs team said, I have a chess player. And then. Since then, we have. Most of what Jay has done is cleaning up and pretty fine. This player that you had that you see.

How does this work in the cloud?

I haven't tested it. I don't know.

You got to do that. You got to.

I'll test it.

Don't do that. I mean, I don't know if it will work.

Yeah, yeah. My guess is that it won't work to begin with, but it wouldn't be too hard to make it work because it's. It's just. Well, I don't know.

Actually, I'll test how good is the engine.

There's anything client server does particularly worrisome. I. I don't see any reason why it would fail over.

Okay.

The integrated chess engine, I think is okay, but there's also a plugin. So you can. If you have Stockfish, which is the premier chess engine, you can run against Stockfish.

And that's something that you can download to run on your machine.

Yes.

But we can't or shouldn't include it for some reason.

I think we would have to license it.

Okay. It's not worth at all, but there's some standard interface to the engine.

Yes, that's that UCI. There is a universal chess interface. Anything that complies with that we can work with.

Well, okay, how do I get. I mean, how do I get something that is us computable out of this.

Now under the new game, there is next to the new game in Your interface, there is a dropdown. Now you can save your game. Currently we save to a pgn, but I guess we can save to a chess.

What I care about is some kind of symbolic representation of the game where you can actually do some computation as opposed to just, you know, oh, I'm just playing a game just for fun. Which is not what we do.

I mean, that's where chess game and chess board objects come into play. Those are the other two objects we have. There are ref pages for all four of these that you're seeing. And then I was thinking it would be nice to say I pick the next 10 moves from an engine and then I can add one of those moves to my current chessboard or chess game and get a new board or a game.

So I want to get the game graph from somewhere.

Right.

How would I do that? I mean.

Currently all. All the moves. What's. What gets stored in a game is a. Is a sequence of moves. And you can. You can. We can visualize that.

What's this?

That's because probably we evaluated it earlier.

Well, so. But why. That can't generate a message, right?

No, for sure, I'll look into that. It might be just an older gui, but I'll look into it.

Oh, boy. What the hell? No, we can't do that.

That's not implemented yet. That's what I said. The chessboard recognized we have an implementation of it. But Chessboard plus a move, that. That's something that is up for discussion. I don't think it's crazy. Why do you think it's crazy?

Because I don't know what else is plus. You know, we're going to do plus like that. Doing plus with strings would make a lot more sense. That's a lot more useful. Don't do this.

Okay.

Don't do this.

Sure, fine.

I mean, we could consider plus with strings. I don't know how much stuff we break by doing that. We probably break a ton of stuff by doing that. By having that be a panda.

You mean string join?

Yeah, string join. Okay.

Oh, but I guess append a chess game comma and move could be. Could be one thing we could do.

But.

But in general, I was thinking the chess game.

What. What's in the. The.

You can open the ref pages. Chess. Chess game is basically a sequence of moves and a bunch of met.

Why isn't a list of moves.

Just a plain list of moves in that first argument?

Yeah, I would think so. One of the keys could just be moves and then it's a List of moves.

That's what it is.

Symbolic representation of a move.

There are different notations. There is the standard notation.

I know, I know. I'm asking you know, a notation that says, you know, F1, C4 or something is not our kind of computable representation. It's like the relationship.

You go ahead.

No, I mean it's like a regex, you know, some compressed regex type form as opposed to something which is actually usable. I mean like I want to know, you know, let me draw the trajectory of every pawn. Okay, how do I do that? So I'm asking. I mean, without writing, without taking apart strings and stuff like that.

That's not something we had thought about or is in this proposal. But I do see that.

But I mean the thing is this is like, you know, it's like the Emacs editor having a game of life built into it. You know, it's just one of these things. It's kind of weird. It'd be one thing if this was actually set up to be, you know, I mean, it's kind of like know the whole sort of computable games thing. I mean, you know, I wrote this whole piece about. About game graphs and so on. Right. Why don't we have tic tac toe in the thing as well? Why don't we have Hangman? Why don't we have, you know, Monopoly? Why don't we have, you know, I don't know what. I mean this is much more. I mean, I don't know that you know, this is just computable game stuff. In this case with tic Tac toe. Okay. I mean there's somebody, some student at the summer school who did this comparable things with chess.

Right.

But I mean that's not.

Which is a mini, mini, mini version of chess, I think. Chess as it is.

Well, it was, it was end games.

Right.

Oh, I don't know what to think of this. I mean I. Honestly I'm. I'm. I mean as you know, WFR like functionality A. Okay. As built in stuff. It's. It's just kind of weird.

Could as well be a tackled repository, I guess at this point if you really don't want it built in.

Yeah.

I kind of think that's a better place for it right now. I mean, I feel like if we're going to build it in, it's kind of like got to be one of like. Like in Wolfram Alpha, right? We have a bunch of information about all kinds of games. Did you even know that?

I knew that, yeah. Yeah, just like meta information, not right.

That's correct. I mean, you know, let's say I want to take a chess game. What's this? Chess Tools backtick. What the hell?

That's just. Yeah, that's just because we put it in the context until design review happens. Well promoted to system once approved.

So I'm going to ask you some questions here. So this. Let's say we do this and I import this game. Oh, what's that?

That's bad. It worked yesterday.

Well, I don't know whether I have. Maybe I don't have the most recent version.

Let me quickly check in today's. While you're checking. Your. Your version is.

No, my version.

Okay. I'm today's. And it works.

All right, well maybe this thing here will work. I don't know. Let's take a look. Maybe works. Okay. All right, so now I want to get a picture of the sequence of boards from this. How do I do it? What the hell is this? Date. Why is that not on our standard date format? What is this?

That's a good point. I actually. Yeah, that should be.

And why are these not entity names? You know, why are those not entities?

They will become entities as soon as you ask for that specific. If you ask for meta game or. Yeah, assign that to a variable or just say percent, then property gain. I'm sorry, metadata. Then we do the interpretation at that point. We didn't make them entities in the summary box because it's slow to interpret them.

Why is it slow to interpret them? Not after they've been made into entities. If there's some slowness, that should be fixed.

But then you're saying at the time we import as chess game, which a PGN file could be hundreds of chess games. We should interpret every name.

Well, I don't know. I don't know. Seems reasonable to me.

But we thought, we thought this is kind of an okay compromise. We show whatever is stored exactly as.

It's a lot of hydration here. I think there's a lot of, you know, all of these things make this a packlet, not a built in thing. Every time you tell me there's a hack, I'm going to tell you, you know, then it isn't. I mean, like, why is that not. You know what. Why is that? I mean, I don't know what that even means. And that's like, why is that not an entity?

Well, if I play a game at my home, that can't be an entity, right? Not, not everything can be a entity. Well, that perhaps.

Geolocation. Where's the geolocation? We know the geolocation.

Again, not explicitly stored in the file. We can. We have other functionality for people to say, interpret this site and give me a geolocation for that.

Show me the sequence of boards.

As sequence of boards. I didn't think about it. It wouldn't be hard to add. We have the rules.

How I compute the sequence of boards. How do I get the sequence of boards? How do I get the sequence of moves? Game states here.

If you ask for moves.

Okay. Okay. Now how do I get the sequence of game states?

Yeah.

How do I apply a move to a, to a game. To a game state?

We can't. We thought about actually.

Why not? I mean, you've got an engine that can do it with, with all kinds of weird GUIs.

Why can't you just take a move.

And apply it to a game state.

And you applying. Applying it to a game state, you mean? Exactly what I thought as adding it to a game state to a chess board object.

Yes, but. And I also want to know, I mean that adding is also silly because some of these are not compatible. You can't do that move.

Right.

Right.

So how do I get.

How do I take AXB5 and I say, you know, and write that out as, you know, knight to, you know, Queen 6 or something. Whatever the, whatever the terminology is.

The change of annotations, you mean, or notations.

No, I want to know what's the human readable version where you, you know, when you, when you see this written out, there's a human readable version that's sort of a whole sentence.

Well, that's a great suggestion. We. We don't have it now, but if you're getting excited about those things, we can get them at it.

No, no, I'm. I'm telling you this is not well thought out. That's the main thing I'm telling you.

Okay, not, not claiming it's well thought of. It's okay.

Look, it's a great packlet. It's not something for the system yet. I mean, let's get it as a packlet. Let's plan on having this, you know, it's a magnificent packlet. But, you know, the thing I'm telling you is this needs some symbolic representation. We can absolutely use this short form just like we do with Reg X's. See what I'm saying? Did you understand what I'm saying? Yes, but it needs some symbolic form where I could compute like I want to be able to Take this game. I want to be able to plot where the pawn, you know, where a particular piece moved and, you know, progressively with time. That's super difficult to do from what you have here. Essentially impossible. See what I'm saying? I have to. I might as well go back to the original chess, you know, import format to do that. Am I making sense?

We have the fence that we can extract from the whole sequence of moves. The fence strings. That fan was a format we introduced before.

Okay, so what? I don't care. I mean, it's like I want to get, you know, give me the, you know, the white queen. I want to see. Where did the white queen move on the board?

Mm.

Right. I don't think you can tell me that that's not our kind of computable system. It's just, you know, it's something very different.

You're. You're right. Some. Some query like that.

Yeah, I know that's what it means to be computable. This isn't. It's a, you know, it's a hack into a chess playing program, which is fine, but not what we would normally have in the system. Make sense?

There are, There are things, There are still things that are really computable, like getting a score for every step of a game. And then. Yes, Fisher, show me.

In this game, show me the scores as a function of time. Is it scores? What is it?

Yeah, I'm trying to score to me. Okay, well, no, I'm, I, I'm not arguing that this is really complete.

If you can give me the score time series, basically, or the list of scores here. Right. Then that's another thing I can compute with as it is right now. What am I doing with this? I mean, like, like, for example, let's say I want to find out which moves involve a bishop. I think that's what a B means. Is that right? Or we have no idea.

Yeah.

Okay, but I mean, then I'm hacking strings. Right?

You are. And one piece of our future proposal was that we could even have things, you know, another symbolic representation in this world could be a move. And a chess move, I think would make a lot of these queries that you're making easier to find.

Then the game becomes a list of moves.

Yep.

But I mean, I just don't see. How do I get the sequence of boards corresponding to this game?

I'm. Well, I'm just trying to get that. And for some reason, I'm failing to do that. Chess. Chess board is supposed to work with a fence string. And I'M mapping that over the list of fence strings and that doesn't work.

Is a fence.

I don't know why.

Is it a board state as opposed to.

That's a board state. Yes.

Okay, so how do I get the.

Fen strings from this, just as a property. Say fens, fen capital, lowercase s. Something wrong in your prototype? I'm getting a list of strings.

Okay. This is a wonderful packlet. That's where it should be for now. And let's, you know, if we want to develop this towards something which will be part of the system, it's got to have these things that we're describing of having a sort of full, explicit symbolic form make sense.

Okay.

Not just strings like this. Like, then when we do tic tac toe, we'll invent our own string notation like this, and nobody will ever understand it.

Can you get Chess tools packlet and see if your. If everything starts working in your build.

Packlet or.

Yeah, that's it. Just evaluate that and go back to the chat to the input field of balls. Okay, so again.

All right. Okay. Magnificent. Okay, so now what do I do with these?

Okay, now. Now just map chess tools backtick chessboard over that. Lowercase b.

What's that? I mapped it. What the heck? Oh, no, no, no. Sorry, sorry, sorry, sorry, sorry. We're waiting. We're waiting, we're waiting. Well, what the heck? Where are the.

So that. That's a currently proposed typesetting for a chessboard. If you open any of those summary boxes, you see a graphics rendering, but that is also a property of those chessboard objects. We can Graphics.

Yeah, this is not properly thought through. I'm sorry. It's not properly thought through. Okay, now what would. How would I take these? And for example, how would I superimpose these so I can see, you know, with some opacity? Do you get what I'm saying?

Over time, you want to make a video or just everything on top of each other? I'm.

I'm happy with everything on top of each other. So can I just say show?

Yep. That would do it without any opacities.

Well, that's not very useful because it's only showing the last thing. Stack them all up and show the last thing. So how would I get it with opacity? I would say. What could I do? Could I say opacity of one of these? No, no.

Opacity doesn't work with graphics.

You have to.

You have to inject opacity inside of each of the graphics. So if you.

Yeah, okay. So it's a list.

List.

This probably should work. This thing that, that. Something like that. Just like it does with a color, maybe. That's crazy.

Probably for graphics. I think it's crazy for image that could work and just call set alpha channel.

Yeah, I mean, I presumably could do this if I said in, in this case here, I mean. But you know, the whole point is, the point I'm making is this isn't very computable. This is basically pictures. It's quite nice. But it's. It's not ready. It's not ready. Are you going to try and disagree with that or not?

1. I guess one meta question is that do we ever want to have this in the system once it's ready?

Possibly.

Okay. Do we want to make that appearance gradual or do we want everything to be super ready? We've. We've done gradual introduction.

Ready. It's not ready. It's not the right design. It's not ready.

Okay.

Okay. Do I have to say that any more? Clearly there's a million things here that are not thought through, that are not set up so that they can be, you know, so they can interoperate with other things and so on. Right. So how do I do the set alpha channel here?

I'm actually just second looking at image compose and trying to figure out if you can just more easily do that with image compose. It requires some. Yeah, no, we can't have a list of opacities for all of us. But set alpha channel. Yeah, if you map thread, set alpha channel slot come up or a kind of a map index. Oh, you want everything? Everything. Okay, yeah, just do that.

Okay. And then what do I do then? I just say image compose or.

Yeah, just image compose should do it.

All right.

Image. Yeah, no, I realize that image compose requires a background, so maybe the first of.

Well, why am I just adding these images? I don't understand. Why am I not just adding these images? Why don't I just say apply plus.

To percent 19 there will be a white image. Maybe do mean of person 1990.

Why will it be a white image?

Because you're adding all of those intensities. It soon goes above one. When you add images, what do I do then mean? Instead of plus, Just. Sorry, just mean of percent 19.

Okay. I mean, it's slightly amusing, by the way. I could. If I say image adjust, I don't know, how do I make it darker? Yeah.

Darker. We have a function darker.

That's not what I wanted.

I have to tell you, it gets pretty exciting. Pretty Soon.

What?

This whole chess business that we're adding, you're already hooked into it to some extent too. Just. I completely agree it's not ready, but I don't see why it can't be a gradual introduction of this in this.

We're not going to like many other things we haven't thought through properly.

Sure. Okay, fine.

I mean, you know, like for example, can I do something where I have, you know, voxels, where it's black and white voxels. So what I'm saying, stacked here. How would I do that?

I cannot imagine what you're asking for. Sorry.

No, no, no. I'm, I'm. I want to show in each successive thing, I want to show the black army on this side, the white army on this side, and as we go down, a stack of pixels, right. I want to show the black army, you know, diffusing out to the. The other side type thing. Get what I'm saying?

Still no, no.

Think about these boards, okay. And think of stacking them in 3D space. Uh huh. Think of turning every black piece to a black voxel, every white piece to a green voxel or something, you know?

Okay. And then slice by slice, show them.

Yeah, we're showing each, each board state.

Kind of like he plots cellular Automata, like a 2D seller automata evolving over time. Yeah, this is kind of like that. But you know, but so you need to stack them up a little bit. Probably have, I don't know, 15 pixels high per game states.

Right. If you just make it an array plot 3D. If I can just extract from this. Okay. I mean, the point is these games are not huge, so there's no, there's nothing. You know, storing the thing in association with lists, this and that and the other is not difficult. Right. So there's a dense representation, there's a sparse representation of each one of these things. Am I making sense? And it should be possible to just get, you know, the, I mean, I don't know exactly. It requires thinking it through. You know, how do you get. If you have the dense representation, you have something which is a list and you have blanks and you have, you know, a piece representation of each piece. That's one thing. Or you can have a sparse representation, we just say the, the location of each piece and so on. And then, you know, and then you should be able to search for the, you know, the kind of thing I'm talking about, namely, you know, the path of a piece, the stack of, you know, black and white cells, perhaps something about, you Know the set of all possible moves from a given place. You know, these kinds of things that.

That we have from a chessboard already you have properties like legal moves or even best moves, which comes from.

Okay, so from a given chessboard. So if I, if I go, if.

I take one of those chessboard objects.

Okay, if I take, near the end of the game here, I take this thing and I ask for legal moves. Can I do that?

It might be a today's feature thing. Ask for properties and see if you have it. Okay, great.

That's nice. That's cool. Quite cool.

And, and you can ask for best move or best three moves.

No, no, but this, this allows you to make a multi way graph.

Exactly.

So, so for each of these, how do I apply each of these moves? Ah, you can't do that, can you?

That we don't have. Tell, tell me how can, how we can inject it.

I don't know. I don't know. I'm telling you, this needs. I don't know. I mean a chess game is a complete history. A chess board is an instantaneous state. I don't understand why what you're doing isn't. Like a chessboard. Chess game can be a list of chess boards.

Yep. I actually think chess boards is a property of chess game and if it's not, it can easily be.

The fundamental problem here is these are lovely. You know, RA6 plus is lovely, but it's not relevant for our language. Okay. What's relevant for our language and what could come out here is something which is a representation of a bunch of moves. Each one is a summary box. It could say RA6 plus at the top. But the whole point is it's got some internal symbolic representation.

I completely agree. I think moves could be added as a symbol.

Yeah. Why don't you work on the packlet for this and then let's pick this up again and you know, in a few months perhaps. And I, you know, I do think it would be a nice source of example data if we can really make it computable. But this is not that. Do you get what I'm saying? And we should just think through, you know, other board games. I don't know, go this, that and.

The other, I don't know, go would be my. Yeah, next pick as far as at least complexity and popularity. I think that's, that's next Tic tac toe, you know. Sure, sure, that's simpler and also simpler.

But I mean, I, I don't know what. And if we have a symbolic Representation. I mean, you could even look at that blog post of mine where I have exactly that for like tic tac toe and a bunch of these other games. Right? Yeah, I, I'm rolling my eyes slightly because I, you know, we've got so many critical things to do and it's like, fine, this is amusing, but I wish it was done in a way that was more kind of, rather than just sort of hacking in a chess program here.

I don't think what we have done is a whole lot of hack, but your feedback, I think is nice. I, I, I think we should do more. We can do more. And that's if you want to wait on releasing in the system, that's fine.

I mean, we do.

Sure, that's fine.

Okay. All right, let's go back to what else we had to look at today. How am I going to store this? We have a section whole thing called chess. I think we're going to end up having to do that because where would it go? What, what guide page would it go on?

I was thinking about that, not sure. And we're getting bugs for it. I was thinking, do we want a chess category of bugs or games? I don't know. It all depends. How much do we think we want to continue on that?

We see how popular the packlet is, but I mean, I think it's nice and fun and I don't see any problem with that.

And I think especially our user base is excited about chess in particular games, I think, in general.

Sure, to some extent. I mean, yes, but I mean it's, it's like, why would they come to us to play a game of chess unless they can do things, which are the kinds of things that we can do that your average, you know, game engine, you know, chess engine that will fit on a phone can't do. Right. And the answer is all sorts of interesting visualizations, analysis, etc. Etc. Etc.

Yes, exactly. That's exactly how this whole project started.

Yeah, but we don't have that here.

I mean, well, just, just the import of PGNS allowed. Ruffle, who was the advocate for the, for this whole thing, import his own games and see how he's improving as he's playing. Well, there is the one number you get from like Chess.com that says how good you are in comparison to others, but it doesn't tell you that move in that game was really nice or you have started making less bad moves in your games. It doesn't give you any other details. And that's why he started this Whole project.

Okay. For some reason this community post is not working anymore. That's really weird. It's really friggin weird. This was a summer school project. Multi way graphs from minimal chess end games. That works. But then why, what's going on now? Well, who knows? I mean a good test case for this would be, You know, well, we could take this and we could try and do some end game type multi way graphs, see what they're like. But you know, for that we got to have things like apply this move, otherwise can't work. See what I'm saying?

Yep.

Okay, let's go to our other part of our agenda and the time that remains.

You know, Bobby's here as well.

Okay, let's go back to what we were talking about before, which I think is much more important. Okay.

Okay.

So can you describe this?

So we started by talking about how we convert manipulates into videos. And the one thing that we specifically talked about last time or you mentioned was how do we make voiceovers for these videos Cases like no caption.

Let's, let's, let's be clear on what we're trying to do here. So I mean we might say I'm going to go to the demonstrations project. Actually needs a facelift for its, for its homepage. But anyway. All right, let's take random one of these. Okay, let's say copy to clipboard. This is a lovely mechanism. Okay, there's my manipulate and I want to go like this. I'm going to say, well you know, here we're showing more steps. Boom, boom, boom. Now I want to show this with an obtuse triangle. Okay, let's show it with stars. Let me increase the number of steps there. Nice demonstration by the way. But right, so I'm, I want to give some voiceover here. But what, maybe what I want to do, I just don't know how one produces a video from this. Bob might have a comment about that. In other words, we, if we say, if you say to me, there are these controls, I don't know what the controls look like. We can take a look. Okay, so let's say that I want to script for myself something with a series of, of. I don't know where it is even steps here, right. I want to say step one, step two, step three. And each one of them is going to last a second or something like this or half a second or whatever it is. And then I'm going to say, you know, I'm going to show dots, I'm going to show suns, whatever and how would I normally. How would this normally be done, Bob, if you were making a video of this, what would you do with voiceover?

Traditionally, we would capture it manually and then clean up the capture as much as we could with a lot of masking. And then the voiceover would be read and overlaid and like premiere and then exported. So there's a good amount of steps.

Yeah, okay, but what I'm asking is this, that what we can do here is script the thing with time marks. Okay. I don't understand how that relates to a voiceover. How does one do that? Because, I mean, I. You know, if you say you've got 1.5 seconds to read these two sentences. I don't know if I can do that in 1.5 seconds.

No, not in 1.5 seconds.

NoSCH I understand. I understand. My point is this. If we are writing a script here that says. This is what I'm trying to understand what we would want to do. It's like, for example, we could have something where somebody. We could have. First of all, synthesized voices are coming. And, you know, if we say, for example, here, let's say we say speech synthesize. I mean, I think we should assume that the synthesized voices will be here soon enough, you know, and let's say we say speech synthesize. This is a tiling. Let's see more steps. Okay.

So it would need to know the duration, overall duration of the video, and just like a waypoint or time marker to know when to start reading that.

Yeah, but.

Right, but. So the thing would be, I would imagine what we would do is we would say, you know, in the script, so to speak, what we were doing is we'd be saying this is the state of the. You know, what we do is something like this. We'd say, but it's a little tricky because, you know, we would say this is the state. This is the voiceover. But somehow the voiceover. Do you get. Do you get the point I'm making? I mean, you could say, here's a state. Static. It's a shot of some kind, right? Then you say, yak, yak, yak over that shot, right? Then you change to another shot. You say, yak, yak, yak. Right? And we could do that here in. By saying, this is this. This. These are the collection of settings at this time mark. This is the voiceover that begins at that time mark. Shoddy. Does that make sense to you? So in other words, we're giving time marks interspersed by voiceovers. We're not Saying what those times actually are. 3.2 seconds. Whatever. We're just saying you're composing this thing where you're in this state. You read this. After you finish reading that, you go to the next state, so to speak.

So sun has. Sun reads one part of the script, Star. When we get to star, it does the next part.

Exactly. Okay, but I mean, so that would be. That's more like the slideshow mechanism. Right? So, you know, if we go to. What the heck is happening? What on earth? I don't even want to know. Slideshow video. What is going on here?

You had one casing different. You're.

I don't care. I. I want to know why it's spinning like that. Oh, that is a bug.

We weren't seeing your spinning.

Right. So please report that as a bug in the prototype build.

Okay.

Okay. So, Bob, I don't know if you've used these things. What the heck is it doing? Look at this. What is it doing? Is there some dynamic that's. That's frozen here? Okay, so here we go to slideshow video. Last five seconds. It shows those three slides. Okay, so what I'd like to know, is this already possible? Shadi, if I want to say, this is the earth, here's a speedboat, you know, et cetera, et cetera, et cetera, Can I do that? How would I do that?

You can do that. You could speech synthesize the three things. So. Not within. So these are. Will be two separate steps. You speech synthesize the things, you join them, figure out their durations, and then.

That's why we already lost the game. Because what I want to be able to do is I want to be able to say, this is the voiceover for this slide. This is the voiceover for the slide.

That is not possible at the moment, but that is like two lines of code.

Okay, fair enough. Great. All right, so now what I want to know is, you know, and this is a question kind of for Bob, about how video, how scripts work. Okay? If we've got static slides, I understand you have a voiceover for one slide, you change slides, you have a voiceover for the next slide, and so on, right? So now the question is, what if you have a dynamic thing that's moving from here to there, and as it moves, you want to be saying, you know, and now we increase the number of steps, blah, blah, blah. And you're saying that while this is moving, you get what I'm saying? Is there a standard way in kind of, you know, movie scripts or Whatever video, you know, to represent that thing. In other words, I, you know, I've seen in scripts, I've seen, you know, there's a picture of, you know, there's a, you know, the person is on a horse galloping from here to there and then there's a bunch of dialogue that's written there.

Right, A storyboard. Yeah, I guess typically it's a storyboard or it's just an alt text outline script. And that's in traditional editing. That's what informs what goes where.

I understand, but so for our purposes, so you understand what I'm saying, that we would have something where you're going, We've got a series. Guys, walk me through how we're planning to do this. So we've got one thing is waypoints that you can define. Am I making sense? States of the manipulate that we can define. Shadi, are you with me?

Yes. And that's bookmarks or time marks that we were looking at last.

But how do you go between those time marks? Is it linear interpolation or is it a jump?

I guess it could be arbitrary, but yeah.

So at the moment it is by default linear interpolation. There is some degree of control that you have over which interpolation. And the plan is to expand that control over which interpolation.

By which interpolation you mean. So two bookmarks could have three controls that are in different states. Right. So in other words, x equals 1, y equals 2, z equals 3, and then the next one something else. Right. And so clearly there are multi many paths to get from one XYZ point to another XYZ point.

Correct, but the point is from the specification language, so I thought about that too. You know, the. If you only want one control to move at the time, which is what you do when you manually manipulate the controls, then the state that you want to interpolate between should only move one control. If you. If. Yeah, if you change the setup to move simultaneously. Several controls. We can do that with interpolation, but you can't manually do it unless you have like three hands.

Yeah, right. But I mean, that's what we could do with gamepads, for example, we did have ways. We do have ways to simultaneously change multiple controls.

Right.

That could still work. Yeah.

Right.

So, okay, but.

But often. Okay. Like your thing here or a bunch of that I wrote too. There are more than one. You can kind of see the simplicity of it. If you kind of clearly define some setup. Start here, then change this, then change that, and it becomes obvious the Thing that you wanted to explain.

I understand.

Kind of like an experiment or like a scene or something like that.

Can I ask something about scripts and things or storyboards or whatever, about camera moves and so on. So if you say I don't know all the terms, you know, dolly to hear, etc. Etc. Etc. Does it, how do you say, you know, like as the person says, you know, there's something amazing here, you know, and you zoom in on the person or whatever. Is there a way of representing the fact that as they start saying that you're zoomed out and by the time they finished saying it, you fully zoomed in. Is there some notation for that or something, some way of describing that?

I mean, those are the names of the moves that, you know.

No, I know. Is there some, is there some way that you would write that if, if you, you know, in a. Whatever. I don't know what the right term is, a storyboard or something, you know, to describe that, doing that, that that's what the final video is going to look like. Is there a way to describe that?

Right.

I mean, you were pretty much saying it, right? It's zoom in, zoom out, pan left, pan right, tilt. There's a lot of things that will be written in the description. So if it's a storyboard underneath each cell of, of like a visual, it would say give direction on what's happening there. And then it would be up to the discretion of the, the director as it's being produced to see it and see how it's going to work. I mean, in, okay, traditional production.

But I mean, so, but what I'm asking is if during the time when something is being said, you want a camera to have moved in a particular way, zoomed in, for example, is there a standard way or does it just say sort of in two separate threads it says on, you know, one place it says, you know, zooming in on the, you know, pet dragon or something. And then, and then somebody is saying, you know, the dragon is saying there's something amazing here or whatever, and that the, that it's then, you know, up to the director, you know, at that moment to sort of say zoom and finish your zoom and just say in words, you know, finish your zoom when the, when the dragon has finished talking type thing, is that, is that what would happen? Or is there some more formal notation.

For that that would be more like just specific timestamps to denote, like if you're working on production that's going to be used for a 30 second ad it would be very specific and narrowed. And it would just be timestamps, just traditional general timestamps.

Okay, but then when the voiceover is given and the person has 22 seconds to read that, how does that usually work? Does the person read it and then you figure out how the zoom is going to work? Or do you have the zoom and then you time stretch or whatever what the person says? Or you have a person who's really good at reading it in exactly 22 seconds?

Yeah, a lot of that's pre production planning. So it could work either way. It could be where you end up time stretching the footage or it's just that you coordinated it well in advance and were able to make it work.

Okay, all right, so what does that mean for what we're doing here? I mean, recognizing, by the way, that a lot of these kind of video direction type things in the, in the time of LLMs, there are going to be more and more automated ways to deal with this. I mean, I've already seen, I mean, you know, there are, there are already sort of proto examples of text to video type stuff, although they're not particularly good yet, I think.

I mean, even in, with real camera moves, when you, when you don't do cgi, they use robots to very precisely control the camera path and those also controls the camera. So zoom and things like that.

Right, but the points that I guess.

Were making similar to what we are proposing here, like defining key positions and then interpolating between those.

Yeah, we're trying to build the simplistic.

Right, but. Right, but the point that I keep on coming to is for many of the things that people want to build as videos from manipulates, they're going to want voiceovers. And I'm trying to understand how on earth we merge those in.

Yeah, so I think, I think one of the things that you seem to think that, I don't know, maybe Bob has an opinion, if that is the majority of cases or not, is that the generation of the video and the voiceover happens at one time. I was under the assumption that a lot of the times the video is generated first and then the voiceover is recorded by looking at the video, someone enunciating while looking at video recording during that to ensure synchronization.

Well, I'm sure that's true. I mean, isn't that what, you know when, when you know, movies are scored or whatever, you know, don't they, don't they have, you know, the orchestra watching the movie when they're doing things or something?

But what I'm saying is this would work with a real person enunciating, but obviously it would not work with automated speech synthesis. So I think the.

I'm not sure it would work with a real person because, I mean, you know, if you allow yourself 22 seconds for this zoom and you've written some text and by golly, that text takes 34 seconds to say it's not going to work.

No, but I'm assuming, like, if, if the text somehow fits those.

It won't. It won't. Because nobody's going to know. Nobody's going to know. It's. No, it's going to look like this. It's going to be. Here's what's going to happen. It's gonna. And I've seen people do this, right? It's going to move. It's going to go to the, the endpoint and the person's going to say, look at it moving. Silence. Because it's still moving at the point where they finished saying let it. Look at it moving. Right? Silence, silence, silence. Five seconds later, it's finished moving. Oh, let's see what it did. Three second pause. I mean, that's what it's going to.

With the design, we can address kind of all of that. Let's say that you wanted to go from state one to state two and you have some voiceover that you want to incur there.

Yep.

Okay. So you want them to line up somehow. So for instance, let's say that you use speech synthesis with normal speed and not sort of legally speed.

Right.

That gives you a time. Exactly.

That's what I'm saying. That's what I'm saying. I mean, that is one way to do it is what I was describing for the slideshow, right? Yeah.

That is easy to do by just computing durations. And I mean exactly that.

But.

But that is something that is speech. Synthesize compute duration, figure out what the time marks are then.

Exactly. You know, and make sure there's appropriate padding so you're not running the sentences together and all that kind of stuff.

That could be a configurable setting as well.

But yeah.

Yep, that's reasonable. But I think that if we're looking at manipulate or manipulate video still kind of hints at keeping the two aspects separate, like the synthesis and then being able to pass the audios to the generation.

I'm not buying it. I'm not buying this. I just don't think this is right because I think the durations are in many cases, as soon as there's an audio track. The durations are going to be determined by the audio.

And also, I mean, I think that we could make a design where it's, you know, that manipulate video. If you give sort of time and states, and if you also give another thing that's marked up to be that generates speech or an audio file, whichever you care for.

Exactly.

Then there could be either default that says, you know, that's the time to take plus padding, or there's something that says, you know, add more.

Right. So. So one way to do this would be. I think maybe this is exactly what you were saying. You know, there's this time mark. Now, the one thing is, it's sort of a fence post thing because these are the betweens, right. You've got a, you know, a state. You've got. It's between, which could be three seconds, just the raw number, or it could be this audio file, whatever its duration is. Yeah, right. The thing that's bugging me is this, okay? And maybe I'm being silly about this. I'm trying to understand if it's all about waypoints, maybe it's okay. Maybe it's okay if it's properly, if the motion is properly parameterized. What I'm. What I'm worried about is that it's kind of like you have to have finished your. Your, your what if you have a piece of audio that is supposed to span four waypoints. So I'm saying it goes from here to there to there to there. And as it's going around those waypoints, you're giving a little speech.

Yeah, so you're getting closer and closer to my original, which is more like, kind of like an orchestral piece where you, you know, orchestrating a lot of stuff. I think that we were trying to get to right now is something that's a real easy spec and then building out from there and more elaborate.

Okay, but let's go through this. I mean, basically what we're saying is. And have we had to address this issue in other places where we have state, state to state to state. But I mean, it's like in a graph, we have edges and we have vertices, right? Have we had to do that for audio? For any purpose of having, you know, a state, or video for that matter, a state and then things in between. Didn't we have to do that for tour video?

Yeah, yeah. And uses that essentially just takes the approach that we just use the keyframes, just the times rather than just the versus rather than the edges.

Okay, but the problem is Let me, let me point out into a video, we're going to have the exact same issue.

Slideshow video, if you wanted to annotate that with text, I mean like speech to. Sorry, text to speech or audio, one could put that in instead of those time steps. Right. Then see what I'm saying? The second use case there, whether it's time, time, time.

Well, but I don't think so because it goes in right here. It's the thing that goes between these two. I mean, it seems like it would make more sense. Like it's vertices and edges listed in a graph. Although this is just linear. Right. To have a separate list, a third element here, which is the audio, for example, for every one of those. I mean, one possibility would be that you have in this thing you say to a video. Well, let's just say slideshow video. It doesn't matter either of them. Right. And then what I get do next is to say comma, you know, audio one, audio two.

Right.

Those are the audios for those durations between this and this. And then maybe what we could do, I mean, I'm sure slideshow video has a. A time step type thing, does it not?

Mm. That's that second syntax you're looking at.

Yeah. Right. So, so then what we would do.

Oh, slideshow video was slightly different. We looked at it last time.

Okay. But it has this image decode.

There's really durations there because those are very static. Right. Whereas the other ones you might be rotating and translating. There is action happening.

Yeah. Okay.

Slideshow, there's no action happening. It's just flipped.

But either way, you could say, like we do with sound note and things at this time, marker, start going yak, yak, yak. I mean, by the way, sound note is a model of this.

Yeah. I mean, it's. It's kind of. But it's a little bit more elaborate than sound note is what I'm saying. Because you're trying to synchronize different things.

I know, I know, I know. So that's the problem is that. That we could say here, comma, automatic for this DT one, and then it could pick up its time from the duration of such and such an audio. Do you get what I'm saying? Yeah.

The problem I see with that is when you want one of the audios to span two of the things you have or multiple of the things you have in that first argument.

I know. So. Right. So that's why it's tricky. I mean, and maybe we have to name a score.

Is the word that I was looking for, what's that score? You know, when you write.

Yeah, yeah, yeah, yeah. Right.

And there. Things can happen simultaneously or not.

I know, but I don't know the notations. I mean, you know, all of these music notations, but they tend to be start at this time, go for this time, go for this number of beats or whatever.

Yeah, it's all relative.

Right.

So you can speed up or slow down the whole thing. Right.

But that stuff works as well. But it doesn't work for voices. We're not going to be able to do that. But I think what we want to do here is it takes a little bit more thought, but I think what's going to be needed is those times that you're specifying here are really times to. Times to audio marks, whatever. I don't. Is there a term for that? In other words, I'm reading, hello, this is the Earth. Hello, this is. This is a speedboat. Right? Each of those things, there's some kind of marker at the end of when I've said, hello, this is the Earth type thing. Do those have. Does that have a name in any video production? Bob, anybody?

We just call it a queue.

So what is. What does it mean? What does a Q mean? A Q just means that's the time at which something's going to happen.

Correct.

Okay, so, okay, so that's what we're talking about here, is we're defining certain cues. And a cue, when that cue occurs would be determined by how long the audio took to say that. A fair characterization. In other words, you say cue. Cue the speedboat or something. Is that the right terminology? I don't even know what that. Is that the right way to say it?

Yeah, that works.

Okay, so then, then that would mean now it's time to put the speedboat on. And that Q is determined by how long it took to say, here's the Earth. This making sense to people. So, I mean, what we would have is a list not of times, but of C. And then we would say kind of what determines each cue based on audio. That making sense to people.

Shadi, I'm really starting to see a DAW kind of a thing, an interactive thing with multiple tracks, multiple pieces and segments in each of those tracks. And. And I can't see that immediately in. In a one line of function. How can we specify that? But overall, I think we are. At least we know how. What we want to combine. We don't know what kind of a syntax would work for us.

No, no, no.

I Do the comp.

The syntax is a little bit tricky and I don't quite see it yet. Okay, right. But, but I think what we're all saying is there are certain, you know, shots or something which might just be a static slide on the screen. It might be something which is, you know, moving sliders to get from here to there in some trajectory. Okay. And there might be a series of shots. And the length of that series of shots would be determined by a piece of audio. So it might be three shots and we say distribute them, you know, have them of equal length in the time it takes to say hello, you know, welcome to the solar system or something. You see what I'm saying? And because I really think this idea, maybe I'm wrong, but somebody can correct me, but this idea of time is determined by how things are spoken seems really quite useful for what we're trying to do here.

I agree. I would not forget though the case where we are just generating video. We don't have any audio. And the plan is generate a video through the demonstration project, for example, and then do the post production and editing in an external software.

I understand that, but it would be really difficult. Okay, so in the external software we could time warp the video if we needed to. Right. So you fill in the voiceover and then you time warp the video. Is that, is that we can, we.

Can do it as well. We have.

I'm not talking time working, just, I don't know, adjusting colors, adding the voiceover, whatever.

I'm just saying you can't add voiceover if, if the thing, if the video you recorded took three seconds.

Well, I mean things. Hopefully the director, programmatic director of whatever would know what they would say and they would know how long roughly would take. Right. And the planning for the video would be sensible rather than produce something completely unusable.

I, I don't know what. Yeah, I think I'm in the process of losing my voice, which is very odd. I don't know, maybe I got sick here.

What I'm saying is the audio case is useful and interesting and, and definitely should be discussed. I'm just saying let's not leave the just video generation behind.

No, I agree, but the video generation is with pure time marks three seconds to do this and so on.

Okay. Okay, perfect. No, that's, that's. I just, it just, that's what we need.

I just, I'm just saying that I think it is super useful to be able to, you know, and we'd have a workflow by the way, where A human could record, you know, this audio file. It requires this. It requires three audio files, you know, for this thing. You know, record them, then stick them in here.

Now.

Yeah. Okay, we need to wrap up. But, but I think so. There's a little bit of design thinking here about how these cues and, you know, the, the gap type thing is going to work, right? Because it's got to be a spanning thing. It's got to have, you know, and perhaps in the score idea, it's like there are three elements here. And maybe the way we do it is we say, just like we do in a table, we say it's this one and there's audio that starts here and then it spans this one, for example. Do you get what I'm saying? Span from left type thing. Right? Because I don't think, you know, having the audio stop halfway through. No, we don't need to do that. That's too detailed. But we might want it spanning multiple things. So we could have. And much like sound note, we could have intrinsic times there. And if the audio, you know, doesn't follow those times, we could time warp the. The video, so to speak. In other words, if the Earth was up for two seconds and the speedboat was up for one second and it turned out that the audio was 4.2 seconds long, you could time warp those two times.

Correct.

So that it fitted 4.2 seconds.

So let me give a partial proposal. Not very well thought out or anything, but. Okay, say that we have a. Manipulate that has some amount of name bookmarks, and then we have a bunch of audio recordings. What if we have to indicate the evolution of the video in its soundtrack? Just time goes to bookmark and time goes to audio. And these would be placed completely independently. And the duration of the audio would count if it be maintained.

But I don't understand that.

No, because once they do, bookmark one starts at zero seconds. Audio one starts at zero second, Bookmark two starts at two seconds. But audio two starts at four seconds.

No, it won't work. No, because what we need to have is that the duration of audio one is what determines the second mark, the second time mark.

Okay, but that, that, that doesn't work if we want audio to span, overlapping different.

Yes it does. Yes, it does. All you do is in the thing, you'd say, you know, the, you'd say, let's say we've got this. We'd say audio one, audio two. And then we would say, let's imagine we have another, you know, a couple, another picture. Let's put another speedboat there. Okay. Then what we would do is we would say audio one, span from left, let's say bad word audio two. Okay. What does that mean? That means audio one is continuing through this first speedboat.

But what if you want audio 2 to start in the middle between of the time where the speedboat is displayed?

Then you got to do it over time. Marks. I'm saying you get to your.

I see these both being complemental potential syntaxes that could go in the same function, right? You could like this. And if not, you do time one goes to audio one, time two goes to audio two.

That's fine. Okay, that's fine. But what I'm saying is I think it's really important to have this kind of just sequence, sequence, sequence thing.

So you're proposing a way to closely align the audio to the slides. And if you want to disengage that alignment, you just specify the times.

Yeah, exactly. So in other words, normally it would just be. You list the audios and they default to, you know. Right. So. So this would be audio one plays through the end of that speedboat. Then it's audio two playing through this slide and audio three through this slide. Now, when it comes to the manipulate case, we've got this, you know, vertices and edges problem, right? Because the. The length, the number of elements here will be one less than the number of waypoints. Do you get what I'm saying?

Yep. I mean, unless we hold the last waypoint.

Which we shouldn't do. Which we shouldn't do because people are not always going to want it. If you want to hold it, you put it in there twice.

Yeah, yeah, yeah, that's fair.

The. I think this looks pretty good and I think we should make a function for this and we should try it out with speech synthesize.

Yeah. So there are a couple of things, right, that need to happen. One, the. The internals of manipulate need to start supporting all of this. That will be step zero. That is step zero.

Okay, but.

But you can do all this audio stuff with slideshow as well, and with animation, video and so on. You could. You could start putting that in. Okay, Right.

Animation, video is just a one piece thing at this point.

One what?

One piece. It doesn't have multiple pieces.

No, I understand that, but you could put in one audio thing and the animation could happen not at an absolute time, but at the time that it took you to say yak, yak, yak.

Sure.

Seems useful to me. I don't know. I mean, I don't Know, does Bob wants to comment on. On this whole general approach of synchronizing, you know, of audios and videos synchronizing and so on. Does this seem useful or is this like. No, we never use that because we'd always have to tweak it by hand.

No, I'm actually interested in trying it out. I think, I think most of the time when I'm producing like a explainer video or something that I start with the outline, of course, but then the audio gets voiceover will get produced and then, you know, the clip getting adjusted to fit the voiceover. So I mean, you're on the right track, I believe.

At the same time, Carlo mentioned this once. Let me reiterate, a lot of the times, my understanding, same as Carlo's, is that Audio Edition happens as a separate step for animation, video and slideshow video. It's really easy to say what should be the duration of the whole thing, or small pieces of the slideshow video, for instance. And then that duration could be exactly the duration of an audio that is pre generated. Do we want to smoosh all of these into these functions?

Yes, we do, because we're trying to get something. Okay, look, I'm planning for the time when audio generation, voiceover generation is really good automatically, which I don't think is far away. See what I'm saying?

With the services that we're adding, it is already pretty good.

But okay, so. So like for example, if we could train it on one of us, okay. And we could, you know, we could have the Carlo explainer video, right. Trained on your voice, for example, that's coming within, you know, within the year. And it'll be quite good in my view. And you know, right now we probably have it for some standardized voices and you know, and that's what I'm. And then we can, you know, it becomes very, very fast to generate videos from, you know, from demonstrations and so on. That makes sense.

Yep, I guess.

Kind of a new world. It's. It's not the world where you have to do video editing every single time you want to produce a video.

Right. Currently what you're asking is one option in these video generations and a video combine after what you're proposing that we add is this kind of a second argument in so many functions, some of which already have second, third.

Well, I know. So you've got to look through it. I don't know how to do that. I'm not solving that problem. You got to look through it to figure out how this works. Maybe it's an option. I don't know. Okay.

Okay.

I need to go to something else. We're going to need to come back on this another time. Can you please think through what it's going to take to put these in? What we did not address here was the name Manipulate Video, which I don't like at all. And. What, what, what are some alternatives to the name Manipulate Video?

If you open the page, I think I put one or two other things.

I mean, it's kind of like tour Video, isn't it?

No, my suggestions in the notebook are bad. This is in a sense tour video. Bob, you were suggesting something else in email. Interpolation video.

Interpolate video. I just.

But they're all interpolate videos.

Right, Right.

You know, and they're all keyframes. You know, all of these things that we, we're producing are sort of easy specs, lot of automations and they all.

Right.

Interpolate to varying degrees and they all keyframe to varying degrees. So we can.

So to a video is really an image to a video. Yeah, or graphics. Maybe we should change it to be image to a video.

It can as well be a video to video where the video is moving and then we're zooming into a part. But, but I. Yeah, I see your point. It. It can have a different name.

Well, manipulate. The problem is a manipulate as a noun is really just an us thing. It's very hard to understand.

True.

And we're planning a 2 or 3D video, right. That does the same in 3D.

Okay. Well, I mean, this is, you know, it's more like a controls video or something. Because we do have the term controls that we use in connection with manipulate.

We do.

And really what the thing is doing is it is picking up certain controls and moving them around. I really need to go. I'm sorry. I mean, we're getting that. I think that's closer. Controls video seems more useful. Your comments.

I'm thinking the word animate. We do have animation video, but animated video, something with animate might be.

The whole point is that what this is doing is moving some controls around automatically.

Okay.

Controls, animation, video or something.

Well, obviously when I hear controls, I don't think of those.

But I know when you hear controls plural, you don't think of control theory necessarily.

Someone had proposed manipulate two video. I'm not sure that makes it any better.

No, it doesn't.

That sounds so kludgy.

I, I'm the one that I'm liking best right now.

How about exploration video?

Maybe.

And this is, you know, we. We take and manipulate and we. We create an exploration.

Yeah, I. I like this concept somehow, of moving controls, because I think that might even be more general than manipulate. Maybe not, because manipulate is the thing that attaches an actual symbol name to these things. I think we should do a little bit more. Okay. Let's bring this back in the next few days. And. Yeah, Okay. All right. We should wrap up here. All right. Good stuff, guys. But. But we're going to need to come back to this one more time here. Okay, thanks.

Thank you.

Bye.