Live CEOing Ep 843: Design Review of Sound & Vision

Hello everyone. Welcome to another Wolfram language design review. The one of the early members of our 14.2 series. This is about sound and vision, otherwise known as audio and image and video processing. All right. The video object itself. Okay, so I've just been reviewing, I was reviewing earlier today some of the UX for this save in Notebook and so on. It looks good. I don't understand the stuff about asset objects. Is that going to be importing?

It is not relevant for now. Until the asset system is ready from the front end, it will be relevant. As soon as that is ready, we have hooks to just use it.

Okay. Now I do want to make sure it's not really this team, but I'd like be good for you guys to drive it that you know, if I try and save or if I publish to the cloud a something which contains videos, it's obviously going to be hopeless. The videos are just going to go away unless something specific is done for them. And do we have that in place yet? The, you know, the cloud publishing.

No, I mean that needs to be manually updated. If the video gui's are in notebook objects, it will work.

But if not, I'm saying if you take a notebook that has videos in it that are saved in local files and you say publish this video to the cloud, something has got to happen that allows you to fairly automatically be able to upload those videos and turn them into cloud objects.

That doesn't happen now. It is.

I know that.

I know that it is. We've mentioned this to John multiple times and I don't know.

Okay, but mentioned is not driving it to completion. I mean, so what meeting is that going to be in? The user Assistance meeting. Is it going to be an ILD meeting? Where's that going to be?

We have bi weekly meetings with John.

I think User assistance make make sense. So for that.

Okay, fine. Then make sure it's there and on the agenda. Okay. Video object track. Video object track. Should I look at its rough page?

Yes.

I wonder if there's any way we can set this up so I don't have to retype these things. Yes.

You highlight the object and then you click the dot.

Which object? This one here.

Like. Yeah, highlight that. And now click the dot in your docky tools.

Click the dot. Okay. All right. Okay, I believe you. I'll do it the next time.

Okay.

Okay.

I think maybe go and check some examples before drilling into too much. There.

Yeah. Some of these sub options, we were just talking whether even they should be exposed or not, but I agree Examples show that big picture.

Now what do I do at this stage? I click this.

Click the arrow.

Shouldn't there be words here? Because I don't really understand what's going on. In other words, because that looks like it just was an error. See what I'm saying? I don't even know.

Yeah, we could. This was what design provided us for that specific use case. We can revisit that and see if there is a more helpful icon that. That we could use.

I think it needs words because it's not obvious to me that I can just transparently just press download and be happy, so to speak.

That's fair.

So it's harder to see in this video. And I should swap the first example. But the dog has a number. The people. The handbag has a number. People have a number. If you scroll down to that cars example, that's a more apparent one to see. So the first case, we're just highlighting the detected objects. If you scroll a bit more, I.

Don'T understand why the detected objects look like they are. Are those the common names of those concepts?

Yes. And then once we track them and highlight them, then you see car one, car two, car three.

Meaning that there is a difference between the things that are the same object but are different instances.

So that's image bounding boxes. That just gives the bounding boxes of recognizable objects correct. Okay, so what's the data structure returned by image bounding boxes?

That's the next topic on the agenda. Currently in your prototype build, it returns a list of times and detections. But if you open Find faces, we have a tabular object coming back from faces, and this has to be universal for all the functions. But scroll down to the scope.

You know, I just don't think we should have literal examples here. We've got. We've got an example section. That's where you put the literal examples. We need explanatory examples, but not example examples here.

I think this is definitely a prototype illustration at this point. And we should. Whenever I look through, I usually write that in. So we don't have this argument until.

The. Well, fine. I mean, unless we change our policy. I mean, after 35 years, we might want to change the policy. But unless we do, we. We want only sort of. Only things that would be in words, but we use a picture instead of words, so to speak. But not things which are. Hey, here's an example. Because that's what the examples are for. We don't want a creep of the examples into the. Into the detail section, right?

Now, usually illustrations are of two, maybe three kinds. One is conceptually understand what comes in, comes out. You know what the transformation is. Sometimes you need to explain a bunch of concepts that have been given names. And so you want to highlight those.

Call them out and, you know, I think we agree, Roger. We've been doing this together for decades. I mean, I, I just. Unless we are thinking we're changing our policy, I think we all agree what we're doing.

Okay, good.

Okay, so right now, this is giving a list of bounding rectangles. Are you proposing to change that?

Not for an image. Not for an image. We are upgrading a whole bunch of image functions to work on a video Object now in 14.2. And if you scroll down to scope, I have a tentative example that shows fine faces on a video there.

Okay, so the big issue is, is it a time series or is it something different?

So the issue with time series is that at the moment it doesn't.

I don't care about the at the moment. The fact that time series is broken right now is not a reason to make something is a time series. What's that?

It is a time series.

Right. Well, so we have to find a way to have it. I don't know how we make that bridge. Right, because eventually time series will be based on tabular, there'll be time series and everybody will be happy. But what do we do in the interim period when time series is not based on tabular, but where time series as such gives us a bad result?

It even fails to be constructed in cases like this, where in one frame you may detect no faces, in another one you may detect four. Because time series expects same dimension for all values.

Yeah, okay, I understand.

Well, so we're going to do exactly what you say, modernize time series in that sense and a few more senses. But tabular will also be an excellent way to deal with time series from certain perspectives. It's more explicit. And we have date and time columns for that very reason. We can actually do a lot of that.

So let me ask you a question. If you're prepared to support, you know, things like time series predict or something like that, taking a tabular as input, which I think is not, you know, that's actually not far away from the fact that it probably takes a list of rules as input, then I think it's okay. Then, then find, you know, faces on a video returns a tabular, and one day it'll return a time series. But every operation you can do will somehow be, will hopefully be able to work the same.

Well.

That'S what we're signing up for if we think we can make all the operations basically work the same. So long as you don't go into individual part numbers.

It's a bigger discussion because think about all the operations that you do on time series, like standard deviation. What is the standard deviation of a list of rectangles?

I say error, it just doesn't do anything.

I'm. I'm just saying that.

That there is no. But I mean, one of the rules in terms of compatibility is if it's an error, who cares? So to speak. In other words, something which was an error in version 14.2 could become a shining, you know, thing that does something in version 15.1. See what I'm saying? I don't think there's anything wrong with that. I've never thought that sort of filling in what was an error is bad. So I'm saying. So. So anything that was an error, not a problem. Now the issue is something where it does something definite and then we want to change what it does. That's where there's a problem.

Part of what we've ensured right now for this release is that you can convert to tabular from time series and event series, etc. And you can convert away back, you know, from tabular to where sometimes those conversions will fail.

The conversion back will sometimes fail. Right. You can represent more in a tabular than you can at a time?

Oh, absolutely, yes.

Yeah. So it will sometimes fail, which is okay. Yes, that's okay.

But that was a challenge that we had before with data set like that wasn't very doable.

Yeah, well, fine. So, I mean, my statement is, yes, you can return tabular so long as we're signing up to make these functionally compatible. The tabular in the future time series. Yeah. Okay. Okay, okay. By the way, I noticed Ethan on our live stream asking, what about video stabilization? Can we have that?

Yeah, we've talked about it. In general, cameras are getting a lot better at stabilizing on the fly anyways. But I. I think as a function, we should have it. Yes.

Okay. Yeah. I'm outraged by how good cameras are at it. It's just crazy. The. I just wonder in the. I'm. I'm charmed by the question of what it means to say there's no AI in this video, so to speak. Just. Just one of those, you know, philosophy meets the real world type issues. Okay, so. Okay, so what you've got here with video object track is what does it actually look like.

So if you are moving to tabular, Carlo, I'm thinking video object track should also return tabular.

Yeah, that is on the list. Absolutely. We just were waiting for deciding on the output format, but I.

The default, I think it probably should be.

I think it should.

Yes, I think it's the most useful. Okay. All right, so what else should I look at here? I mean, you know, there are many, many, many details, but I'm sort of assuming uses detector to find objects of interest. So that's like image cases or something.

The default is using image bounding boxes. So detect objects, generic objects, and track them. But one may want to just detect people or faces or text or whatever.

Right. So how would I do that? I mean, is there a nice syntax for that that's similar to image cases? If I look at image cases, I can just say as a second argument, people or something.

Oh, I wasn't thinking about that, Carlo. I think that would be doable. We could have concept.

Why not? Yeah, yeah, I think so.

We can do that. The thing with video object track is a bit more general detector. You could use your own neural network.

Right.

But we could definitely support string shortcuts for what you just said.

String and concept entries.

Yeah. So it's basically video object track of video comma, car.

Yeah, yeah. We can add. That.

Is the name. Right. Video object track. What is it actually doing?

So we've had a little bit of discussion about this because the main use case is obviously video, but it works on other data types. The input could be labeled matrices. So something that comes out of the.

Of component morphological components, watershed components, or.

It could be a bunch of bounding boxes. So could be generated from frames of video, could be from wherever.

But I understand it's what it's taking as a video because it's giving a time series or a tabular time series.

Yes, that is the main case. Yes. But you could also input boxes that you've detected in other ways. And it doesn't know where it comes from. It just sees the boxes and assigns the boxes into tracklets.

That's that first.

What is a tracklet?

Sorry, sorry, that was unnecessary detail. It essentially, if you have a list of boxes for each frame it identifies.

It tries to make tracks. Tries to make tracks of the things to associate what, you know, what zebra.

Yeah.

Walked behind what zebra and came out the other side.

Each box in a frame will have a unique identifier, essentially.

What are those identifiers?

Are they integers if they didn't have labels before? They are integers. Otherwise it's their previous label. So if you had a box that was a zebra that was labeled as a zebra, it will become zebra comma integer.

So a list. The label is going to be a list of label comma integer.

Doesn't that get ugly when. When the thing was. Okay, so even if there's only one zebra, it's zebra one.

Yes.

Okay.

But now we never know if a second zebra is going to show up.

No, I understand that, but I mean, I don't understand if we don't know that it's a zebra, but we can. I mean, we don't know it's called a zebra, but. And an antelope is also in the picture. Are we. How are we representing? So we've got zebras, we've got antelopes. Are we is even though we can plainly tell that zebras and antelopes are different. Well, because we don't know what the level of distinction is. Is it a whatever revis zebra or a whatever other kind of zebra it is forgotten the other kind the. A common zebra or something. So in the case where they are anonymous, where it's just objects in the image, they are just given sequential integer IDs.

Yes.

In the case where it has been identified as a known named object, it is given the name of the object and then identifier starting at 1. Yes. Or those objects. I guess it's okay. All right.

And then if you noticed, highlight video automatically converts that list of label comma one to just a row of label space one. So at least there I think the ugliness goes away.

I don't quite understand that.

Scroll down to that first car example.

That one.

The next output that has the indices. Okay, so those in the return type.

Row of that thing. Yeah, right. Okay. I mean an interesting question is let's say I want the workflow of Im. Trying to just show, you know, as if my camera was on one of the cars and I'm just making the sub video that is just that car showing the very, you know, highly focused driver on that car type thing.

Scroll down. That was one of the last examples I made before the meeting. Scroll down to Applications. Those first two are. Or first one was just counting that last one with the white car. That's a zoomed in on that one tracked car.

By the way, if I pressed play and then I got the download thing once I've downloaded. Why doesn't it start autoplaying?

I. I think we should look into that. I was thinking the same.

Okay.

We. We need to Keep track of that. State that right before download we press play.

Yep. Okay, where are we going here? Let me look what this looks like. Oof. Oof. This is pretty complicated. Yeah, that's really unnecessary. I mean, let's think through how to make that a lot simpler. What are you doing in all of this?

So actually, Shadi, there is something that might make this potentially easier. It doesn't exist yet. So we have two video of an image. Right. And that follow. You can move a rectangle across an image as a function of time.

All right.

There is an easy extension of that, like having the same thing, but instead of the image input, it will be the frame at time T. So that.

And potentially we were getting that for 14 too, right?

Yeah. Rebecca is working on it, seeing how problematic it will be. But yes.

Okay, so that would simplify the last three steps of this. All of the positions equal and things like that would just be tour video of the input video and the bounding boxes.

Okay, yeah, that makes sense. I mean, either that or video object trim track would be. Would have a property that basically is those images. Because clearly you want those images. Right? The bounding, the. That object, sub. Whatever. I mean, what I would have thought is that you should be able to generate. For every tracklet, so to speak. You should be able to just generate a video for that tracklet, for the bounding box of that tracklet over time.

You're essentially proposing to introduce different properties. Their video object track could be.

That's what I'm suggesting. Just like we have for fine faces or facial features or whatever else. I mean, find faces and facial features are two different functions there. And so, you know, maybe we want. But I don't know if that's correct. But I think, I think this thing about wanting to have a whole collection of little videos for every zebra, so to speak, its own personal video would be the right thing. That seems like a very useful thing.

I mean, I think, I think in making these examples, just a simple transpose of the current result would be interesting as well. Currently we do. Time goes to bounding, time goes to labels, to bounding boxes. But just labels going to times and bounding boxes would be interesting.

So I understand.

Okay. Do you want us to think through what properties would be useful?

I think so. And maybe we don't want properties. Maybe we want some different functions. I mean, I think that the. You know, and also you raised the question of, you know, this isn't just. This could be for a disembodied image represented by morphological components for example.

Yes.

Or just boxes with no reference to any frame as just rectangles.

Yeah, I understand.

Well, the application section here is starting to kind of evolve. Interesting use cases. There's one thing a little bit farther down that shows tracks of objects that they followed.

That one. I was going to ask about that because that's an obvious thing to do as well.

Right.

Particular once you've done image stabilization, You know, doing image stabilization. And then. Because those tracks, if the camera's wobbling around, those tracks are going to be kind of stupid for scientific purposes. You might have mounted a camera on some microscope and you might be looking at little critters running around underneath it, and then the tracks will be just fine. But if in a more, you know, in a sports setting or something where the camera is panning around, you want something where you can also lock, you know, you can do effective stabilization and lock the background, so to speak.

And in fact, camera, this kind of camera compensation is one of the features that we're planning for the next version of the. This function because that will improve the tracking quality as well, in addition to, you know, producing smoother tracks.

Well, okay, so look, I'm wondering if this is more than one function in the end, because I feel like.

But, but also we, we don't necessarily have to introduce them all now because I think we are. We're sort of evolving use cases.

No, I agree, but I, I mean, the question is whether we have a. Yeah, I don't think it really helps to call it image object track. I think video object track seems more sensible and we just have to explain that it can be disembodied. I mean, if it's disembodied boxes, there's some complicated heuristic for when two boxes are the. You know, when you see two boxes approaching each other, the zebra from the left, the zebra from the right, they cross over in the middle and then they. Then, you know, the left moving zebra and the right moving zebra are distinguished zebras rather than at the moment where they. I mean, if they were particles, it would be unclear what the identity of the things that come out is. Given that they're zebras, we think we know the identity. In other words, when they crossed over, they didn't exchange their souls. Yeah.

And the algorithm backend does. Does make some assumptions about movement and velocity and direction.

No, I get it, I get it.

But there is, There is sort of other things too, like the occlusion problem that someone, let's say it walks behind a box or something.

Sure. Sure. And then, then, I mean, in the magic trick, the, the zebra that comes out is not the zebra that went in, you know, and it pretty soon it turns into it. It becomes a Trojan horse or a Trojan zebra and lots of things come out or something. Who knows? Okay, so the question then is. I mean, I feel like facial features, for example. I like the fact that I can use this one function. I get out these different properties very obviously in an association and then I can, you know, that gives me a discovery interface basically to go and dig in and you know, look for specific properties and so on that I want. That makes sense. So I think for this, if what comes out can be something that is somewhat, you know, revealing of, You know, what's going on. I mean, clearly the highlight video thing just gives that. But I, I would say, you know, I don't know, it maybe it's too sort of expensive to have those individual sub videos be generated by default.

It is not a particularly cheap operation. I would definitely not do that by default.

Right, but what you could do by default, I mean, the thing is, If we had. Well, the graphics of something is not that useful. What we want are those tracks. I don't know what to do. You see what I'm saying? I mean, if we had a. Well, if we had a video track object that came back, then we could format it however we want. But again, I don't think that's particularly desirable. Well, let me see. So one thing is we've got sort of the time on top the. And then for a given time we've got a list of objects named objects. Perhaps an association. Is it an association where the key is the object name and the value is the. Is the rectangle and so on. Right. Is that correct?

Yep.

One day those rectangles will probably be masks that aren't necessarily rectangular. Right, Correct.

Yeah. So that is already. They can already be labeled matrix as the input. Right. Because.

No, no, there are already things like YOLO8 that can generate mask the object track.

How are we going to represent the mask as output? Because a rectangle is one thing, but what we need to have is some kind of construct that is a region and an image defined by a mask.

Yeah. Having a binary image the same size of the video is too expensive. I'm thinking offset, offset and a binary, something like that.

Well, I think what we need is a opaque object that basically represents, you know, an image ROI or a, you know, image mask or some such other thing where it's just a thing where, you know, given A particular image. This is a mask within that image. And we're not really telling people. You know, you can render the mask or whatever, you could probably extract some properties of the mask, but we're not really telling people. This is a list of pixel locations or whatever. We are storing it in an optimized way. So I'm saying. So then you could take out that mask and you can say, show me the bounding box of that mask and give you the rectangular bounding box. You can say, you know, measure the color levels in that masked region of the image, et cetera, et cetera, et cetera.

Yeah, I've seen a few data set that deal with segmentation that store as this kind of information, bounding boxes and segmentation polygons.

Okay, but why don't we have a nice symbolic object that just is called, you know, I don't know what it should be called. Not image region, image.

Not some beam.

Image, I don't know, sub image mask or something. Do we have an image mask yet or are they. I don't know what we should call it, but I mean it's an image region, image sub region, something like this. And what it is is a thing which would have a display that would just be that little torn piece of the image, if you get what I'm saying.

Yeah, I mean that sounds good. I'm not sure that is realistic for this release.

No, I understand that. I'm not talking about that. I'm saying what I think we should do is something where we have this tornado, probably we have a visual for these things which are some kind of gray rectangle with a torn piece of treasure map or something on it. Where that torn treasure map is the actual collection of pixels and where this thing is an image mask object or something, or an image sub region.

Objection. Object.

Possibly. Possibly. What object?

You say?

What's that? Vision object, Segmentation object? I'm not sure that's quite right. But. But in case we. We all know what the concept is. But then the idea would be that many of these things could return these objects, unlike this, could potentially return this. These objects and so on.

That sounds good to me.

Okay, but that doesn't solve the problem of we've got two very different things. We've got kind of the time series of what's in each frame and we've got. Here's an object, show me where it goes in all these frames. Now by the way, it could be the case that the zebra called Fred is the only thing we care about. We want to we don't even want any of the other data returned. We just want Fred's track. And so I have to say I would suggest a, you know, video object time series. That's not the right name but I would suggest maybe two functions. One of which is sort of time series first and then what's in every picture which is kind of like image contents. There's kind of an analog of image contents. That is on a per frame braces with a time series wrapped around it. I mean even, you know, one possibility would be image contents, time series, Image cases, time series.

We could also use video cases. Video contents.

Yes, we could. Yes we could. I mean and return the same kinds of things with the same kinds of properties as image cases and image contents.

Yep.

And then that's that way round. But the other way around is, well actually maybe that is the, the way around because contents gives you the frame by frame. Here's everything in it. And image cases is more like give me the, you know. Well, I could certainly imagine give me the zebras. I'm not sure that that might be a way to do that. To have those, you know, different transpositions of the, of the output.

And for those would we want to just have video input accepted and all the rest kind of ignored?

What, what's the rest?

I was gonna say keep in mind and label. Yeah, keep in mind image cases. Image contents already work with videos in 14.2 or is planned to work as there is in the agenda. So I really think what we care about is the tracked objects. So some kind of a tracked video cases or tracked video content seems to be the function we're after.

That's, that's right. I think that's right. Tracked image cases. Tracked video cases. Yeah, probably tracked video cases, tracked video contents. Video contents. I don't know, something like that. Yes.

Yeah. I'm not sure how image contents was returning it data set of, of stuff is going to be useful here. So I think we'll start with tracked video cases and see what other ones would be useful.

All right. Okay. But I, I think we're going to end up with different functions here. I don't think we should try and stuff everything into every one of these.

I, I, I have a counter proposal that maybe you probably won't like. But the two results that you're talking about are one that more or less one that transpose of each other. We want some way Nvidia object to control to switch between these two output formats.

Not particularly. I think we should have two functions. I Think they're really different. I mean, yes, we could switch, but I think it's kind of a one part. I think we all end up with a, you know, image measurements type disaster, where it's like there's 75 different, you know, jokes by numbers that you can ask for.

Yeah. Yeah. Okay.

I think. I think we should try and operate on the idea that most of the time a function, you know, yes, you can ask for this property and that property, but they have to be very similar kinds of things. Not transposed data, for example, usually.

So, going back to the names of the proposed new functions, in addition to video object track, I think Shadow, you proposed tracked video cases and tracked video.

I don't think these are proposals yet. I think you guys need to think about it.

Okay.

I think video cases and video contents would be good with the assumption that everything that you throw a video at that has a video in the name will take tracking into account.

Probably.

Oh, interesting. Okay, probably.

But yeah, we can discuss more and come back to this.

Well, this is exactly what we've talked about.

Yes.

No, on that one.

I agree. That's. That's the current default in prototype builds. It's really ugly and hard to work with.

We only put it there because Time Series can handle some of these Artworks formats.

Okay. But that's not. I. We went through this. Okay. Yeah. If we want to use tabular for this right now, which I think would not be a bad idea, we can do that.

Yep.

With the understanding that all these functions have to support, you know, transparently support either, you know, Time Series head head wrapper or Tabular head. I noticed from Ethan, our live stream asking is the way to AI upscale video to 2160 DP pixels. I have no idea. I assume you do it frame by frame, but it certainly can be done better if you're doing it. I mean, it will work better if you're doing it across the video. Do we have anything to say about that?

I think at the moment there is one upscaling neural net in the repo, but it does work on images, so you'd have to apply it frame by frame. We don't have anything.

Yeah, but that won't work well because, you know, that car will be upscaled to have a, you know, whatever on it.

There will be time artifacts.

Yes, yes. So that's. I think we just don't have that technology at this time. Are there any edit functions to do deep fake style replacements or alterations of an object in a video across multiple frames? Not built in.

Correct.

And I don't know how. Well, I mean, I don't know when people are doing that. I, I haven't tried doing it. I mean, I, I, you know, the early things that I saw put this face on this person, which is totally crummy. They were just texture mapping on, you know, from a fine faces type thing. I don't know what's happened since then. Can somebody comment on that?

Yeah, stuff actually works quite well right now. The results are much more impressive than what you're talking about.

I understand, but so how does it work? What do you give as input?

So I haven't played that. So there are several pieces of software, some open source stuff that you can run on your machine, but for example, Adobe products can do that. I haven't tried on videos yet, so. Okay, I have a good answer.

But I mean, the basic point is for us the kind of knitting of one thing onto another. Right. Which is kind of a generalized texture mapping in some sense. What function should we provide that does that?

I think I'm having trouble imagining that because I have. We don't have the technology to do that yet and I haven't played with it.

All right, okay, fine. But I mean, I just try to understand functionally what might we do? What we might say is this is a tracklet. Okay. Replace this tracklet with this other process tracklet. Yes.

Or this box or stuff like that. I'm aware images, image synthesize, essentially. But you can pass it a masked image and a prompt. In this case, you pass it, I guess.

I mean, I think, you know, it could be replace this car with a car with a giraffe or something. Instead, I don't know. I mean, it's sort of reminding me of things like color replace, which I hadn't really thought of before we, we had it, so to speak. I mean, it's sort of obvious that you can do that. But color replace is in my experience, moderately useful. And. But it is replacing not for a very specific, you know, single point in color space, but so some kind of range. Okay, Let's see. So I mean, that's why I'm saying, I think that we might find something. That. Right. Can replace. Just like we can do a color replace, we're essentially doing a tracklet replace. See what I'm saying? Or is that crazy?

Yes.

Stuff that I seen doesn't even need to specify a track. You specify a mask in a frame and then the tracking happens.

No, I get it. Okay, so it's, it's a, it's a Track object replace, basically. Yeah, well, that'd be great. I mean, I think that's, that's. I mean that. But we need to understand that because as we're building out these tracked object things functions, we got to think about that one. So that's also essentially a tracked object video replace. Because video replace as such is not particularly. I think it's not. Is that useful. I mean, I suppose what would happen if I wanted to put a big black bar over this region of the video? How would I do that? Just an image overlay video. Sorry, what?

Overlay video. And that can be timed or not timed. So you could put another video which moves or could be a still thing.

Okay. Okay.

Some ideas for I. To play with. Yeah, no, I, I think these are some good ideas to maybe prototype now and then build functions on later.

Okay.

It.

Let's see. Sorry, could you repeat. I lost concentration here.

Yeah, I just appreciated the conversation as fruit for thought.

Okay. All right. Video. Oh, I was looking at image key points and things frame by frame analysis. Yeah, I mean image key points is kind of a very poor man's, you know, object track, I think.

Yeah, but, but then as we talked about, stabilization key points are one of the key points we use for simplicity. So the bottom line is that just like you looked at Find faces that now works with video, we extended a whole bunch more functions as.

Let's see. Okay.

However, now that I'm looking at it more from your screen, it helps to think. Actually, I'm wondering. Text recognize and barcode recognize as two examples may be very sparse and we probably don't want to have a dense tabular. We only want to report stuff when we recognize them.

Kind of agree. Well, we don't want a dense tabular. So this is another, yet another issue of sparsity. Like it's in image cases. There is a giraffe that you see for just a moment in this two hour video. So I agree. That's another point that, that we want. That's a case where we, you know, we might want to prune the tabular to only show, you know, places where there's success type thing. Yeah. Okay. All right, let me see. I think there's more work needs to be done on a bunch of these. Okay. How much more do we need to do right now? I think we, we, we've got some more. I mean if I. Roger, are you going to have to go in two minutes?

Yeah, pretty soon. Yeah.

Well, I, I need to know because I, I'm trying to decide what I'M doing right now. So.

Okay. Five minutes, you know, five minutes.

Right.

What I think we have. It seems to me that we need another go around for this anyway.

Yeah.

Look at the rest of Notebook.

I'm looking at some of these. Okay. Dollar Sound styles. Well, it should be called sound note styles. I know I've been pushing. I don't think this is the right solution. I've said about 500 times that I think we've got so many of these, we should be using, you know, WDR for them.

Okay.

I mean, we can use this one. It should be sound note styles. Okay.

We actually just put it on the agenda because you were asking if you really think WFR would be better. We're happy to move it there.

Not. Well, this one has been such a painful one. I don't know. I mean, can you inventory other ones that you have? Because there's just so many of these things. The setting for a methods option. The setting for, you know, method option. That's setting, you know, Etc. Etc. Etc.

Yeah, it really feels like there should be a system function to provide all of this information.

I agree.

Well, which is WDR basically because each one is its own separate zoo.

But wouldn't it be nice if we could start having one function? I was even thinking extending information information of image, resize, comma, method or something like that.

Yeah, possibly that wouldn't be bad. Well, maybe this is a design, an LCC design discussion in which somebody should bring all the different cases. I think we, we tried to start doing that at one point, but let's bring these cases and. Fair enough. I kind of agree. It's almost like a system information type of thing. Okay. What target device to utilize? Gpu. I mean, obviously we should. Now in the modern times, machines like the iOS machines that have sort of very built in GPU capabilities, we should just do that by default.

Yep, that's. That's the work that's happening now.

Okay, Well, this is all good, but we need to talk about it some more. There's. There's a lot here we're going to need. Yeah, this is going to.

Just a quick check. You skipped past highlight video, but you did see it in other functions.

I know, I saw that.

Is that an approved function now or you want to look at it again?

We'll look at it again because it's going to be affected by whether we're doing sparse or dense or tabulars or this or that or the other.

Sure.

Right. There's a lot of good functionality here. It's going to take us another two hours to go through this.

Okay, thank you.

Okay, sounds good. Thanks a lot. Thanks, everybody.