Invisible digital twin brings next stage of augmented reality

This is the first company I've seen that explains why what could come next year as Meta and Apple bring new kinds of augmented reality devices to bear. What are we seeing here? First of all, what's the company?

Oh, we're Singulos Cingulos Research Inc. We're based in Vancouver, Canada and we working on the Perceptis platform, which allows architect to understand objects in the environment.

So you're using a standard camera, standard iPad, looking at a physical chessboard that only has pieces on one side.

That's right. And I'm playing against a virtual opponent. And in each move, the perceptive platform is understanding where the objects moved in three dimensions so that it can tell it to the other side and then the other side can respond.

So there's a lot of computer science going on here. First of all, it's recognizing each object and knowing that that's a pawn or that's a rook or that's a knight or a king or a queen, Right. And then it's also building a digital twin of the checkers of the chessboard. So it's, it's keeping track of both the physical things and it's putting these virtual opponent on the board, right?

That's right, that's right. So it's keeping track of a merged view of the physical and virtual worlds aligned to real world 3D.

Right now people might think you're building a chess app, right? But you're actually building a platform that other developers can build all sorts of different things about the physical world, recognize things in the physical world, and then be able to do all sorts of fun things with them. Right?

Exactly. Yeah.

So we.

We are actually about technically objects and allowing application developers to build on top of our platform. So we're excited to get someone else to build a, a chess app and take on the next level. This is pretty raw. We got the virtual pieces, but you could imagine if you wanted to really have a chess app on the app store, you'd want to build it out, you want to automatically connect to virtual opponents and you'd want to do all this. So that's what we're trying to enable. One of the interesting things about this one is we don't even care about the board in this case. So we don't detect the board. We actually detect the pieces, and based upon where the pieces are, we know what the board must be.

Oh, that's interesting.

Yeah. So we could actually project a virtual board if we wanted. So you only need the pieces, you don't need the oh.

So if I just laid the pieces out in the right order on a.

Tabletop, it would just project a board.

Oh, that's cool. Let's say I am a developer. Let's say I really understand, I'm a senior developer, I understand how to do some form of AI. And this is Unity. I would assume, or something that you're.

Presenting the 3D we're using ARKit for, to manage our virtual space stuff. So it's quite nice. So it's really not much beyond. If you've been developing AR apps in the Apple infrastructure, it's basically one more piece of information and we already align it to the ARKit worldview. Right. So we're finding the object, ARKit has a view, and we're basically constantly updating our data structure that we provide to you that says, hey, here's the objects, here's where they are, here's what they are, and you move along.

Where I was going, let's say I am a developer like that. How long would it take me to take your platform and do something physical and something.

It shouldn't be very, very long. I mean, it took us, one of my developers, a few weeks to put together this chess demo. You had to get the engine and do all that. Yeah. So if you have the idea, it really is not a huge amount of development because there really is a simple data structure that we're providing. We're saying, look, there's a pawn. The pawn is at this location in three dimensions. And then from there you say, oh, well, if the pawn is there and the rook is here, then I know that this must be the way it is.

Switch over to the LEGO demo if you could.

Yeah.

Because this is great. The LEGO demo shows that you can deal with very chaotic group amount of physical things. Right. So here you're going to open up your app.

Sure, sure, yeah.

Start looking at legos and the computer that you built starts recognizing all of the physical objects there. You're building a database in 3D of all those objects, Right, Exactly.

What we're doing.

Your system is keeping track of each of the blue square there.

Blue square is just to tell the user that, yeah, we've got that one. But under the hood, we're remembering where each of them are so that when we come back, or we zoom in or we zoom out, we're able to not re inventory them and then we know where they are when we want to go build. So let's say we go ahead and we want to build this moose. So I'm Going to press the moose. You can see that this is actually popped up from where they're supposed to be. And if I press this, you can see that it remembers, of course, where they are because it discovered it in three dimensions. So it knows where each piece that we're going to need is. And as I move around and I can even look up, right. I don't have to stay focused on the Lego because we understand the world. Right. And so then I can say, well, no, I'd rather have this duck. So it's going to build the duck. And I can say, oh, there you go. There's the pieces that you would require for this specific.

Could you use this for stuff that doesn't have a strong physical form like, like a hamburger or, you know, because Legos have a very strong physical form. You know, every Lego has a very specific shape and all that that your computer vision can see and understand. If I was looking at something like, you know, sand or pepper, you know, could I do that?

It's a great question.

Yes.

Especially the framework, the framework of dealing with three dimensional objects and understanding them over time and space. Now the difference is, at the base of our understanding are these neural contacts that we build by processing three dimensional models. The question is, do you want to understand every hamburger in the world or just a subset of hamburgers? You just want to find McDonald's hamburgers because you could easily have a model as you move to broader things.

I'm trying to think I have some, some apples and a banana next to, next to my computer. And so you start thinking, oh, could you recognize those kinds of objects and do things like, you know, oh, here you, you could make this kind of meal because you have these things in your house, right?

Yeah, and we believe so. Right. I think that the fact is you just need to have a nice model. And then it's also down to what are you trying to distinguish? So LEGO is actually a super hard problem because one by two plates and one by two bricks are very similar. So to make that work, we actually integrate a lot of information over time to get to the right answer. And bananas versus oranges, you can be less picky, right? Yeah. And so there's different criteria there.

That's really great and crazy. I hope people learn more about you. Where do they learn more about you, by the way?

So the place to find us starting tomorrow is perceptusai.com on there we'll have a bunch more videos that show off different capabilities, different applications, and for developers who are interested in licensing this, you can sign up and request a live demo where we can show you some more stuff about how it all works.

It's very rare that an entrepreneur makes me this excited, so congratulations. This has a huge impact on next year. I'm excited to see what people build with you and where you take your company, because this is right on point for what's coming from Apple and others.

Cool. Thank you so much for taking the time to look and think about it so deeply.