An Avatar Engine for Real Bodies with Katy Schildmeyer, Apotheosis | Seamless

Hi, Katie. Thank you very, very much for joining me today to talk about your 3DRC Grand Challenge win. Let's talk a little bit about, about you. So please, can you introduce yourself and tell us a little bit about how you got into this industry?

So my name is Katie Schildmayer, and I am the Chief Innovation and Research Director for Design Cycle Solutions and founder of a subcompany of that called Apotheosis, which is dealing with avatars. Um, and I actually got into it, uh, through my main work, which is as a technical designer, pattern maker, uh, for apparel design. And I've been in the apparel industry since I've been about 16.

Uh, so nice.

Every job, uh, that could possibly be involved in either retail or design I've kind of worked. So I understand fully what the design cycle process looks like and how it impacts consumers and the globe and the planet and all the good stuff. But I am also a person who has been known to go in and test ideas, test theories, and see where they break down in a system and then how can they be improved, right? So I did that in retail for many years where I would go in and I would harness departments to become very good at selling, as well as in the design process where it was effective, you know, textile testing, pattern making, standardization. And so I got into the standards wheelhouse through helping several smaller companies that were startup phase learn how to do the work that we do in apparel because it was kind of madness to an outside entity and therefore standards became a big part of the work and that's how I got into this.

Amazing. So yeah, in the industry for 16 years. So as you've just said, really seen the full spectrum of challenges but also opportunities that there are in this space. You know, I think we all know that this is an industry that is really keen on adopting new tools and tech, but obviously the realities of that are pretty slow, especially in this space. It's a very, very old industry, so these things take time, but amazing. So I think I'm right in saying you won the Grand Challenge in 2023 with Apotheosis. Can you give us a brief overview of the technology or solution that you submitted for the Grand Challenge that year?

Well, it's a multi-phase one that we submitted, and, but the first part of that was to deal with the avatar solution because in the apparel fitting cycle, the biggest challenge that we've had, at least since I started in 2015 in 3D, was the avatar bone system. Because bones carry skin and skin, flesh is what we fit on. And it was really born out of if our base body is wrong consistently, then anything we do in 3D is going to have a hip problem in relation to the quality fit where we're going to have waste. We're going to consistently see that. And we actually saw that in our surveys before we even kind of went down this rabbit hole, was to really survey what was happening with fit, with companies utilizing 3D in this space before our system came along. And it was very eye-opening and very sad. To say the least, and response from several companies to change this cycle and process of the actual bone structure was one that was, you know, it was met with silence, large part. So that's where we came from, was born out of a very specific need for a solution that was based on our industry needs from a production standpoint.

Absolutely. And I think it's a testament to how big a challenge fit is that, you know, I've been talking to past winners of the 3DRC Grand Challenge. You're now the third one that is focused on fit, of course, coming at it from a different angle. But yeah, fit is just such a huge, huge challenge and we know leads to so much waste. So can you walk us a little bit through the specifics of the tool. So how does it work? Aha!

So I developed and designed the tool in relation to a chiropractor, so my sister who happens to be a chiropractor, where she was also struggling with 3D tools that you use for scanning and phone apps and things like that, right? She also needs posture recognition in her system. So it was kind of this happy medium. But our tool actually has what I call size adjusters in it. And this is part of a patent-pending solution that basically allows us— we've done something that I've invented called graded weighting, where the graded weighting is attached to the size adjusters And size adjusters allow us to expand and contract the body in a pose solution, so it's very agnostic. And it allows us to, to test the boundaries of a fabric, to test the boundaries of a size, and ensure that whatever fabric we're utilizing along with the body and the design style can be harmonized. So it's a lot more organic in that process. We can test for the, like, hip curvature, so we can change hip curvature. We can fit corsets. We can, we can add soft tissue aspects to it. So you get more of a, the compression aspect to it. And then we've also added in 3 center points of gravity. So the center points of gravity also allow us to extend, move up and down the crotch bones. So now we can get pelvic tilt. We can move the buttocks apart and together for fitting in underwear, swim, and otherwise. So it's it's really developed for how, how we actually fit in reality. So it's truly phygital in that sense.

Nice. And I love that it was, uh, the love child of a technical designer and a chiropractor too. I think that's a really interesting take, uh, very unique, I'm sure, of an angle. So I love that. Um, and just out of interest, I mean, is the idea that this would be utilized to create new standards for mass production, or is this much more kind of leaning into the on-demand and personalization theme?

So it's actually both. So we work both ways and we do extrapolate the body out of real people. So we actually use working fit models in the industry today that work as a standard fit model for like ASTM, those types of things. And we essentially are able to, you know, grade those things out. We do tackle harmonized grading in much smaller pods than other grading charts would, and they're not typically parametric grading. So again, we're moving back to actually how we have to work in apparel because the What the marriage is between a parametric form and grading in reality is very different. And that's where we see parametric patterns kind of consistently fail in regards to a size category or charting when they're scaling in and out because they're actually unaware of the logic of pattern making. And so, like, our next one is dealing with pattern making, but it's— there is a misunderstanding of a parametric form and the pattern sequence when you don't understand it. And so that's a big part of what we actually used in our approach was How do we grade? So we use 376 measuring landmarks because of grading. So grading is embedded in this. We have 580 measuring names and protocols. So it extends way, way beyond just an avatar. There's a lot of underpinnings that go into it. That are totally associated with pattern making.

Absolutely. So a lot of kind of granular, technical-specific data that you have to use and capture, which makes sense. You know, fit is a big, big topic, as we keep on saying, and that makes a lot of sense. I wanted to also ask, you mentioned a moment ago that it's agnostic, and you mentioned also 3D as well. When you say that, are you referring to the fact that it's agnostic with regards to being able to be, uh, plugged in and work with other 3D modeling systems that are available out there?

Both. So it's kind of a both, a tandem thing. So what I mean by agnostic is I can go in and I can move the joints more freely, um, which is why I do like Clo, um, and Style3D, because I can like do some of that. And in Browzwear, you can use the rotational tools. Really, it's just the rotational tools right now for posing, trying to work on getting them into much more of the free-flowing agnostic movement that needs to occur for us to just expedite the design and fit process. It's a lot easier. And being able to pose and adjust the body in, you know, and save those poses for, let's say, a dancer versus an elderly person. Again, this form can go into both those zones. So, and we, funny enough, are in a beta project with a dancer right now. So it will be really interesting to see how that comes out. But we can go into those zones which typically the current avatar systems can't. And some of the dress forms that can be brought in through the API systems, you can't really change their pose and put them into a dance position or a running position or a bending position like with a walker. And I did motocross for a while. What we did was we actually put the figure onto a motorbike in the system and then took the motorbike out because it's too much data and held that pose and then designed the pants around that pose because—

wow, yeah, yeah, they—

that's, that's the, the posturing they need for those pants. And so And there's a lot of like other 3D components that go into those types of products anyway, and that's a very technical thing. So being able to go into that agnostic place with something of that nature is a really useful thing. And that's kind of what I mean by agnostic is that the designer themselves can become much more organic with it. But yes, it is more interoperable where we work on the API chains that would say you can bring it into Browzwear as well as CLO or Style3D or whatever you're using for design.

Thank you. Amazing. Thank you for clarifying that. So I think I'm fair in saying that you are in build mode. Is that fair to say? You're still very much in build mode at the moment?

We actually do make it for brands right now. So there's again more of the brand standard and working with the fit models to create it from the scan to then, you know, put it into their system and get it all situated. And then we are working on the cosmetic features and things like that to be launched more towards August and September of this year. Nice.

Okay. And up until this point, what have you found to be kind of the, the, the biggest creative or technical challenges that you've come up against when developing the solution?

That's a— well, there's a lot, but I'll, I'll keep it actually to the one that's probably the, the most blatant one, which is how different teams in the design cycle look at a dress form and evaluate fit. So one thing we found was when we made the cosmetic form and we put it into a brand, and a brand, let's say they had an evaluation with a head of design, head of merchandising, and then the TD room, right? And they were all evaluating the final fit. When it was on the cosmetic avatar, the design, so the creative ingenues and the merchandising ingenues, the marketing people, were so focused on the avatar and wanting to change it and play doll and dress up and do all this stuff. They were overlooking the fit aspect of the clothing and the technical designers were going crazy because they wanted them to just focus and approve the fit. Please, no discussion about how this person looks. And that made me realize that happens in reality. That when a creative designer is thinking of this idea in their head, from inception, they never ever think of how it's going to grow in mass, which is what we talked about in our, our, uh, challenge is that you really should design for your target market. If the target market is a size 14, design on that first, make your idea come true on that first, and then see how that plays out cosmetically after you do fit. So that you can see the impact on what a real person looks like, not a model. Because again, they were picking apart the person, and that's probably something we should walk away from and not do, uh, because from a, a realistic standpoint, a real person in a real fitting room somewhere in the world is trying that on, and they are not a model. And they shouldn't be picked apart. But again, that's a context of fit that we don't talk about, and that is aesthetic fit. So is it the right fit for the person, and are we telling the truth of what that looks like on that person? Call it skin color, hair color, eye color, whatever. In reality, when we go to sell at market, And oftentimes we're not, and that creates a dissonance between the consumer and what they expect to get. And then when they go to try it on, if it's not even made for their body shape, I mean, you, you have a first-stage problem right there. So that was the, the most blatant problem we saw.

Yeah, thank you. Um, and so I guess what I'd love to know now is what made you want to take part in the 3DRC Grand Challenge? You know, what was it about, about it that kind of stuck out for you and you thought would be a good opportunity?

Well, I sit on standards committees, so I am part of the P3141 standards with the IEEE, which is 3D body processing, and that's how I kind of found out about it through my IEEE emails. And then I also sit on the digital wearables. I'm a co-chair for the digital wearables in the Metaverse Standards. So it seemed like it was a really smart thing to do in relation to a standard to help along a standard to bring it to a place that apparel needed because we are lacking some of that in the digital space for apparel. And we really do need to have clarification in the avatar genre of things to say, we have this, it does fit, this avatar is for gaming. And unfortunately, they are not the same. Because computing variables that go into one need versus the other need are very, very different. So that's, that's kind of what got me into it was to try to clarify the differences and create a standard.

Amazing. I love that IEEE have been such a good partner for the 3DRC Grand Challenge for so many years. It's always great to have them involved. And yeah, you know, the 3DRC is all about that itself, trying to set standards around the people, process, and technology behind DPC. 3D. So a good, a good marriage there between our, our values for sure. Um, what did you find as being the, uh, the biggest benefits that you got out of taking part in the challenge?

Um, well, one, uh, it was being able to open up the conversation and bring in, uh, new awareness, or also have people who are kind of in agreement with the struggle of needing a better tool and having them test it and give feedback on it, which so far so good has been excellent. So it allowed us to open up those conversations a little bit wider, and a lot of technical designers who have not been taught really how to do digital fittings, mind blown. So for them it was like, oh, I didn't even realize. And then it started snowballing and I've seen since several experiments and ideas come into life on LinkedIn and other spaces. So it's really good in that sense. I love opening up the Pandora's box of this.

Absolutely. And I think also, like, us at Pi Apparel too, like, all of our events have always been trying to, you know, just create that space for a conversation to take place. I think this industry, as I said, I think there's a lot of people in it that want change. There's also a lot of people that don't want change. But, you know, creating kind of space to just have those conversations and open the door is so important. So I love that that was kind of your, your main driver for taking part. And of course you won. So how did it feel to be recognized as the Grand Challenge winner in 2023?

Um, I think it validated a lot of, uh, the intuition, but it, it also validated, as you said, change. Um, and I'm, I'm a big believer in if a door is locked that shouldn't be locked, like to, to hide or to keep change away. That's probably the one I'm going to open or work on picking. Pick it down. Pick it down. By the way. Yeah. So it was a catalyst in that respect where it supported having change. So I wasn't some random voice over in the corner going, I hate this. It was saying, okay, is there a tribe out there that also feels this way? And that also wants change because, I mean, it's going to come regardless. And you might as well, as I say, like jump out of the plane and go and take the ride, or you can stay hidden in the locked room and let time pass you by. And then when you walk out, time has passed by and you don't know what to do. So which one is it? And so we were very supported in that sense.

Nice. Good. I'm really pleased to, to hear that. And I, and I'd love to know, since winning, how has your solution changed, scaled, developed? Like, what's changed since? What's going on?

So quite honestly, We, when we first did it, we had much less landmark points because we said, let's see if we can get away with this process. But since doing that, we actually added more. And the way we test for everything is much more in the 4D than just 3D. So we added in the 4D components of measuring the body in motion. so that we could weight it a little bit better, a little bit more correctly, um, in regard to motion and movement. Uh, and then, uh, again, it was— we first wanted to just offer a cosmetic, uh, relationship to the rig and the solution, but out of Again, working with the technical designers, seeing how it plays out in a fit review process, we found that it was much more important to have a non-cosmetic dress form body, still human, still with all of the nuances, but just take away all of the features because then the conversation could be around fit and scale and all those things. And then what it also led to was, again, Design Cycle Solutions, which was how do we have a greater discussion with brands about this process and using these tools correctly through their system where it becomes like an iPhone of tools where it's simplified and you know when and where to apply the cosmetic features, when not to. Um, and then how does that apply into AI? Because we understood that, um, AI was going to utilize the, the body and redress the form and do all that stuff. I already knew it was going there, but I knew that if I had scale, just out of like testing my own self in an AI system, if I took the scale of my form and body and then pushed it through, I found that it stabilized. So I could stabilize the shape of what it was designed for and then push that through. Now with our avatar, you can again change the shape of that figure So you could do an hourglass, an athletic, and otherwise, but test that, that product on those different variables. And if it fits two of those variables, show two of those variables in your e-com through the magic of AI, which is our other company that we kind of have called Weekends Inc. And they just launched Content Plus, which is an amazing, amazing AI. But it, it does pull everything through in that sense, uh, where it marries everything together.

Yeah, it's funny because, uh, just before you brought up AI, I was thinking in my head, this is probably the only conversation of late that I've had where AI has not come up yet. So I knew it was going to make an appearance at some point. Uh, but no, great, great to hear. And also, like, I want to mention too, I think I'm right in saying this, your website went live not very long ago as well, right? So for anyone who, uh, watches this, uh, or reads, um, the summary, apotheosisgroup.com, I believe it is, which I'll also put in the notes. But everyone should definitely go and have a look and have a play around with it, which would be great as well.

We also, we also will be at DiBarto in New York for an art installation known as the Planetary Self that's hosted by UNESCO. So yes, that's going to be a very brilliant thing where there's some textile experts and our founder Sue, who's a master fit expert.

So is that, is that why you're working with a dancer at the moment, or is the dancer a separate thing?

Actually, so we're working with a studio that does something called bullet time photography and scanning. So bullet time is something that comes from The Matrix, where he's, you know, pulling back and there's a 360 spin, right? This gentleman named Eric Parr in Toronto is is a National Geographic Award winner photographer, and he works alongside a dancer, and they capture movement in light and dimension. So we found that it was the only, like, right thing to do where—

Absolutely.

These variables of how far can we expand 3D and the body and the challenges that the body goes through, which is very much how that duo works together. They're very ingrained in being present and bringing that into their art forms of photography and also working with time. So they look at nature in the form of time, which is really like a very interesting thing to me. So it's more of a, I think, an art, artistic passion project, but we are testing it mathematically for, you know, outputs related to the body and how we weight. And again, furthering our skill with that.

Nice. That sounds amazing. Well, I'm sorry that I'm going to miss that, but it sounds good. Please send me a link to his work because I'd love to see it. As well. I feel like it sounds like something you need to see to understand it. But yeah, send me a link, I'd love that. I also wanted to ask you, we only have a couple more things to cover, but just out of interest, earlier in the conversation you mentioned something about being part of a standards committee around the metaverse, which is a term I have not heard for a while. Can you tell me just a little bit more about the work there that you're doing? I find that very interesting. Interesting.

Yeah. So Metaverse Standards is a very, very big collective of digital artisans, engineers, and of course, recreational gamer artisans that hack their own things together in Unity and in Unreal. Essentially, the standards groups work largely on trying to create and understand how these digital assets move through the world, right? So I work— well, I'm part of the avatar division, but I chair in the wearables because there's a huge, huge, huge problem in the ontology. So how we name something in AI and its relationship to the metaverse as well, because we have to think, how do children today learn about clothing? I know I learned about it through a catalog or through a magazine. Or watching MTV Runway, like shows on, you know, runways, or obviously learning the craft in home economics. So I learned the nomenclature of clothing through that process. But today they learn it through going and buying the skins and the wearables that a game character has. Or that an app has, right? Like a Fashionista app for kids. Those are the way they learn about clothing. And so definition becomes very important in that sense, because when you go into reality, the definition can be different. And a good example was that I Googled, give me the top 10 colors used in apparel today. Right. Gave me 10 names, but the top 5 were all associated with a men's dress shirt collar.

Right. Okay.

And they, they were just descriptive terms of that collar. And it's because people are not tagging things with metadata correctly. Again, meta, metaverse, metadata. And metadata will be exceptionally important in the future, not just for the metaverse of things, but for DPP, the digital product passport required. Like, you, you need it in how you tag your product in the e-commerce space to be sold, right? And again, if it has a digital twin in a gaming space, so let's say it's Ralph Lauren doing their clothing and they turn around and sell the jacket in Fortnite, which they've done. Yeah, that digital twin entity has to carry the— it should carry the same information across. So, uh, that's a standard that we're trying to set in the metaverse in relation to language of the wearable. And then the other aspect is, uh, like wearables that have to transverse between apparel product creation and move into a Roblox or, you know, Ready Player One type of situation, or even into like a filter or a digital fit kind of system. Where are the underpinnings going? How much data does it compute? How How big is the file initially and how do we make that more simplified for a multiplayer game? Because all of these things have different underpinning computing that is required. And in, I mean, e-commerce alone, we figure 2 years from now, it's going to need to utilize probably over 2 trillion equations that happen daily on online shopping.

Wow.

And so these variables that we're talking about become very, very important across that process.

Yeah, absolutely. Thank you for going into that. It's a space I find so interesting. I actually, the reason why I'm in Amsterdam is I actually left Paya Paro for 2 years but then, but then I came back. But, um, I left 2 years because I went to work for The Fabricants, which is a, a digital-only brand that is based here. Yeah. And, uh, so I worked with them for 2 years. And yeah, I'm not a gamer, uh, you know, I'm not from that world at all. I'm like you, you know, I, I, I, I go in— I, I grew up going into brick and mortar shops and doing all of that kind of stuff. It was a brave new world for me, and it was just such a fascinating space to work every day between a fashion design group, a very tech group, and then a gaming group, and kind of like finding the intersection of all of that together. It is a very, very interesting space. So thank you for going into that. For my own, uh, uh, interest, is it's great. Um, so let's just finish off with these two then. Um, I guess number one Looking back, what was the biggest learning that you took away from taking part in the 3DRC Grand Challenge?

Well, I mean, it's the process of, I think, taking a very big system and giving birth to it. It is labor. Intensive. And, um, because of the level of quality that I'm, I'm used to wanting to output for the industry in terms of general work that I do in general, uh, I am more perfection-oriented that way. And so I had to more internally give myself leeway because anything that evolves has to have those imperfection stages. Because again, I'm also in that evolution alongside the idea. And so it, for me, it was much more personal because I had to let go of perfection. And open the doorway again to, uh, change and understand what those were for me and be more present to that situation. Uh, so the process for me was definitely more, uh, aligned with how change occurs. And how— what I fear in that and letting go of that, uh, to allow progress of something greater than myself to come through.

Yeah, nice. Good answer. Um, I feel like I keep on going off on tangents, uh, during this conversation. But another thing that I meant to ask you before is, um, are you already working with any schools, uh, or students at all with your tool? Because I feel like this is something that they'd love to get their hands-on and play around with.

Uh, that— speaking of which, uh, I was actually going to be sending this off to Dr. Simeon Gill, who is a peer and fellow researcher on a few other papers. Um, and then the other school that I was intensely looking at, uh, because we do the IEEE P31 4-1 does a little kind of posturing research group and fit group with Cornell in May. So, I was looking at discussing those options with Cornell and their human ecology division.

Yeah, amazing. I just love— like, I feel like one of the things that we see a lot at our events is, um, you know, more often than not, these students, assuming that they get given the opportunity to play with these tools, they can then go into brands and into the industry, and they're already equipped with all of these new ways of thinking, new ways of doing things. And I know that technical design is obviously a huge area, and a lot of schools offer courses in that. So Perhaps if anyone from any of those schools is listening to this too, I'm sure that you wouldn't mind if they reach out to you.

Oh no, I don't mind.

Whether there's a way that you guys can work together. Yeah, it'd be great. Perfect. So let's end on a high note then. As you know, the 2025 Grand Challenge is live. It's the 7th year. I believe I saw on LinkedIn that you're also going to potentially take part again, which is exciting. I look forward to seeing you there. To see what you submit. What advice would you give to other startups, researchers, or innovators that are considering taking part in the challenge?

You— I mean, it's important to get ideas out into the world regardless of whether you win something or not. You're creating a point in space and time for your idea to grow. And that is the start of how you get it into being and get it into the next stage and the next stage. So it's baby steps along the process. But the Grand Challenge is a wonderful group of people. I think everybody in that group is interested in change. Like, there's no— I don't think anybody in that, that, uh, party of reviewing these things is somebody who wants to hold anything back. Um, so it's open up the floodgates and go. And again, if you don't get chosen, it doesn't mean that it's not viable. It, it actually means you've introduced it into space and time and reality.

Amazing. Well, thank you so, so much. I'm sorry that I took our conversation all over the place, but it's been super interesting to talk to you today. I've loved it. I've loved it. So thank you very, very much for your time.

Yeah, thank you very much.