Jay Hack demos Mira, a search platform that pulls together beauty videos, reviews, and product listings. It automatically extracts product mentions from videos with timestamps, analyzes sentiment in reviews by specific attributes like blendability, and matches users with influencers and content based on their skin tone and type.
Chapters
Hi there, I'm Jay, and today I'm going to be demonstrating Mira's vertical search platform for beauty content discovery and exploration. I'll show how we can take the many different types of fragmented content that exists within the beautyverse, if you will, such as influencers, videos, images, products, reviews, and stitch them all together into a single unified data asset, then surface that data asset to users in an informative and fun experience. Cool. So I'd like to start with the user. And one thing that we have typically seen is people's beauty search process will begin with them looking for a look that they might enjoy. So a night look. And they'll start with a video. So here, this video particularly appeals to me. And one of the things you'll notice is immediately what we're able to do is automatically extract all of the products that exist in the video, as well as the point at which they occur. So for instance, I can click here and immediately see the point at which she's using this palette in the video. Now this is based on our observation that traditionally what users do as they watch a video, if they have a specific portion that they're interested in, they'll actually scan through manually until they find that. And this is a, you know, sort of a tedious and not fun process to go through. Now for instance, for this video, I might be interested in what concealer is she using because I need to buy a concealer and she's done quite a good job of it. So I can go over here, say, "Okay, face." Here we have the concealer, click on it, and immediately it jumps to the point in the video where she's actually using this. So I can see this product in action. Now, sort of the logical next step for a user, which we often observe, is if they see this product, they think, "Oh, it looks great on her. I wonder if it would work for me." They want to do more of a deep dive on this product. Now, the alternative is I open up 8 different tabs for Sephora.com, beauty blogs, etc., online. However, here we can just click directly through to this product and see a view that brings up all the relevant information about this. So here we have details about the product, price, etc. This is integrated with external e-commerce websites. Then also a bunch of different videos that describe this product, this product, excuse me. So clicking on this brings up the point in the video where she's actually describing this product right here. So I can get many different visual examples of it working on people who are already accredited beauty influencers. Or have some sort of community validation placed upon them. And as you can see, there's many different videos here. For any given product, we find, you know, between maybe 15 and 100 videos will actually mention it explicitly and talk about it. Now after you've sort of seen this visual confirmation that people who you like or you trust have tried it on, they look great, another thing that users always want to do is confirm through textual reviews that this product is good quality. And so it's sort of a broader community validation behind the product. Product. However, what we notice people traditionally do is they'll scan through reviews one by one until they find one that addresses the things that they actually care about. In a concealer, the things that people might care about are, is it long-lasting, is it blendable, is it creamy, etc. What we've done is performed an automated linguistic analysis of all these reviews that surface the parts of the product that people actually care about. So if I search for reviews about blendability, we'll automatically highlight the portions that talk about this. And then furthermore, do things like for the Cremis, I guess this is no negative, crease resistance, cool. We pull up the reviews that are actually negative about this as well and separate them or segregate them from the ones that are positive. So here you can see, it creases on my eyes. Here's one individual who thought it was bad. And this gives us the ability to do things like rankings under different properties here. So this has 29 positive reviews that mention the long-lasting content and 3 negative ones. Now we can not only do this for text but also videos since we, you know, can see the captions of the videos. So if I cared about the crease-resistant properties, here are all the videos that address that fact. And furthermore, since I'm now looking at individuals, I can say I want to filter by skin tone, so people who might have a similar skin tone to me, and furthermore the skin type, so I only care about people who have dry skin. And this makes sure that the person who's talking about it is somebody that I can personally identify with. Now, as I'm sure you're aware, there are many more dimensions to "do I identify with this person" that might have to do with aesthetic preference, does she actually look like me, etc. And so to that end, we put together a personalization workflow that I'd like to show you here. Cool, so for this onboarding portion, I thought I'd bring in a friend of mine who's actually an active cosmetics user to demonstrate what it's like for her. This is Evelyn, and here we go. Cool. So we ask to mark the brands that she most appreciates. So for Evelyn, this is going to be— looks like Benefit and MAC. And next, what we're going to do is we're going to run an actual visual analysis of her face. So you can see it's tracking her using Snapchat Flower Crown. She clicks through it. And we've run this against the backend and found people that we think she's likely to visually identify with. And then not only that, but share a similar aesthetic. Preferences based on the brands she marked. So we can see here, from head to toe, has mentioned Benefit 16 times and MAC 20 different times. Oh, that's interesting. I actually follow her on Instagram. So I really— I don't know if you heard that, but Evelyn just said, "I actually follow her on Instagram." So I would count that as a success. Cool. So now what we've done is we've actually stored a profile of what she looks like and what her, quote, aesthetic preferences might be. And we can actually integrate this into her search process as she hops across our data assets. So let's say we go back to the NARS Creamy Concealer. And what you'll see is that the people that we pull up for representing this in video are actually people who we think she's most likely to visually identify with here. So right away, bam, she hits the page. And she has something tangible she can look at. She can live vicariously through these influencers. So if you go ahead and click on that, it'll pull up again the point in the video where she's actually using Cool. And then this also sort of integrates down below where we say, if you're watching videos, you want to see people who you actually look like. I can flip this switch, post to people who look like her. Got it. And then furthermore, what did you say your skin type was? I have oily skin. Oily skin. Cool. So we can say filter to people who only have oily skin. And it looks like we have the same woman here. Oh, that's exciting. Cool. Great. So thank you so much for watching our demo. I hope you enjoyed. And I'm looking forward to hearing your feedback. Thanks so much.
