Jay demos Mira, a tool that pulls product mentions from makeup tutorials with timestamps, links them to retailers like Ulta, and personalizes recommendations by matching users to visually similar influencers. The system also surfaces trending products and reviews in a BuzzFeed-style top-10 list.
Chapters
Hi, I'm Jay, and today I'm going to be demonstrating a prototype of Mira's web-facing UI, as well as demonstrating a few of the technologies we've put together to address issues of aesthetic preference and identity in the cosmetics world. So here's our website, and the first thing I'd like to demonstrate is how we can go through videos, search through for anything. So let's start with Smokey Eye. And one of the things we've put together is we can go through a video and automatically extract all the products that exist within this video. So, for instance, one of the things we observed as we were doing user testing is if I'm watching a video for smokey eye, what people tend to do is they scroll through the video until they find the portion at which somebody is actually applying something to their lid. We can alleviate this pain by finding the exact portion of a video where somebody is talking about the product by clicking on the product, and then it pulls up that portion. Now, you'll also notice over here that we've integrated different e-commerce websites So if I find a product that I like, I can go over here, click on it, and it'll pull this up in different e-commerce context. So here we have it at Ulta.com. Now one thing we noticed when we put this in front of consumers was they sort of have a rolling tally or histogram in their head of what brands are appearing in these videos. And they want to know what products are the most hyped, or at least for their community, what products are being most frequently used and which ones are most positively reviewed. And furthermore, one thing that's really important to people once they found a product they like, so, you know, for me, for instance, I'm seeing an Indian woman currently. Put on makeup, um, what's really important is that you can see somebody who looks like yourself putting the same thing on. So you can identify with them and confirm the fact that it works for somebody of your skin type or your face shape, etc. So to that end, we put together a different skin on top of the data asset we've amassed, which is focused on product-based search. So here you'll see several different products, and you can think of this as something along the lines of a BuzzFeed top 10 list for, like, for instance, top 10 mascaras, except this has been assembled with data. So for any of these, we can click on it and see immediately all the reviews discussing it. And if I click on one of these, it'll pull up to the point in the video where she's actually applying this mascara. So I can see some sort of objective support for this as being a positive product. And furthermore, we've, we've integrated a bunch of reviews down here. And likewise, if I want to go buy this on Ulta.com here's the obvious way that I would go about doing that. So this solves the problem, or at least attacks the problem, of we want to know what people are saying about a given product and which ones are the most hype. So here you'll see we've also pulled out highlights from videos, and it demonstrates what people are saying about this product. But there's the remaining issue of how you can tie an individual's identity into their product search. If we really want to make this comprehensive, we would not just want to know which types of mascaras are the best, rather which mascaras apply most to me. And so to that end, we've also put together a personalization workflow. Now for this portion— Anne? For this portion, instead of using myself, I thought I would use a friend. So here we go. Here's my friend Anne, and I'm going to go ahead and take her photo. Thank you. Cool. So what we've done here is these are all the different influencers that we have access to. And you'll see around our site every now and then we'll have a little toggle option. What this does is it toggles to people that we think she's likely to visually identify with. So as you can see, one of the things it's captured onto is perhaps her ethnicity and then furthermore eyebrow shape, etc. Cool. So now that we do this, if I search for something like a palette, what we'll see is these are the palettes that are most advocated for by a community of people she's likely to identify with. So as you can see, all these women here share complexion, ethnicity with her, etc. However, if I had just done a search without considering, you know, her cultural affinity or her aesthetic preferences, these are the people who would pop up. And likewise, I think this is particularly poignant if you go to something like Foundation, where here this is one that's advocated for by everybody. And okay, yes, as we sort by people who look like her, we can still see that there's a large portion of the Asian community seems to be advocating for it. However, this is the view that she would see if she had not gone through the personalization workflow. And it's hard to personally identify with any of these videos if you're an Asian woman and you're looking for people who you think look like you. Great. So this is— that's our prototype. I hope you enjoyed checking it out. And let us know if you have any feedback or questions. We'd love to answer. Thanks.
