#21: Improve your customer feedback with Kraftful

Man, this is going to be fun show. Because at Rackspace we always asked customers for their feedback using a variety of different ways. My guest today is using AI to get customer feedback. Who are you?

Who am I? I am Yana Wellinger. I am founder and CEO of Kraftwerk. Graphl is a copilot that helps now thousands of product teams build better products by listening to users at scale, which is asking them for feedback and helping you incorporate that feedback into the product roadmap. So that's me.

And you're not just doing Net Promoter score, right? That's one thing we did at Rackspace. We wanted to know how likely you were to recommend our product to other people. Right. And I guess that's a core piece of it, but you do a lot more than that. Give me an idea of what I could do with your service to get customer feedback from various places in my apps or services or communities.

Yeah, so we do both. Helping you collect data, new data, and then use the data you already have. So the first step is really to just connect all of your sources of feedback to Craftful. So that may be app reviews, support tickets, call transcripts, survey data from various sources, and lots and lots of different sources. And then what Craftful does, it goes out every day and collects your data from all of those sources. And then we crunch the data and we show you product insights, basically prioritized lists of feature requests, complaints, loved features, competitors mentioned, that sort of thing. And then we let you chat with it and ask follow up questions. And then once you've really exhausted that and you're sort of like, okay, that's cool. I know everything everyone's ever told me about my product, but I still want to know more. We then have these AI generated surveys and interviews. The AI generated interview is actually new, we launched that this week where you can say I want to learn about this topic or I want to learn more, kind of dive deeper on the feedback I've already gotten and we automatically generate a survey for you. And then another option now is to then also have the survey kind of interact with the user and ask follow up questions and really personalize the interview to the user and make sure that everything you can learn as much as possible from this particular user and not waste their time with something that's not, not, not interesting to them. They can really complain about the product and the way they want to do. And, and you will learn so much more because you could do that with each individual user.

Yeah, tell me, let's go back up to 100,000ft. Because you're building a tool that's using AI for people who are trying to get use AIs to do new things. What are you learning about the market and what are you learning about businesses by doing this? And maybe what's the challenge? Well, let's just cover that one question at a time. Scoville.

Great, great questions, all of them. I think that, you know, one thing we're seeing that's sort of like the maybe not so great news is that, that companies are really consolidating and are trying to use AI for, for a lot of things. And, and of course we're helpful there, right? Because we help you do a lot of things and learn a lot of things and then still have time over to then act on all of those insights. And so I think that that's kind of. We hear that this is, you know, we used to have big teams that used to do this kind of stuff, but now you guys, you guys are there. So we could just sign up and get the same insights in a couple of minutes and then start acting on them, right? So that's, I guess that's kind of like what I'm seeing in the industry more generally. I think the other thing is that there's this sense of everyone needs to use AI, and so everyone's kind of looking at, okay, well, the developers got their AI, right? Like, that's great. They got the best, the latest, greatest stuff. But like, what's there for me, right? Like Intercom has their own AI for support, right? And so they see us as this solution for product teams that is sort of an equivalent of that, but specifically to serve the needs of product folks to be able to use AI. So we're sort of like this human AI interface for product teams.

No, that's awesome. How good is it at prioritizing? Because how good is AI at doing this, at doing this kind of work?

I actually think that AI today would be like a mediocre pm, but who knows.

What I could do? Maybe.

I don't know. I don't know. I haven't discussed you, but I think that at least where we're at now with AGI and things like that, things are going to get very different. But for now, we're at a universe where humans are still expected to figure out how to build a car when what they're hearing from users is, I want a faster horse. I think there's still that at a high level. That's the difference there. Our AI will tell you, here's the Things that are coming up most often and soon it will be telling you here's the things that are coming up most often with this particular sentiment so that you could kind of act on that data and you can kind of slice and dice it and see here's the kind of thing that's coming from these kind of customers. But generally kind of this more high level prioritization exercise is still left to the product manager to do because there's still some work humans got to do.

Yeah. Is that employee feedback being fed back into the system to, to help train future versions of crapful?

Yes, in some sense, yes. There's definitely. We don't have. We don't attempt to do that kind of prioritization yet. And as a result we're not exactly collecting data on that kind of prioritization. But the prioritization we do do, which is based on number of mentions and the analysis that is made in sort of like the way in which the we surface dimensions, all of those things are editable and when they're editable that is sort of the, the intent is for the model to get better and better over time based on how the user interacts with the data so that it can even like it's not even so much as the model gets better, it just gets better at interacting with this particular human and behaving like this particular human wants to behave.

I'm getting feedback from a variety of places from LinkedIn messages to Twitt DMS to emails. Can all of that be fed into the system as well or does it have to come into a structured place.

So far it has to come into a structured place and I'm with you most of my. Because kind of our, our user base found us mostly originally through Twitter. Most the, some of the best feedback I'm getting is coming via Twitter DMs and we have a. We've sort of. I've hacked together our product to, to basically be able to take that in when I just like every time I see that I copy and paste it and put it into this, our freeform text analysis so that it all ends up in the repository. But there isn't a way yet to connect Twitter DMs, LinkedIn DMs and some of these better sources. For me it is really mostly connected to kind of support ticket sources and app reviews and survey data and some of these other ones calls you can, you can transcribe calls and they, they automatically land in graphical and get analyzed.

You mentioned that you're going to turn on some sentiment features. So could you listen to a call and understand that the customer is angry or joking around happy? Right. The difference between the two often is tells the support team, you know, this is a more important thing to solve.

Yes, exactly. We're going to do both a binary sentiment from positive to negative as well as the actual emotions that this feedback exhibits so that you can get a sense for. This type of feature is usually associated with this specific emotion, this anger or this frustration or this excitement. It also helps if folks are requesting features to get a sense for like, what, with what urgency, Right? Like what is the motivation there? Is it sort of just like, ah, man, you know, I guess I wouldn't mind dark mode, you know? Or is it like I need to be able to log out of this thing so that it's not recording me every single second? Right. Like something that feels a little bit more urgent.

This stuff is real important. You know, I was thinking about new bank down in Brazil. They built their own customer relationship management system. So if a customer calls in, they get routed right to an employee without going through a phone tree. And that's one of the ways they, they differentiated themselves from other banks in Latin America. And they're now a public company doing that. What, what makes. Give me some best practices. What, what would make a great company in your eyes for getting really great customer feedback and being able to act fast on that customer feedback? Because I think people are like product managers or executives are looking for that kind of let's get this product, this company much more in tune with the customer. Right?

Yeah. I think it is the interest in learning what users are saying. Because a lot of companies, particularly big brands, there is a bunch of feedback out there. And I think I as a product manager, the reason I'm building this is before starting craftful, I was a product manager and this is my pain point. I always wanted to know what are people Every company I would join and be like, where are all the places where users are giving us feedback? How can I get into all of these different places? And support teams would be like, well, PMs don't really do this here. You need to have certain credentials and stuff. But I think the number one thing is to have that curiosity to know that our users have spent time giving us some feedback and we should care to make sure that we've listened to all of it and addressed everything we can and think is important and then also then try to learn more. Right. So I think that curiosity is really, it's more of a culture element. But I think that's critical for then being able to set up. Right. Like, if you, if you have the curiosity, it's really simple. You go into craftful and you connect some sources and it takes a couple of minutes and you're, you're up and running. Right? Yeah, but I think, I think, I think it is the motivation to, to learn is really, really important.

Yeah. You track if an employee has gotten back to that customer and answered the question or complaint or problem or whatever they're giving feedback about.

Not yet. Definitely on the roadmap to be able to close the loop with the user and let them know. Because now with AI, the really cool thing is that you can really let them know, hey, we're going to build this feature, or hey, we're not going to build this feature, but we're going to build this other feature that actually addresses your problem in a much better way, and here's why. And you can do this in a really personalized way that takes into account what they said and what, like what you're doing and what maybe what the team talked about. And you can do that with millions of people. Right. So that's the really cool, cool time we live in. Right.

Does it warn exec or do you build a dashboard for executives to know, oh, your Net promoter score is going up today or it's going down today or something like that. So that, you know, an executive, particularly at a bigger company with a, with a bunch of people doing this kind of work, could they have a dashboard where they can see the health of the product and see, you know, whether people are just reacting negatively to do it today or particularly to a new release that gets put out, that kind of thing.

Yeah, absolutely. So the particularly that sentiment analysis that I mentioned, that is going to be a dashboard that will let you track this specific sentiment of users over time and also see what's happening today? Because so far, for most part, we really focused on the builders, helping the builders learn what users are saying and then using that data to write good product documentation, be able to push that to the project management tools, write the good prd, act on the data. But we're hearing a lot from execs. I just want to see that my people are doing, doing the work. So part of the sentiment is going to be to have an Insights panel where you can kind of just keep a pulse on what users are doing. I also say that you can do NPS surveys via Graphful. You could do any kind of survey. You can use it as a standard Survey builder if you wanted to. But I'm not a big believer in mps and so I think that while we will continue to support like if people want to collect that data, it's not probably ever going to be kind of front and center. And when we say you're tracking the sentiment of your users, you can learn so much more based on actually listening to what they're saying and their sentiments and what they're requesting at what frequency. There's so many different signals now that I think is going to be much more interesting to teams.

Could you use this to run a restaurant chain? Because a lot of that customer feedback comes in on Yelp or Google reviews or some other place like that. Can I hook it up to Yelp and get and watch my restaurants review? The better question is what kind of business is this great for and what kind of business does it suck for?

This is great for every business. And I think that's sort of the thing that was really eye opening to me when we first launched it. And we had thousands of users sign up. We sort of broke every API that we were using, we broke through the rate limits and we had to kind of go out and try to get extensions. And what we saw was that we had product teams from every tech company sign up. Not surprising. But then it was restaurants, restaurant brands, global retail brands, financial institutions, car manufacturers, public agencies, cities. Because everyone has something that's a, that's a product. A restaurant produces their food. Right. Or they also have websites and sometimes apps, which is kind of what I think they're more likely to be looking for with craftful just because back then we particularly supported app reviews. And so yeah, it's really everywhere. Pretty much everyone's a builder, even if they don't have specific titles like product managers or other kind of what you would traditionally see in tech companies. But I think we did a good job of positioning it as something that they would want to use. And then we saw them come, which is really exciting.

How much does this cost to build into an app or into a restaurant chain or something like that?

So we have a number of plans, including a free plan which kind of individuals often use until they upgrade to a higher plan. But right now we have a Pro plan at $15 which analyzes up to a certain volume of data. And then once you're through that analysis, then you go to teams plan, which is $300. So it's. And then we have an enterprise plan, which is sort of a little bit more.

Yep. Do you, do you build components to Put on web pages or into apps where, you know, you can actually become a ticketing system. Are you trying to compete with the ticketing systems like that were out there or.

Yeah, we're not trying to compete with ticketing systems, but you can embed our surveys. So you can embed our surveys in the web web app and then obviously over time you'll be able to embed them in more places.

Do you think you're going to ever incorporate an AI chatbot to talk to the customer directly and say, hey, I noticed you're having problems here. Here's some things you could try while you're on the phone with the customer service agent or, you know, that kind of thing?

Yeah, I would never say never, but I think that that's such a broad use case. Like there's so many problems to solve with just serving the product teams and making this a really, really helpful experience for them that I think that serving the end users would probably be outside that scope. But, you know, who knows, at some point maybe we're just like giant company that does everything. AI that's related to products and people. Right? I don't know.

You probably have a small. Tell us a little about the company itself. How did you get this started and how did. How what? How did you get it funded? That kind of thing. Talk to other entrepreneurs.

Crapful started as a different solution for product teams and really was kind of predated both ChatGPT and the GPT3 API. And so originally it was more of a standardized user experience that learned based on how users were interacting with the product. We then evolved into more of a basically an analytics platform. I was really fortunate to be one of the very first people to experiment with the GPT3 API in the very early days. And this is the first problem that I gave it a bunch of app reviews and saw the promise and kind of continued iterating on that. We then launched this is kind of in early 2020. We launched the first GPT powered feature in 2021 and we then decided to completely pivot to this. Last year it's really been this, this version of Craftful has been a 13 month wild journey. But we've really gone the. Yeah, we've been through a few pivots along, along the way started the company out of Y Combinator. So I kind of had had this idea that, you know, I was head of product at ifttt. I had this idea that we could build, that I could help product managers build better products applied to YC on a whim got in and so had to quit my job and incorporate and quickly get started. I think back then we probably were like the youngest company in the batch. Now it's a little bit more common to be kind of idea stage, but it wasn't just at that time.

That's awesome.

Quite unusual.

Y Combinator sure has helped a lot of entrepreneurs get going that way. Right. Um, since you've been playing with AI for a while, what, what frustrates you about AI? What you know, in the technology stack that you're building, what, you know, what do you wish that the AI, you know, the different models that you're using would do or, you know, where do you think the industry needs to go to help people help entrepreneurs like you?

Yeah, you know, I think, I think I'm. I have two. Two ways I think about it. One is, one is we're always kind of. We've built the best possible thing you can build with like the best models today with GPT4, and we're already thinking about what we're going to build with GPT5. Right. So that's part. Part of me is kind of constantly thinking about what's the next thing. And most of that is around, okay. There's certain types of features that we would want to build with a model that can reason better. And so I will think about that sometimes. But I think that I'm actually much more in the super appreciative camp because I had to build for GPT3 and kind of work through the really complex prompts that you had to do back then just to get something somewhat meaningful out of it. And then, and then in the Beginning, when the GPT3 model kept improving, that improvement would break our product. And so every time there was a new model, like a new version, you'd be like, ah. And I remember like, when DaVinci3 launched and that was the first time that the model didn't break the product because it was like, it just got smarter and it could take the prompts as is and we didn't have to change anything. And so now it's. And then it became more of like, well, how do we pare it down so that it becomes less complicated? Right. So I think that now I'm sort of looking at it, I'm like, they can do so much. It's pretty incredible, right? Like, yeah, I suppose. I think someone who's starting out right now and would just look at it and be like, oh, I had the. All of these expectations, right. I think my expectations are much Lower.

Yeah.

As a result of having tinkered early.

For people just starting out with AI, do you have any tips for people that you know you've been doing it for a while? The prompting tips or the tips on how to. Are you, are you playing with agents? Are you playing with rag, you know, which are some of the terms. Like I see some of the technologies I see entrepreneurs doing.

Yeah, we have rag in our. So you get the feedback and then you can ask follow up questions. And so we had to build in quite a lot of kind of complex things with embeddings into our product and we are using the Agent API. So I think we're using all of this available technologies that you can to put together this whole like pipeline. I would say, you know, I probably play around with it. I don't think that, I don't think that any of those things are absolutely necessary to build a great product. Like we started just building with. This is before, you know, before GPT3, before, sorry, before GPT4, before GPT 3.5. You can get really far with just like just taking a model and figuring out how to solve a very specific problem. I think it is about identifying a cool problem that you can solve with this technology. I sort of wouldn't complicate it, I guess.

Speak English to it, right?

Speak English to it or not English speak any language actually. That's another thing that I would say what I found was fascinating. We launched and then we had users all over the world and I say like suddenly had to figure out how to go global in like as a tiny startup which you have to. Right? Because LLMs can do, can do this work, right? Like we, when you're doing an account.

Japanese, you could talk German, whatever, right.

And which is great because people have users all over the world and they may be an English speaking team, but they have their product in different countries. And so it's really helpful for them to be able to interview their users in their native language and then get the insights in English or whatever language their team speaks in. But, but that was sort of a really interesting thing where suddenly people were sort of like giving us feedback in like languages we've never spoken and on the analysis in languages we've never spoken. It was pretty wild.

It's a new world that AI opens up to all of us. Anything else? Because I'm not a product manager, it's hard for me to figure out what a product manager would need to know about you. Is there a question I didn't ask that I Should ask you.

It's a great question. That's a great question. Trying to go back and think about what, what you've asked most of the.

High level stuff, sort of like, what.

Are, you know, what are product managers doing today and what are they going to do in the future and how, how, like how should they adjust for this world where AI is going to.

Be.

Like a major contributor?

If I was building a humanoid robot, how would I use your product? Would I talk to my humanoid robot and say, hey, humanoid robot, you washed the dishes badly today?

Absolutely. That's the thing. I think that eventually, even if the AI is sitting on the other end and listening to the feedback, they still will need something that collects the feedback from the humans. Right. And figures out all the different ways to do that. Because there's a lot of complexity there, right. Like around if you're just specifically asking questions in a survey and getting very targeted answers back versus having a call and people are sort of, there's two people talking and you don't know who's kind of the person. The person asking questions is the person providing feedback. And we humans will have all of those problems still even when robots are providing services. Right? So I think that a lot of those same pain points will need to be solved. I do think that, and I know that this is sort of like somewhat controversial view on the world, but I do think that when even in a world where AI and robots can provide a lot of the services and do a lot of our work, I do think that there is this human element of building that is so innate to particularly folks who love to build, that folks will continue wanting to create things for other humans. And so I think that we will see builders build even if they maybe don't have to. Right. Or don't have to make a living, but they will still want to do it because it's fun.

Even if I had a humanoid robot that could clean the toilets and wash the dishes, maybe a builder teaches it to dance. Now I have to rate it on its dancing technique. Nice dancing. Or could you teach me how to line dance? Like a country line dance? That's feedback that a builder might need to grab and do something with.

That's right. I love it. Cleaning robot is going to have apps that humans are going to be like, yeah, this is all you AI have thought about, all the useful stuff, but let's provide some entertainment here.

It's a new world of product management. Thank you so much. This is really fun getting to know you and your company. I've seen people use it and they're, they're getting a lot of value out of it. So thank you. Where do we learn more about you, about craftful and about you?

Yeah, probably for craftful. Just craftful.com has, has all, all the relevant information. I think I'm probably the most active on Twitter, which is still Yana tweets. I guess I should rename myself to Yana X. But.

It'S hard to stop calling it Twitter. I've been on Twitter or x for, what, 17 years. It's hard to get stop calling them tweets and stop calling them Twitter.

I still call them tweets. I still call them retweets. I'm not sure I am going to stop.

Well, thank you so much for spending some time with me. That was awesome.

Thank you. Really enjoyed it.