Prototyping AI Apps with @CreatewithPlay -Part 2 | By Harcourt Allen

Today I'm going to give a super brief overview of play. Sounds like a lot of you.

This is one of the first in depth tutorial series that I've seen that actually even utilize an AI framework, the ikey.

So I have the link for that as well that you'll want to.

So if I'm understanding this correctly, then.

I'm on the container.

Okay, perfect. So that if you're on the container then it doesn't have a value. So that's why that one's going to be red. Absolutely. Great question here. So I think if you had a.

Custom symbol that you added to SF symbols, would it show up or could you show up in here? Like if you had a custom icon.

Thank you so much for your time and I know it's your busy schedule, need some extra time after work, so really appreciate that prototype and you know, ship your product as well. Hello everyone. Thank you for joining. Hey Harko, nice to have you here.

Absolutely, thanks for having me.

Yeah, my pleasure. Awesome. We'll wait for five minutes and then I'll just start with the presentation and then we'll go into the workshop. Awesome.

Perfect.

And meanwhile guys in the chat, feel free to drop where you are from, what you do, network with each other. So you are like building your professional network. We have some people from the part one workshop and now some new people on the part two. But I'll share the part two resources here. Awesome. Hey Kate. Yeah? Feel free to drop us some good energy in the chat and then feel, introduce yourself, connect, share LinkedIn, get to know each other, all that good stuff. Okay, so like in 6, 5 we'll get started so we can quickly get into the part two of the workshop and it's going to be more hands on session so it's going to be a little time consumed. So we'll stay on track with time. So yeah, we have our guest also here, Harkut. And we'll get started at 6, 5. I have a quick presentation just to give some context to you all and then it's going to be hands on session. We have Paulo here. It's my neighbor. Carolina from San Francisco. That's awesome. From colorado. Denver. Nice. Oh that's, that's you.

Nice.

Cool. Yeah, some familiar frame faces, some new faces. I'll introduce ourselves also so if you're not familiar and so yeah, I think we are almost there. We'll wait for two more minutes. Feel free to drop in the chat and connect with each other. Share your LinkedIn and all that good stuff. Okay. One more minute and we'll get started and then. And meanwhile, if you all have any questions, feel free to drop in the chat. I will take them at the end, but in between if somebody is stuck, we can help you all as well. Yeah, that being said, let me share the present slides. Okay, you all see my screen right? Which is loading.

Perfect.

Awesome. Let me see sometime people in the waiting room, I need to approve them so I'll make sure I have access. Okay, I see it here. Perfect. Hello everyone, thank you for joining. Super excited to for the part two and our special guest Harut also is here and we did a part one and this is the part two. So if you have not seen the part one yet, I'll share the recording as well. So this session is going to be prototyping AI apps with Play. Play is one of my favorite prototyping tool and I connected with Dan who's the founder of IT and then Harkud is a design educator. So I did this first workshop. The first one was like prototyping focus for AI apps and then we did like a quick int to play but and people started asking like we want to like do more hands on session how see how to do the AI and LLMs and stuff like that. So we called someone directly from Play. Such a pleasure to have them. And Play also won the Apple Design award. So it's a very well designed product. So I'm super excited about this part two of the workshop. A quick outline is going to be we're going to have like intros, I'll introduce myself and our to the workshop and then we'll do the hands on session. Not 20 minutes to be 40 minutes and we're going to dive deeper into a hands on session where we're going to open up Play and create an AI app. And then last Q and A would be you can ask anything related to Play app or you can also ask about AI and things like that. So that's the overall structure and play. Like I mentioned, it's a design prototyping and a tool. You can also ship your app which is very exciting. Last time when I did the workshop there were a lot of founders and they're like very excited about the shipping the app as well. And it lets you create all delightful interactions and so this tool gives you the ability to prototype and ship your product as well all in one place. And then also if you see some of the interactions the community has created, it's very beautiful and helps you create better good user experience and stuff. And I'm John. So I'm a product designer and I'm a founder of design studio called Human Inspired Studio for JP Morgan, Citibank and stuff. And I'll pass it on to Harkut. She's a design educator at Play and I'll pass it on to her to share a little bit about her. But I watch all her videos on YouTube. Big fan, super excited. Have you Harkut and love to hear a little bit about you and share about yourself with the community.

Yeah, absolutely. Well first, super excited to be here everyone. Thank you so much John for inviting me and for setting this up.

Oh good.

Love that Caroline, fresh off watching 10 of my videos. Nice. Yeah, as John said, I'm education and support lead at Play, so I make a lot of the videos that are on YouTube, make some of the sample demo projects that we have in the lobby of Play if people have played around with that, and then also run user support. So if you're ever emailing the create with Play support email, that's usually me on the other end. Little about me I am located in San Francisco but originally from Kentucky. I think most people here are in San Francisco. So I'm bummed to have missed the last meetup in person, but hopefully we can do something again at some point soon. But yeah, I'm super excited to show you guys more about Play generally and AI specifically within Play and then answer any questions you might have.

Awesome. Thank you. Looks like Caroline also your fans. So we have some of your fans here and yeah the last one was in person and we'll plan to do more meetups also in the future. Future around AI or new tools like Play or more tools. And super looking forward to do more events. Just do this for fun for the community and to bring people together. This is going to be a workshop time, but the end if you stick around, I'm going to share all the resources for you through this slide. So make sure you stick around till the end. And so that being said, I'll pass it on to Harkut and we'll dive deeper into the play's AI capabilities and feel free to follow. Follow along as well.

Awesome. Yeah, thank you again John.

So stop sharing my screen and I'll.

Share mine in just a second here. Awesome. So today I'm going to give a super brief overview of Play. Sounds like a lot of you were out the last session, but we'll do a super brief overview of Play and then I have a file, a blank file, no interactions in it that I was thinking Everyone could work off of. I'm going to be following this file during this session, so I'm pasting it into the chat right now. Anyone who has Play installed, if you click this link, it should duplicate it directly to your team. And it's just going to be a super basic chatbot here that we're going to set up together. Now, the other thing that I'll say is, in order for this to work, you're going to need an OpenAI API key. So I have the link for that as well that you'll want to go check out and make sure you have your own API key that you can use. If you don't have one, you can still follow along, but. But it won't actually work in the end because you need your own API key. So with that, I'm going to go ahead and share my screen or. Sorry, I'm going to request to share my screen from you, John. Awesome. Okay. And let's go ahead. All right. Are you guys able to see my screen right now? Yes.

Excellent.

Okay, awesome. So what you'll see on my screen right now is the Play app. So, as John said, Play is a tool. It's a collaborative tool, meaning multiple people can be in any file at the same time for designing, prototyping, and then shipping your mobile iOS apps. And pretty soon we'll have iPad, iPad viewer out, which is super exciting. But yeah, for now, iOS apps. And today we're going to talk about AI. So when you're designing things in Play, you're going to do that in design mode. So if I wanted to add some objects to my page or my canvas around here, I would do that in Design mode. I could add objects from the add panel over here. Maybe I want to just quickly add an image onto my page, and then I can adjust any of the settings for that image using these properties up here. So something like a image, I can add the actual image data. I can change the rendering mode, the content mode, the layout, position, appearance, all of that. I can also add native elements in Play, which is something pretty exciting. So let's say something like a segmented control. I can add that in and then automatically it's going to work. It's a native segmented control rendered with the same technology as a mobile app. So I say all of that just because we're not going to really talk about Design mode much in this session today. We have a ton of videos online that you can go learn more about Design mode in Play. But that's a brief overview of what you can do in design mode, design anything from scratch or use Apple's native iOS element. But we're going to spend most of our time today in Interaction mode, and this is where we're going to set up our whole AI interaction. So were people able to open that file? I don't know if it needs to be everybody, but hopefully the people. Okay, I'm seeing a couple thumbs up. Excellent. Okay, so I'm going to have everybody meet me in Interaction mode, as I said, and we're going to start off by adding a prefab to our page. Now, a prefab is a prefabricated interaction that the Play team has already created for you. So with something like OpenAI, we've already created a prefab to make it super easy. All you have to do is just hook up your API key. So let's go ahead into our Prefabs tab over here and in our API section down here, I'm going to add this OpenAI API. I can do that in a couple different ways. I can either drag and drop it, that'll add it. I can double click it with an object selected and that will add it, or I can go into my Quick Add menu and just search for it here and I can go ahead and add that way as well. So this is our prefab. You can tell it's a prefab because it has this blue double node looking UI here and you're going to be able to unlock it if you're on a pro plan. And if you're able to unlock it, you can see everything that we've added inside this prefab that helps create this interaction. So just a general note about prefabs, any prefab you can actually create totally from scratch and play, we've just done it for you, to save you some of the legwork, making it easier to create interactions faster. So this Prefab is for OpenAI specifically for chat, but you could create one for an AI tool like Perplexity or for images for OpenAI Claude. Anything that has an API, you can hook it in through a fetch node. So I'm just going to quickly show that before we get back into this prefab. A fetch node is something that can be added onto any trigger in play. And this is where you're going to add in the URL of the API and then follow along the documentation of that specific API. So if there's any headers in there, if there's a different type of body or like maybe sometimes the URL you have to add in the API key in the URL. So this fetch node is pretty generic and you just follow the documentation for your specific API and that is how AI is integrated into Play at this time. We don't have AI in the tool right now, although like most other tools right now, of course we're exploring AI. But the really cool thing about Play is being able to use AI in your products. And the fetch node is the heart of that. So that's a super brief overview. Again, we have tutorials on how to use Fetch in more detail. If you wanted to use a different AI service. Not OpenAI, but if you want to use OpenAI, which is what we'll do here, you can just go ahead and use this prefab. So now that we have this prefab added, I just want to talk about a couple things. First, is this fetch on load? This just means it's going to fire automatically. Now I don't want that here, so I'm just going to go ahead and turn it off. Then we have a prompt, and this prompt is what we're going to change by typing into this text field in order to send something to the AI that the AI can then respond. Then we'll have the API key. So if you went ahead and got that API key, we're going to go ahead and paste that in here. Don't look at my screen for a second. And then you can change which AI model you want to use from ChatGPT. And then you can also tell it a role, which we'll talk about in just a second. I also have remember context on, which means the AI is going to remember everything I send to it. So it's going to create more of a conversation here. And then you can have actions fire when the API is loading, when the API responds, and if there is an error with the API. And then you can also select a spinner, which we have an activity loader, which basically just spins, indicating that the API is loading. So when you add this, by default it's going to have these two nodes attached to the API response. And we are just going to focus on this set text for now. So we can go ahead and delete the set variable. Now the set variable is there because from this OpenAI, you can either save the response that the chat gives you into a text element, which is what we're going to do here, showing it in a chatbot, or you could save it as a variable to use elsewhere in your product, but for now we're just going to do this text. Okay, so that's all we need to know for now. We're actually going to add interactions not onto this prefab here, but instead onto this text field. But before we do that, on my iOS device, I'm just going to connect to macros because we're ready to start with our prototyping. So this is going to load and it's going to show me whatever I have selected on my Mac OS app. It's going to show me on my iOS app as well. So let's select this text field. Now, you'll notice it's in this text field container, but that's not what we want to select. We want to select this native text field. And if I go over to my iOS app, you can see this is actually a native text field. I can type anything in. When I tap to select it or set the focus on it, you'll see my native keyboard pops up. And then I could press that send or return button here. But I don't have anything set up yet, so it's not going to do anything right now. But because this is a native text field, we can use a native text field event. So the basic fundamentals of an interaction in play is a trigger which is the input, and an action which is the prototype's response. So we want to add a trigger onto this text field that is going to be this native text field event. And this lets us fire an interaction when one of these text field specific actions is completed by the user. So focusbegan is going to fire when the user focuses on the text field while typing is going to fire while the user is actively typing. Value changed will fire every time the value is changed, and so on and so forth. The one that we want here is going to be return key pressed. Now this is going to fire when the user taps on this return key in the native keyboard here. So when that happens, meaning the user has finished typing everything that they want to type into this native text field, that is when we're going to fire the interaction. So now we need to have actions on this trigger. So the first one we're going to do, and pardon me, because I'm going to go back to this other node quickly, first thing we're going to do is set this prompt. So the first time this fires the prompt is hello, chatgpt. But we want to update that prompt every time based on what the user typed into this text field. So we just need to set that prompt, which is a prefab variable, because it's on this prefab node, we want to set that prompt. So I'm going to go over back into this text field and we're going to add a set variable action. Now, with set variable actions, you can just click on the left side here and choose any variable that's in your project. So if I already created a variable over here, or if I wanted to create one, I could just select that variable. But something that not everyone knows about play is you can also type in an expression that you can set using a set variable action. So for example, if I wanted to set the background color of the page, there's not a set color node in play at the time at this moment. So you would set the background color using a set variable action, just to give a super basic example. So in this case we want to set that prompt. So I'm going to use an expression, and as a fun fact, you can always move this expression editor around, and when you do that, it's going to pin it. So that way when I select something else, it's always going to stay in view. And then I can always close it just by clicking that pin again. But let's go ahead and open it. So what we want in here is to target that prefab variable. So to do that, we're going to start by targeting the full page, because that's where the prefab is placed. So I'm going to type in page. When I start typing that in, it's going to give me a list of everything that matches in my expression editor. The page is an object. So when I select that page and then I do a period, it's going to give me a list of all of the object properties. So any properties that I can get for the page. So something like that, background color or the border, bottom left, radius, you can get any of these properties. But the one that we want Here is this OpenAI API, which you can see is the trigger. This is the prefab that's placed on the page. So we're going to go ahead and select this next. Now, when selecting the prefab, we can also do a period and see the prefab properties and any prefab variables. Now, remember I said that prompt was one of the prefab variables, so you'll find it here in the list. So now we can select that and then I can press return to save it. So this variable action is going to set the value of that prompt to something else. And what we want to set it to is the value of that text field. So on the right side here, I am going to type in. I can do this two different ways. One, I can type in text field and I can actually target this text field again. That's an object and then dot, and then this gives me the text field specific properties. And the one we want here is value. So that's one way that I could do this is actually type in text field, select the text field and then dot value. But another way to do this is because this interaction is placed on the text field itself. Instead of typing in text field, you can just use self and that's going to know what object this is on because the interaction is placed on self and then we can do dot and that's where we can find value. So either one works here. I personally like to use self. So we're saving that variable here. Now if I go back to my full page and I go back to my API node, now every time I press return on my text field, it's going to update that prompt, which is going to make my prefab fire, which is going to give me text back. But now I need to do something with that response text. So that's what this set text action is. So this response text is another prefab variable. This is what the ab. This is what the AI is responding with basically inside here. And I'm not going to unlock it right now because it's for pro users only. But if you wanted to upgrade to a Pro Pen and Play and unlock this node, you'd see the whole interaction all set up and be able to see that we're saving from the API, we're saving it as a dictionary, and then I'll get your question. Just a second. We're saving it as a dictionary and then we're grabbing the response value from there. So that's what this response text is. Now all we have to do is just choose a text element to display it in. So I'm going to select this response text here. And now once I have my API key in, I have my text field interaction and I have this set text on my API response. Now I can double tap to reset my prototype and I can type something in here like hello, and when I press send, then my gosh. Of course, always tech fails when it's not supposed to fail. So it will respond and it will write that in there. I think I messed up my API key here. So that's probably what's happening because it was working right before this with a different API, but I don't know if anyone else has tried this and had it work on theirs, but it will respond to you in that space there. Now, quickly, I just want to answer question. I saw someone was raising their hand. I'll just take a second to do that. Wanted to understand a bit about text. Oh yeah, absolutely.

Yeah.

The pretext and another question was what prefabs are?

Yeah, absolutely. So prefabs are just prefabricated interactions. So it's basically if I were to create a tap trigger plus a set text action or even this interaction here, we've basically created this whole thing for you, wrapped it in a prefab container and now you get access to the entire interaction just through one node. So all you have to do is add the prefab from here, change some of those prefab settings. So I don't know, let's do drag and drop list, for example. It's not going to work here, but we have put these properties out here so you can change and customize these properties. They're going to affect values that are inside the prefab, so inside the nodes in the prefab and then it's all just going to work like magic. So we have a bunch of videos on that that I can go ahead and share with you guys later. And then it sounded like the other question was this self value. So I guess I can go through each part of this separately. So first we have the self part. So self is just going to target whatever the interaction is on. So because this whole interaction is on the text field, when we type in self, we're actually just targeting this text field. If we instead created an interaction on this text element and targeted self, it would always be targeting this text element instead. And then every single object has object properties. So because self in this case is a text field, we're able to access all of the text field specific properties. Does that answer your question? A little bit. So yes. Excellent. Glad to hear it. Okay, understood. Wonderful. Okay, so that should be saving our. That should be saving our response in here. I'm just gonna see if something on my end is causing this. I'm sure you guys have all been in situations where you're presenting something and everything was working right before you hopped on the call. And then of course you get on the call and is not working, which I love. Okay, give me just one second. I'm a shot sharing my screen just to make sure I can get this API key working for us all.

Anyone doing follow along or try doing it?

I'm trying, but I'm catching up. I think I put the API on the text field instead of the page, so I'm like trying to move it.

Nice. Okay, y', all, it turns out I did just use the wrong API key. So here we go. Now, when I type something in here, hello, and send it now, it's going to respond, how can I assist you today? If instead I put in like, what is the weather in sf? But I got to write that correctly. Now it's going to tell us, oh, it's actually not able to do that. But if we use a different API, it could do that anyways. You can see it's responding back to us, though, and it's printing everything out in this response text. So that's the fundamentals of this AI. You're basically going to feed it a prompt and then you're going to display that response text. But there's a couple other interactions here that we can add in to make this whole flow a little bit smoother. So I'm going to go into this text field interaction again. And something cool in play is that you can add a bunch of different actions onto one trigger. So when the return key is pressed, we're setting this variable, but we can also do other things as well. So in this case, maybe I want to take whatever we typed in and we want to put it in this text right here so we know what we're responding to. Well, we can go ahead and add a set text action. In the settext action, we're going to go ahead and target this question text. And then for the text, we can open the expression editor by clicking on this left side here. Because we want to use this self value, the object property, to give us whatever we typed in that text field. We want to use that as the text value. So we're going to use expression here. And then in here we're just going to use that exact same thing. We're going to say self dot value. So now when I type something in here, hello again. Now it's going to set that to be hello. Now, something cool you can do in play, because this is an expression editor is inside here. I could say in quotation marks, you space plus. And I can actually just make this an equation with strings. So now it's going to say the words you, and then it's going to give us what we typed in. So let's try that one more time. Hello. So now it's going to say you plus hello. And then underneath that is going to be the response from the GPT. Now, also, when you send something like a message or a search, oftentimes you want it to automatically clear. And you can do that in play pretty easily as well. We are going to reset the text field by using another set variable action. So let's do set variable. And just as a side note here, inside set variable you can create several different lines just inside one node here for set variable specifically. But because I want to set the variable and then I want to set the text, and then I want to reset this text field, I'm splitting these set variable actions up into two different ones because everything goes top to bottom in play. So this one's going to fire first, then the settextaction and then the second set variable action. So for this one, you know how I said earlier you can actually target object properties? Well, we are going to take this self value and we're going to set that to blank. So on the left side here we're going to use an expression. This is we're going to type in that same self value, the text fields value, and we're just going to set it equal to just quotation marks that are empty. Now I'll press return. Now all of that set up together, we can type something in it is going to send it to the GPT that's going to respond. It's then going to type it in here so we know what we sent. And then it's going to reset our input text field so now we can get started typing something else. Now just pause for a second there. Does anyone have any questions with what we've done so far?

I had a question like, is there a way to create like a set of variables that would allow, I guess, for the creation of a conversation history?

Yeah. Yeah, okay. Absolutely. Absolutely. So we actually have a full other video on that. I'm just going to pull up my messaging prototype quickly. And we're not going to combine both of them at the same time right now, but I can show you how both of them work together, or not together, but I'll show you how both of them work. But yeah, so you will basically save each of the chats that you send in and the responses, you'll just basically like duplicate the, the message that you're sending and create UI that makes it look like a text field, like you're chatting back and forth. And then you're just going to set the value of each one of those individually. I know that sounds a little vague, but I'm going to show you in a second. And then after this, I'll also make sure to post post the tutorial link to our whole chat project. Yeah, Caroline, looks like you have a question as well.

Sorry, I was getting the audio.

Right.

So if I'm understanding this correctly, then the process is create the API link and then go to the exact spot in the interaction where the event would happen, open the event and then put in a sequence of the program's response. So refer to that, what the response is to that event and then what the styling of that text and then the reaction if they were to leave that text field.

Yes. Yeah. Basically, the main thing you need to make sure this particular prefab works with OpenAI is you just need to set that prompt somehow. So in this case, we're setting the prompt by typing something into an input text field. But if you had like 10 buttons or something, you could also set the prompt when you tap a button. So there's a bunch of different ways you can set the prompt. We're just doing it. So it's a chatbot. But yeah, you're exactly right. You're just going to set this prompt and then any other actions can just follow that. And whenever you're setting that prompt, it's just going to make the. This prefab work automatically. Gotcha.

And then this part is when you're setting things at this level, these are just reference points for you to pull in when you're creating a variable.

Correct. Sorry, say that one more time.

Like these hashtags over here are kind of reference points for you to pull in whenever you actually build out that interaction.

Yeah, exactly. So inside a prefab, each of these, which are called prefab controls, which are also function the same way as any type of variables, but they're only within the scope of this prefab. So all of these are connected to things inside the prefab itself. So when you're typing this in, it's not just going into the ether, it's being specifically fed into that fetch node. Gotcha.

Thank you.

Yeah, absolutely. Just to quickly show this chatbot here, it's saving text fields. Let me go ahead and switch the project now. It's going to open just a second.

And.

Yeah, so now when I type anything in here, then it's going to send this. It's going to be added into that chat. I could continue adding things in and then if I wanted to connect this same chat feel or the same interaction to an AI prototype to have it chat back and forth, you could figure that out by basically copying both of these projects from our learn section or demo projects and you could mix them together. Maybe we'll do a live stream on that at some point. Creating a full chatbot experience together.

Another in person workshop. You're done.

That'd be a lot of fun. Nice. Cool. A couple other things as part of this same A couple things as part of the same prototype that I just want to show how play can control some native behavior. So as you all saw, when we open this text field, it opens this native keyboard. But let's say I wanted to dismiss the native keyboard other than just sending it, it's kind of hard to do that with the ui. But I can create an interaction in play that anytime I tap anywhere on the full page, it's going to dismiss that keyboard. And all of that is done totally natively because again, it's a native text field straight from Apple. We didn't have to prototype this or code it or change the way it looked. All of it's Apple pops up automatically because we're using their technology. So we'll go back into interaction mode with the full page selected and we're going to create another trigger on the page in addition to this prefab. So we'll go ahead and add a tap trigger here. This is a quick add menu. Tap is at the top. But if I wanted to search like scroll or something, I could find that as well. But I'm going to do tap and on the tap trigger I'm going to add a dismiss and keyboard action. Now there's not any pre or there's not any controls in here. So like for tap, I'm able to change the number of taps or change the number of fingers. I'm tapping this with dismiss keyboard is just going to dismiss any keyboard that might be open. So now when I open this, I can tap and it's going to close that. Now the other thing is by default it just shows me to this page. But what if I wanted to open that keyboard and focus on this text field automatically when the page opens? Well, I could do that with yet another interaction. So on my page again, I'm going to add an event, but I'm going to spell event right? It's going to be a page event and it's going to fire on load. So these page events and view events are all part of the view's life cycle. So this is just one moment. This is when the page actually finishes loading. So when that happens, that's when we want to set the focus on this text field. So I'm going to go ahead and add a set focus action. So I can choose either to target a specific text field or I can just go through each one of the text fields if my text field tags. So this one's really helpful if you're using a. If you're creating a form that someone has to fill out and you want them to be able to go to each input one by one, this is a really great way to do that. Just forward. But in this case, we want to target a specific text field that we want to set focus on when we load the page. So we're going to select that text field. It's going to be this one here. And now when I double tap to restart my prototype, you can see the text field automatically opens. So it prompts us to know exactly what to do. We'll start typing in, ask any type of question like what does red and blue make? And then it will respond back with the whole thing. Now, one last thing I want to show, and this is always my favorite part, I think this is so fun to be able to do is inside our prefab, you can change the role, which I mentioned earlier. So right now ChatGPT is responding to me like a helpful assistant, but if I instead wanted them to be a sassy and rude assistant, I could just change that role in there. And now when I go in here, I'm like, what does red and blue make? And then it's going to respond as if it's that. As if it's that prompt or if it's that role. So, oh, look at you trying to mix colors. So it's a little sassy. I think this is so fun because it's super easy to customize your AI response not just to what you input, but also the vibe you want to give. So let's say you were creating, I don't know the legality of this, but let's say someone was making like a therapy app, something like that. You could have it be a lot more respectful or affirming or push back, you know, any of those things versus if you're trying to make like a snarky assistant app, you could change the tone to that. So it's a lot of fun you can do just by manipulating some of these different prompts. So that is our overview of play and AI inside play, specifically using this OpenAI API. I hope you can see from this quick demo. We built a lot around it, but the fundamentals of setting up an AI chatbot in play are super, super simple. We have it already for OpenAI. We'll probably create some more prefabs for different APIs in the future. But yeah, would love to hear any questions you guys have now and if there's anything that comes up after the session, would love for you guys to join our community. Just hosted on Circle or email me and I can answer any additional questions as you keep building.

Yeah. Carolyn, you have a question?

Yeah. For the roles, I was curious if it was just tone or can you have it embody other Personas like a marine biologist?

Ooh, I actually haven't tried that. I think it's tone, but it could have the tone of having a marine biologist. And you can give it pretty detailed instructions in there. So if you wanted to write a whole paragraph like you're a marine biologist named Carolyn who loves dolphins, you could write that in and see how it responds.

Sounds good.

Thanks.

Yeah. Heidi, do you want to ask the question?

Yeah.

I was wondering if there are currently any companies that you partner with that are currently leveraging Play for their AI assistance. I'm mainly here as like a. From a UX design perspective and currently working on a project to develop an AI assistant, so. Nice.

That's awesome. I don't know if I'm publicly allowed to share that.

Okay, sorry.

Totally fine. You can definitely go check out our website and we have some of the companies using Play there. But yes, I can definitely say that just people are using AI functionality inside Play.

I had a question. Sorry. So if I want to connect my custom database, can I do that or are you, like, planning to bring that?

Totally, yeah. So there's two different ways you can use that Fetch node. So if you have access to any API, so a public API or an API that you've created on your end. So this is really helpful for a lot of companies who have their own API. You obviously will have the documentation that goes with that API. So you'll be able to use the Fetch node, same as you would for any public API, for your own private API. Now, separately, if you have a JSON file instead, you can use a Fetch JSON node. You just upload the JSON file into Play and then you could connect it, pull the data from there, and we have a bunch of projects and videos showing how to do that in Play.

Oh, wow. Like, I didn't know that. That's quite an unlock because, like, I've been creating like an AI mentor app.

Yeah.

I'm connecting it to the database of my book, which I'd written in the past.

Oh, cool.

So I think, like, I could. I didn't know that. So now I can, like, fetch like a JSON from there.

Exactly. Yep. You can fetch it right from the book and you can use that little role there.

Yeah, act like assertive mentor or like straightforward mentor or something. I love that. I did, like, a session on AI agents internally at work, and defining the role is very important in AI agents. It's good that we have this capability. Yeah. Jetted. Good.

Oh, sweet.

Sorry.

Thank you. I got stuck on that first part with the input field.

Yeah.

My set variables, they're like red where it says equals self dot value.

Yeah.

I'm in dark mode. I don't know if it's red by default, but like.

Yeah. So red typically means that there's an error. So that's probably what this is. So a couple things we can do to debug that quickly. So let's just make sure. Are you selecting this text field, not the text field container, but the text field itself?

I'm on the container.

Okay, perfect. So that if you're on the container, then it doesn't have a value, so that's why that one's going to be red. So if you want to save yourself a little bit of time, you can just copy the whole interaction by selecting the trigger and just paste it.

Oh, copy the trigger, hit delete, and then double click on the text field or get the text field, hit pay. Oh, wow.

Yeah.

No more red.

Yay.

I love that. Love debugging live.

That was cool. Now I'm going to see if it all works.

Nice. Did anyone else get stuck while creating that and want to maybe quickly walk through how we can unblock them or any other questions?

Yeah. And feel free to share if you're building anything that you all want to discuss as well.

Yeah.

See how we can. See how we can use play to prototype.

Yeah, exactly. Another fun thing. While I'm waiting, if anyone wants to know, we actually have this. This prototype set up for light and dark mode, so it inherits your phone settings. So I don't know if anyone has dark mode right now, but it should probably be showing up in dark mode for you. If I were to change into dark mode here, you can see all of my colors flopped, and when I type anything in, it's going to work totally in dark mode. Could also just go ahead and switch back. So that's. That's a fun accessibility thing in play. I don't think I have it set up, but you can also use dynamic text in play. So if I had that turned on right now and I was using large text or something, then it'll show up much bigger. So it's basically instead of giving every user the exact same prototype when we know that not every user is going to experience a prototype in the same way. Some people have dark mode, some people have light, some people have large text, some people have teeny tiny text or regular sized text. We obviously want people's feedback on the product they're actually going to experience. So Play does all of that stuff natively for you to make it a little easier.

Yeah, the accessibility is a great add on like Apple's we use here. They use the Apple's native component so they are accessible out of the box. And I want to make a use case for this app. Advocating for accessibility. This is super helpful feature. Yeah, Jared, go ahead.

Oh, I think my hand is still up. Okay, never mind.

Yeah, as designers I think we have two kind of audience. Before designers and founders. I think today mostly are designers, but I think yeah, you could probably use this in your prototypes, on your portfolios, your presentation or internal innovation teams. I've done a lot of companies are trying to see how we can use AI, so I think pitching your own product internally is going to be super helpful at your work. And if there's founder, I'd speak a lot to a lot of founders like this is super helpful to prototype and test your ideas with your customers and potentially when raise investment even before you hire a full fledged engineering team and stuff. So it's a lot of value add for prototyping, especially now with AI since everybody's trying to figure it out. So you all can't be the thought leaders. Yeah. Pass it on to Kate. Yeah.

Thank you, John. Hey everyone, I have a very basic question. So somehow I noticed that you have this right panel with device. So you actually have this device frame. I only have this iPhone 16 Pro and that's kind of all I was trying to find. Where is this exist or probably if this inability to fix this panel.

Yeah, so this is actually my actual iPhone right now that I just have plugged into.

Oh, I see. So you.

So that's, that's how I'm doing it right now. Just because I like to type with the actual keyboard. But we do have play simulator. So from this plus button here, top corner, it's going to open this panel and you can open a new simulator. Now by default it's always going to be iOS 16 Pro. But once I load my simulator in, I can click on this here and I can choose any of these simulator windows. So if I wanted to see what it looks like on an iPhone 8, I can do that here. Or I mentioned iPad Viewer, an actual app to view iPad apps on an iPad coming soon. But for now you can view what this would look like on an iPad size as well. So if you wanted to do the full tab here, you could do that as well.

Perfect.

Yeah. Our simulator is a lot of fun too, because you can probably not going to be that helpful here, but you can use Slo Mo, which I always love. If you want to see what an interaction looks like, but you want to fast in the actual app, but you want to see every step of it, you can do it super slow, super, super, super slow or even like extra fast as well. So a lot of fun inside the sim.

That's cool. Thank you.

Thank you so much. Absolutely.

And the Play, everyone is so excited about it.

Exactly. Yeah. That is one thing I'll say about Play is if we love hearing our users feedback and we're pretty good about implementing it quite quickly. So if anyone gets into Play and is using it quite a lot and you have thoughts on something you think could be better or feature you're looking for, definitely, definitely encourage you to reach out to us because as I said, we love hearing from users.

Yeah. I think you also recently launched the iOS, the Liquid Glass. That was very quick.

Yeah, yeah, exactly. So if anyone is curious about using Liquid Glass prior to Apple publicly releasing it, you have the beta on your computer and on your phone, you can actually download Play's beta for Liquid Glass. You could probably prototype with that. And it's not like obviously Figma has a Liquid Glass simulation that looks pretty cool, but Play is actually using Apple's real native glass technology or Liquid Glass technology. So it's going to match one to one with what it's going to look like in the end. And every release they do our Liquid Glass changes as well because we're using that real technology. So after this again, I can send over a bunch of links to some of the videos I mentioned, but also we can send a link to that Liquid Glass beta if anyone's interested in that.

Nice. I tried it out. It's very super helpful. It's very quick. You just like toggle one button and turns into a Liquid Glass. And it's quite interesting. You can prototype and design with it very quickly. I had a question. I know the answer for it, but I'll just ask for the community, how do we share this prototype with others to get Feedback and stuff.

Absolutely. Great question here. So I think this is one of the coolest features of Play is how easy it is to share this with other people. And we do this natively with something called an app clip, which is a piece of Apple's technology that allows you to experience a full app without actually downloading that app. So what that means is you can send an app clip link to anyone who has an iPhone. All they have to do is tap the link and it'll work like magic. They don't have to download any type of Play app or viewer app or anything like that at all, or create an account. It literally just opens and they're going to feel as though they're using a production app. So everything that you're experiencing here on your phone, they will experience on their phone instantly. So the way you'll do that is you'll go to this publish up here in the top right corner and it's going to open this modal and I'm going to talk about this in just a second. But on the left side is this app clip. All you have to do is just click Publish for that app clip. It's going to copy a link. If you want a little bit of extra privacy, then you could add that in here, copy that link like I said, and then you just paste it in to a. Let's see if it's gonna. Let me do it. You would just paste it in right here. I'm not actually sending it because I don't want my API key stolen. But you'll get this API or you'll get this app clip link, send this to anyone and then they'll be able to open it and experience it again, just like magic. So this is great in this example, but it's really helpful when you're trying to get stakeholder buy in, or if you're trying test with users and get real user feedback. Or as John mentioned earlier, I think a lot of people were founders in the last session. This is so great for sharing POCs super early on, with basically no developer resources at all. But you're giving it to an investor, they can interact with it and they're like, wait, it's ready already. You already did it When? No, you just created a prototype and play, which is pretty cool. And then the other thing I'll mention in that publish panel is this export to Xcode. So again, on our Pro plan, if you export this, you can create a whole Play package. So I can just go through, add this and it's going to Pull every asset I'm using, every style in my project. I don't have any components, but if I had components, it'll bring those. Same thing with any pages in my project and any variables. And I can export all of those as a package into Xcode. And just by default, when I open my Xcode, I can press Play on that simulator and you'll get the exact same thing that you saw on your prototype that you just created. It's already going to be in Xcode. And John or Jared, I saw that you have a question. Go ahead.

Yeah. Can you do the reverse too? Like if you change something in xcode, bring it back in and it'll affect your prototype?

Not right now, no. You can't import anything into Play, like any code into Play, but you can. Once you've exported something from Play, you can go in and you can make changes to your Play prototype and then re export it back to Xcode. Instead of making changes in Xcode and going back and forth, we recommend making your UI changes and prototyping changes in Play and then exporting those. And then of course, you can still add additional things once you're in Xcode.

Thank you.

Yeah.

And compared to like wipe coding, this feels more natural workflow aligned with like how designers are like used to designing and stuff. So it's much of an easy translation. And Shift UI is like very easy as well. So if you are a designer Interested to learn iOS, I think ShiftUI is pretty good to start and this translates really well to shiftui very quickly. Yeah. Instead of spending too much time prototyping in code, just prototyping here. It's much natural role and it's much faster.

Totally. Yeah. We've had a lot of people say, especially like designers who are more interested in that design engineering role that I personally think is going to be like quite, quite popular in the future. They use Play to learn how a developer thinks without actually having to write the syntax code, which obviously that's where many, many issues come into. It's kind of the mental model of how a developer thinks things. But you're still doing it with shapes and designs and properties like you might be used to in a tool like figma.

Yeah. And play. You articulated well that design engineering, how developers think that this ties up really well here, especially here we use stacks. So also While coding with ShiftUI, you use hstack, vstack, zstack, and the same model is followed here. So if you're learning a lot as you kind of design things as well.

Yeah, yeah. Exactly.

And I know a lot of iOS developers, they get very annoyed by designers when we start using custom elements and we create like a custom arrows and everything. But here it's tied up to the native components, like the SF symbols and everything, so the developers don't have to create everything from scratch. They could just like, you know, use those modifiers and APIs in while development so it's much easier for them. So it's. Yeah, here are the SF symbols.

Yeah, so as he said, there's a bunch of SF symbols here. You can search from all. I think there's over like 6,000 or something crazy at this point, we load all of them in from. From Apple and then you can also. Or let me get something that's a little bit more fun here. Oh, my God. There's too many to choose from. I never know which one I want to choose. I'm just going to select one. Um, but yeah, so you can play around with the rendering mode. So by default it's monochrome, but you could change it to this hierarchical or palette or multicolor. So with palette you could actually change. Like if I wanted one of these to be teal, one to be green, and one to be purple, you can actually play around with all of that. Hierarchical is obviously just gonna change the opacity of some of the different layers. But then one of the things I think is super cool is I'm just going to change this back to a heart. For now, you can play around with the native animations as well. So if I go into interaction mode with this selected and I do a tap trigger and then I do a set SF symbol. Let's do effect first. I can play around with one of these animations. So I'll just target myself here. I can change the tint color if I want, but I don't really want to. And then I can play around like, maybe I want this, this to bounce. So now when I tap this, you can see it's going to bounce. And that's the native bounce behavior that Apple gives you. And you can just play around with all of these. You can change like how many of these you want to repeat. So it's going to do this like four times. Now I could make it like really, really fast if I wanted to. So a lot of playing around that you can do there. You can also change which SF symbol. So let's do a set SF symbol rather than a set SF symbol effect. Just quick little keyboard shortcut here. Sometimes I like to disable things you can do that. Just by selecting the node and pressing D, just the letter D. And now it'll be disabled. And so now this is what's going to fire. If you have them both enabled, then they'll both fire. So we can target self and I can choose a different SF symbol. So if I wanted a heart that's filled in here could do that. I could add a replace animation. This is awesome. Like, for a like button. Now I tap that and it just fills in with that little animation. So super easy to make something like a like button in play all natively. Jared, I think you had a question.

That's amazing. I was curious, if you had a custom symbol that you added to SF symbols, would it show up? Or could you show up in here? Like, if you had a custom icon?

That's a good question. An actual ss? I don't. I don't think so. Right now. I think what you would do is just upload that as an image. So. But you could do the same thing. It's not going to be that native behavior there. I'm actually not 100% sure on the custom SF symbols, but let's say you uploaded an SVG or an image, you could still make it do that bounce behavior. We have several prefabs for that. So if I wanted to just add an image quickly and we'll just set this, go into interaction mode. Now I can do this, like, pulse prefab on here. And if you look on my iOS device, you can see just by adding that prefab, it's doing this, like, pulse behavior here on my phone. So pretty easy to do that. I could also just add a tap trigger with an animate block. I could scale it up during animation. So let's scale it to like 110% and then duplicate this. And when complete, I'm going to set it back to 100%. So now when I tap, you can see it scales up and then all the way back down. I would probably want this to be 100% instead.

Wow. Faster.

That's awesome. Yeah, it's a lot of fun. And then you can play around like, this is my favorite thing about play. Now I can play around like, maybe I want there to be a lot more spring here now, you know, I probably don't actually want that, but you can play around with all these values, test out like 15 different iterations in a matter of of seconds. So when you actually ship something, you're pretty sure that's what you want because you've tried out all the other combinations again. So Fast.

Awesome.

Awesome. So I'll just quickly share some of the resources and if you have some questions we can take them or I think it will do some like what's your takeaway kind of a question to the audience and then we'll wrap up. Can you see my screen with the Figma screen slides?

Yes.

Okay, so I'll walk you through the resources, I'll share the slides with you all. And so we. You can download the Play app from here, the workshop part one that happened. You can access the recording from here. It's a substack that's my. Where I share all the workshops and content. And then this is play's YouTube channel and you're going to see All Hearts video there. And then Place Community is a very exciting space. People share their work, take feedback. If you get stuck, you can ask any questions and you get to meet design engineers, designers and engineers and stuff. So it's quite a vibrant community. You can connect with Harcourt form on LinkedIn here and then you can connect with me on LinkedIn. So these are some resources for you. You all get this email after this session. Luma sends out the email. I'll just put the link then. Yeah, I think we covered most of the questions, but I have a question for you all. What was your takeaway for the audience and what are you excited to build? So if you can share that, it'll be great here. Anyone like to share any key takeaways.

Alarm? There is a pretty fast to do stuff if you know what you're doing and that the UI is very beautiful in the app. I'm not like a pro user, but I've been really interested as a follower of all the updates over the like the year. And yeah, I'm excited to try it and get this kind of demo working and see if I can get it to like render images and stuff that the AI delivers. And it's pretty neat. Thank you guys for doing this for us too.

Yeah, absolutely. And I'll just say on that note, if anyone is playing around with Play after the fact and wants to use it with their team, we do offer team demo sessions for teams of five or more. So if that's something anyone is curious about, you can just either DM me on LinkedIn, email us, email me specifically, ask John about it and we can set up something for your whole team. We'll go into even more detail than what we went into today, so. So.

Awesome. Anyone else would like to share your takeaway things that you learned? Yeah, I'll share mine. Sorry, go ahead, go ahead, Paul.

Oh, sorry John. Yeah, so my takeaway from this is I really appreciate the ability for Play to use a lot of the native Apple features and deliver something that is a native experience, especially just by using these drag and drop nodes and prefabs and all these containers. Like I had a bit of an, a little bit of a sneak peek seeing Play pop up on the app store on LinkedIn and got to try it out a little bit and I think this is one of the first in depth tutorial series that I've seen that actually even utilizes an open AI framework. And I really, really appreciate that insight to see like, oh, this thing is powerful and it's just a matter of time where people just familiarize and get used to it. So yeah, that's my takeaways. It's very exciting. It has a lot of potential and I'm excited to see where it goes and try it out for a lot of the things that I'm working on.

Awesome. Thank you.

Paolo makes Vision OS apps as well, so if you want to talk Vision os, he's the guy.

Thank you for that plug.

I got to say that must have been one of the top 10 moments of my 2024 was putting on those Vision OS goggles for the first time. I was so mind blowing. That was crazy. So props to you.

Awesome. Anyone else? The last one Takeaway. I am excited. I didn't know about the JSON fetching thing. I think I can make my AI mentor with Play now so I'm very excited to try it out and let's share it with you all on LinkedIn. Let's see how it goes. Yeah, I do have some designs but I was quite like I didn't know how do I connect my own data? Like, like data. So now the fetch I came to know. So that's one of my takeaway and we love to do more events and meetups and sessions like this just to bring community together, learn together and just like it's kind of fun. So yeah, any final takeaway? Awesome. I think we're good. I think this was a great session And Harkut, thank you so much for your time and I know it's your busy schedule, you need some extra time afterwards work. So really appreciate that and we'll do more in person events as well. We did one in Palo Alto and a lot of people registered so we'll plan to do more sessions like this and just like, you know, build the design and engineering community here in the Bay Area and like learn about AI and tools like play and you know, all those cool tools that are coming come to la.

Okay.

We do have a meetup group I used to be part of in Southern California. Where is that? Huntington Beach, Orange County. But yeah, LA might be a little further. But yeah. Thank you everyone for joining. I'll share the resources with you all and then please do connect with Harkur and me later if you have any questions. And then yeah, you're going to get the email with all the resources and yeah, stay in touch and looking forward, forward to have you all in more meetups like this and we'll probably do another in person meetup so if you're interested, let me know again, thank you. Arkut, any final thoughts and takes that you want to share with the community before we close?

Yeah, yeah. I just want to say thank you guys all for attending this session. It's so cool to find people who are interested in learning more about prototyping and AI and wanting to do that after work hours. So super grateful that everybody joins today. And John, super grateful you set this up up. It's always cool to meet designers and developers and founders and just general product people in the industry. So really grateful to be here. Thanks again and thank you all.

Thank you. Bye.

Thank you.