OpenAI Canvas First Impressions: Will It Replace Your Coding Assistant?

OpenAI just released Canvas, a new way of interacting with ChatGPT. The focus of Canvas is collaboration between the user and ChatGPT, which goes well beyond a chat interface, which I think is the right take when it comes to coding. Think about this as a cursor alternative, but it lacks a lot of features which I'm going to highlight in a practical example that we're going to build later in the video. But let's quickly look at some highlights from the blog post. Although the chat interface is easy and works well for many tasks, it's limited when you want to work on projects that require editing and revisions. So Canvas offers a new interface for this kind of work. In terms of the implementations, it's very similar to the Artifact feature from Anthropic, but in the current form it lacks a lot of features that are available in Artifacts. When you are working with Canvas, it will basically open up another video on the side that will show you the code or the writing and then you can make edits and ask it to review the code for you. Now, for writing, it can suggest edits, adjust the length of your document, change reading level, add final polish and add emojis. It seems like there are custom prompts which run behind the scene and when it comes to coding, it can help you code review, add logs, add comments, can also fix bugs, and port your code to other languages. We are going to look at how you can activate these features later in the video. Let's look at how this was trained. So this is based on GPT4O trained to be a creative partner, or more like a copilot. This is trained on synthetic data generated and distilled from O Preview. And with the recent API updates, you can actually use distilled outputs from the bigger models to train smaller models. So they're using a very similar approach that is now available to developers. In some cases you will actually need to ask the model to use Canvas as very similar to artifacts, because in some cases even the CLAUDE fails to use artifacts, so you will have to trigger it. The behavior seems to be very similar here as well. Now Canvas is in early beta. That means it's going to be lacking a lot of features, but hopefully OpenAI is going to be making very quick updates. I'm going to put a link to this blog post, but now let's look at a practical example. If you are a plus user, it should be available to you. Now you can click on the model selection and Then click on GPT4O with canvas. In some cases, if you don't see this, log out and log back into your account. Now, when you select this, you don't really see any difference between the Interface of normal ChatGPT GPT with Canvas. Let's try to create a coding project and the Canvas will appear on the side. For this project, I want to use the Flux model to generate images based on a text prompt and the app should be able to download these images. Okay, so here's the prompt that I'm going to be using. I usually find that if you provide very detailed instructions to these models, especially for coding problems, they tend to do really well. So the task is to create a Python based web application that allows the user to generate images from text prompts using Flux Model hosted on Replicate. The application should have a modern responsive interface built with HTML and as you can see, I am specifically asking it to use certain technologies, not letting it decide what to use. So it's supposed to use the Flux model that is available in Replicate and then refer to the official documentation for instructions on how to integrate and call the model. Now I'm providing the official API documentation. Later on I'll show you how you can edit that or provide further instructions. When we are using the Canvas feature, the UI should be a modern and clear user interface using HTML. The main interface should include a text box where the user can type their input the image generation prompt. Once the prompt is submitted, the app should display the generated images. Now, it's also supposed to display two buttons. One is the download button, the other one is regenerate button which will let the user regenerate the image with the same prompt. Okay, so I copied my prompt here and let's run this. When we run this, you're going to see it opened up another window on the side. This is basically the Canvas and even the UI is very similar to something like cloud artifacts because you can click on this and then this will show you all the code. Now from the code I can see it did not use the Replicate Python client. The API documentation provides the Python client. So, so this is one of the first issue that we are encountering in the code. The second thing that I can see here is I think it started the HTML file but did not finish. So I'm going to ask it to provide the complete code and see what it does. It's actually just going through the same code base and you can see the changes it's making. Now it's rewriting the HTML code. This looks good. It gave me two different segments. One is the HTML file, the Other one is the Python file, but here it doesn't really tell me how you can run this. It's a little different than the usual GPT4O experience, but I think the main part is here. If you come here, here's the code preview, then port to another language. I think since it's both the HTML and Python, it doesn't really give me another option. So I'm going to click this again. There is Fix bugs, add Logs and Add comments. Let's start with the Add comments as we'll go through the code again one by one. It starts adding comments. Now, since it's a collaborative tool between the user and ChatGPT, I'm going to suggest some edits here. I went to the official documentation of for using the Flux model to copy this. Here I'm going to just select this specific function and then either you can edit or ask the ChatGPT to explain this code to you. Okay, so instead of the REST API, I wanted to use the official Replicate Python client with the Flux documentation. So I'm going to provide that Python code example and let's see what it does. As you can see, it added Replicate and it's updated all the code. Now it's directly using the Replicate Python client. Now, I still have to run this to make sure that it works. Let's see if we ask it to add logs, what it does. So there are pretty extensive comments, and right now I think it's simply adding more print statements, not actually adding any logs using the log functionality within Python. Okay. There is also an option of Fix Bugs. If you click on this, I think it goes through the code one more time and tries to identify if there are any bugs in there. It's making some small adjustments here and there, but I think probably it's not going to be as robust as you identifying the bugs in the code. So I ran this a couple of times and it's changed the way it's saving images. Now it's using the webp version, but it looks good to me. Let's try to run this code and see if it actually works. Okay, so I'm in Cursor. I created two different files. One is the Python file, which I'm calling Flux UiPyT, and the second one is the index HTML, which is the UI. The backend is based on Flask API and Index HTML contains our front end. I had to create a new virtual environment. I installed Replicate and Flask in this virtual environment based on the documentation from Replicate. I had to set my environment Variable which is basically my Replicate API token. And now let's just run this. So another thing, I created a new virtual environment which is in there. Let's run this and see what happens. Okay, so here is the ui. It's pretty simple. I'm going to say create an image of a Llama with sunglasses. Alright, let's click on Generate Image. I don't see any issues in the back end. It generated the image for us and then we can regenerate and see. Yeah, this is pretty neat. So it's using the Replicate API with flux on the back end and it works. Let's download the image so we can download the image as well, which is pretty neat. Now I just had a quick test on it. I will play around more with this and I think I'll definitely find some more stuff. It's a very neat idea. I think we collaborate with ChatGPT in a very different way, but I don't think it's as feature rich as artifacts or cursor. Another problem is that it puts everything in this single canvas, so if there are multiple files they're going to be listed in here. Unlike Claude, which can create multiple artifacts for different files in the same chat session. The ChatGPT canvas in the beta does not have that ability yet, but hopefully OpenAI will be able to add this pretty quickly. Here I was using Claude to create a UI for local GPT vision. It's now available as a standalone repo on GitHub. I'll put a link to local GPT vision which is basically creating an end to end RAC pipeline using vision models. So both the retrieval as well as generation happens through a vision model and highly recommend to check it out. Okay, so this was my very first impression of using this new canvas feature within GPT4O. I think it definitely has a lot of potential, but at the moment it's missing a lot of features. OpenAI has been pretty slow in releasing different features, but with the dev day it's definitely catching up and seems like the focus is shifting to developers, which is always great news. These were my quick impressions with the first use of Canvas. I'll be creating some more videos when I figure out some more details. I hope you found this video useful. Thanks for watching and as always, see you in the next one.