Anthropic's updated Skill Creator lets Claude build and test its own skills. A PDF-filling skill initially failed but after evaluation correctly populated fields, with metrics showing the improvement. The tool also tunes skill triggers to prevent misfires when multiple skills exist.
Claude skills just got 10 times better. So what did Anthropic actually do that made all these skills better? They updated their Skill Creator skill, which is an official Anthropic skill. And if I open up the actual SkillMD, you can see this is what it does. It creates new skills. It can modify and improve existing skills. It can measure skill performance. So use this when you want to create a skill from scratch, if you want to update or optimize one, if you want to run evals to test a skill, if you want to do benchmarks, or if you want to optimize a skill's description for better trigger accuracy. So I'm going to talk about what each of these little elements mean, but I just wanted to show you that this is the actual Skill Creator skill. It's basically just all of Anthropic's best practices on how to build better skills. So what the evals do is it lets your agent actually evaluate the quality of your skill and then make improvements. Here's a quick example that Anthropic actually ran with this eval. The skill for filling out some PDF stuff was having trouble finding the right spot to put the text. But then after they ran the evaluation on the skill and it was able to improve, now you can see all the text is accurately being placed whether that is a checkbox or just a fill in some sort of field. Here's an example where they said benchmark the PDF skill with and without the skill loaded and show me side-by-side results so I can see the uplift. And we get all this information about these different evaluation metrics. We get the pass rate, we get the total time and the total tokens. So here you can clearly see that with the skill you're getting much better results. And then the final piece is skill trigger tuning. So once you've got a project filled up with, let's just say, 10 or more skills, you might notice sometimes that you get false triggers or you get misfires, meaning you wanted it to use a skill and it used the wrong one, or you wanted it to use a skill and it just didn't use any at all. So using the trigger tuning, the skill creator will basically analyze your skill. It will test out different prompts that you might use to trigger that skill, and then it will edit the description so that that skill gets called more accurately. And this is an actual evaluation that they ran. We have the test score and the train score, and the green and blue are basically the results after it has been analyzed and fixed with the trigger tuning. And what I think is really cool and how I want to end off this section before we get into a live demo is where this is going. And at the bottom we have a quote from Anthropic themselves that say, over time, a natural language description of what the skill should do may be enough with the model figuring out the rest. And I really think that this word may should actually have been will. If you wanna watch the full breakdown, then click on that play button right here and I'll see you guys over there.
