Ethan LockshinSales at NVIDIA focused on AI adoption

Ty Dunn, Co-Founder of Continue, on how he evaluates LLM providers

Name: Ty Dunn, Co-Founder of Continue, on how he evaluates LLM providers
Uploaded: 2025-08-26T13:52:27+00:00
Duration: 1 min 32 s
Channel: Ethan Lockshin
Description: In this interview, Ty Dunn, co‑founder of Continue, explains how his team assesses foundation models and inference providers for AI‑powered coding…

26 Aug 2025 · 2 min

Watch on YouTube

1:32

In this interview, Ty Dunn, co‑founder of Continue, explains how his team assesses foundation models and inference providers for AI‑powered coding assistants. He argues that the most reliable benchmark is everyday developer usage—whether developers gain tangible value—rather than synthetic benchmark suites. Continue offers maximal flexibility, allowing models to run locally (e.g., Olama, OLM Studio, llama.cpp) or via cloud services (AWS Bedrock, Google Vertex, Replicate, Grok) and directly from model labs such as OpenAI, Anthropic, and Mistral. Dunn emphasizes that model choice is context‑dependent: cost, required accuracy for specific tasks, and available compute (GPUs or partner resources) all influence the decision. Because new models appear rapidly, his team continuously re‑evaluates both models and inference providers to find the best fit for each user scenario.

Chapters

You were a founder, you went through Y Combinator, you raised seed funding from Heavybit. You're one of 10 AI coding tools that's really gaining traction in the market. As a founder building with AI, how do you evaluate foundation models or.
Inference providers? So when you think about it, right, like the best benchmark, there's all these benchmarks where it's like put it through a series of tasks. And often kind of the way that models get better is by having benchmarks. But I always like to say that the best benchmark is are developers using it on a day to day basis, are they getting value from it? Right, because we allow people to have maximum flexibility of where they run their models, how they run them, say locally with Olama or OLM Studio or Llama cpp or do cloud deployments, like to gather replicate Gro, put a model that you can find in aws, Bedrock or Vertex, or even use it directly from the labs like OpenAI and Anthropic and Mistral. But then also there's kind of the zooming in for every individual and it's about playing with it, seeing what works for you, trying it on different tasks. And for certain people in different situations, one model might be better. It's not always about what is the best, but what's the best for them. And as costs grow, need certain accuracy on a particular task, you kind of have certain compute resources, whether those are GPUs or a partnership with someone like Grok, right? You end up in a situation where the best might not just be purely which one gives you the greatest code. You're just constantly evaluating models because there's so many new ones coming out. Constantly evaluating, evaluating infants providers and figuring out which one works for your situation.