Ethan LockshinSales at NVIDIA focused on AI adoption

Scott Williamson's Advice for Product Managers Leading SaaS Companies into the AI Era

Name: Scott Williamson's Advice for Product Managers Leading SaaS Companies into the AI Era
Uploaded: 2025-07-31T13:23:09+00:00
Duration: 1 min 28 s
Channel: Ethan Lockshin
Description: Scott Williamson says product managers at SaaS companies need to deeply understand how LLMs work and their non-deterministic nature. Since AI outputs…

31 Jul 2025 · 1 min

Watch on YouTube

1:28

Scott Williamson says product managers at SaaS companies need to deeply understand how LLMs work and their non-deterministic nature. Since AI outputs vary run to run, teams must define clear bounds of correctness and decide which responses are acceptable. Product design has to rethink how to handle multiple possible outputs and iterate on prompts to steer outcomes.

Chapters

I've heard people talk about how there's pre AI companies and post AI companies and really that ChatGPT magic moment in 2022 was kind of the pre post LLM company. If you were product manager at a SaaS company, a pre LLM company who was trying to guide their company towards the future in an AI world, where would you start? As a product manager.
I would study like hell. Gotta. You gotta learn the ins and outs of AI systems. You have to know how LLMs work, you have to know how to prompt them. You have to understand that LLM AI systems are non deterministic and you have to manage that differently before either the feature passed the test or it didn't. It either behaved like you thought it should or it didn't. With AI systems you have bounds of correctness, basically and you have to decide what those are and you have to think about the product experience differently because you might prompt the LLM twice and get two different responses. And so which one are you going to allow? Which ones aren't you? And how can you improve those responses so that you get the range of outcomes you want? And so it's a different way of thinking and it is incumbent upon you and the engineers on your team to really understand how these systems work and be able to deliver a great user experience through that.