Library of Minds: 100x AI Engineers Will Be Treated Like Elite Athletes | Kian Katanforoosh (Workera, Stanford)

I don't think humans are good judges of skills. I think they hurt other people. It might become illegal at some point to have a human judge certain skills. You're looking at Cristiano Ronaldo going to the World Cup. How many people work on him being ready for the World Cup? I think the same thing could happen for the corporate world. For every one super worker, you might have three culture architects that are like, how do we entertain them? How do we make them happy?

Super excited to have my friend Kian Katanforoosh here. He's the founder and CEO of Workera, a platform that helps organizations assess and measure and develop skills in the era of AI. And previously, he was a grad student and teaching assistant to Andrew Ng, where he helped co-create CS230 and Coursera's deep learning specialization, which has been taken by millions of people. It's great to have you here.

Thank you for having me.

Would love to start at the beginning. Like my own, your parents fled Iran during the '79 revolution. And had to completely start over. I think your dad switched from having a science degree to selling clothes. What did that teach you about starting over that you think helps you today as an entrepreneur?

Yeah, you know, I think that a lot of people equate starting over with starting over financially. I think in the case of my family, maybe yours and many other Iranian families, it was more of an identity crisis to a certain extent. You're leaving your home country, you're leaving your family, you're leaving your friends, you're leaving your network, you're leaving a language that you speak for a language you don't speak. And so I think the hurdle is much more than financial. It's everything all at once. And my father was a great student at one of the top universities in Iran and found himself in the middle of France, in Toulouse. In literally the middle of France, learning a new language and having to figure things out. And I think what maybe he would say is nobody owes you continuity. What is going, what made you successful before is not what's going to make you successful tomorrow. And so the only thing that you can take with you is not your titles or pedigree, It's your ability to learn, work hard, earn trust from people. And that's the concept of reinventing yourself in any environment.

Seems to be a theme in your life. Yeah, yeah, for sure.

For sure.

Enjoying the conversation? Got a question? You can ask Kian's digital mind. Just check the link in the comments or description. I mean, you were able to build a new network pretty fast. In 2015, you went to Stanford. You were Andrew Ng's grad student and mentee. I feel like most grad students kind of strive for the kind of relationship that you had with him. I'm curious like what you did for that relationship to form the way it did.

Yeah, so I don't think there was a secret in the sense that I was his student first and I think I did well in one of his classes. I then was his TA and I had good reviews. I love teaching. I spend a lot of time thinking through How can I take this complex neural network subject and make it easy for people to understand? And then as a result, I think one day he told me, come to my office. And I went to his office and he said, hey, I'm starting a new class. I'm starting a new company in education. I know you're passionate about education. Do you want to do it with me? And I said, yes. I'm not crazy. It was sort of my dream. And then I think I ended up having a very thorough relationship with Andrew as an advisor, as a manager at deeplearning.ai, and also as a board member at Workera. So I guess he manages me in many different ways, I would say.

Yeah. And you guys created one of Coursera's top classes in AI, Deep Learning Specialization. Millions of people have taken it. And I'm curious, At that scale of teaching people, is there anything that surprised you about the way people learn?

Yes. So first, I think people underestimate the difference between teaching a class to 20 students and teaching a class to millions. It's so different. To give you an example, the first class I taught at Stanford, CS230 Deep Learning, had 38 students. It was in 2017, '18. And I would go and maybe sometimes I would miscalculate some activation function and maybe at the end of the class, one student would notice, would come see me and say, hey, I think your notation was wrong here. And I say, oh yeah, you're right. And then I send an email to the 38 students and they're fine. Nobody was confused too much. The second quarter at Stanford, we had 550 students in the class. So first, any mistake spreads so fast. Like you can get one student confused, two students confused within 5 minutes if you haven't fixed the thing. Half of the class is confused because they talk to each other. That already requires a different level of rigor when you show up and teach. And then you take that to online learning where I remember that day super well. It was a Tuesday. We were in middle of August 2017 and we launched, we pushed a button on Coursera and we launched the deep learning specialization's first 3 classes. I think within hours there were like tens of thousands of people on the platform learning.

Organically or through marketing?

I think like Andrew, everybody follows what Andrew does. And so when it goes out, there's just like a flood of people that are interested. And, you know, like we had to raise the bar of quality so high because we knew that when, if an assignment has a bug in it and it always happens, there's just no classes I've ever seen online with that level of student sort of participants that doesn't have an issue to fix, that can like get tens of thousands of people stuck in one place in an assignment. And then it's not like you're in a classroom and you can go and mentor the students yourself. It's like they're remote somewhere elsewhere in the world and they're stuck on their assignments. And how do you fix that? So the level of quality is super high when you teach at scale. Like everything needs to be bulletproof. And I think people underestimate because Andrew makes it feel very natural. Like he teaches at scale and the slides are polished and everything is great. He makes it look easy. It's actually really, really hard. And then the second aspect is maybe personalized learning. I think personalized learning is by default when you have 38 students, you can do personalized learning.

You can do office hours and one-on-ones.

Yeah, you just adapt to the question that is asked. When you teach online, it's very hard. I mean, you know this meme of the women's basketball coach that like shouts at her player like crazy? You've seen that one?

I don't think I've seen that one.

I don't know if people have seen that one. It's like a basketball, women's basketball coach that literally screams at her player and her player says yes and goes back to the field and like does what the coach said. That level of coaching is only possible usually when you're coaching an Olympian or someone who's reacting very positively to that level of feedback. It's not for everyone. Um, in fact, if you look at a million students, you have dozens of ways of coaching that would benefit them. Some people need nurturing, some people need challenges. Some people need other things. And so how do you do that at scale is really, really complicated.

Was that even possible with Coursera's class at the time?

No, at the time it was not possible. I think with Workera it's possible now because we plug in the flow of work and we understand people's preferences very well. But I don't know, at the time it was impossible for sure. Yeah.

In 2024, you launched Elo as the first AI mentor for workforce skills. Someone who's a big believer in AI as mentors, there's still pushback and I'm curious what you think the person loses by replacing a human mentor with an AI mentor.

Well, definitely loses a lot, but there's no choice. Like there's not enough human mentors. How many Andrew Ng's are there out there? Very few, if none, you know? And in fact, at Stanford, I remember when I was starting Workera, I was interviewing a lot of students around the world that were taking our online classes and I was saying, okay, tell me what's the difference between you and a Stanford student in your opinion? Every single time it would come down to mentorship. It would be like, I have access to the same content. It's all here. But I feel like Stanford students have really good mentorship and I miss that. And I think, so it's not like what are you losing is they have no choice. If you're in the Bay Area, you're probably in a bubble. There's a density of mentors that is higher than anywhere else in the world, especially for AI. But the rest of the world, they are, Mentors are scarce, so I don't think you have a choice. Agreed.

And where is AI not just a better assessor of talent, but more unbiased than humans? And where is it hopelessly worse, hopefully worse?

So generally I think AI, not every AI, but AI trained well with good alignment and guardrails is better, is a better assessor of talent than humans. For sure. And it's easy to see, you know, it's already proven that humans are sort of hiring people and selecting people that sound like them or look like them or believe in the same things they believe in. And to a certain extent, there's some normality in it, like company values, etc. But there's certain things you don't want to be part of your hiring process. Humans also judge people differently depending on the time of the day or depending on what they ate for lunch, literally. And if you've ever tried to run an interviewer training program at one of these big companies, you know that it's very hard for it to pay off because you train 1,000 interviewers and then you look at the data and still the consistency is very low. So it's a very hard problem and I think AI is definitely better than humans. It just has to be with the right guardrails. On your second question, where does it fail and the risk that I'm worried about? Well, one is bias. You know, a bad interviewer can hurt the career of a handful of people. A bad AI can hurt the career of a lot of people. Like it's bias at scale, right?

And so based on the data it's been trained on.

Correct. And so the level of scrutiny you need on these systems is exponentially higher than the level of scrutiny you need on a human interviewer. The second aspect might be sort of conformity. Like you don't wanna lose that genius expert just because they fall out of distribution. So you need to have AI that allows for those edge cases of people that might not conform with the regular skills you're measuring, but might spike incredibly in certain areas and they might fail on certain dimensions that you care about, but they would actually be an amazing hire or an amazing employee or an amazing colleague for other reasons. And that's not so much of an AI problem as much as it is the human configuring the AI properly, like telling the AI what I wanna measure. Yeah.

I mean, one of the moats in startups is finding undiscovered talent. So how does it find outliers and undiscovered talent? Is it purely just me telling the AI what I look for, or even that has its own biases?

Well, one of the advantages comes from the scale. Like AI can measure thousands of people in a single day. No humans can. So by the fact that you look in a bigger pool, you actually have a better ability to find those hidden gems. That's one of the biggest advantages of AI. On top of that, you know, AI now, like our systems are multilingual, they can be deployed globally. You know, brilliance is everywhere. And so AI allows you to sort of measure people anywhere they are in any language so that you sort of do not judge them on their ability to speak English if it's not necessary. Like obviously there's a minimum, but some people, you know, you ask them to jump on an interview in English and they struggle even if they're a genius at whatever they do. So AI helps with a lot of that type of stuff because you can cast a wide net, you can find the hidden gems, but it also has the issues I just talked about.

You have floated that it might one day be unethical for humans to judge human capabilities instead of machines. I'm curious, what's the tipping point where everyone else starts to believe that as well?

I fully believe that. I don't think humans are good judges of skills. I think they hurt other people because of their biases. I think there is a tipping point where the science will prove that a certain AI or Kera or others is already better than humans and there is no coming back from it. Like I do think it is possible that it might become illegal at some point to have a human judge certain skills.

Wow.

And if you're a company, public company, maybe you will have to be compliant. You will have to make sure that you're hiring people based on the best practices and because the best practices are documented out there, you have to follow them. I don't think that's a crazy idea.

Yeah. I like it. Hot take. So you've assessed thousands of people. What is a signal of future success in young people that we don't pay enough attention to?

I like what Jensen Huang said around suffering, I think it's true and it builds resilience. And I think a lot of, I think that's an important thing to look for. Totally agree. The other one is vision. I think one of the curse of, let's say undergraduate students that come to the job market is their life has been sort of charted up to that degree that they're getting. And they always wanted the best, the best high school and the best undergrad degree and maybe the best major. And they didn't have to think a little bit more about what do I want to contribute to society? And you can tell the students who have figured it out and have a very clear vision of like, in 5 years from now, I would love to be able to show that I did this and that and this and that. And those who are still a little bit lost. And I think they need to take some time to figure out what they want to do and they will perform better on the job if they figure it out. Because on the day-to-day, you will have lows, you will have highs, and in a low, you need some guiding light to remind you why you're here. And the students or entrants to the job market that have figured out why they're here tend to be more resilient and be better performers.

That's funny. My first two interview questions are, what's the most you've ever suffered? And what's your 5 to 10-year vision?

We're aligned.

And on the opposite side, what is something that a lot of most ambitious and smart people are trying to improve in, but you don't think it's worth the effort?

I think people get obsessed with the latest trends.

Hmm.

Maybe too much. Like I use the OpenClaw example, uh, a lot. There's other examples, but at some point when OpenClaw was so popular, people would mention it on their resumes. And I'm like, it's great. It tells me like you're watching the news and you're trying the latest thing, but it doesn't give me any signal really that you have the ability to continuously learn. And so I think I would not index too much on the latest shiny object when I'm hiring.

Yeah.

I'm actually interested in the underlying skills and behaviors that the person might have. And maybe an underlying skill or behavior is durable learning. It's like, oh, this person has built, has set up a Mac Mini with OpenClaw on it. That's not interesting to me. I don't care that they know about that stack. I care how do they decide to do that? Why did they decide to do that? Is it because they just have this insatiable curiosity to always look for the latest thing and stay up to date? That I'm interested in, you know? Yeah. Yeah.

I mean, it's related to the learning about your parents, like learning velocity and curiosity. Yeah. How do you evaluate that in like an interview with someone?

So first I think you cannot measure learning velocity in like 30 minutes or 60 minutes, but you can estimate it. The way we estimate it at Workera, I ask that in every single interview where I'm part of, is a few questions like, okay, what's your learning strategy? Just super open-ended question. And people go very different routes when you ask that. What is one thing you did not know that is non-trivial that you did not know 3 months ago that you know now?

I like that one.

Yeah. Teach me something you've learned over the last 30 days. There's a bunch of questions like that that can be a proxy for learning velocity, but you will not measure it unless you have a longitudinal way of measuring it. And so that's where Ambient comes in. Elo would estimate it. Ambient will actually look longitudinally of how much progress you're making, and you can tell how someone's learning velocity is way higher than someone else.

Yeah. Last 4 questions. Name 3 or 4 roles that you are most confident will exist in 4 years that do not have titles today?

Ooh, I would say one would be the CTHRO.

Wow.

Chief Technology and HR Officer. I think the CTO and the CHRO are merging.

Interesting.

I think it's just AI is a technology that impacts people's day-to-day so much that it doesn't make sense to have two separate roles. It's like the chief technology officer should also know very deeply about human psychology, organizational behavior, and incentives.

Isn't that kind of rare? Someone who's just like super technologically advanced, but also people advanced?

It's very rare, but there are some out there. I mean, you know, we work with Jackie Kenny, the CHRO of ServiceNow. I would put her in that category. She's responsible for the company's AI transformation. She's a CHRO. She's effectively a CTHRO, I would say. And she has the dual capability to understand technology and people. And I think that's really powerful today. And I think there's gonna be more of that. And then the question is, are these CTO-trained individuals that like learn HR, or are these CHROs that learn technology? I don't know. I think it could be either, but that role needs to live in a single function, in my opinion. Mm-hmm.

Yeah.

And then I think there's gonna be more roles around culture architect type of thing where, you know, we're gonna have employees that are 10Xers or 100Xers and the real focus will be on how do we retain them. I mean, you see the talent war in the Bay Area right now.

I see it. Yeah.

And so I wouldn't be surprised that for every one super worker, you might have 3 culture architects that are like, How do we entertain them? How do we make them happy? How do we, you know, I wouldn't be surprised.

Like professional sports.

Yeah, exactly. Like professional sports. And it's fine. Like you can have an entertainer, you can have a masseuse, you can have a, you can have a lot of these roles that are critical to the economy. And frankly, like you're looking at Cristiano Ronaldo going to the World Cup, how many people work on him being ready for the World Cup?

A lot.

Dozens, right?

A lot. Yeah.

Well, I think the same thing could happen for the corporate world.

I like that one. So the half-life of digital skills today is 2.5 years, down from a decade generations ago. To the listener who is anxious about their skills becoming irrelevant, what is the adaptation playbook on what someone does this week, this month, this year?

So the framework of perishable and durable skill is important in my opinion. So durable skills are skills that have a half-life of over 5 years and they're gonna be useful for the next 5 years or more. Perishable is usually much lower. As you said, there's a lot of perishable skills right now. Uh, to innovate you need both, especially perishable skills. But today I think the way you split your time on a daily basis depends on whether you're at wartime or crunch time, or whether you are at peacetime or normal time. If you're at peacetime, I would advise in corporate terms, I would advise to spend 80% of your time building your durable skills and maybe 20% on the perishable. Those normal times are great times to invest in your future. And the durable skills are the one who will compound. How good of a communicator are you? How much trust are you able to establish with your peers? How good are you at coaching? How good are you at learning? How good are you at problem solving and critical thinking. I would invest a lot in those. In crunch time, I think the equation flips. I would probably spend 80%, if not 100% of your time on perishable skills. Like you have a project, it needs to get out. It's mission critical. You don't have time, unfortunately, to invest in your communication skill. You have to use the best and latest technologies that are out there that might die in a year from now, but will be critical to get that project out at the right level of quality. And so I think that's the framework I would look for. Perishable, durable, and then crunch time, normal time, pretty much.

I'm gonna take that one. And my last question, if your Delphi could help one person learn one thing, who would that person be and what would be that thing?

That person would be someone that none of us have ever met, probably somewhere in the world, not in the West. And I hope that my Delphi would be able to turn someone who lost confidence about their career into a high confidence person who's gonna wake up tomorrow and learn and then change their life 3 months from now, 4 months from now.

Oh, we'll make sure that happens. All right. Thank you so much, Kian Katanforooshoosh.

Thank you all. Yeah, thanks, Darrin Morris.

Thanks for tuning in. If you enjoyed today's episode and want personalized advice from Kian, head to the link in the comments or description to ask his digital mind on Delphi.