Karina Nguyen is the Research Lead at OpenAI

Model training is more an art than a science. You train hundreds of models and then you learn so much about what could possibly go wrong. And most of the time it comes down to model behavior issues. So one of the things that I've learned early days at Antarctic was you taught the model some of the self knowledge of hey, you actually don't have a physical body to operate in the physical world. But then at the same time you had data that kind of taught the model some of the function calls which is like this is how you set the alarm. And so the model would get extremely confused about whether it can set an alarm, but it doesn't have a body in the physical world. So it's like the model gets confused and sometimes it will over refuse. It's like, I don't know, sorry, I cannot help you. And so there is always like a balanced trade off between how do you make the model to be more helpful for users but also not being harmful in other scenarios. So it's always about how do you make the model more robust and operate across variety of divers scenarios.