Shagility challenges the myth that ML engineers spend most time building models. A colleague claims 100% modeling time because a centralized data team feeds him clean data in minutes, but that only happens at a few top companies. Most teams spend around 80% of their effort acquiring, cleaning, and wrangling data.
Chapters
80% of your time's getting the data and wrangling it right before you actually get into the machine learning. And the guy goes, no, no, 100% of my time's writing machine learning models and testing them. And Sean said, well, you know, how does that work? And the guy looked at him funny again. He said, well, it's easy. Every time I want data, I just log a request and then one of the data team give it to me in the next 10 minutes. Yeah. Because, you know, they, they had hundreds, if not thousands of people sitting there serving the machine learning engineers with data and., you know, maybe the top, the fan companies do that, but the rest of us don't. And so I think often when we read these papers, we, we forget that it's a different organization with different context. You know, the stuff that they have in place, the people they have, we probably don't. Um, and then how much of it's buzzwashing? Yeah. How much of it's really true? It's true, but how much of it is warts and all, right? I think there's a lot of, uh, happy stories out there, happy pubs, and not, not the, uh, horrendous
