Mike Olson explains what big data is and why it matters. He covers the challenges of handling massive data volumes, the role of machine learning, and how systems like Hadoop make it possible to process different types of data at scale.
I'm going to give you sort of the broadly accepted view and then I'm going to. I'm going to actually tell you what I think. So it's possible to get your hands on lots of data. It's possible to get your hands on lots of kinds of data. And it's possible for you to get your hands on a fire hose of data these days. Right. Any one of those can happen to you. Right. You can get transactional data in tweets and weather models. And that complexity, that's just, my goodness, it's a bunch of different stuff. And you can get literally petabytes easily. It's hard not to get terabytes these days. That used to be a scary number. Now it's, oh, yeah, it's only 100 terabytes, it's hardly anything. Right. And the volume, the velocity rather, at which it arrives can be overwhelming. Any one of those by themselves could feel like big data to you. If you've got two or more of those problems, it can be absolutely overwhelming. So volume, variety, velocity, those are the memes that the industry has generally embraced. My point of view is if there's data you want to work with and it doesn't fit where you want to put it, it's big data. And that might be because it's complicated and you got lots of different kinds, it might be because you want to ask way more complicated questions of that data than were previously possible. So there's a very common, until recently, research technique now widely used in the industry called machine learning. And actually some of that work has been done here at Stanford. I would say probably better work had been done at Berkeley. But machine learning requires good systems for scaling out very broadly. And machine learning is an unbelievably powerful technique for deducing facts from data that aren't obvious. Using other techniques, you can let the data tell you what it's about and what's happening. And machine learning really, really likes Hadoop. It really likes these scaled architectures. So if you want to do that kind of analysis, if you want to store that variety, if you want to land a petabyte somewhere and you can't afford to spend $40,000 a terabyte to do it, turns out this platform is pretty good. And big data is.
O processo de reconhecer limites, mesmo quando a vida parece exigir sempre mais.
Depois de tantas histórias, aprendizados e pausas necessárias, o @interior_iza está de volta
