Search and SEO in 2013, 2023, and 2033

Oh shit, there's actually people in here. I was hoping to talk to an empty room, but here we go. Hello everybody, I'm between you and one more talk and then it's lunch, so I'm gonna massively overrun, just so you know, because I timed this talk at 28 minutes and I've got 20, so we'll see how well we go. Who here was working in SEO in 2013? Show of hands. There's a few hands going up, right? Who here was newer than that, like 2016, '17, '18? Yeah, a bit more. Yeah, I was working in SEO as a hobby in 1998, which makes me a really old motherfucker. I didn't even know it was called SEO back then. I just, you know, tried to get listed on the Yahoo directory and ranked in AltaVista and all that shit. And of course Google came along and changed everything for everybody. So let's look back at the last 10 years first and foremost. 2013, who here remembers that? Feels long ago, but wasn't actually that long ago. Because in 2013, these were the trending keywords in Google: Oscar Pistorius just murdered his girlfriend, Nelson Mandela died, the worst of the three Iron Man movies came out. Was a pretty shitty fucking year, wasn't it? Boston blew up, Boston Marathon. Yeah, it was just fucking terrible. But this is what Google looked like back then. Remember these? The ten blue links and the ads that were actually looking like ads that you can. You know, distinguished from normal organic results. Ah, the good old days. Lots happening there. Before that, Google had all— you know, this is their timeline up to 2013, what they were all doing and all the little interesting things they rolled out, all the new features, all the new benefits. This is basically Google's initial evolution, and after 2013, things took a bit of a different turn. SEO in 2013 was Fairly simple, I'd almost like to say. It was quite deterministic, as in you did A and B would happen. I mean, we had just gotten out of Panda. Remember Panda? Fucking hell, what bullshit that was. Panda was basically Google's attempt to do what Google is still trying to do, is filter out shit content. They're not very good at it, but Panda was their first proper stab at it. And this was like Panda had started rolling out several years, they had newer and newer updates. Penguin had just arrived. Remember Penguin? Penguin. Penguin was more or less aimed at over-optimization, as in too many links or too many exact match anchor text and that sort of stuff. And Google said, right, we found a way to get spam out of search results. Yeah, right. And of course, Hummingbird was confirmed in September. Hummingbird was an interesting update. Hummingbird was Google getting a bit smarter about looking at keywords and looking at search terms and understanding what you actually meant when you typed anything into Google. So it got a bit smarter and a bit better at understanding search intent. And we had the Knowledge Graph, the early stages of the Knowledge Graph, which I will be harping on about quite a lot today, because the Knowledge Graph I think is something people are sleeping on a little bit, and it is quite crucial to how Google still today works and will continue to work in the future. So SEO in 2013 is more or less business as usual, I think. It was fairly relatively straightforward. You know, you wrote content, you published it, you built links, and you got your site ranking on the 10 blue links in Google search results. And then 2015, everything changed. 2015, RankBrain arrived. And RankBrain was a fundamental shift in how Google approached search. Not because RankBrain itself was particularly good, it wasn't, but it was Google's first foray into using machine learning, or AI as it was then known as, which is fucking bullshit because this was just machine learning. Was just simplistic machine learning models. Well, it was the first time they actually used machine learning in their search results. RankBrain was primarily intended to allow Google to serve up relevant search results for when they didn't really understand what the query was about. When they couldn't properly assign search intent, they had this machine learning system called RankBrain that basically enabled them to drive relevant results anyway, despite the fact that it was a bit fuzzy what the user actually meant. And it was, like I said, the first time Google used machine learning in their search results, and that was a reluctant step from them. And the reason why it was reluctant was because up to that time, this guy, Amit Singhal, was in charge of search. He joined Google, I think, in 2001, and the first thing he did was basically rewrite the entire Google search engine. He looked at what Sergey Brin and Larry Page had written and said, this is fucking bullshit, and just rewrote the whole thing from scratch. So Google up to 2015 was basically Amit's little toy. And Amit was very much against using AI and machine learning in search. And he sort of got overridden in the end because, you know, we had RankBrain. But he was still quite resistant to it. He was very much a fan of manually crafted ranking algorithms, of human editorial input in how search and search results were being shown. And then Amit got fucking sacked for being a sexual predator and got replaced by this guy, John Giannandrea. John Giannandrea, up to that point, was the head of Google AI. And Google said, right, why don't you go and take over search? And you can imagine that was quite a drastic shift of direction all of a sudden. You had a very anti-AI guy who got shown the fucking door, and then you get a very pro-AI guy who's replacing him. And that's really when things started really accelerating for Google in terms of how they embraced machine learning and AI in their search ecosystem. And then 2018, something major happened which went mainly unnoticed in the US but had a massive impact here in the UK. This is a Sistrix visibility graph for a whole bunch of news websites. You probably can't read that, but there's The Sun, The Daily Mail, The Express, and The Mirror, basically the tabloid news websites. And in March 2018, They got fucking sledgehammered. Absolutely sledgehammered. And they have not recovered since. Before then, those news publishers could basically rank for whatever the fuck they wanted to. And not just in news results, but also in the 10 blue links, the regular organic results. They wrote an evergreen article for like, what's the— I'll give you an example. The Telegraph had a number 1 ranking in Google for top 100 songs of all time. The Telegraph. Yes, the Daily Telegraph. Not Rolling Stone, not a music magazine, no, the fucking Telegraph. Because news publishers could just rank for everything because they had so much authority, so much link value, they could rank for whatever the fuck they wrote. And the smart ones made really, really good use of that. And then March 2018, Google decided to roll out what I think was the very first of what we now call core algorithm updates, which is a machine learning model where they basically let machine learning interpret search intent and improve rankings. And I think this was the very first one. And it had a big impact in the UK. It was almost impactless in the US. But I think this was a real moment for the SEO industry when things moved away from manual ranking factors and went into the machine learning era. And from this moment onwards, websites news websites specifically, but basically all websites as well, were put in a certain topic area. As in, you can rank for like celebrity news, but not for this, not for like music stuff. And that's when Google introduced what I think they, they call topic authority. They recently actually admitted that they have a thing called topic authority, and they've had it for years, which is not a surprise to most of us because we've been optimizing for topic authority for Well, the last 5 fucking years at the very least. So this was a big, big shift. And since 2018, we are now in the era of core algorithm updates where machine learning is increasingly driving rankings in Google search results. There's less and less reliance on manually crafted ranking factors. And if you've heard of the acronyms like BERT and MUM, there's also machine learning systems. Again, improvements on the systems to enable Google to understand search intent. BERT and MUM are very much focused on understanding search intent, as sort of follow-ups on the Hummingbird that way, and allow Google to understand what you actually mean, and allows Google to understand longer queries and longer sentences as well, so that they can, you know, serve up relevant search results. Every core update, major core update that Google either pre-announced or admits after the fact, is, I think, an improved machine learning model that they roll out. And this stuff is a constant churn. Google is constantly improving these models based on human feedback. Google employs thousands of quality raters through outsourced companies who get paid to perform Google searches and then fill in a form to give feedback on Google on how good those results are, how good those websites are. We've all heard about this, the quality raters. Google publishes their guidelines. In public for all of us to read. So the things like EEAT we're all talking about, that's people using those factors to send feedback back to the machine learning systems. So the machine learning systems get better, Google rolls out a new version, i.e., another core algorithm update, and rankings go all over the fucking place again. And Google has to do this because machine learning systems can do what manual ranking factors cannot. Because the web is huge and chaotic and a big fucking mess. So Google realized those manually crafted factors are just too gameable, too manipulable. So let's just let the machines take over. They can make more sense of the web than humans can. They're better at doing that sort of stuff at scale. So that's trend number 1. But then you may ask, what about all these other major algorithm updates that Google has done, like the mobile-first indexing stuff, or HTTPS, Or the Core Web Vitals stuff? Well, I'll let you in on a little secret. Those are not so much major algorithm updates as they are social engineering updates, where they basically pre-announce them well in advance to impact our behavior as people who work on the web, to make us do things that make Google's life easier. Google wants us to do mobile-optimized websites because Google wants to serve mobile-friendly results because most Google traffic is mobile traffic. Google wants us to serve over HTTPS, not just because it's more secure, but it means that Google only has to crawl over HTTPS. Google wants Core Web Vitals because it means Google can serve up fast-loading web pages in search results and doesn't have to deal with people bouncing back to search results because the user experience is bad. So social engineering updates. And if you start really looking at Google's public communication about basically anything to do with SEO, most of that is geared at changing our behavior as SEOs to make us do things that are compliant with what Google wants from us. We are being played, people, played like fucking fiddles. And it's working because we're all dancing to their tunes. Google says we need to do this, oh, let's go and do this. So that's the additional side of it. It's not just the machine learning updates, it's also the social engineering updates. So yeah, trend number 1, machine learning and AI. That's been a key trend in the last 10 years, especially the last 5 years. Google moving away from manual ranking factors and embracing the AI side of the equation. Now let's go look at today's search results. This is the current search result for website development, at least if you're based in New York, because I use this with a VPN. And you can see there's a lot going on there. We have this big fucking block of ads, we have the Knowledge Graph there to the side. If you scroll down, we get a local business map pack as well. We get web development frameworks, on the side and other entities, and then you get a People Also Ask box, and then finally there's some organic search results as well. There's a load of shit going on there. If you take that as one big search result, that's a fucking huge search result. You can't even see that, can you? We have ads at the top. This doesn't really format properly, but there we go. We have a local pack. Yeah, this doesn't really match up with what I'm showing, but hey, let's go with the flow. People Also Ask, we have an organic search results box, we have related searches at the bottom, we have a Knowledge Graph there as well, and we have related entities there as well. So The image was supposed to be bigger than this, but there we go. This is just one search result, and there's all these kinds of different elements going on, and that's SERP features. Google loves throwing more SERP features on the page. SERP features like this: we have a top stories box for articles. For example, do a search for Arnold Schwarzenegger, and who doesn't like Arnold Schwarzenegger, right? You get a top stories box, but you also get his movies. You get people also search for box with some related entities. You get a shop for Arnold Schwarzenegger t-shirts. You get images for Arnold Schwarzenegger. You get video results for Arnold Schwarzenegger. There's a load of shit going on there. All these SERP features, all these enrichments to ensure Google will present you something of value at some stage in that search result. So that's the second thing that's been happening for years and years and years where these SERP features keep being introduced. And the SGE, Search Generative Experience, is just another search feature. It's just another layer on top of that, another way to interact with Google's index of the web. So more and more diversity in the types of content and the types of stuff that Google is showing in search results. That is the other major trend. And I want to talk a little bit about the Knowledge Graph, because you saw this as one of the SERP features for the Arnold Schwarzenegger search result. And, well, Maria Shriver is his ex-wife. Sylvester Stallone, we all know, because it's his big rival in the 1980s and '90s action movies. But who is the third guy, Joseph Baena? Anybody know? That's his son, correct. And who's Ronnie Coleman? Anybody here know that? Hmm? Mr. Olympia. Mr. Olympia, exactly, yes, Mr. Olympia. Ronnie Coleman has more Mr. Olympia titles in bodybuilding than Arnold Schwarzenegger, because Coleman has 8, Schwarzenegger has 7, I think, 6 or 7. So Ronnie Coleman is a more successful bodybuilder. And why is Google showing this? Why is Google showing these as related searches? It's because of the Knowledge Graph. It's how these entities are related to one another. Arnold Schwarzenegger being the key entity here, that's what you searched for, is related to the sport of bodybuilding because that's how he originally became famous, as the best bodybuilder in the world. And then Ronnie Coleman is a more successful bodybuilder than Arnold Schwarzenegger. And therefore Ronnie Coleman is very closely related to Arnold Schwarzenegger through all those connections. Of course, Maria Shriver there as well, his ex-wife. And then there's a connection there to all this other stuff. Terminator movies, his time as governor of California, et cetera, et cetera, et cetera. The Knowledge Graph powers things like this. It powers things like top stories boxes. 'Cause a website's connection to the Arnold Schwarzenegger entity determines whether or not they are seen as having topic authority on Arnold Schwarzenegger and whether or not their stories will show up. So things like topic authority, things like authorship, things like EEAT, are all expressions of the Knowledge Graph. These are connections of the Knowledge Graph, entities connected to other entities that feed into Google search results and allow Google to populate these SERP features with relevant pages from relevant websites. Not just pages, but also videos and images and all the other stuff going on. I think the Knowledge Graph underpins most SERP features in Google. 'Cause it enables Google to understand entities and relationships, and that's how Google tries to understand the real world, how things are connected to each other in the real world, and things you and I take for granted. Google needs the Knowledge Graph to make those connections and understand how the world basically works. Structured data helps there a lot. Structured data empowers the Knowledge Graph, and it's a key aspect of Google making sure their search results remain high quality. So that's the third trend in the last 10 years, the knowledge graph. It's not so much about keywords, it's about entities and the relationships between entities. Now, I can't go on stage in 2023 and not talk about generative AI. So sorry, people. I asked ChatGPT what SEO will look like in 2033, and it gave me voice search. It gave me personalized search results, and it gave me more emphasis on user experience, which isn't actually that bad, but the others are just fucking bullshit. So yeah, there is a key problem with these large language models. They're basically predictive text on steroids. They're not actually intended on a fundamental level to give an accurate representation and accurate answers. They're intended to generate content, to generate text. The key problem they have is that, as you can see, they just keep churning out bullshit, industrial-grade bullshit. And they haven't quite solved this yet. I mean, the newer models are getting better and better at this, but it's still the fundamental problem that these models have is that they're not intended to be accurate reflections of the world. They're intended to generate content. Based on the likelihood of one word following another word. Now, Google is in a unique position to solve this conundrum because they have something that helps them understand the real world and can inject truth and accuracy in search results. And I've just talked about that. It's the fucking Knowledge Graph. The Knowledge Graph is the solution to Google's challenge when it comes to ChatGPT or SGE or whatever the fuck they call it next week. That they can just— if they can somehow marry up the knowledge graph with the large language models, they will have a system that can accurately reflect the real world and generate content that is more or less accurate and reliable. Now that's a really fucking interesting approach, I think. So let's talk a little bit about search in 2033, where we're going to be then. I think machine learning and AI is of course going to continue to grow naturally. I mean, They've opened Pandora's box, so they can't close it again. The problem I foresee with those systems is that they are basically black boxes. Google's own engineers don't necessarily know how their own machine learning systems work. Because it's output-based. They put stuff in, i.e., the whole corpus of the web, they let the machine learning thing do its thing, and then out come rankings or SERP features or whatever it is. What happens in the middle is a total fucking mystery. You can't open that box and look inside because that's not how these neural networks work. It's a black box. It's an intricate system that does its thing. They can only improve its output by making it learn new things. But what actually happens in that brain of it, nobody fucking knows. So they are black box systems, black box algorithms. And I think we have a fundamental challenge there not just as a search industry, but as a society. And we are allowing these black box systems to control more and more aspects of our lives. And I think there's going to be a reckoning at some stage. But with the slow pace of movement of things like legislative changes, I don't see in the next 10 years we're going to crack that particular conundrum. So yeah, we're still married to machine learning and AI for the foreseeable future. We will get more and more search features which will be powered by AI and powered by the Knowledge Graph. SGE being one of those new search features. I don't think SGE is going to replace search. For those of you who are worried that the large language models are going to make SEO obsolete, rest assured that's not going to fucking happen. You're going to be fine. It's just another box to play in, just another feature to try and get ranked in. That's my prediction. At least if in 10 years we're all out of work, you can blame me for that. And yes, I think the Knowledge Graph is going to stay a pivotal fundamental aspect of Google Search. The Knowledge Graph is really how Google tries to understand the real world— entities and relationships between entities. That is not going to disappear. If anything, it's going to become more important because it is really the only way where Google can ensure their results are accurate and trustworthy and reliable to a certain degree. So the Knowledge Graph will stay, and the Knowledge Graph is probably going to even become more important. So get your structured data markup right, people, because structured data feeds into the Knowledge Graph. It improves Google's understanding of entities and relationships. So the more and better your structured data implementation, the more you're going to reap the rewards later down the line. Now, this is a search result in 2013. Nice 10 blue links. This is the cluttered mess of a search result we get today. So what are we going to get in 2033? Are we going to get like a human brain interface with Google? I sincerely fucking hope not. I'm not going to get a chip implanted into my brain. Nobody wants that shit. But I'm going to make some crazy predictions. Insane predictions, maybe, about what I think search and SEO will look like 10 years from now. I think we will have conversational search. Finally, at long fucking last. We've been promised this for the last 10 years, aren't we? With voice search and all that shit. That never really materialized because the systems weren't smart enough. That's one of the advantages the large language models have, is that they can actually understand conversations. And conversational search. I think we will get to a mode where we will talk to our machines or have a conversational chat interface with machines, like we're already having in SGEs to a certain extent, and can ask questions in a very natural human way, and the machines will understand perfectly what we mean and enable us to get the right results, the right information out of them. I also think we'll get predictive search because these machines know an awful fucking lot about us. If you have an Android phone in your pocket, you're basically carrying the world's most powerful spying device right in your pocket. That shit knows everything about you. And it will know stuff that you need before you even realize it yourself, and it will just prompt you proactively, like, oh yeah, your train leaves in 10 minutes, better get to the fucking station. It'll know you're at the afterparty of the Nottingham Digital Summit, it knows exactly when you need to catch the last train, so it'll tell you that beforehand. Which is really, really handy and really, really fucking creepy at the same time. Predictive search, I think, will definitely be a thing. And lastly, and this is probably my boldest prediction, we will have headless websites. What do I mean by that? We will have a website without or with a very basic frontend. Websites that are primarily intended to be a system for machines to interact with, a data source for search engines and other machines. Like a whole set of JSON files or XML files or whatever it is, or just structured data markup that makes their content machine-readable. Or maybe through an API of some sort. And all you're showing to your users is like a landing page saying, hey, go on Google, use that search feature to interact with us. You see this happening more and more already where you can have an entire shopping e-commerce experience in Google Search without ever visiting the retailer's website directly. I think this is going to accelerate. We're going to end up in a state where some websites, it'll make commercial sense to not build a front end, to just have the website as a data source for Google and other search engines and other search machines. Headless websites. That's what I mean by that. A website without a human navigable frontend. I think that's a real reality. We're already going in that direction to a certain extent by websites, e-commerce retailers especially, providing feeds to Amazon and using Amazon as a storefront. But they still need their own online catalog to feed into Amazon. They still usually have an own e-commerce website based on that. At some stage, it won't make sense for them to maintain their own e-commerce website as a website. They just need the catalog, and that generates the feed that goes into Amazon or will go into Google with product schema. And we'll get to that stage more and more as more and more SERP features will mean more clicks stay on Google's own properties, and therefore we need to make sure we are visible on those properties, which means the case for having a website of your own becomes less and less unless you have something that's genuinely value-added. There. So yeah, in 2023, I think search is going to look the same but also drastically different. I still think we will be doing search with a search box mostly. I don't think search is going to disappear because when you look back at 2013, 10 years ago, feels like a long time, but wasn't that long ago. So 2033 feels like very far into the future but isn't that far. I think it'll be a gradual evolution of search as it will be gradual evolution of user behavior, but fundamentally search and SEO will still be about more or less the same things, especially if you're doing things the proper way and you create stuff that's worth creating in the first place. There have been a few people here talk about, you know, mass-produced mediocrity. I love that quote, by the way, Julio. I think if you're going down that route, you will set yourself up to fail because there's no way to distinguish yourself from anybody else, and search engines don't want to rank that mediocre bullshit either. You gotta do something exceptional, and if you don't have anything exceptional in your business to shout about, then why the fuck are you in business in the first place, right? Do something exceptional. That's really all that I have. Thank you very, very much.