Technical SEO for News Publishers

Hello guys, welcome back. We're not— we're keeping you busy, I hope so, right? Not much time between breaks. We got a ton of questions. I'm so excited and pleased actually to present my friend and my co-founder here at Ness, Barry Adams. I keep calling him the man, the legend. And truly he is, right? 23 years in SEO. I think he started when— before AltaVista or something. But this is like Barry Adams. Barry is a veteran of this industry, right? He has been active since 1998. He specializes in technical SEO specifically for news publishers. He worked with some of the largest news publications in the world, including News UK, The Guardian, Fox, MailOnline, and TechRadar. Barry really enjoys sharing his knowledge and experience in these kind of conferences, and he writes a lot as well. He's a regular speaker of digital conferences around the world, and he delivers annual guest lectures for several universities. He has written hundreds of blog posts. You should check them out for sure and subscribe to his newsletter. It's seoforgooglenews.com. I'm going to be sharing it, uh, in the chat as well. Uh, and without further ado, Barry, the floor is yours.

Cheers. Uh, thank you very much, uh, John, for that lovely introduction. Yeah, I've been doing this for, for way too long. Um, and as a result, um, you know, SEO is hard enough as it is when you've been doing it for as long as I have. And yes, I, I first tried to optimize the website for AltaVista and trying to get it listed on the Yahoo directory. That's how old I am. Um, and this is where you say, but Barry, you don't look that old. Um, but as a result of doing SEO for way too long, I have no social filter left, which also means there might be the odd swear word here and there. I always feel inclined to put that warning in there in advance, especially for the American audience who are a little bit more sensitive to the odd profanity here and there. So if you're on a speaker at the moment in a public space, maybe put on your headphones. Um, with that out of the way, let's talk about technical SEO. Why is technical SEO important and why should we focus on it? Uh, it's, it's an important area for news publishers. And this is one of the areas where news definitely differs from, from regular SEO, where news has a bit of a unique challenge to overcome that, that regular SEO doesn't necessarily struggle with as much. Um, and when we talk about technical SEO, we talk about some specific aspects of Google's various processes. Now, in a very, Simplified model Google has has three main processes, as all web search engines do: the crawler that finds content, the indexer that makes sense of the content, and the ranker, the query engine, the front end that shows the content. For technical SEO, we're primarily concerned with the crawler and the indexer. It's about optimizing your website for crawling and optimizing the content on your website for indexing. When we talk about Googlebot, the crawler. Um, there's actually a lot of stuff happening behind the scenes. It's, it's a very simplistic state of, oh, Google just crawls your content and off you go. But there are actually hundreds of different processes that go into the crawler. Uh, you know, it's about discovering URLs as fast as possible by finding links and looking at XML sitemaps, about prioritizing which URLs to crawl, doing some deduplication there as well to make sure that not a lot of crawl effort is wasted on URLs that are very likely duplicates of existing URLs. For example, if Google finds URLs with tracking parameters, it might not necessarily crawl those URLs if it has already crawled the clean version of the URL, because Google's pretty sure it's the same URL. Now, Google might still crawl it after all and decide to do something with it, but generally speaking, Google tries to deduplicate that and not waste effort on URLs that it's pretty sure are duplicate URLs. The queue management is about which URLs Google crawls in which order, parsing the HTTP responses. Is it a 200 OK? Is it a 404 not found, etc.? We'll talk a bit about that in a second. The crawler is also constantly monitoring the time to first byte, i.e., how fast the website responds, and adjusts its crawl rate accordingly. Again, I'll talk about that a little bit more. Resource management, both for the website so it doesn't overload the website and Google itself, so it can crawl. As fast as it can without trying to crash its own data centers. And there's a lot of other stuff going on that we don't really have a lot of visibility on. And all of these we've just gleaned from the things that Google engineers have said over the years, the things that they say in talks, that they say in Webmaster Hangouts and in podcasts. So there's a lot of stuff happening in Google's crawling ecosystem before Google even finds the content on your website for indexing. So this can be very overwhelming, but it's basically for news publishers, a very simple challenge, making sure your new articles get crawled as fast as possible. That's the big challenge you have as a news publisher, uh, with, um, classic SEO, as I call it, or general SEO, you know, getting a new product page crawled instantly or a new section page or a new blog post isn't usually a big deal. You know, you can wait a couple of hours, a couple of days, a couple of weeks even sometimes, and you'll be fine when it comes to SEO. Google eventually crawls it and indexes it. In news, we don't have that luxury. When an article is published, we want to make sure Google pretty much crawls it instantaneously, as fast as it possibly can, so we can index it and start ranking it straight away. One of the more straightforward ways, not necessarily easy ways, but more straightforward ways we can optimize for crawling is to make sure our website responds blindingly fast. We see a strong correlation between Google crawl activity and the server response time when Google tries to crawl your website. In this case, we have a strong negative correlation in this crawl stats report. We saw that when the response time went up, the crawl requests went down. Now, this is still a fairly fast response time. You know, we're talking less than 200 milliseconds, which is more or less the recommended speed that Google advises. And even then, we see that when the crawls Average response time goes up to about 200, 250 milliseconds or 300 milliseconds. Google actually slows down how it crawls the website quite significantly. So, uh, this is an interesting one and it's something we also used to see in the old crawl stats report. You may remember this from the old version of Search Console or even Webmaster Tools. If you've been around long enough, there was an old crawl stats report in there. And these are 2 screenshots from years and years ago from, uh, 2 different client projects of mine where you see a very strong correlation between page speed and how many pages Google is able to crawl on any given day. On the left there, this is an old screenshot from 2013. We saw that when, uh, the time spent downloading a page, basically that's the server response time. When that went from about 2 seconds down to half a second, so significantly faster, the amount of pages crawled per day went from about 700, 800 to well over 2,000 per day. And that was a fairly small website, but even in that scenario, Google could crawl the website much more effectively. On the right-hand side is a more recent one. From 2016, 2017 even, when we saw that a website launched a new version, which was significantly faster and more optimized than the old version. Time spent downloading a page, the green line there on the bottom, went from again around 2 seconds to about half a second. And the amount of pages crawled per day went from about 100,000 per day to a temporary peak of half a million pages per day until it settled to a new sort of equilibrium. of around 200,000 to 250,000 pages per day. So more than 2.5 times the amount of pages per day that was the case before the new version. And that was basically a result because the website could handle it better. The website responded a lot faster. Googlebot noticed that the website responded much faster. So Googlebot increased the crawl rate until it found a good equilibrium and a good static level of crawling the website. So, you know, if you look at your own crawl stats report in Google Search Console, Uh, there's a lot of interesting information in there. The server response time is probably the single most important aspect so that you make sure Google can crawl basically to its heart's content and you don't have to throttle it in any given way. There's an awful lot of extra information in there, by the way. I love this new crawl stats report. And when this was first launched, I, I almost fell on my knees and thanked the Google gods for giving us this level of information, because before this, we only had server log files really to, to do deep dives into Googlebot crawl behavior, and that wasn't exactly a hard science. And this report is much more useful than what we could clean ourselves by looking at server log files. And you see, this is a fairly typical Google crawl stats report for a fairly decent website, fairly heavily focused on 200 OK status codes. It's fairly heavy on the JavaScript. It's something that I do have concerns about for this particular website. Because keep in mind that Googlebot crawl behavior is shared across all file types as well as all different responses. So, you know, if your website is one HTML file and 25 images, that is 26 crawl requests for just that one page. So you have to make sure that there's not a lot of waste in the amount of requests per page that Google has to just load the single page and render it and get a decent overview of what the page is about. The same with client-side JavaScript. I have historically not been a fan of client-side JavaScript, putting it mildly. And you see in this case that Google spends a lot of effort crawling the website's JavaScript and not as much crawling the website's HTML or images or CSS. So that would be an area where I would then look at and say, right, is this necessary? Can we optimize this in any given way? Is this negatively affecting us in any way? So, you know, that's why this report is so useful. The response times as well, 200 OK responses, 304 Not Modified responses, 301 redirects, that sort of thing. It is important you serve the correct HTTP response to any given request that Googlebot or any browser really sends you. This is an area that doesn't often go wrong, but when it does go wrong, it tends to go catastrophically wrong. Especially on older websites with technology stacks that were, shall we say, playing loose with web standards. A very common issue is, for example, when you serve a not found error message, but the actual HTTP response is 200 OK. This is a soft 404 error. That's what Google calls that in its Google Search Console index coverage report. And it's generally a bad idea because the 200 OK status code basically tells Google This is a valid web page, and you should crawl and index it. But then the actual content on the page says is not found error. So Google shouldn't be indexing that page. So that's a problem. You want to make sure that the not found error actually serves a 404 or a 410 status code, so that the HTTP status code matches with the actual content and behavior of the web page on the website. The same with redirects. The 404 not modified is an interesting one. You know, there's a bit of a debate whether or not Google. adheres to that as we wanted to. 304 Not Modified basically is an HTTP response that says, oh, the page has not been changed since the last time you crawled it. So Google shouldn't really have to then actually crawl the whole page and index it. However, by the time Google reads the HTTP status code, it has already received the full HTML of the web page as well. And generally, Google will still then crawl and index the page just to make sure of it.

Yeah.

I am still a fan of the 304 Not Modified page as well, because it also can serve you as an interesting data point for your own internal optimization. And I know that some CDNs like Cloudflare are quite good at serving these 304 Not Modified status codes. So they can be a useful source of information for yourself to see where there's potential crawl waste issues on your website and what you can do to optimize crawling further, or when, you know, you need to maybe be more aggressive with your CDN Caching to make sure that there's not too many requests made of your server, and the CDN does most of the heavy lifting. So yeah, HTTP status codes is one way to optimize for crawling. And keep in mind that all page resources, like I said earlier, consume crawl budget. So all the images, all the JavaScript, JSON payloads, you name it, they all will be crawled by Googlebot, especially when it goes to the rendering stage of your of the indexing process, Google will request all the resources associated with the page. So if you have a lot of images or a lot of client-side JavaScript, that will also consume your crawl budget and could lead to Google wasting a lot of its crawl effort on those sorts of resources rather than actually trying to find new pages and new content to index. Um, so keep that in mind that a crawl budget is shared basically across all aspects of Google's ecosystem, including the Google Ads bot. If you run ad campaigns, and I know some publishers who run campaigns for, you know, some monetized aspects of their website, or like events that they run or subscriptions that they have, you got to make sure that your ads campaigns are correctly configured so that AdsBot isn't doing too much heavy lifting there. AdsBot and Googlebot share crawl resources. And one particularly notorious wastage of crawl resources is when you have dynamic search ads running that map search ads to your website. And you got to be very, very careful with that because It can gobble up your crawl budget quite significantly. I've seen a case recently where 70% of a website's crawl effort was AdSpot. Suffice to say, the remaining 30% that Googlebot needed to crawl the website was not going too well, and that website had severe indexing issues as well. And keep in mind also that link equity, PageRank, basically the old-school term as we used to call it, impacts on crawl budget as well. The more link value a web page has, the more often Google is going to crawl it. This is one of the reasons why internal linking is so incredibly important for news publishers as well. Uh, what Christine also adhered to in her talk, good internal linking is fabulously important for good SEO. Uh, if you have really good internal linking structures that sends link value to section pages, to tag pages, those pages will receive link equity from those internal links and, uh, will be crawled more often, which can help with A, Google understanding the structure of your page and your topical expertise and your editorial specialties, and B, allows Google to find Google in their technical recommendations for publishers say that they want you to have static URLs for your section pages. And that is primarily because Google uses those section pages to find new articles. Uh, you know, there's a bit of a misunderstanding that all you need is a new specific XML sitemap and Google will use that to find new articles, which is true. Google does use the new specific XML sitemap to find new articles, but I learned, uh, entirely by accident that that's not actually the main discovery mechanism that Google has for finding new articles. And I learned that because a client of mine accidentally deleted their news XML sitemap. So they didn't have a news sitemap for Google to discover new content with. And nobody noticed until weeks later because traffic remained the same. Google crawled and indexed articles without any trouble at all, uh, and, and ranked content without any trouble at all. Which told me that, okay, those news XML sitemaps, they're handy. They're the nice supplementary signal, but they're not a must-have. Google actually finds the content by crawling your homepage and crawling your section pages and finding newly published articles on your homepage and on your section pages. The XML sitemap is an additional signal for Google to find more articles. But generally speaking, most of your new articles will be discovered the moment you publish them on your website and they get linked to from other pages on your website, primarily your homepage and your key section pages. Pages, because Google crawls those very often, sometimes several times a minute even, to find new content and then discover new articles to index. And that is primarily due to the link value that those homepages and section pages have. The more link value a page has, the more often Google is going to recrawl it to try and find new content there. So yeah, optimizing for crawling, very important. The next thing then is optimizing for indexing. Again, The indexing process, you know, we can look at it very simply and saying, oh, Google reads the content and ranks it. There's a lot of stuff happening there as well. And this is arguably even more complicated than the crawling system. Uh, Google selects which pages to index in the first place. It tokenizes and parses the HTML, basically cleans it up to make it easy to parse. Google tries to render pages. Now I put a bit of a, of an asterisk in this with the news publishers, because Google says they render all pages that they crawl within a few seconds. I put my doubts with that particular statement from their end. I do think that Google skips the rendering phase for news publishers because rendering is resource intensive. And there is often a bit of a delay between crawling and indexing and crawling and rendering, I should say. Uh, and rendering is basically Google loading all the resources, executing the client-side JavaScript, and basically behaving like a browser. To a certain extent, to get a complete picture of what the page looks like and where the content sits and all of that. And because that is fairly resource-intensive, Google doesn't necessarily do that in real time. It can be a bit of a delay—a couple of minutes, couple of hours, sometimes even between initial crawling and the full rendering of the web page. And of course, in the context of news, you can't wait a couple of minutes or a couple of hours before content gets properly indexed. So I'm pretty confident Google skips the rendering phase for news articles for inclusion. I'm pretty confident Google only looks at the HTML source code and tries to extract the articles from the HTML source and indexes it based on that, which is also why structured data is so important. And more on that later. As part of the indexing process, of course, all the meta tags get processed as well. Uh, Google tries to canonicalize ranking signals on, on the canonical version of the URL, which by the way, is not always the same. Is it necessarily the same URL as you put in your canonical meta tag? Because the canonical meta tag is a hint, not a directive, and it can be overridden by other ranking signals. For example, internal links and XML sitemaps, they're also canonicalization signals. So you have to make sure that you use consistently canonical URLs across all aspects of your website to make sure Google selects the correct one. PageRank gets calculated as part of the indexing process. Google, of course, does its quality evaluation, something that So a lot of stuff happening here in the indexing stage as well. But we can simplify this for publishers to a large extent by focusing on the key attributes of indexing, which for news publishers is making sure your articles get indexed flawlessly and rapidly. Of course, the rapidly is part of making sure your articles get crawled. And then the indexing is about making sure that the articles get processed properly and Google can extract the content without any hiccups and doesn't have any trouble understanding what the article is about. So it can rank those articles straight away in top stories and in Google News. In a nutshell, indexing is basically extraction plus semantics. Extraction is extracting the article from the HTML source or from the document object model. Semantics is understanding what the article is about. When you talk about extraction, it's basically about making sure the article can easily be identified as it sits in the HTML source code. And that Google doesn't have any hiccups with extracting that article from the opening headline all the way down to the closing paragraph and understands then what that article is about. Which means, you know, on the assumption that Google doesn't necessarily render articles when it comes to indexing for news and looks at HTML source code, it also means that clean HTML is actually still important. This is a very old-school aspect of SEO, and I've been around long enough where this was all the rage. And people forgot about clean HTML code for a very long time because they thought, well, Google can parse everything and anything. I have seen quite a few examples over my years working with news publishers where improving the structure of the pure raw HTML source code actually helped articles get indexed a lot quicker and helped articles rank better with almost no other changes done at all. Um, and primarily by things going wrong, we learn what the limits of Google's extraction process are. When it comes to indexing articles from HTML source code. And again, I'm talking about speculation here, nothing that's ever been confirmed by Google, but I'm pretty sure Google has an upper limit into the amount of HTML that it actually will parse before it says, you know what, fuck this, I've got better things to do than go through this mess. And that limit for me is roughly around the 1 megabyte mark. I've seen cases where the HTML source code was more than 1 megabyte and the articles just weren't being properly indexed at all. And primarily that was the case because the head content was too big, because there was a lot of inline JavaScript and a lot of inline CSS styling in the head of the HTML. In fact, so much on some pages we're talking about 800-900 kilobytes of CSS styling in the head before the actual content of the web page started. Now, the developers in that particular project said we do that because it's faster for then for those pages to then render in the browser. And I put my question marks with that as well, because really, 900 kilobytes of CSS, is that really that fast to download, especially on the mobile connection? Yes, you're saving yourself one HTTP request by not having a separate CSS file to download, but Jesus fucking Christ, people, one megabyte of CSS, are you joking me? It can be neater than that. One specific example comes to mind where we had about 450 kilobytes of inline CSS styling in the header, and it was very hit and miss whether articles would be indexed fairly quickly. When we narrowed that down to about 100 kilobytes of inline CSS in the head, the problems basically disappeared. So yeah, there is a bit of a limit there on how much junk you can put into HTML source code before the actual article starts. And you want to keep it as clean, as optimized as possible, so that Google doesn't have any hiccups finding the article in the first place and then extracting the article from the HTML source code. You'll also want to optimize the head in general. I'm quite a fan of putting critical meta tags quite high in the head, so your title, your description, canonical tags, your structured data. Yes, Google says you can have your structured data anywhere in the HTML source code. I'm a fan of putting it in the head again so that there's no hiccups there and Google can very easily find it and parse it. And Open Graph tags— this is an interesting one. I have Again, we're talking anecdotal evidence here where, you know, there's nothing officially confirmed, but they have a nagging suspicion that Google looks at the Open Graph tags, which is basically Facebook Open Graph tags, and uses them as a fallback mechanism when it can't parse the HTML properly. The Open Graph image often gets used as the headline image in Google Discover for articles, which I found to be very interesting, especially in cases where that image differed from the actual structured data image that we had In that same article, so I'm pretty sure those Open Graph tags are sort of Google's fallback mechanism for very quick and easy parsing because Open Graph I think has a broader adoption than news article structured data has. So that's why Google tends to fall back on that a bit. Again, no confirmation on that on any official source, so take it with a grain of salt obviously. But anecdotally, I can tell you those Open Graph tags are quite important. So yeah, start your head section with those crucial meta tags and crucial metadata, and then any inline. CSS and JavaScript, put them lower in the head and make sure they're only valid head tags. I see a lot of web pages that have HTML tags in the head that don't belong there, like div tags or image tags or iframes, and they're not header tags, they're body tags. And when Google sees that, it assumes the head has ended and the body has begun, which could lead to Google ignoring any other header tags that may exist in the head. Below those body HTML tags. So keep your header clean and don't put any tags in the head that don't belong in the head, that belong in the body. Uh, so yeah, doing that sort of a test can sometimes tell you where things might go wrong and it sort of minimizes problems. Uh, you know, generally speaking, 99 times out of 100, if you have invalid tags in the head, Google can parse it still fine and doesn't have much issues extracting the content. But the one time where you definitely want an article to be indexed properly, You'll see Murphy's Law. That's the one time Google actually fucks up because it has a hiccup over a body tag in the head that doesn't belong there. So it's about minimizing those errors as much as possible and minimizing the chances of things going wrong there. And then the article itself, uninterrupted article HTML. I see this again a lot where you know in the HTML source code you have the headline. There's a huge block of HTML below the headline, which is usually your social media sharing icons, and then the actual content of the article. And that's not actually ideal. There's no need for that huge block of HTML to sit there. I'm a fan of using CSS or sometimes even using client-side JavaScript to insert those social sharing buttons or whatever it is after the page is rendered, or, you know, making sure that it doesn't actually sit in the HTML source code in the actual article block. A clean article block in the HTML source from opening headline, which should be an H1 header, yes. All the way down to the closing paragraph, one interrupted block of content makes it easy as possible for Google to extract the actual article content from the webpage. And then you can do whatever you want after that and use CSS or JS to insert image galleries, video blocks, related content, social media icons, whatever it is, whatever junk you want to put in there to ruin your user experience, by all means, throw it in there. But make sure that in this HTML source code, it's either above or ideally below the actual article content so that the article itself is in one clean block of HTML for optimal extraction by Googlebot. So that's the extraction part of the equation. The next part is then the semantics part. So basically, can Google understand what the article is about? This is, you know, I hate telling journalists what to do because I'm not a journalist, and usually I don't have to tell journalists what to do. It's just about writing good content. If you write content that talks about the topic in depth, you tend to automatically mention related entities and relationships between entities. It's not something that often goes wrong. Sometimes you have to tell journalists to be a little bit more explicit than they might otherwise be. I always tell journalists, if you take a sentence from an article in isolation, you should almost be able to get the context of the sentence without reading the rest of the And if that's not the case, if there's too many sentences that lack any context, you may need to clarify them a bit and put a bit more value in there by explicitly stating what you mean. The example that Christine Liang gave in her talk about the Raiders coach headline versus the actual name of the person headline, that's a fantastic example. The actual name of the person is a much stronger signal for Google because it takes away any ambiguity. It mentions the person. In question and makes it much easier for Google to understand what the article is about. And the same goes for actual content of the article. Try and be a little bit more explicit than you would maybe be if it was purely for humans, but don't cater to search engines too much. It's about finding that balance a little bit so it doesn't look too clunky and artificial. But I like giving the advice of, you know, providing just a bit more context for search engines to work with when it comes to writing the article. I'm also a fan of using semantic HTML, HTML5, basically with header and navigation and sidebar and all those sort of heading HTML tags you can put in there. It's not necessary, but it does help Google understand where the actual body of the article begins and ends. So it's just an extra nice bit of signaling you can use in the HTML source to make it a bit clearer for Google. And structured data markup, again, it helps Google understand the article. The more data you can give to Google in the structured data snippet, the better. So don't skimp on attributes too much. There is a sort of a bare minimum you can get away with, which is only like half a dozen attributes or so. But I find that if you give more attributes, you know, you don't have to give a description, but it helps if you do. You don't have to give the full article content in the structured data, but it helps if you do, unless you're a paywall, in which case you probably don't want to do it because then people can still bypass the paywall by just throwing the article into a Structured Data Testing Tool, which is one of my favorite methods, by the way, of reading paywalled articles without paying for them. Sorry. So yeah, the more value you can give to your structured data, the better. Talking about entities and the relationships, by the way, I don't know if you've ever had a play with Google's Natural Language API demo. They have a Natural Language API where you can use for all kinds of different purposes. They have a live demo version. You can throw a piece of content in there and Google will tell you what entities they find In that particular piece of content, and I find this quite useful as well because it sort of tells you what Google thinks the article is about and how confident Google is about those entities in the article. Now I cannot talk about technical SEO for news publishers without at least mentioning Core Web Vitals. I'm going to be fairly brief about this though because I don't know about you; I am fucking fed up talking about Core Web Vitals. The Page Experience update has been live for a very long time now, since August, which in SEO terms is bloody forever. Uh, it's of course more than just Core Web Vitals. We're talking about mobile friendliness, safe browsing, HTTPS, interstitials, which can still be a bit of a problem for some publishers. But Core Web Vitals are 3 load speed slash usability related metrics that Google has managed to classify: Largest Contentful Paint, First Input Delay, Cumulative Layout Shift. Basically, if you're red in all 3 of these, your website is really bloody terrible and you probably should work on these. Google Search Console will tell you exactly what Google thinks about your website in terms of Core Web Vitals. Um, if it's mainly green, you're doing okay. If it's mainly orange, you can probably put some effort into there. If it's mainly red, you really need to get your ass into gear and fix that shit. Uh, huge FAQ. Google's provided a lot of information about Core Web Vitals in multiple FAQs. They also had a recently published Search After Backup podcast about this, which is worth listening to. Now, the real question we of course had now that Google has completed the page speed the page experience update, and we have a couple of months into it now. What has the actual impact been? Can we actually measure the impact of bad Core Web Vitals on search results and rankings? Well, the folks at Sistrix have actually done this study not too long ago. They wrote a blog post about that. And it basically shows that yes, it does actually have an impact. This is a very telling graph where the green line is websites with really good Core Web Vitals. The red line is websites with bad Core Web Vitals. And you sort of see where things are going there. All websites managed to improve over this period. People are actively working on their SEO, but the good websites improved a lot more than the bad websites. So yes, there is definitely a measurable impact from good Core Web Vitals. Now, Core Web Vitals in the context of news has an extra dynamic to it, which of course is— yes, I'm going to use this rare word here— AMP, Accelerated Mobile Pages. A lot of stuff has been said about AMP in the last few days. I'm not going to go into that in too much detail because my face would just be this Jeremy Clarkson smirk all the time. I have gone on record many times saying I'm not a fan of AMP, and I can honestly say this is— we're living through the death throes of AMP as I speak. Suffice to say, Core Web Vitals are measured from the Chrome User Experience Report, people who use Chrome, and that then means that those Core Web Vitals are taken from the version of a page that a user interacts with, which in the context of AMP is often the AMP version of a page, When Google shows AMP on mobile, people end up interacting only with the AMP version of an article. So people, yeah, the Core Web Vitals are often extracted from the AMP version of an article. And AMP has a performance cheat. I'm going to call it a cheat because it's not a real performance advantage. It's a performance cheat because of the preloading and prerendering from the AMP cache, plus all the other shenanigans Google has been up to behind the scenes, if the allegations are to be true. But AMP is no longer required for top stories on mobile. So does it mean non-AMP can rank? And should you switch over? Should you kill off your AMP pages in favor of regular pages? The jury's still out on that one. Uh, the NewsDash tool has a great AMP tracker, which shows that on average, 27% across the world of top stories results are now non-AMP on mobile with a few outliers. I love the fact that the Netherlands, my own native country from way back in the day, has 73% non-AMP articles. Now, I think that's primarily because most news publishers in the Netherlands never actually adopted AMP because it wasn't actually ever a proper requirement for top stories in the Netherlands, I think. So most publishers were like, yeah, fuck that, we're not doing that. But yeah, suffice to say, if you have AMP pages at the moment, interesting times ahead and you may want to consider turning them off. Briefly then, before we run out of time, the future of technical SEO. We may have seen this, the index now statement from Microsoft Bing in collaboration with Yandex, where you have sort of a live indexing API so that Microsoft doesn't actually have to crawl your website anymore, but you just ping Microsoft or Bing in this case when you have a new update. Google, of course, have had their own live indexing API for a very long time, but it's in a beta stage and in a very limited use case for job postings and broadcast events. And they've been very hesitant to roll this out to a broader audience. I do think in the long run, We are going to live indexing APIs, especially for large enterprise-level websites, because it makes more sense for Google to do things this way. It'll save them resources in the long run rather than trying to actively find pages to crawl. With live indexing, you can just provide the pages to Google for them to very rapidly index them. Structured data will continue to evolve, and this is something you do need to be on top of. The folks at Sitebulb have a fantastic page which monitors all the changes in the official documentation that Google publishes on structured data. A fairly recent one is when they started recommending the author URL attribute, uh, specifically for news articles or articles. So that's something you'll want to look at and implement because those recommended properties very quickly tend to become required properties. So that's not something you want to be tardy about. Uh, I'm very excited about edge SEO, which is basically SEO done on content delivery networks like Cloudflare. So rather than you having to adapt your own codebase, you can just basically make changes and manipulate pages as they sit on the edge nodes of your CDN without having to make code changes on your own end. Bit risks involved there, of course, like your live version of a web page that Google and your end user sees differs from your own codebase to a certain extent. But the speed with which you can roll out and test changes as well Uh, to decide then whether or not you want to roll them out in your production code is a huge advantage. Uh, this, by the way, this article from Search Pilot is a great primer on why you should consider SEO on the edge and what the advantages are. And it also allows you to do A/B split testing with SEO, not necessarily for articles because, you know, Google will only see one version of an article, but for like section pages, static pages, or commercial evergreen content, you could do A/B split testing to see what works and what doesn't work. which is fascinating. And it sort of opens up SEO to a whole new wealth of experiences that we can try stuff out and see what works and what doesn't work. Google is getting better with JavaScript. So I may need to shut up about JavaScript at some stage, but for the moment, I still love griping about it. We're getting better and better reports in Google Search Console. Was it only yesterday that Google said they now have Google Discover and Google News data in their Search Console API, which is Awesome. Big fan of that. Much better documentation that Google is constantly rolling out. So I'm a big fan of that as well. It makes our life easier. It allows us to build stronger cases to development teams as well, if we can supply official Google documentation about how stuff should be implemented. So yeah, it's a good time to be in technical SEO and it's— our lives are getting gradually easier and more effective in the kind of changes we can recommend and the benefits we see from those changes. Very, very briefly, technical SEO tools that I use. I use a variety of crawlers. DeepCrawler is my go-to cloud crawler. Sitebulb and Screaming Frogs are my go-to desktop crawlers. I use Little Warden to monitor web pages and websites to prevent me from going totally apeshit when stuff goes wrong. SEOInfo is the only plugin that I use in my browser to tell me everything I need to know about a web page. SEO Browser is a great tool where you can monitor search results across all kinds of different countries by just, you know, sort of live VPN to see how web pages, how search results look like in different countries and different nationalities. PageSpeed Insights is very, very useful nowadays. It gives you really good insight into what you need to fix to make your pages more performant. Same with WebPageTest, which is really good, also partially maintained by Google. And I like GTmetrix as well, because they are very hard to please. And if you score well on GTmetrix, you tend to do well in all load speed measurement tools. I'm going to plug my own article here, Technical Optimization for Articles on the seoforgooglers.com newsletter. If you're not already subscribed, please do. This is me. If that talk wasn't impressive, I hope these logos impress you. Thank you very much.

Oh, wow. Wow. Uh, uh, Barry, this is, this is awesome. This is amazing. Um, um, uh, so we have a ton of questions, so I'm going to get into it right away. Uh, let me show this very— the most voted question. This is coming from Tina from, uh, US. Can you give us some insights, uh, onto how much of an impact duplicated content will have on SEO? For example, a news organization that takes syndicated content or a PR company that is sending out press releases or news articles that are published in full.

Yeah. Uh, in a nutshell, very little. Um, Google tries to do deduplication, but deduplication is a resource-intensive process. And in the context of news, Um, I've seen cases where identical articles sit next to each other in top stories carousels with no discernible difference between the two. Um, so by all means, take a feed, publish it as your, as your own, and try to get into top stories with it. I don't think Google has such a thing as a duplicate content penalty. So it's a fairly low-risk endeavor.

Another question here from Pat in UK. Uh, what schema would you recommend other than news article for a news site?

The only one you really need is newsArticle. Um, it's good if you put that as part of a nested structure with like webpage, which is something that, for example, the Yoast SEO plugin tends to do for you automatically. Beyond that, I mean, um, yeah, I mean, for author pages, the person schema markup, I think, is useful to connect the dots behind the scenes and allow people to identify individual journalists and authors. and see their topical expertise. But yeah, news article really is the big kahuna. And I don't think you really actually need much else than that.

Would you recommend, like, if you have a video on the news article to include the video schema and so on?

Good point. Yeah. Video object markup, very useful if you have an embedded video on there as well, uh, to give Google a bit more context. If it's a live update article, live blog posting is a good version to use for, uh, live update articles as well. So yeah, that's a good point.

Awesome. Uh, question from our own Shelby here, which is one of the speakers, uh, coming, uh, later in the event. What is the first step you take when looking at a medium to large news publication from the technical perspective?

Uh, Google Search Console's Index Coverage Report is my starting point. Uh, it generally tells me how healthy the website is in Google's eyes, what kind of errors Google finds, how, how robust its indexing is, how much crawl waste there is. It's, it's a very useful report for me to start. And, um, second step is to crawl the website myself with a crawler like DeepCrawl to see what else I find in terms of like internal redirects, 404 not found pages, et cetera, et cetera. But yeah, start with Google Search Console. The index coverage report is an absolute must-have. Yeah.

I wanna say we, we're gonna have a lot of like, uh, uh, lounge roundtables. Uh, Barry was there. Our sponsors will be there. A lot of them touch on technical SEO, so. So if we don't get to some of your questions, please, in the break, head to the roundtables and there we'll be happy to answer your questions. Okay. We have a question from Arthur here from Brazil. Is there a way to maximize SEO gains through syndicated content? Should we advise any tag used, link on syndicated content?

I'd say just pass it off as your own. Maybe rewrite the headline a little bit, rewrite the opening paragraph, and publish it on your website as if it was your own content. If you know your syndication partner allows you to do that, and a lot of them will just yeah whatever they'll they'll very happily let you do whatever the hell you want with it because it's your content at that stage. Just pass it off as your own and try to get into top stories with it. If your syndication partner insists on things like canonicalization tag proper attribution, the usefulness of your syndicated content becomes significantly less. But you can still just let Google crawl and index it and see if you get any traction on it with. There's not a lot of downsides to just publishing syndicated content as long— and put this caveat in there— as long as most of the content you publish on your website is actually your own unique content. You don't want to have the majority of your own content be duplicate content published from other sources, because that's not a good way to build editorial expertise and trust in Google's eyes. Majority of your content needs to be your own, and then you can just publish syndicated content alongside content you publish yourself.

A question here from Ruff Morlock from Germany: how do you avoid cannibalization if you are writing about the same topic several times, for example about R. Kelly? I think this is a question for Christine, but I wonder, Barry, if you can answer this question.

Yeah, I like the Spoken Wheel approach to that, where you have sort of one master article which is sort of summarizing the whole news event as it unfolds. You have spin-off articles which go into depth into specific areas of that issue. So the summary article, the champion article, is just short summaries of all the different things that are happening with links to these spin-off articles that then are a bit more focused on one specific aspect of the developing story, which of course slightly different headlines and fairly good structure, I think. Like The Guardian is absolutely ace at this, especially when it comes to breaking news stories where they have like one live update article, which collects everything that's happening. And they have spinoff dedicated articles, uh, that go in depth into more specific aspects of the developing story.

Uh, another question here we have from Mara from US. What are good resources, classes to learn more about technical SEO for publishers?

Good question. Um, I think there are a lot of resources. I hate to toot my own horn, but I'm going to do anyway. I actually do trainings, technical SEO trainings for news publishers. So do get in touch if you have a big tech team to train. Yoast SEO, I mentioned that before, they have a technical SEO training course, which is somewhat geared towards news publishers. So you could get in touch with them as well. And there's some general technical SEO training courses out there in the wild, which are fairly decent as well, but maybe not necessarily geared towards news publishers.

Gabrielle from Brazil, more about structured data. How much authority does a linked author receive on an article marked with structured data?

That's a great question, and I have no idea because it's still fairly new. I couldn't tell you how much of an impact that has at the moment. In a couple of months, hopefully we'll have some more information on that, but at the moment it's too new. I don't know yet.

For inclusion of videos, need to include your own original video, need to publish on YouTube. I think this is for Christina, but I have one last question for you, Barry, before we go into break. What are the common technical SEO issues you see across many publishers? You have worked with a lot of them, like big, medium, small. What's common you see?

Pagination is something that a lot of publishers get wrong. You know, they have these automatic load more things and they depend on like user actions like scrolling or clicking on the load more button, which Google doesn't do. Google doesn't scroll, Google doesn't click on buttons. So Google doesn't go beyond page 1. That's a pretty common issue that I see. A lot of internal crawl waste issues as well with broken links and internal redirects that And like I said, that server response time, um, very rare is the publisher that has a server response time that's better than 200 milliseconds. You see in the Core Web Vitals report. And, uh, for me, that's a, well, I'd say easy. Of course, it's, it's not always easy to improve that, but it's a straightforward thing to improve. If you can improve your caching systems to give you better server response times, it just increases your crawl weight dramatically. In measurably good fashion. So that's a common issue I see with a lot of publishers.

Barry, thank you, thank you so much. This has been a wealth of information. I know I'm gonna watch this session once again. All the recordings will be available. Now we have a break till 12:15. Please head to the lounge area, meet with others in the industry, ask Barry and the speakers and sponsors' questions. Thank you again so much, guys. See you at 11— 12:15 for a session with Valentin on analytics.