How To Find and Fix Common Technical SEO Issues - Webpromo.Expert

In this webinar, SEO consultant Barry Adams walks the audience through the most common technical SEO problems and how to fix them. He begins by explaining crawl budget and how Google discovers URLs via sitemaps, links, DNS and user behavior, then shows how duplicate URLs, improper redirects, and missing SSL can waste crawl budget. He demonstrates Apache .htaccess rules (and equivalents for Nginx) for canonicalizing www vs non‑www, trailing slashes, and HTTP→HTTPS redirects, and stresses testing SSL with SSL Labs. The second part covers on‑site issues: poorly built XML sitemaps, mismatched canonical tags, internal redirects, pagination, faceted navigation, robots.txt/X‑Robots blocking, page‑speed diagnostics (WebPageTest vs PageSpeed Insights), and structured data implementation (preferring JSON‑LD and using Google Tag Manager). He also discusses how to handle expired pages (404 vs 410 vs 301). The final segment focuses on international SEO, covering domain strategy (ccTLD vs gTLD), geo‑targeting in Search Console, sub‑directory vs sub‑domain structures, language attributes, and correct hreflang implementation. Throughout, he shares real‑world examples, tool recommendations (DeepCrawl, Screaming Frog, Majestic, Google Search Console, etc.) and answers audience questions about link metrics, sitemap strategy, schema latency, and title overrides. The session ends with contact information and a promise to share the presentation.
Chapters
Let's absolutely disregard if more people join us inside. They can pop in later. We'll go on with our presentation. OK, guys, we just started. And actually, so we have Barry Adams with us today, or tonight, depends on your time of day or night where you are now. So this is a funny story, okay? I already— we already discussed it here before starting, but I can't resist. I can't resist. So when I hear Barry Adams from Belfast— Barry is from Belfast, from Northern Ireland— I immediately thought about he's Irish. But he is not Irish, actually. He's Dutchman. He's a real Dutchman from Holland living in and working in Northern Ireland. So because he's Dutchman, I decide as a matter of respect, I dress tonight or today in orange. It's a national color. Yes, yes. And look, look at Barry. His beard is orange. So he is really, he is really a Dutchman. Now we can see. Okay, we have another person joining us from India. Our colleague in India and our regular. Okay. Nitin, can you hear us? Nitin, are you with us?
Yeah, hi.
Hi. OK, it's Nitin from India, SEO from India. Musa is from Pakistan. It's our expert panels. And obviously Barry Adams, Barry Adams, a Dutchman from Northern Ireland, who is our speaker tonight, or today, depends on where you are now. OK, I guess our preliminary stage is finished and we can go on with our presentation. Guys, are you ready?
Yeah, pretty ready.
Okay, we're all ready. So I ask, guys, you to mute your microphones during Barry's presentation and when he will finish we can talk and you can have questions and I'm pretty sure Barry will be glad to have answers.
OK.
I'm muting my— I wanted to say I'm muting my microphone, but I muted yours. No, I'm muting my microphone, and after that I muted— and Barry, it's all yours.
Thank you very much, Anton. Thanks for having me on this webinar. I am going to be talking about technical SEO, obviously, and about some of the more common problems that I encounter whenever I do an audit on their website for technical SEO issues. First, a little bit about me so you know who this guy is who's talking to you for those who don't actually know me. My name is Barry Adams. I work for Labic Digital where I do SEO audits. And I also am an editor for State of Digital. I speak at quite a few conferences both here in the UK and Ireland as well as abroad. I do some lecturing for local universities. And I've been lucky enough to have won a few awards here over time. I'm not saying that makes me particularly awesome. I think it just makes me good at writing award submissions. But there you go. This is the agenda for today. Basically, I've divided it up into 3 main sections. The first section is about server-side technical SEO and talking specifically about Apache web servers. Then a little bit about on-page and on-site SEO, what you can do to make your website more search engine friendly from a technical point of view. And lastly, a little bit about international technical SEO, very specific things that apply to international and multilingual websites. And then towards the end, we'll have a bit of a Q&A to see if anybody has any questions about that. So server-side technical SEO, what do I mean by server-side technical SEO. Primarily, in my view, it is about optimizing your website for crawling so that when Google visits your website and crawls your content, it spends the most amount of time crawling the right pages and the least amount of time crawling pages and URLs it shouldn't actually be crawling. Because every site has a crawl budget. That's a limited amount of time that Google is going to spend on calling your website before it just gives up and goes somewhere else. And this correlates more or less with how authoritative your website is, how many links you have pointing to your website from other sources. The more links you have, the higher your crawl budget is going to be, and the more time Google is going to spend trying to crawl your website. Having said that, you know, the crawl budget is never infinite. Google will at some stage give up. And go and do something else, no matter how well optimized your website is. So you need to make sure, especially if you have a larger website, that that time frame is used optimally and that Google doesn't waste a lot of time on pages and URLs it shouldn't actually be crawling in the first place. And how your website is set up and configured plays a very big role in that. And this is what I want to talk to you about today. First, we need to understand where Google gets sources from when it decides what pages to crawl. Roughly, these are the main sources that Google uses to discover new and changed web pages. Of course, it can always start on a website homepage. It does do a regular site crawl, which is what we usually associate with the Google crawl. XML sitemaps that you supply to Google through Google Search Console are a very good source of URLs for Google to crawl. And of course, links from other websites pointing to your website. When Google finds that link, it'll follow it and start crawling your website from that point in. Google also maintains name servers, which means it keeps track of DNS records. So if you create a new subdomain, for example, Google will know about that through the name servers and will be able to crawl it. This is very important if, for example, you have a staging website where you develop a new version of a website for a client and you don't lock it down. You have it, for example, on a subdomain like staging.website.com. Google will know that it exists and it can crawl and index it, which is of course problematic if you're still building and developing a new website. So you need to be a bit careful with that. Always make sure you lock down your staging environments with robots.txt blocking or other mechanisms. Google is also a domain registrar, so it knows when new domain names have been registered and can add these new domains to the crawl queue so that at some stage in the future, the crawler will go and visit that new website as well. So just because you've registered a domain and haven't actually verified it in Google or done anything else with it doesn't mean Google can't see it. Google knows about new domain registrations and can crawl those new domains. And lastly, Google harvests data from your browsing habits. If you use a browser like Chrome, Google will know what pages you visit. And if it's new, interesting content, Google can add that to its crawl queue as well. So lots of different sources that Google can use to find and discover new content on the web. When it comes to crawling your website, you'd be surprised how much time Google can spend crawling URLs that it can't actually do anything with. This screenshot is a report from DeepCrawl, which is a tool I tend to use regularly. It's sort of a Screaming Frog SEO spider, but then on steroids. And it's hosted in the cloud, so you don't have to install a local program like you do with Screaming Frog. In this particular instance, we see that on this web crawl, DeepCrawl has crawled nearly 200,000 URLs, but more than 150,000 of them, so a good 3 quarters were non-indexable. That's a lot of wasted time spent crawling pages that can't actually be included in the index for all kinds of various different reasons. And DeepCrawl, of course, tries to mimic a Google crawl when it crawls your website. So if DeepCrawl has this issue with your website, then you can imagine Google has a very similar issue. That, of course, is a problem. We need to investigate why that is the case and why there are so many non-indexable pages on the site. The most common issue is that there is a lot of duplication on URLs. In an ideal scenario, one piece of content has one URL that Google can see it on and that Google can rank in its search results. For example, your website homepage might just be something like this, website.com. However, often you find that websites can also be reached on the www.website.com subdomain, which basically means you have 2 versions of the same piece of content right now. With some content management systems or bespoke-built websites, you also have a 3rd option for the homepage in this case. That could be index.php or maybe it's default.asp or home.html or whatever it is. And that then creates a 3rd version of the homepage. And all 3 of these show the exact same content, It's the exact same web page, but because there's different URLs, they get treated as different websites— different web pages, sorry. And therefore create a duplicate content issue. Some websites also have another problem, of course, that you have something like /category or whatever it is underneath there. But then there's a second version with a trailing slash. And if the website is not configured properly, you will see that these 2 URLs, even though one has a trailing slash and one doesn't, they serve the exact same content. They have the exact same page. And therefore, Google would treat them differently because if there's even one character different in the URL, Google treats it as a separate page with different content. And of course, then you have versions with and without www. And then You create all kinds of different multiple categories there. Now add to this, for example, if a website has an SSL certificate installed, then all of these URLs will also function with HTTPS rather than HTTP. And you can imagine there's quite a massive duplication error right there. Google will try to crawl all these versions unless some sort of measures are put in place to prevent Google from doing that. Some of these— the most effective way to prevent these things from happening is to make sure your Apache, if you use an Apache web server, that your Apache .htaccess rules are properly configured to intercept requests for the wrong version of the website, the non-canonical version, and send them on to the right version. This .htaccess rule right here, for example, will intercept any request to the non-www version and send it to the version with www. with a 301 status code, which means permanent redirect. This is a very simple rule and basically prevents all kinds of duplication errors for website versions with and without www. This is one of those things that I come across very often when I do technical SEO audits on websites. This is the redirect code you need to use if you want to use trailing slashes and default to trailing slashes. This basically intercepts any request for any URL without trailing slash and will append it to the version with the trailing slash. And you can also create exceptions for this rule. For example, the homepage or specific categories that you don't want to add a trailing slash to. For example, you have HTML pages on your site. You don't necessarily want page.html to have a trailing slash after it. So again, you can create exceptions to this rule. So that only folders and categories have a trailing slash appended to it. And .html pages or .asp pages don't get that added to it. And this is an interesting one where you can basically redirect all requests for the non-HTTPS version to the one with HTTPS. Of course, HTTPS being a small ranking factor nowadays, You will find that when people install an SSL certificate to enable HTTPS for their website, they tend to forget to default their website to the HTTPS version. And Google can therefore see 2 different versions of the same website, one with the SSL certificate active and one without, which can create all sorts of problems as well. Not to mention a whole load of security warnings if people try to access the content. So with this simple rule, you basically intercept all requests to your website that do not use SSL, that do not use HTTPS, and then send them on with a 301 redirect to the version with HTTPS. So Apache .htaccess rules are quite powerful. It's something you definitely want to learn more about. Speaking about SSL and HTTPS, by the way, you should always test whether or not your SSL certificate is properly configured, properly set up, and does what it's supposed to do. This SSLlabs.com, they have a fantastic SSL report. You can just throw your website in there and it will tell you whether or not it is a proper SSL certificate that passes all the tests and doesn't have any known issues. Recently Google said, I think through John Mueller, that if there are problems with the SSL certificate on your website, Google might not necessarily associate the ranking boost it does with SSL with your website. So you want to make sure that your SSL certificate is properly configured and properly set up and passes all of these tests. So I would definitely recommend using this sslabs.com test to make sure that your SSL certificate on your website is properly configured and gets a good rating here. Like I said, Apache .htaccess is very powerful stuff, and these are 2 really cool websites you can look at to learn more about .htaccess. Apache is one of the most used web server platforms out there. It's not the only one, of course. You also have Nginx, and it's very similar to Apache in a lot of ways. So you'll find that a lot of things you learn with Apache you can also apply, maybe with a slightly different syntax, also apply to Nginx. If you are unfortunate enough to be working with a Microsoft .NET server, ASP.NET, well, you're in for a whole load of trouble anyway. And I do recommend you get away from that as soon as possible. I'm not a big fan of Microsoft's .NET websites because they tend to bring a whole raft of problems with them. We spoke a bit about the 301 redirect status code. I want to basically emphasize which are the most important HTTP status codes for you to use. Your website— your web server needs to be properly configured to serve the right codes. In the right circumstance, in the right context, so that when your web server interacts with Googlebot, Google gets the right signals from your website for the right pages, and it doesn't get confused about what it is you're trying to accomplish on your website. Of course, 200 OK would be the standard HTTP status code. That's just basically the website saying, right, everything's fine. Here's your content. Go ahead and show it. And this is, of course, for Google. A good signal that it can just take the content and put it in its index. If a page doesn't exist, you should always serve a 404 Not Found error unless you want to redirect it to a better alternative page. Quite often I get websites where when you try a URL that doesn't work, you get redirected maybe with a 302 status code or a 301 status code to a human-friendly error message. But that's of course not what we want. It's called a soft 404 error. And basically, it can confuse Google about what it should actually include in the index or not. Roughly speaking, if someone tries a URL on your website that has never existed before, that is truly a broken invalid URL, it should serve a 404 Not Found error message. And there's 2 forms of redirect, the 301 permanent redirect and the 302 temporary redirect. They do basically exactly what it says. The permanent redirect is a signal that the old page is permanently replaced by the new page. And Google will adhere to that and throw the old page out of its index and replace it with the new page. The 302 redirect is basically saying to Google, yes, this page isn't here right now. You should use this page instead. But keep the old page in the index for the time being. That's the 302 redirect. In most scenarios, you will want to use the 301 redirect. Although sometimes, for example, if you do automatic redirection based on what country a user is from, you might want to use a 302 redirect for that so that Google doesn't take your website homepage out of its index. Lastly, of course, 500 server errors are not something you really want on your website. If you do serve an error message because maybe your site is being moved or there's a catastrophic error somewhere, try to serve a 503 error message, because Google doesn't necessarily act on a 503 error straight away and will know that it's just a temporary flaw on your website. So it's important to understand what each HTTP status code does, how your website responds to different scenarios, and what codes it serves, and how Google will interpret these codes when it tries to crawl and index your website. That is the server-side part of it. Now I want to talk about the on-site technical SEO, which is a lot more interesting, I think. We talk about things like XML sitemaps. I presume everybody knows what XML sitemaps are and how they're being used. So I'll show you an example here of when a website gets it wrong. This is a website that has an XML sitemap which is submitted to Google Search Console and has 2,910 pages in it. However, only 8 of those are actually included in Google's index. And that really hasn't changed in the last several months. As we can see, that is not a good thing. So we want to know why this is actually the case. You give nearly 3,000 pages to Google, but it doesn't really do anything with them, and only 8 of them are indexed. In this case, this was an interesting little conundrum. What I did is I exported the sitemap, and I fed it to Screaming Frog, the Screaming Frog SEO Spider, and see what happened. And what happened was every single one of these was 301 redirected to another page. It turns out that this website was, of course, configured to use trailing slashes so that every request without a trailing slash was intercepted through the Apache .htaccess configuration that I spoke about before and sent to the version with a trailing slash. Unfortunately, the XML sitemap contained all the URLs without the last slash at the end. And therefore they all resulted in a 301 redirect. This of course is a bit of a problem because it makes Google do the work twice. It tries to crawl the URLs in the sitemap, but all of these are redirected to versions that are slightly different. And it basically means double the work. And that's a bit of a waste of your crawl budget. Now for less than 3,000 URLs, it's not really that big of an issue for Core Web Vitals. But if you have very large websites with hundreds of thousands of URLs, you want to make sure that your sitemap is fully optimized and only has final destination URLs in it. And that all of them serve 200 OK status codes. That way you can be sure that your sitemap is optimized and gets properly indexed. Google recommends a maximum of 50,000 URLs in a sitemap. I tended to recommend that as well. But a recent case study from Nick Eubanks changed my mind a little bit. I thought, well, maybe we need to lower that limit a little bit. Because he just showed that if you have 35,000 URLs maximum per sitemap, you get a much higher ratio of indexed URLs. Google seems to struggle with 50,000 URLs in a sitemap. But if you have slightly fewer than that, like 35,000, you have a much higher chance to get nearly all of those URLs actually indexed. So that would be my new recommendation. Make sure you have 35,000 URLs per sitemap, no more than that. And if you have more than that, then you should create a second sitemap. This was another interesting one that I found out where there was a very big gap between the amount of URLs submitted in the XML sitemap and the amount that were actually included in the index. And when I first crawled the sitemap with Screaming Frog, I got only 200 OK status codes. So I thought, hmm, what's going on here? That's a bit weird. So I did a little bit further digging. And it turns out that all of these pages had a canonical tag on them that didn't actually match the URL in the sitemap. So basically, this website was sending a very mixed signal to Google. It was giving all these URLs to Google in the XML sitemap. And then when Google was to crawl them, it found the web page. But all of these pages had a canonical tag on them that said, no, you need to use this version for ranking purposes. Again, a lot of duplicate effort, a lot of confusion for Google, and it might lead to all kinds of problems later on. So you need to make sure your sitemap has the final destination canonical URLs in there only, and you're not wasting Google's time by giving it URLs you don't actually want to have included in the index. The XML sitemap is there for you to basically say to Google, these are all the web pages I want you to index and rank in your search results. So make sure they are the final canonical URLs only. Speaking of canonical tags, some SEOs recommend that you don't use canonical tags unless you have to. I am one of those that disagrees with that. I would say always use canonical tags because it's very easy to create duplicate URL versions of a page without you meaning to. For example, you take this webpage, which is of course a perfectly normal page. And if this is hosted on a website that doesn't use parameters or anything like that, you can think, right, there's only one version of this page. I don't need to use canonical tags here. But what happens when you, for example, share this page on social media? You might get Google Analytics tracking code appended to the end of the URL. And that effectively creates a duplicate version of the URL for the same page. So you can create duplicate versions of a page just by sharing it on social media. In this case, using a tool like Buffer. If you have a canonical tag on your page, you prevent any duplicate content problems from occurring in the first place. So even if you don't think there's any duplicate content issues, I would always recommend to implement canonical tags properly. Speaking of canonical tags, you need to make sure you use the right ones. Only use complete destination URLs in your canonicals. This one here at the top uses a relative URL, and that's the wrong thing to do. Never use relative URLs in your canonicals. The other one I sometimes see is that people leave out things like the HTTP aspect of a canonical and just use a domain name. That too is incorrect. The one at the bottom is the one you want. The full destination URL complete with HTTP.
Yeah.
Or HTTPS, whatever your preferred version is, of the page in the canonical tag. You'd be surprised how often this goes wrong as well. Now, sometimes you see versions that don't have HTTP, HTTPS, and just have 2 double slashes at the start of it. These will also work, but there are some scenarios where those are not ideal. So I would recommend just having the full HTTP or HTTPS address in the canonical tag. Smiley face there. Sorry, skipping a bit ahead. Now, I did say always use canonicals, but I also think you should be very smart about how you use canonicals. I think there's too many cases where SEOs see a canonical as a quick fix for all kinds of different issues, but don't necessarily understand that it doesn't necessarily cool solve crawl issues on your website. Canonical tag needs to be seen by a search engine before it can act on it. So Google needs to crawl multiple versions of a website and then act on the canonical tag to crawl the canonical version. So when it comes to saving crawl budget on large websites, the canonical tag is pretty useless. Google still can crawl duplicate versions of the website unless you actively prevent it from doing so. With maybe 301 redirects or robots.txt blocking. Canonical tags are primarily for preventing multiple versions of the same page appearing in Google's index, but they don't help you save crawl budget. So it's good to understand that canonical tags are very useful for index issues, but not for crawl issues. What I also see sometimes is that websites have redirects on internal links. Sometimes even on links in the top navigation. Or you click on a link in the top navigation and instead of going directly to the page you're meant to go to, you end up with a 301 redirect to another page. I use the Ayeema Redirect Path plugin to discover internal redirects when I'm just browsing a website. Of course, you can also use Screaming Frog or DeepCrawl to do a crawl on a website to discover any internal links that result in a redirect. In an ideal scenario, you want to make sure all your internal links go directly to the destination page. So they all serve a 200 OK status code. Internal redirects can be leftovers from previous versions of the website or an old fix for canonical issues, et cetera, et cetera. I would always make sure that all your internal links are straightforward and do not use redirects. Because these redirects again use up a little bit of crawl budget and can make the website perform less effectively. Of course, having a very flat site structure also helps with that. That every page can be accessed within a couple of clicks. It makes the website very easy to navigate as well as very easy to crawl so that there's not a lot of effort involved for all pages to be discovered. When it comes to pagination, this too can be a bit of an issue when it comes to both crawl optimization and preventing duplicate content issues. Take this instance where you have a list of men's shoes products, which is 5 pages long. Now there's multiple ways this listing can be sorted. You can sort it on position, on name, price, et cetera, et cetera. And that means these 5 pages can be sorted in 5 different ways. So you have quite a few different variations of pages just showing the same products in slightly different orders. And then on the right-hand side, you also have an option to show multiple different numbers of products per page. So you can imagine that just one product category on a website can generate hundreds of different pages of product listings just because it allows you to sort it in different ways. You don't necessarily want Google to crawl all these different versions. In fact, you want to make sure Google doesn't crawl all these different versions, because it's a massive waste of Google's time. You're not presenting anything new or different to Google. You're just sorting products in a slightly different way. So you need to optimize how Google will approach such a paginated listing. The easiest way is, of course, show more products per page. I don't know about you, but I'm quite comfortable scrolling down on a large web page. Whereas clicking on the next link takes more effort. It means that a new page has to be loaded, etc., etc. I prefer it if you just show 20— rather than 20 products on a page, maybe you show 50 or even 100, and just allow me to scroll down through that. Of course, when you use multiple pages to show a product, you want to make sure you use pagination meta tags so that Google understands it's looking at a series of paginated listings, so that it knows why this is page 3 out of 5 or whatever it is. And, you know, I'm looking at multiple pages which all belong to the same category. Google will then deprioritize deeper paginated pages in its crawling so that it'll spend most of its attention on the main page. As an optional extra, you can include a canonical tag that points to the version of the page that shows all products. This is not necessary. But more of a nice-to-have that you can do. And that could be the version of the page you want Google to include in its index. Lastly, the brute force approach, which I'm actually quite a fan of, is to just block the sorting parameters in robots.txt. If, for example, you can sort a page on price or on color or on size or whatever it is, and that creates a new version of the URL with a different parameter at the end of it, You can just create a rule in your robots.txt file to block that parameter with a disallow rule to make sure Google just can't see it. And therefore, if it can't see it, it can't crawl it and can't index it. Now, sometimes these pages still end up in Google's index because if there's an external link from another website pointing to a page that is blocked in your robots.txt file, Google can still follow that link and try to include it. include the page in its index. There is a second rule in robots.txt you can use, the noindex rule, in addition to the disallow rule. It's not 100% foolproof, but it can serve as sort of a backup mechanism to increase the chances Google really will not include that page in its index. So it's worthwhile having both of them in there if you really want to prevent Google from indexing a specific page. OK. Of course, before you make any changes to your robots file, make sure you test them out first. Google Search Console has a great robots.txt tester, and I would recommend using that and trialing it out with different pages on your website when you make any changes before you actually put a disallow rule live on your website. You'd be surprised that if you— especially if you use wildcards and asterisks, in your disallow rules, how easy it is to inadvertently block pages on your website that you don't necessarily want to block. So always make sure you test changes out first before you actually roll them out. As an alternative, you can use the X-Robots tag as an HTTP status code, which again can be done in the Apache .htaccess file. In this case, with a very simple rule, you can prevent Google From indexing, in this case, PDF files. This is a very simple rule to implement in Apache .htaccess and has a very powerful impact on your website. Because Google just— every time it tries to access a PDF file, in this case, it just gets the HTTP status code, noindex, nofollow. And it will adhere to that. It will say, OK, I won't index this page. And it won't follow any links, obviously. Yeah. So that is a good backup mechanism if you want to prevent Google from accessing, for example, certain file types or certain URLs that match a specific string. Another thing that leads to problems with crawling and indexing is faceted navigation, which is where, for example, people look at a certain list of products and they can filter it down on different attributes. In this case, I'm taking the example of a jewelry website where you can filter down on different types of earrings, different color of earrings, and different style of earrings. Each of these tick boxes you can select creates effectively a new page that theoretically Google could also crawl and try to index. And because you can mix and match these different facets in all kinds of different ways, you can create literally millions of different pages For one product category. And none of these actually have any new content to show. All of them just show the same products, just filtered down on specific attributes. Excuse me. And the screenshot below that is from Google Search Console, from the Index Status report. You can see that there's 2.5 million pages indexed. Now, if you know that the website only has about 10,000 products, you realize that's a bit of a massively inflated number. It shouldn't have that many pages indexed. So you need to make sure you manage faceted navigation really well. From a user experience point of view, faceted navigation is very powerful and recommended to use because it allows people to very quickly find the product they're looking for. But from an SEO point of view, it can be a bit of a nightmare. The easiest way is, again, if faceted navigation uses URL parameters, to simply block these URL parameters in your robots file. That is a very effective and straightforward way to prevent Google from ever trying to crawl them in the first place. Of course, you need to make sure that Google can still crawl the default product listing so it can see all the products and can crawl all the products, and you don't prevent it from crawling deeper into the website to pages you do want it to crawl. But generally speaking, a robots.txt block is the most straightforward way of dealing with fast navigation. The second option is to just tag the links with the rel=nofollow tag. And Google will adhere to that. When it sees a link tagged with nofollow, it will not follow that link. It will not crawl the destination page. This is not entirely foolproof because sometimes you might have a link on another page that doesn't have the nofollow tag. And then Google will follow that other link and find the page that way. But nonetheless, it can be a good backup option in addition to a robots.txt block. Sometimes you see people use JavaScript or CSS tricks to hide faceted navigation from Googlebot. But this doesn't really work anymore. Google has gotten quite good at executing JavaScript and following links that are embedded in it. It's not quite perfect at it yet. But it's good enough at it that as a means of preventing these faceted links to be crawled, it doesn't work anymore. On top of that, Google is now actively discouraging you from blocking JavaScript and CSS in robots.txt. So it can properly render a web page and see exactly where content sits and how the page behaves, which it uses for all kinds of mobile usability as well as general quality analysis. And of course, JavaScript can add load speed to your website as well. So don't use JavaScript to hide faceted navigation. You have to use robots.txt block or the nofollow link tag. Speaking of load speed, like I said, crawl budget is a time-based measure. Google spends a certain amount of time before it gives up and goes to another website. So if you have a very fast loading website, it means Google can crawl more pages in the same amount of time. This is one of those reasons why having a very fast optimized website is so worthwhile. The PageSpeed Insights tool that Google itself recommends is pretty okay, but I am quite a big fan of webpagetest.org. Primarily because it gives me these waterfall views like you see on the screenshot here, where you see exactly how long each element of the page takes to load and how long the different elements of a particular load time actually take. For example, the green section of each is the actual time to first byte. That's how long it takes for the web server to respond for something to be loaded. And then the blue line is actually how long it takes to load it over the line. This is very powerful stuff because it allows you to very quickly identify where things go wrong. In this case, we see that this website has 2 very big images that take more than 2 seconds to load. That is obviously a bit of a problem. We want to make sure we address that as soon as possible. Then, of course, at the top we see the initial page load at 227 milliseconds. We think that's quite fast. But, you know, the time to first byte, how long it takes for the web server to respond to that initial request, is actually quite long. We want that to be 100 milliseconds or less. So maybe we need a more optimized hosting environment, a better web server to host this particular website. And at the top, we also see those A, AA, and FF marks. That's basically how well webpagetest.org grades different aspects of load speed. In this case, images and static content are not compressed and don't use caching, which can also add to the load speed. So those are things you might want to look at yourself as well. For me, this is a much more useful report than Google's own PageSpeed Insights report. This is why I always recommend using WebPageTest to evaluate load speed. And of course, the code itself. I literally took this screenshot only a couple of days ago from a website that was built on ASP.NET. And this is one of the reasons why I hate ASP.NET. This is ViewState code, Microsoft .NET ViewState code. That is embedded on every single page of the website in a given session. This is just totally useless. It adds a lot of weight to a page, makes the page slower to load, and just is of so little value that I can't even begin to express how frustrated I am when I still see this. There's no reason why anyone would have this sort of gibberish in their code. You need to make sure that your HTML code that you're serving to users as well as to Googlebot is clean and concise and optimized and as lightweight as you can possibly make it. And in this case, that is obviously not the case. So don't do this or I get very angry. Lastly, for on-site stuff, I want to talk about structured data. We all know schema.org markup, you know, the extra code you can put on a web page to help Google understand what it's about. In this case, this is the product markup to help Google understand it's looking at a product and how expensive the product is and how many people are offering it, et cetera, et cetera. This will make your page a little bit heavier because there's all these extra code you need to add. And there's quite a lot of things that can go wrong with this code if it's not properly implemented. So I am a fan of the JSON Linked Data markup instead, which is just a bit of JavaScript notation you can put in the header. Head section of your HTML pages. Keep it entirely separate from the body code. It makes it much easier to implement and much easier to fix any problems and errors that you might encounter before you roll it out live. So I'm a big fan of JSON-LD. And for any future structured data implementations, I would always recommend to use this rather than the old-fashioned HTML notation. Speaking of JSON, you can now use Google Tag Manager to inject it directly into pages. So you basically set it up once, configure it once with Google Tag Manager. And if you use Google Tag Manager on your website, the schema code, in this case JSON Linked Data format, will be injected on the page and be configured properly to use the right product name, product price, etc., etc. So with one setup, you basically solve the whole problem in one go. I would definitely recommend you read the article at the link there at the bottom of the slide. It makes life a lot easier if you use structured data markup on your website. And of course, as with everything, test it before you roll it out. In this case, Google has a great structured data testing tool that allows you to identify exactly if a a piece of structured data is properly valid. And if not, where it actually goes wrong. This is very easy to use. You can either tell Google to fetch a live URL. You can just copy and paste a bit of code in there to validate it. And it makes it very easy to see if the code is useful, if it's properly validated, if Google recognizes it. And it can prevent a lot of issues later down the line. I also briefly want to talk about expired pages. You mentioned 404 Not Found. Google advises that, for example, you have an e-commerce website and you remove an old product that's no longer in stock. They tell you just serve a 404 Not Found error. I'm not necessarily a fan of that because I think you could lose link value if certain product pages have accumulated links over time. That link value will be lost if you serve a 404 Not Found error. Yeah. I'm more of a fan to keep the page up and just have a very clear message, as you see in this screenshot, that the product is out of stock or you don't offer it anymore. And then make sure you have related alternative products for people to look at. This way you maintain the link value of those specific pages and you provide a good user experience as well, because people can arrive on this page and find related products that appeal to them as well. For very high-volume and high-turn e-commerce websites, for example, you have a classifieds website, an online auction site, or maybe it's a job listings website, whatever it is, it's really quite impractical to keep all these job listings or online auctions online all the time. So you might want to use a slightly different approach. In this case, I would 301 redirect old URLs to the most relevant new URL, for example, to the main category that that product listing belong to. And keep that 301 redirect online for at least 180 days so that Google understands that it is a permanent redirect and has properly deindexed the old URL. And after that, you can serve a 410 status code. A 410 is slightly different from 404 because a 410 status code basically tells Google, yes, I know this page doesn't exist. It really is gone. Please remove it from your index. So it's a bit more of a strong signal than a 404 page is. OK. Which is why I quite like to use it in this particular scenario. For example, on a job listings website, when the job has been fulfilled and goes offline, you serve a 410 error and Google very quickly removes that job listing from its index. Now I want to conclude with a little bit about technical SEO for international websites, because this is an interesting can of worms that brings a whole new problem with it itself. You'd be surprised how often people get this wrong, that they pick the wrong domain. And for example, I see this a lot, that people have a .co.uk domain name for their website, because they're based in Northern Ireland, which is technically part of the United Kingdom. But then they also want to rank in google.ie because they have a lot of customers based in Ireland or in Dublin. And then they don't understand why their .co.uk website is not performing well in the Irish version of Google. Well, quite simply, if you have a .co.uk website, Google will automatically assume it is targeted at the United Kingdom and will rank it in google.co.uk almost exclusively. It's very hard to get that ranking in google.ie. So before you buy a domain name for your business, you need to have a good clear understanding of what your long-term plan is with regards to your digital marketing. Are you going to be location-specific? Are you always going to be content to target a specific country? If so, then yes, by all means, choose a country-specific domain name like .co.uk or .ie or .ru or whatever it is you want to do. If you, however, have intentions of marketing yourself internationally later down the line, you might want to consider a generic top-level domain like a .com. Or at least be prepared to buy local country-specific domain names later down the line. Dynamic domains can also be geo-targeted though. So you can have a .com domain and have it targeted at the United Kingdom, which is something that I've done myself. My previous version of my website was barryadams.co.uk, which as you see in Google Search Console is automatically associated with the United Kingdom. Whereas at polemicdigital.com, I can manually set that to target users in the United Kingdom. It's quite powerful stuff. And you probably want to make sure that you have this properly configured. Because if you do select this to be targeted at a specific country, then it will help rankings in that country, but it will hurt rankings in other versions of Google. If, for example, I untick this box and say, no, my polemicdigital.com website is not geo-targeted, then it will likely hurt my rankings in google.co.uk, but it might help my rankings in google.ie or in google.com. How you structure your website is quite important as well. You can, for example, have a generic domain name like website.com with subdirectories or subfolders like /gb for the United Kingdom, /it for Italy. And you can verify these separately as separate websites in Google Search Console. And each associated with a separate geo-target. So Google knows that the /gb version, or maybe it's the gb.website.com subdomain you're using, is targeted at the United Kingdom. And the main website.com is not. Ideally, when you do use subfolders, try to make use of official ISO country and language codes if at all possible, because it helps simplify things and And make sure there's no ambiguity about what you're actually trying to accomplish. For example, Belgium is one of those strange countries where they actually have 3 official languages: French, Dutch, and German. So you, in theory, you could have 3 different versions targeted at Belgium, but all of them have a slightly different language. So, you know, if you can use the ISO code BE for Belgium and FR for French language, nl Dutch language, de German language, et cetera, to make sure Google understands which version it should show in which search result. And also make sure you use the HTML language attribute. A lot of websites that I analyze that have multilingual content don't actually use the HTML language attribute. Search engines don't necessarily pay a lot of attention to it, although I know Bing does pay a bit of attention to it. Google depending on the context. So I do try to encourage people to have this properly configured, that if you serve, in this case, an Italian language web page, you have the actual language attribute at the top declaring that, yes, I know this is an Italian piece of content, so please treat it appropriately. And of course, we all know the hreflang meta tags. This is something that was introduced a few years ago and is very powerful stuff. But often is not correctly implemented. Hreflang tags can be a bit difficult to wrap your head around. Basically, you can tell Google that, yes, I know I have multiple versions of the same piece of content, but they're all targeted at a different country. In this case, we have pieces of content that are all in the English language, but aimed at different countries. In this case, Ireland, Canada, and Australia. And that can present a problem again for duplicate content reasons. But you still want Google to use different versions of these in different versions of its search results because they might, for example, have different currency and different pricing on there. So with hreflang tags, you can prevent duplicate content issues and make sure Google understands which international version of your content it needs to rank. A rich version of its own search results. And Google Search Console again is your go-to place to make sure that your hreflang tags are properly implemented and that there's no massive errors on your website with your hreflang tags. Google Search Console can give you quite useful information about whether or not the hreflang tags have been properly recognized and if there's any other geo-targeting issues. That is really me for now, and I think it is time for us to have our Q&A if everybody is happy.
Yeah, I hope everybody is happy. We have— before we go to all the questions, typical technical questions people are asking, if your presentation will be available. Barry, will you pass May I have your presentation so I will be able to upload it?
Yes, will do. I'll send it after the webinar.
Okay, this is done. I will upload it after a couple of minutes after the webinar will finish. But if you look at the chat, you can see audience questions. Before I ask guys if they have questions, you have to read it out loud if you can.
OK, Barry, I got a question.
Moosa, before, before, because it's already a question from audience. Barry, can you read it in the chat?
Yes. Oh, I wonder if it's on chat. Audience question here. Why do I get the different results in the analysis of inbound links to my site? Semrush counts 2, Alexa counts 7, VB12, WooRank says 15, et cetera, et cetera. Well, that's because all of these have different data sources. They all use different data to determine how many links you have pointing to your website. I'd like to use Majestic as a tool, but even Majestic is not entirely accurate when it comes to measuring inbound links to your website. Google itself doesn't give you a lot of information about your links. It gives you a bit of a sampling of links in Google Search Console, but that too is not a complete report. So in short, there is no one perfect source of link data. I prefer Majestic because it has the biggest database of links. But like I said, it doesn't find all links. Either. So we are unfortunately stuck in a position where the link data we have to work with is incomplete, and there's no such thing as one tool that will give you all link data, unfortunately.
OK, I don't know if it answers your question, Dejan. If it's not, you can put follow-up question. Musa, what was your question?
OK, Barry, here's the scenario. Let's say we're working on a huge ecommerce website where they have tons of products, tons of categories, and then the main pages. And the way for going XML sitemap, when I'm creating an XML sitemap, so I put the main links in one sitemap, and then products on the other one, and then categories on another one, and connect those sitemaps together. Is it the right strategy, or what should be the right strategy, especially when you're dealing with links more than 35 or 50K links?
Yeah, I think that is the right strategy to have separate sitemaps. For different types of pages, because it allows you to be a bit more specific in analysis when things go wrong and when you get a report in Google Search Console about how many pages are actually indexed of each type of page. So yes, have one for your static pages. Have a separate sitemap for the category pages. If you have a very large e-commerce website, you could even say, right, we have one sitemap for main categories, another sitemap for subcategories, maybe even specific sitemaps for brand pages or product feature pages, and of course, separate sitemap for the actual product pages themselves. And you will then need to split them up over multiples if you have more than 50,000 URLs. Cool.
That's good.
Thank you.
Guys, Nitin, Rabbel, do we have any questions to Barry?
I see a question in the chat from Nitin. Do I have any idea why Google takes a hell of a lot of time to detect and start showing schemas in SERPs? That is a very good question. And no, I don't have a clue. It frustrates me a lot. I always recommend—
I actually—
Yes?
Yeah, I actually implemented a schema on my website. And it is not showing up in the SERP. And it's been more than 1 year. And still, I can't see— I don't know. I read it at many places that you need to have patience for that. How much patience? I mean, it's more than 1 year, and I'm still waiting for it. You can't see it there.
I feel your pain, man. I feel your pain. I've been having schema.org implemented on my own website for a year and a half, and it's not showing anything. I think it depends on the context of the website. In some websites, when you have a lot of reviews, a lot of different types of schema.org code, Google will be a bit quicker to start showing it because it thinks it will actually add value in that particular context. If, for example, you only have a handful of reviews and maybe a handful of product listings, Google doesn't think it adds a lot of value. It could take a bit longer. But I don't really have a clear answer because I'm not entirely sure myself how Google decides when a website gets a rich snippet and when it doesn't. I wish I could help, but I'm sorry. It's something I still struggle with myself.
All right. I don't know, Nitin, does it answer your question? I guess, well, well, well, well, we don't know. Yeah, we don't know. Rabil, do you have any questions?
No, sir.
More questions? I'll check our website. I don't have more questions on our website. Guys, if you don't have more questions, okay, how does it work, guys? More questions? No questions? Okay, then I'll check it again. No questions on our website, so guess I have to say, thank you very much, Barry. Oh, sorry?
There is a question by Ravi.
Just a question in the chat just popped up from Ravi. Google is displaying site name instead of page title in mobile results only. That can have multiple different causes. Sometimes that happens when you have redirected an old site to a new website. It takes the name of the old redirected site over and shows that in search results. Sometimes it's because there's a lot of links pointing to your website with that name in it, and Or override the title on your own page with the title from those external links. Not really much you can do about it yourself, to be entirely honest. Google has a knack of just overriding your title and your meta description and showing what it thinks is the most relevant title and meta description for your website. So it's not something you have a lot of control over, unfortunately.
OK, but that way, the site which I'm referring to is a responsive website.
I don't think that really makes an awful lot of difference. If it's a responsive website, the code should show the same title and same meta description regardless of if you're looking at it on mobile or desktop. So I do think it might be from external links. Or sometimes you even see internal navigation links, the anchor text for internal navigation links being used as the title in Google search results. All right.
Thank you. OK. We have all questions here. I'll check again. It doesn't seem we have more extra questions. Any extra questions? Okay guys, it's like auction, 1, 2, 3, last questions. Okay, okay, no more questions. But, but sometimes people are kind of slow or they're a little bit shy. But if they will have more questions later on, say tomorrow, some people are a bit slowish like me.
I'm slow.
like myself, for example, will you be happy to answer their questions?
Yeah, of course. I mean, I'm always very happy to answer questions. I always say anybody can ask me questions. That's always free. You only have to start paying when you have to start doing actual work. So the questions always— the knowledge always comes free.
What is the best way to ask you, to communicate with you? Is it what Twitter or what?
I tend to respond on Twitter very quickly, otherwise just send me an email at barry@polemicdigital.com. I tend to respond quite quickly as well.
Okay, I'll put it on our webpage. Actually, what happened with Twitter today? Twitter was kind of down for quite a long time. I don't know, it wasn't only me, was it?
No, I had problems with it as well. It was actually quite good because it allowed me to get work done, but it was a bit distracting. Why is my Twitter not loading?
There you go. Yeah, I'm trying to load.
Not even Twitter is safe. Yeah.
It looks like it works now. OK, I'll put your Twitter handle in our web page. So if somebody wants to ask you more questions or another question, follow up, they can use Twitter. OK, OK, things are coming. Okay, thank you, Barry. Thank you very much. So we had Barry Adams, Dutchman from Northern Ireland, tonight with us, or today, depends on. We also have Musa Heymani. We had Nitin Manchanda and Rabih Dennis as experts. And we had in and out Abbas Rajani, but for some reason, I don't know, he's out now. Okay, thank you everybody. Thank you, Barry, and thank you everybody who were—
Thank you.
watching us tonight. Tomorrow we will have a Google Q&A with Google's Andrei Lipasev, Senior Strategist, with Bill Slavsky, Eric Enge, and Alan Jones. But it's tomorrow. But today we had Barry Adams. Thank you very much, Barry. Thank you, guys. Thank you and goodbye.
Thank you.
More from Barry

NewzDash Webinars: Barry Adams ON Advanced Tech SEO for News Publishers and LIVE Q&A









