SEO for Web Migrations: Recovering your Organic Traffic after a Web Migration Gone Wrong

Hello there. I hope that you're having a wonderful start of the week, a great Monday. Today I would like to share with you one of my potentially least ideal scenarios and how to handle it to hopefully change it to something positive, right? And it is about web migrations gone wrong, right? Because maybe potentially, very likely, they didn't have the correct SEO support and validation right from the start. And you might tell me, Aleyda, what are you telling me? There are so many guides out there sharing about the importance of taking SEO into consideration right from the start, right before the start even, right? And it took again my attention, this topic comes back again because I was actually taking a look at Sistrix IndexWatch with the top SEO losers of 2018, right? And I saw that there was a whole section here, you can see, of those websites that lost tons of visibility here after a web migration. And not only in the UK, but also in the Spanish version. One of those is actually a Government sites, ouch. And and so today, in today's episode, I want to cover this very particular scenario. What can we do if our website doesn't take SEO into consideration right from the start? Our clients somehow they forgot they should tell us about it, and they come afterwards and say, "Hey, hey, can you help me? How can I recover?" my lost organic search traffic? Let's take a look. One of the main reasons and causes of why websites lose, lose traffic after a migration and, and why the migration goes wrong is because they haven't actually taken into consideration all of their top meaningful pages, uh, to be migrated to their new location. Uh, so somehow they have missed those pages that were ranking very well or were bringing lots of traffic, were being referred, were being linked a lot by many other websites. And so the first step should be a little bit of doing forensic work to identify which have been all those old pages that were driving a lot of organic search traffic, rankings, those most links, those really meaningful pages. That the website used to have to check if all of these pages have actually been taken into consideration in the migration and have been migrated accordingly. That will be the next steps to validate all that. But let's first gather all this, all the data. Ideally, we should start with our own internal data with Google Analytics and Google Search Console. Go to the analytic tool pages, those that were bringing most traffic in general and organic traffic in particular before the migration. Uh, happen, hopefully we will have still have access to, to that, uh, historical data. And we can see here per landing pages, uh, which were these particularly important pages bringing more traffic at that point. And the same with the Google Search Console, right? Additionally, it should be ideal to, uh, compensate and complement this information, uh, with the ones of, of, um, ranking indexes and, and competitive, uh, tools like, for example, Uh, Semrush. I love it because I don't know if you have seen, but it doesn't only allow us to understand, uh, the position changes over time, uh, even of any type of meaningful keywords that they— for which they, they can find our, our, our website pages, uh, ranking for, uh, but they actually allow us to change the day and, and take a look at historical data. We can actually also select here the time and see which were the 2 pages, uh, bringing traffic, organic traffic at that point. Another tool that is very, I have to say, useful and visual too, that the one that I prefer from a visual perspective to double-check how the migration happened and when the migration happened, is Sistrix. Because, uh, we can visualize for any type of migration, not only domain, uh, domain, uh, from one domain to another, but also, uh, from protocol change, HTTP to HTTPS, specific path change. If it is a migration from a subdirectory to a subdomain, for example, this tool allows us to visualize the change in a very flexible way. So we can check the before and after and also the URL changes, how these URLs actually migrated or were moved at that point, or at least the information that the tool tracked, right? So we can come and check for, for those rankings before, uh, the migration happened in July 2016 and, and the ones that, uh, the website has right now, right? So to see how the, the, the, the URLs change and the traffic dropped, the, the how they were moved, if they were actually moved. But here in particular, what is important is that we can grab and we can gather all these important pages, former pages that were bringing lots of, of, of rankings, of traffic, meaningful traffic, uh, to the website before the, the migration happened. And not only from a rankings perspective, I have to say it also is always good to double verify from a link, external link perspective, those pages that were bringing attractive, attracting more external links, uh, more popularity, more, more referred users from other sites, right? So again, we can use, um, link analysis tools like, I, I, like for example Ahrefs, or in this particular case Cognitive SEO, they, they also use the data from Majestic. So it's very, very complete and have this very, very nice visualizations. So I can see which word does, which has been the, the top link pages of, of this website in the past, which have been identified in particular that have have been broken, that are broken over time. They're ones returning 404 HTTP status in this particular case. But then again, this one is most important to me, to bring this URL's data, to double-check if these important pages that at some point were highly linked, if they are redirecting somewhere, if they have been taken into consideration in the migration. After you have gathered together all of those meaningful, important top traffic rankings, former URLs that should have been taken into consideration, uh, it's time to validate, verify if they have been actually migrated correctly as they should have been, uh, right from the start. You want to double-check, you want to validate, you want to verify and, and crawl this list of URLs, important URLs and make sure that the redirects are correct, that these are permanent redirects, 301 redirects, that, uh, the, there are no redirect chains or redirect loops. They, they actually go to the final relevant destination directly, uh, that they are actually going, uh, to a relevant new page, to a meaningful new page, right? Uh, so you can do this by going to, to your list, you, you, the list that you have gathered of URLs that you have gathered and put together. Do list crawl with whatever crawler. Most crawlers, all crawlers I think, they support, uh, to, to do list crawl. Uh, Screaming Frog I'm showing here, but you can also use Cyborg, DeepCrawl, uh, Botify, Oncrawl. And, and, and you can, um, import the, the URLs, uh, and use the list crawl mode, uh, to check them. In particular, I love, uh, this report that Screaming Frog has because it's particularly useful in these situations, uh, the, the report of redirects and canonical change. So for example, I, I crawl the list of all URLs of GameFAQs, which we saw was one of the sites that lost, uh, the organic, uh, search visibility when they move, uh, to, to a new domain, right? And, and as you can see, uh, all of them, or most of them, right, they are— they all have, uh, a redirect going on. What we are going to, to do here to double-check where they are going, if, if they are actually going directly to the, to the proper new version of these URLs in the new locations, is to, uh, select the redirect canonical change, um, report. And we will obtain here, yeah, uh, this Excel, uh, and that we can actually filter to see what type of redirect, the, the, the, if it is an HTTP redirect, uh, going on implemented correctly, uh, how many a number of redirects have been included, uh, if, if there are any redirect loops going on. So you can see it's false, thankfully. If there are any temporary redirects in the chain, for example, this is also very, very useful to identify problems. And the initial URL here, of course, it is non-indexable. It is redirected, uh, the, the, the, the, the URL HTTP status code that is at 301. And where is it going, right? So we can see that redirect number 1 is this older, uh, URL. It is going from, from here to here, to us, um, a subdomain as we mentioned before, right? And, and we can see that, um this is the final URL destination, that there are no more, uh, jumps in, in the redirect chain, right? If there were many, like, redirect chains, if, if, if a former URL going to another one that is again redirected to another one that is maybe probably going to a final URL destination that might be showing a 404 HTTP status, that will be shown here. So after we have made sure that all of the URLs have been correctly migrated to their new location, um, is to validate if the new URLs where they have migrated are correctly optimized, if they are still meaningful, relevant, indexable, and, and they are findable, uh, and they actually— they are, they, they can, they are able to, to maintain the relevance and popularity towards the same uh, topics and terms for the— for which the old URLs were ranking in the first place, right? So I would do 2 things in, in, in this situation. I will release a normal crawl, uh, typical crawl, uh, in, in, in the new website, in the new location, um, and I will also probably list crawl, do another list crawl of the destination, the URL destination of those old pages. Why? Because I want to double-check, I want to verify and prioritize and make sure that the list of the destinations of these old pages, these new URL versions are top ones. These top ones are actually manually revised and, and checked and seen if, yeah, the title tag is there, if it is meaningful, it is still relevant, uh, similar to the old one or even better than the old one or worse. Uh, to take the appropriate action. Metascriptions, um, headings, the metadata is correct, right? But beyond that, uh, the, the, if, if, if, if there is any blockage of these URLs, and then of course do the same for the rest with the normal crawl again, and, and, and better understand also how they, this particular pages are internally linked, uh, in, in the new web. Organization to crawl again and check what I mentioned before, uh, titles and descriptions, uh, the, the canonicalization here, if they are self-canonicalized or canonicalized to other URLs, uh, why if so, right? To double-check if the, the, they have any noindex going on, why it's so, right? And check if any of those really top, top 2 pages, high traffic pages, are redirecting somehow to any of this. Why again? And if the titles are the correct ones, the descriptions are the correct ones, the headings are the correct one. Ideally, we should not only crawl and understand the configuration, maybe the configuration, the optimization of these new pages are still correct, are still relevant, but we should find a way. There should be a way to compare versus the old ones, right? Because maybe we see that the title is relevant, the description relevant, content— content is still there and it's relevant, it's not duplicated, it's nothing content. So what's going on? Uh, why they are not ranking as well? Well, maybe the old pages, these old 2 pages, uh, were even better, were even more meaningful, more relevant, more popular. They were, uh, better linked not only to external but also internally, right? So we should be able to compare. Ideally, here is when we should ask, and we should be able to ask and tell a client, look if there's any way to enable and check in a test environment, in a development environment that is closed, of course, and see if the old website was, was, even better than this one, right, to better understand the gap, maybe still existing gap, uh, the old versus the new one. Ideally, that, that should be a possibility. And, and, and, and then we could— we can, uh, double-check versus the old design in a very straightforward way because we can again crawl the old design and compare the 2 crawls. I love how Cybulb allows us to, to easily Uh, check, uh, the, the 2 crawls. Also with DeepCrawl, we have the, the possibility and modify the— check the gap between 2 different crawls, right? Uh, if it is not possible, if there is no access to this old version of the website somehow, uh, we can come and, and check, um, if, if it is something recent, right? Uh, Google Cache. So for example, here we can see how In the case of GameFAQs, it seems that somehow some of the URLs still are shown on their index. But then when we double-check, we can see that the actual cache is for their new URL version, right? So although it might seem that they are still there, the last snapshot is already showing already the, the, the new URL version. We can always go and try to work around here with the Internet Archive. We can go to January here and see how, how it looked back by, uh, back then, uh, the old version of the site. So yeah. So now this is it before the migration in January, right? So it looked like this. So we can see and we can compare all the old pages versus their new versions and actually compare and see how they used to look in the past and how they look now. And yeah, again, technical configuration, content relevance, how meaningful is the content, also the gap between the old location versus the new one, especially if it is a domain, across-domain type of migration. Because again, this, this plays an important role, especially with very competitive domains targeting very competitive queries. So for example, this old website, take a look at the amount of linking domains that it had, the number of linking domains, the, the, the number of backlinks, uh, and, and the influence, the authority of this, of this whole domain. And if We'll look at the new one. Again, it's also pretty good. There's not an important gap, but I'm pretty sure that there should be some important links, former links that are still pointing to the old location, especially if it is a very, very old one. If it has a domain that has a really good authority that has been here for a long time, and we can see how it was already there 5 years ago. So it is important to understand so that if there's a gap, an important link authority, link popularity gap between the former domain and the new one, maybe that is even the reason of why it is not performing as well for top terms, top queries, because the new domain, the new destination is not still as popular as the old one. And then if that's the reason, and of course there's away now, it has already happened, right? Is to better understand which are those top links that were pointing to the old domain, to the old location, all URLs, and, and how you can reclaim them, how you can claim them, how you can, you can recover those links, these links, so they can go to, to, to your new location, especially if you identify that they are going to pages that are not those that have been correctly redirected. Of course, the first step should be to fix the redirect if you can and make sure that the new location is, is as relevant, etc. But if somehow you cannot do it or somehow still like that you cannot recover as much, uh, the next step should be of course to try to reclaim and, and, and recover the, the, the popularity that was lost. And, and hopefully by doing all this Uh, we'll be able to recover and that lost, those lost rankings, that lost, uh, visibility, uh, traffic and, and, and being able to fix. I will say that by doing this step, we'll be, we'll be able to much do much better job to, to, to recover, to fix, uh, what wasn't correctly done right from the beginning, which should have been the ideal scenario, of course. And, and I hope that This was handy to you. Ideally, after this, I will say that is the right time to set SEO alerts in case they haven't been implemented before, right? To understand when the pages changes, when the pages are migrated, your redirects are placed, and nobody can tell, oh, I wasn't alerted about it, or didn't— I didn't know it. So, so there is Actually, uh, Crawling Mondays— this was the second Crawling Monday that I did, or the first one even. Uh, anyway, there, there is a previous episode of Crawling Monday that you can see here, uh, about that specific topic, uh, to be alerted, uh, with, uh, a few tools when anything on, on, on your website changes, including, uh, when a migration happens without even being alerted of it. Right, um, Little Warden, um, Content King, a couple of tools that you should definitely check out. But check out the video. Thank you very much for, uh, being here, for watching this, and I hope that you have enjoyed it. Uh, I look for my presentation about this topic, but of course not only about this specific scenario but a broader, from a broader perspective of a complex migrations from, from an SEO perspective and how to win them, how to take the opportunity to, to make the most out of them to improve actually your, your organic search rankings and traffic. And looking forward to share again with you in the next Crawling Mondays. Thank you very much. Bye-bye.