Jason BerkowitzFounder, SEO Director, Break The Web · Making SEO Less Annoying for DTC Marketers • SEO Director & Founder of Break The Web

How to Crawl Shopify Stores using Custom Signatures

1 Dec 2025 · 1 min

Watch on YouTube

1:27

Shopify stores can bypass rate-limit errors during SEO crawls by generating custom crawler signatures in Online Store Preferences. The store owner creates a signature, names it, sets a 90-day expiration, then shares the credentials with their SEO tool so it crawls without hitting 429 errors.

Chapters

If your brand is on Shopify and you do any type of SEO activities or SEO execution, and you've tried crawling the website, or maybe you work with a freelancer or an agency that has tried crawling the website, they run into various different issues like error codes 429, in which Shopify blocks crawling from crawlers that are trying to browse different pages at a very high rate, going way too fast. Thankfully, Shopify has made this really, really easy in which you can create custom signatures and send these signatures to your SEO specialists so that they can start crawling the website with ease using tools like Screaming Frog, Sitebulb. All you have to do is when you're in your main home dashboard, click Online Store, click Preferences, scroll all the way down, you'll see the section for crawler access. Click Create Signature, name this whatever you want— Break the Web SEO Agency. Well, you got to spell SEO right to begin with. And then you have your primary domain In which they can crawl, and they might crawl quite frequently. So it's best to have the expiration last 90 days and click create signature. What we have here now is new signature fields that if you copy and hit the clipboard button for each output, you can paste this to your SEO specialist and they can crawl your website with the various tools that they may have internally with ease. So go ahead, send this to your SEO agency, and thank you for making crawling super, super easy for SEO companies.