Shopify Robots.txt: What You Can and Can't Control

If you've ever tried to edit your robots.txt file on Shopify and realized you can't find it anywhere in the theme editor, you're not alone. Shopify handles robots.txt differently than most platforms, and it confuses even experienced SEO professionals.

Here's the situation: Shopify generates your robots.txt automatically. For years, you couldn't touch it at all. Now, you have limited customization through a robots.txt.liquid template file — but there are still constraints on what you can and can't do.

Let me break down what Shopify's default robots.txt does, what you might want to change, and how to make those changes within Shopify's system.

What robots.txt Does

Your robots.txt file tells search engine crawlers (Googlebot, Bingbot, etc.) which parts of your site they're allowed to crawl and which parts they should skip.

Important distinction: robots.txt is a suggestion, not a wall. Well-behaved crawlers follow it. Malicious bots ignore it. And blocking a URL in robots.txt doesn't remove it from Google's index — it just prevents crawling. If Google finds the URL through links, it can still appear in search results with a "No information is available for this page" message.

For SEO purposes, robots.txt helps you:

• Prevent crawl budget waste on non-valuable pages
• Keep internal/admin pages out of search results (though noindex is better for this)
• Guide crawlers toward your most important content

Shopify's Default robots.txt

Shopify's auto-generated robots.txt blocks several paths by default:

• /admin (your Shopify admin panel)
• /cart (cart pages — no SEO value)
• /orders (customer order pages)
• /checkouts (checkout process)
• /account (customer account pages)
• Various internal and system paths
• /collections/*+* (filtered collection URLs with multiple parameters)
• /blogs/*/tagged/* (blog tag pages)
• /*/collections/* (nested collection paths)

It also includes a reference to your sitemap: Sitemap: https://yourstore.com/sitemap.xml

This default configuration is actually pretty good for most stores. Shopify blocks the pages that shouldn't be indexed while leaving product pages, collection pages, blog posts, and static pages open for crawling.

What You Might Want to Change

Even though the defaults are solid, some scenarios require customization:

Blocking specific collections or pages from crawling. Maybe you have a "Test Products" collection or a landing page from an old campaign that you don't want crawled.

Blocking certain URL parameters. Shopify URLs with sort parameters (?sort_by=best-selling) create duplicate content concerns. Some SEOs want to block these.

Blocking vendor pages. Shopify automatically creates /collections/vendors/[vendor-name] pages. If you don't want these crawled, you can block them.

Allowing previously blocked paths. Maybe you want tag pages crawled because you've built them into content hubs.

Adding additional sitemaps. If you have multiple sitemaps or a custom sitemap for a specific section.

How to Customize robots.txt on Shopify

Since 2021, Shopify allows customization through the robots.txt.liquid template file.

How to create it:

Go to your Shopify admin. Navigate to Online Store, then Themes, then Actions (or the three-dot menu), then Edit Code. In the Templates folder, look for robots.txt.liquid. If it doesn't exist, create a new template file named robots.txt.liquid.

The structure:

Your robots.txt.liquid file must start with the default Shopify robots.txt content (using Liquid tags) and then you can add your customizations below.

The file uses Liquid syntax. You include the default rules using the content_for_header tag specific to robots.txt, and then add any custom Disallow or Allow rules beneath it.

Important: Don't remove the default blocks. Shopify's defaults protect important system pages from being crawled. Add to them — don't replace them.

Common Customizations

Blocking a specific page: Add a Disallow rule for the path you want to block. If you have a test page at /pages/test-page, add a rule to prevent crawling of that specific path.

Blocking filtered collection URLs: Shopify creates URLs like /collections/shoes?filter.v.size=10. These filtered URLs can create massive duplicate content issues. Block the filter parameter paths.

Blocking search results pages: Your internal search at /search?q=keyword creates thin, duplicate pages. Block the /search path.

Adding a custom sitemap: If you've created an additional sitemap (through an app or custom development), add a Sitemap directive pointing to its URL.

What You Should NOT Block

Some merchants get overzealous with robots.txt and block things that hurt their SEO:

Don't block CSS and JavaScript files. Google needs to render your pages to understand them. Blocking CSS/JS prevents rendering and can hurt rankings. Shopify's defaults allow these — don't change that.

Don't block product images. Image search is a traffic source. Blocking /cdn/ or image paths prevents Google from indexing your product images.

Don't block collection pages. Collection pages are your category-level SEO assets. They rank for broad keywords. Keep them crawlable.

Don't block all parameter URLs blindly. Some parameters (like variant selection) lead to canonical URLs that Google handles correctly. Only block parameters that create genuine duplicate content problems.

Don't use robots.txt to deindex pages. If you want a page OUT of Google's index, use a noindex meta tag or remove it from your sitemap. Blocking via robots.txt prevents Google from even seeing the noindex tag, which means the page might stay indexed but without content — the worst of both worlds.

The Crawl Budget Question

"Crawl budget" is how many pages Googlebot will crawl on your site in a given period. For most Shopify stores (under 10,000 pages), crawl budget isn't a real concern. Google crawls small to medium sites comprehensively regardless of robots.txt.

Crawl budget matters for:

• Very large stores (50,000+ product pages)
• Sites with massive filtered URL combinations
• Sites with dynamic parameter URLs creating near-infinite crawl paths

If your store is large, robots.txt becomes important for directing crawl budget toward your most valuable pages (products, collections) and away from waste (filtered views, tag pages, search results).

Monitoring Your robots.txt

After making changes, verify them:

Google Search Console. Under Settings, then Crawl Stats, you can see how Googlebot is crawling your site. Under the URL Inspection tool, you can test whether specific URLs are blocked by robots.txt.

robots.txt Tester (now deprecated in new Search Console, but third-party tools exist). Test your rules against specific URLs to make sure you're blocking what you intend and not accidentally blocking important pages.

Crawl your own site. Use Screaming Frog or Sitebulb to crawl your Shopify store. These tools respect robots.txt by default, so any page they CAN'T crawl is one that Googlebot can't either. Check for unintentional blocks.

The Meta Robots Alternative

For page-level crawl and index control, meta robots tags are more precise than robots.txt.

noindex, follow — Don't index this page, but follow its links. Good for thin content pages that link to valuable pages.

noindex, nofollow — Don't index and don't follow links. Good for truly internal pages.

In Shopify, you can add noindex tags through:

• Theme code (Liquid template conditions)
• Apps like Yoast SEO for Shopify or Smart SEO
• Custom metafields (if your theme supports them)

When to use meta robots vs. robots.txt:

• Single specific page you want de-indexed? Use meta robots noindex.
• Entire URL pattern you want not crawled? Use robots.txt Disallow.
• Page with links you want crawled but page itself not indexed? Use meta robots noindex, follow (robots.txt would block the links too).

Shopify-Specific SEO Considerations

Beyond robots.txt, Shopify has some SEO quirks to be aware of:

Duplicate URL paths. Products are accessible at /products/item AND /collections/category/products/item. Shopify handles this with canonical tags (pointing to the /products/ version), which is correct. Don't try to block the collection-product path in robots.txt — the canonical handles it.

Pagination. Shopify creates paginated collection pages (?page=2, ?page=3). Google handles these with crawl logic, and Shopify adds appropriate rel=next/prev signals. Leave these crawlable.

Tag pages. Blog tag pages (/blogs/news/tagged/topic) are blocked by default. If you've invested in making these valuable content hubs, you might want to allow them. But if they're thin pages with just a list of posts, keep them blocked.

What To Do Right Now

Go to your Shopify admin. Check Online Store, Themes, Edit Code. Look for robots.txt.liquid in your templates. If it exists, read what's in it. If it doesn't exist, your store is using Shopify's defaults (which is fine for most stores).

Then go to Google Search Console. Use the URL Inspection tool to spot-check 5 important pages (your homepage, a top product, a collection page, a blog post). Confirm they're all crawlable and indexable.

If you find issues — pages blocked that shouldn't be, or pages indexed that shouldn't be — that's your immediate action item.

If you want a full technical SEO audit for your Shopify store — crawlability, indexation, site structure, and performance — book a call with our team. We'll identify every technical issue holding back your organic rankings.