Shopify Robots.txt: What the Default Blocks and What to Change

Shopify auto-generates a robots.txt file for every store. Until 2021, it was locked — you got what you got. Now you can customize it through the robots.txt.liquid template, but most merchants don't know what the default contains or which parts are worth changing.

Here's a precise breakdown of what Shopify ships, what makes sense, what's limiting, and how to override it safely.

The Default Shopify robots.txt

Shopify's default output looks approximately like this (exact formatting may vary by plan):

User-agent: *
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts/
Disallow: /checkout
Disallow: /carts
Disallow: /account
Disallow: /collections/*sort_by*
Disallow: /*/collections/*sort_by*
Disallow: /collections/*+*
Disallow: /collections/*%2B*
Disallow: /collections/*%2b*
Disallow: /*/collections/*+*
Disallow: /*/collections/*%2B*
Disallow: /*/collections/*%2b*
Disallow: /blogs/*+*
Disallow: /blogs/*%2B*
Disallow: /blogs/*%2b*
Disallow: /tools/guided_selling
Disallow: /search
Disallow: /apple-app-site-association
Sitemap: https://your-store.myshopify.com/sitemap.xml

Some entries also appear with explicit Allow: rules for CSS and JavaScript assets needed for rendering.

What's Smart in the Default

/admin, /checkout, /cart, /orders, /account — These are correct. There's no search value in letting Googlebot crawl your checkout flow, account pages, or order history. Blocking them protects crawl budget and avoids indexing session-dependent pages.

/search — Also correct. Internal search result pages are near-empty when Googlebot hits them without a real query context. Indexing them creates soft 404 candidates and thin-content noise.

/collections/*sort_by* — This one is smart by default. Sort-parameter URLs (?sort_by=price-ascending) create duplicate versions of collection pages that differ only in display order. Blocking them prevents index bloat.

The %2B / %2b / + variants — These encode the + character in URL parameters, which some apps use to combine filter facets. Blocking them prevents indexed filter combinations that have no standalone search value.

What's Limiting (and Sometimes Needs Changing)

Faceted navigation blocking

The sort_by rule is right, but Shopify's default doesn't block all faceted URLs — it blocks specific parameter patterns. If your store uses Shopify's native filter system, filtered collection URLs (e.g., /collections/shoes?filter.p.m.product_type=sneakers) may or may not be blocked depending on your theme and app setup.

For stores with heavy faceting — many product attributes, many filter combinations — you'll want to audit which parameter-filtered URLs are getting indexed and decide intentionally which (if any) should be. See faceted navigation SEO for the full decision framework.

The /collections/ block is absent

Notice what's not blocked: /collections/ itself. Collection pages are indexable by default, which is usually correct — collection pages are your category pages and often your highest-converting organic landing pages. But if you have auto-generated collection pages with thin content (vendor collections, automated tag collections), those can become soft 404 candidates.

How the robots.txt.liquid Template Works

Since Shopify's 2021 update, you can edit your robots.txt through a Liquid template. To access it:

1. Go to Online Store → Themes → Edit code
2. Under Templates, look for robots.txt.liquid — if it doesn't exist, create a new template and name it robots.txt
3. Add your customizations inside the template

The template uses Liquid. Shopify provides a robots object you can iterate over to render the default rules, then append your own:

{% assign default_groups = robots.default_groups %}
{% for group in default_groups %}
  User-agent: {{ group.user_agent }}
  {% for rule in group.rules %}
  {{ rule.directive }}: {{ rule.value }}
  {% endfor %}
  {% if group.sitemap %}
  Sitemap: {{ group.sitemap }}
  {% endif %}

{% endfor %}

# Custom rules below
User-agent: *
Disallow: /pages/wholesale-portal

This approach renders all of Shopify's defaults first, then appends your additions. Do not skip the default rendering unless you know exactly why — you'll lose the built-in protections.

Overriding Specific Rules

To remove a default disallow rule, you need to render the default groups selectively, filtering out the rule you want to drop:

{% for group in robots.default_groups %}
  User-agent: {{ group.user_agent }}
  {% for rule in group.rules %}
    {% unless rule.value contains '/search' %}
  {{ rule.directive }}: {{ rule.value }}
    {% endunless %}
  {% endfor %}

{% endfor %}

This is fragile — if Shopify changes the rule format, your filter may break. A safer approach is to render defaults as-is and then add explicit Allow: rules for paths you want to open up:

{% for group in robots.default_groups %}
  ...default rendering...
{% endfor %}

User-agent: *
Allow: /search/

An Allow: rule takes precedence over a Disallow: rule for the same path in most crawlers, including Googlebot.

Common Bad Ideas

Blocking /collections/ — Some guides recommend this to save crawl budget. It removes your category pages from Google's index, which is almost always wrong. Collection pages are your commercial intent pages.

Blocking /products/ — Same problem, worse outcome. You'd be removing your product pages from search.

Disabling robots.txt entirely — Setting Disallow: / for all user agents blocks every crawler. Some merchants do this accidentally when copying configs from staging environments.

Adding Crawl-delay: — Shopify's infrastructure handles throttling. A crawl-delay directive in your robots.txt may slow down legitimate Googlebot crawls without providing any benefit.

Validating Your robots.txt

After any change to the template, wait a few minutes for Shopify to propagate, then:

1. Fetch https://your-store.com/robots.txt directly in a browser to verify the output
2. Use Google Search Console's robots.txt tester to test specific URLs against your rules
3. Check that your sitemap URL appears correctly at the bottom of the file

The robots.txt generator lets you build and validate rules before editing the Liquid template — useful for testing pattern logic without touching production.

How This Fits Into Shopify Technical SEO

The robots.txt file is one piece of a larger technical SEO picture for Shopify stores. Once crawl rules are correct, the next common issues are canonical tag handling (Shopify generates canonical tags automatically but sometimes points product page canonicals to the wrong collection), and faceted navigation indexing.

If you're seeing "Indexed, though blocked by robots.txt" warnings in Search Console for your Shopify store, the robots.txt fix guide walks through the exact decision tree — including the noindex-via-robots-txt trap that catches a lot of Shopify merchants.