SEOPayment protectionDetails
- Revisions within the package are free.
- If you do not respond within 3 days of delivery, the work is accepted automatically.
The money is reserved when you pay and released only once you accept the work.

Example result
robots.txt and sitemap: an online clothing store
Client: an online clothing store with filters and site search. Task: stop search engines from crawling thousands of filter and cart pages, and point them to the sitemap.
User-agent: *
Disallow: /cart
Disallow: /checkout
Disallow: /account
Disallow: /search
Disallow: /*?sort=
Disallow: /*?color=
Disallow: /*&size=
Allow: /collections/
Sitemap: https://example.com/sitemap.xml
Sitemap plan:
| File | What it lists | Pages |
|---|---|---|
| sitemap.xml | index of the files below | |
| sitemap-collections.xml | categories | 64 |
| sitemap-products-1.xml | products | 1–5,000 |
| sitemap-products-2.xml | products | 5,001–8,300 |
| sitemap-pages.xml | about, delivery, returns, blog | 41 |
Why: filter pages repeat category content and waste crawling. Products out of stock for more than 90 days come out of the sitemap but keep their page.
The client and all details are made up.
About this service
Online stores generate endless crawlable URLs: filter combinations, sort orders, internal search results, cart, checkout, account and compare pages, and each platform names them differently. Blocking too little wastes crawl on junk pages, blocking too much hides products or the CSS that renders them. We read your current robots.txt and the URL patterns on your store and write a new file with commented rules for each pattern, plus notes on what should be handled with noindex or canonical instead.
What is included
- What you getA ready robots.txt for your store with a commented rule for each URL pattern, a list of example URLs that the rules block and allow, and notes on patterns that need noindex or canonical rather than a block.
- URL patternsFilters, sorting, search, cart and account
- Commented fileEvery rule explained in the file
- Keep openProducts, CSS, JS and chosen filter pages
- Block or noindexWhen a block is the wrong tool
- Test examplesURLs to check in a robots.txt tester
How it works
- 1PatternsYou send the store and URL examples
- 2RulesWe write and comment the file
- 3CheckYou get the file and test URLs
What we need from you
A link to the store, your platform, examples of filter, sort and search URLs from your store and the filter pages you want to keep indexable, such as brand or colour pages.
We will ask you:
- Which platform runs the store?
- Which filter pages should stay in search?
- Paste a few filter, sort and search URLs from your store
Compare packages
| Packages | Basic$10 | Standard$20 | Premium$39 |
|---|---|---|---|
| What is included | Core rules | Full store rules | Multi-store rules |
| Deadline | 1 day | 2 days | 4 days |
| Revisions | 1 | 2 | 2 |
| URL patterns covered | up to 8 | up to 20 | up to 40 |
| Commented robots.txt | Yes | Yes | Yes |
| Test URL list | Yes | Yes | Yes |
| Order Basic | Order Standard | Order Premium |
Payment protection
You pay when you order, but the money is released only for accepted work.
- Fixed priceThe package price does not change after you pay.
- The money is reservedWhen you pay, the amount is only reserved. It is released to the service once you accept the work.
- Checked before deliveryEvery result goes through a quality check before it reaches you, and you see it on the order page before accepting it.
- Free revisionsAsk for changes within your brief as many times as the package allows.
- Acceptance after 3 daysIf within 3 days of delivery you neither accept the work nor ask for changes, it is accepted automatically. If you write in the order chat after delivery, the order waits; once we answer, we may start the 3 days again and will say so in the chat.
- Redo or a full refundIf the result does not match your brief and the revisions did not fix it, or the work takes more than twice its deadline, write in the order chat: we will redo the work or refund the payment in full.
Questions and answers
Can I edit robots.txt on Shopify?
Yes, through the robots.txt.liquid template. We write the rules in that form if your store is on Shopify.
Does blocking a page remove it from Google?
No. A blocked URL can still appear in results if other pages link to it. Pages that must disappear need noindex and must stay crawlable until they drop out.
Will you upload the file?
No, we do not log in to your store or server. You or your developer replace the file and test it.
Did not find your answer?


