WordPress robots.txt and sitemapRobots & Sitemap

robots.txt and sitemap setup guide for WordPress

Usually ready in 15 minutesRevisions includedQuality check
Packages
Payment protectionDetails
  • Revisions within the package are free.
  • If you do not respond within 3 days of delivery, the work is accepted automatically.

The money is reserved when you pay and released only once you accept the work.

robots.txt and sitemap setup guide for WordPress

Example result

Example for the Standard package

robots.txt and sitemap: an online clothing store

Client: an online clothing store with filters and site search. Task: stop search engines from crawling thousands of filter and cart pages, and point them to the sitemap.

User-agent: *
Disallow: /cart
Disallow: /checkout
Disallow: /account
Disallow: /search
Disallow: /*?sort=
Disallow: /*?color=
Disallow: /*&size=
Allow: /collections/

Sitemap: https://example.com/sitemap.xml

Sitemap plan:

FileWhat it listsPages
sitemap.xmlindex of the files below
sitemap-collections.xmlcategories64
sitemap-products-1.xmlproducts1–5,000
sitemap-products-2.xmlproducts5,001–8,300
sitemap-pages.xmlabout, delivery, returns, blog41

Why: filter pages repeat category content and waste crawling. Products out of stock for more than 90 days come out of the sitemap but keep their page.

The client and all details are made up.

About this service

WordPress gives you a virtual robots.txt and a sitemap from the core, and SEO plugins like Yoast, Rank Math or All in One SEO replace both with their own, so many sites end up with two sitemaps, tag and author archives in the index and the wp-admin rules copied from an old forum post. We look at what your site serves now and write a step-by-step setup guide for your plugin: which sitemap to keep, which post types and taxonomies to include, a clean robots.txt and where to change each setting.

What is included

  • What you getA setup guide for your WordPress site: the current robots.txt and sitemap findings, a ready robots.txt, the plugin settings to change step by step and what to submit in Search Console.
  • What it serves nowYour live robots.txt and sitemaps checked
  • One sitemapWhich to keep and which to switch off
  • Content in the sitemapPost types, taxonomies and archives
  • Clean robots.txtNo outdated wp-includes blocks
  • Plugin stepsWhere each setting lives

How it works

  1. 1
    CheckWe look at your live files
  2. 2
    GuideWe write steps for your plugin
  3. 3
    SubmitYou apply and submit the sitemap

What we need from you

A link to the site, the SEO plugin you use (or plan to), the post types and taxonomies you have, such as products, portfolio or events, and which archives you want in search.

We will ask you:

  • Which SEO plugin does the site use?
  • Which post types and taxonomies do you have?
  • Should tag, author or date archives appear in search?

Compare packages

PackagesBasic$10Standard$20Premium$39
What is includedBlog or small siteBusiness siteWooCommerce or multilingual
Deadline1 day2 days3 days
Revisions122
Post types and taxonomiesup to 4up to 8up to 15
Plugin setup stepsYesYesYes
Ready robots.txtYesYesYes
Order BasicOrder StandardOrder Premium

Payment protection

You pay when you order, but the money is released only for accepted work.

  • Fixed priceThe package price does not change after you pay.
  • The money is reservedWhen you pay, the amount is only reserved. It is released to the service once you accept the work.
  • Checked before deliveryEvery result goes through a quality check before it reaches you, and you see it on the order page before accepting it.
  • Free revisionsAsk for changes within your brief as many times as the package allows.
  • Acceptance after 3 daysIf within 3 days of delivery you neither accept the work nor ask for changes, it is accepted automatically. If you write in the order chat after delivery, the order waits; once we answer, we may start the 3 days again and will say so in the chat.
  • Redo or a full refundIf the result does not match your brief and the revisions did not fix it, or the work takes more than twice its deadline, write in the order chat: we will redo the work or refund the payment in full.
How it works →

Questions and answers

My site has two sitemaps. Is that bad?

It confuses things and often lists pages twice. The guide says which one to keep, usually the plugin's, and how to switch off the other.

Should I block wp-content in robots.txt?

No. Search engines need your theme files and images to render pages. The ready file keeps them open.

Will you change the plugin settings?

No, we do not log in to your WordPress. You follow the steps in the guide, which take about half an hour.

Did not find your answer?

Packages