Robots Meta Tag Builder
Toggle directives, copy the tag. The builder covers the common robots rules — index/noindex, follow/nofollow, noarchive, nosnippet, noimageindex, max-snippet, max-image-preview, max-video-preview, unavailable_after — and emits a tag with only the rules you changed from the default, plus the same rules as an X-Robots-Tag header. (If you change nothing, it shows index, follow and tells you the tag isn't needed.) Rarer rules such as notranslate, indexifembedded and none aren't included.
Why Page-Level Beats Site-Level Crawling Control
The robots meta tag is the precise instrument; the robots.txt file is the blunt one. robots.txt tells crawlers what URLs to fetch; the meta tag tells them what to do with the page once they have fetched it (index it? follow its links? show a snippet? store an image preview?). For sites where individual pages need different crawling behavior — a draft page that should be crawled but not indexed, a paginated archive where deep pages should be followed but not surfaced — the meta tag is the only correct control surface. robots.txt simply cannot express that granularity.
How the Builder Handles Conflicts
Some combinations deserve a second look, so the builder warns without blocking them. noindex, nofollow keeps the page out of results and also tells Google not to follow its links; use noindex, follow if pages linked from it should still be found through it. noarchive no longer does anything in Google, which retired cached links. Invalid values (a non-numeric max-snippet, an unreadable date) are left out of the tag until you fix them, so what you copy is always valid.
Frequently Asked Questions
Where does the robots meta tag go?+
In the head of the page's HTML as the server sends it. Google can read a robots meta tag that JavaScript adds while the page renders, but it may not render a page at all, for example when the served HTML already says noindex, so JavaScript cannot be relied on to add or remove one. For PDFs and other non-HTML files, send the same rules as an X-Robots-Tag HTTP header; the builder shows that line too.
What is the difference between noindex and disallow in robots.txt?+
robots.txt disallow tells crawlers not to fetch the URL at all. Noindex tells them they can fetch it but not include it in the index. If you both disallow and noindex, Google may never see the noindex directive (because it is not fetching the page), defeating the purpose.
Should I still use noarchive?+
Only for search engines other than Google. Google retired its cached-page links, and its documentation now says Google Search no longer uses noarchive; leaving the rule in place does no harm, and other search engines and services may still honour it.
Does nofollow on a page affect external links to it?+
No — nofollow in the meta tag affects how your page's outbound links are treated, not how inbound links to your page are weighted. It is a per-page outbound link policy.
What is max-snippet?+
max-snippet:N caps the length of the text snippet Google shows in the SERP at N characters. Common values: 0 (no snippet — same as nosnippet), -1 (unlimited), or a specific cap like 160. Most sites do not set this; the default snippet length is fine.
Can I target specific crawlers?+
Yes — replace 'robots' with the user agent: a meta tag with name='googlebot' and content='noindex' targets only Google's crawler. Bingbot, Yandex, and others have their own user-agent strings. The builder defaults to robots which applies to all.
What does unavailable_after do?+
It asks Google to stop showing the page in search results after a date and time, given in a widely used format such as ISO 8601 (2026-12-31) or RFC 822. Useful for event pages or time-limited offers you want indexed only while they run. The builder leaves the rule out until the date is one it can read.
Does the meta tag override robots.txt?+
Neither overrides — they answer different questions. robots.txt controls fetching; the meta tag controls indexing of fetched pages. The combination of both is the full picture.
Built by Derek Giordano · Part of Ultimate Design Tools
Privacy Policy · Terms of Service