How this tool works
Combines selected directives into a meta name=robots element and escapes the result as HTML.
How to use it
- Choose only directives that match your indexing intent.
- Add the tag to the page head.
- Verify the live response and inspect the page in Search Console.
Limits to keep in mind
A crawler must access the page to read a meta noindex directive. This tool cannot inspect HTTP headers, robots.txt, or server response behavior.
Frequently asked questions
Can robots.txt replace noindex?
No. Robots.txt controls crawling; a crawler blocked from the page cannot read its meta noindex directive.
Will this remove a URL instantly?
No. Recrawling and processing take time, and this tool does not submit removal requests.
What is the difference between noindex and nofollow?
Noindex asks search engines not to show the page in results. Nofollow asks them not to follow the links on it. They are separate and can be combined.
Can I use meta robots on PDFs or images?
No, meta tags only work in HTML. For other file types, use an X-Robots-Tag HTTP header set on your server.
What the robots meta tag controls
The robots meta tag is an HTML element in the head of a page that tells search engines how to handle that specific page. Unlike robots.txt, which controls whether a crawler may fetch a URL, the meta robots tag controls indexing and how the page may appear in results, after the page has been fetched.
The Meta Robots Tag Generator combines the directives you choose into a single meta element with the name set to robots and escapes the result as valid HTML. It does not edit your site and it does not check how your pages are currently configured. It builds the tag text, and you decide where to place it. The main benefit is avoiding typing mistakes in the content value, such as a misspelled directive that search engines would not understand.
Indexing and link directives
The two core pairs are index and noindex, and follow and nofollow. Noindex asks search engines not to include the page in their results. Follow and nofollow control whether the links on the page may be followed. By default, a page without any robots meta tag is treated as index and follow, so you do not need to add them to allow the usual behavior.
Use noindex for pages that are useful to visitors but have no reason to appear in search, for example thank-you pages, internal search results, filtered duplicates, or account areas. Use it with care, because a noindex page can drop out of results completely. Nofollow on a whole page is rarely needed. For individual links you do not want to endorse, it is usually better to use the rel attribute on that link instead.
Preview and availability controls
Beyond indexing, several directives control how a page can be shown. Nosnippet prevents a text snippet from being shown. The max-snippet value sets a limit on the number of characters in a text snippet, max-image-preview controls the size of image previews, and max-video-preview limits video previews. An unavailable_after value asks that a page stop appearing after a given date.
These directives affect how your result looks and can reduce click-through if they are too restrictive. Setting nosnippet on an article page, for example, can leave a result with only a title. Use these controls when there is a clear reason, such as licensed content or time-limited offers, and leave them out otherwise. Choose only directives that match what you intend, because every extra directive is one more thing to maintain.
Crawl access and conflicting signals
A crawler has to be able to fetch a page to see its meta tags. If the same page is blocked in robots.txt, the crawler cannot read the noindex tag, and the URL may remain in the index if it is linked from elsewhere. To remove a page from search with noindex, allow crawling and let the crawler see the tag.
Watch for other conflicting signals too. A page with a noindex tag should not also appear in your XML sitemap, and it should not be given a canonical pointing to a different page, because the signals disagree. If you need a rule for non-HTML files such as PDFs or images, a meta tag cannot be used. Send an X-Robots-Tag header from the server instead, which supports the same directives.
Targeting specific crawlers
The generic robots name applies to all search engines that support the tag. You can address one crawler by using its own name, such as googlebot, in place of robots. When both are present, the more restrictive combination is generally applied for that crawler, so keep rules simple to avoid surprises.
This generator focuses on the general robots tag, which covers the vast majority of use cases. If you need different instructions for different crawlers, write each tag separately and check the documentation of each search engine for the names and directives it accepts. Not every search engine supports every directive, and supported options can change, so review official documentation when you rely on a rarely used value.
How to use the tool
Start by deciding what you want for the page: appear in results or stay out, links followed or not, and any preview limits. Select only the directives that match that decision. Review the generated tag and read it back as a sentence, for example "do not index this page but follow its links".
Paste the tag into the head section of the page, or into the field your CMS or SEO plugin provides for robots settings. Do not add a second robots tag if your plugin already outputs one for the page. Publish the change, then open the live page source to confirm the tag is present and appears only once. It can take a while for search engines to recrawl the page and apply the change.
After publishing: verify and monitor
Verification matters, because a wrongly placed noindex can remove important pages from search. Use the URL Inspection tool in Google Search Console to check that the live page shows the expected indexing status, and look at the page indexing report for pages excluded by a noindex tag so you can confirm each one is intentional.
Be especially careful after site launches, redesigns and theme changes, since a staging noindex setting is a classic cause of lost traffic when it is carried to the live site. The tool cannot inspect your headers, your robots.txt file, or how your server responds, so those checks are yours. Keep a record of which pages carry noindex and why, and review the list periodically.
More SEO Tools · Browse the tools catalogue