What does noindex mean? How it works, when to use it and how to find it
noindex keeps a page out of search results. How it works, how it differs from a robots.txt block, when it's the right tool — and how to catch the accidental one.
noindex is an instruction to search engines: you may visit this page, but don't show it in search results. It's one of the most useful controls you have over what appears in search — and, left on by accident, one of the most damaging, because nothing on the page looks wrong to visitors.
Two ways to set noindex
The robots meta tag
In the page's <head>: <meta name="robots" content="noindex">. The name robots applies to all search engines; you can target one with its own name, such as googlebot. Directives can be combined: noindex, nofollow also asks crawlers not to follow the page's links, and none means the same as noindex, nofollow.
The X-Robots-Tag header
The same directives can be sent as an HTTP response header: X-Robots-Tag: noindex. This is the only option for non-HTML files such as PDFs, and it's easy to set site-wide in server configuration — which is also how it gets left on by mistake. You won't see it in the page source; check the response headers in developer tools or with curl -I.
noindex is not the same as a robots.txt block
This is the most common misunderstanding. Disallow in robots.txt stops crawlers from fetching a page; noindex stops them from indexing it. For a crawler to see a noindex, it has to fetch the page — so if you block a URL in robots.txt, the crawler never sees the noindex. The URL can even appear in results without a description, if other sites link to it.
So: to keep a page out of search, allow crawling and use noindex. Google stopped supporting noindex rules inside robots.txt in 2019. Our guide to robots.txt and sitemap.xml covers what robots.txt is actually for.
When noindex is the right tool
- Thank-you, confirmation and account pages that have no value in search.
- Internal site-search results pages.
- Thin tag, archive or filter pages you don't want competing with your main pages.
- Staging and preview sites — ideally protected with a password as well, since
noindexdoesn't stop people visiting. - Campaign landing pages that duplicate your main content.
noindex isn't the right tool for duplicates you want consolidated into one URL — use a canonical or a redirect for those, as explained in how to fix canonical problems. And don't combine noindex with a canonical pointing to another page: one says "drop this page", the other says "this page is a copy of that one".
How noindex gets left on by accident
- A staging site's setting copied to production during launch.
- WordPress's "Discourage search engines from indexing this site" box, under Settings → Reading, left ticked.
- An SEO plugin default that sets whole content types, such as products or categories, to noindex.
- An
X-Robots-Tagheader added in the server or CDN configuration for staging and deployed everywhere. - A page template that adds
noindexwhen a field is empty.
Check whether a page is indexable
Rudra's free SEO checker looks for noindex in the robots meta tag and the X-Robots-Tag header, checks robots.txt, and sums it up as indexable or not with the reason.
How to check a page
Run the page through Rudra's free SEO checker. It looks for noindex and nofollow (or none) in the robots meta tag and the X-Robots-Tag header, checks whether robots.txt blocks the page, and notes an HTTP redirect or error status — then shows a single "indexable" verdict with the reason. It also flags a noindex page whose canonical points elsewhere as a conflict, and notes snippet-limiting directives such as nosnippet that don't block indexing but change how the result looks. A noindex costs 30 points in the SEO score, because it removes the page from search entirely.
For a whole site, a Rudra site scan counts indexable and noindex pages and flags noindex pages listed in your sitemap. In Google Search Console, the Page indexing report lists pages "Excluded by 'noindex' tag", and URL Inspection shows whether a specific URL can be indexed.
Removing a noindex
- Find where it comes from: the page source (meta tag), response headers (
X-Robots-Tag), CMS settings, SEO plugin settings or server configuration. - Remove it at the source, and clear any page or CDN cache.
- Re-check the live page to confirm both the tag and the header are gone.
- Make sure the page isn't blocked in robots.txt and is in your sitemap.
- In Search Console, use URL Inspection and request indexing for important pages. Recrawling can still take days or weeks.
Frequently asked questions
Does noindex remove a page from Google immediately?
No. Google has to recrawl the page to see the directive. For urgent removals, Search Console's Removals tool hides a URL temporarily while the noindex takes effect.
Do noindex pages still pass link signals?
Crawlers can still follow links on a noindex page unless you add nofollow. Over a long time, search engines may crawl a noindexed page less often, so don't rely on noindex pages as your main route to important content.
Should noindex pages be in my sitemap?
No. A sitemap should list pages you want indexed. Listing noindex pages sends contradictory signals.
Is noindex the same as nofollow?
No. noindex keeps the page out of results; nofollow asks crawlers not to follow its links. They can be used separately or together.