Noindex Tag: What It Is, How to Check It and How to Fix It

What does a noindex tag do?

Search engines handle a page in two steps. They crawl it, meaning they fetch the HTML, then decide whether to index it, meaning they store it so it can appear in results. Noindex acts on the second step only: the crawler may read the page but is asked not to list it.

People say noindex tag for two different things. Strictly, the noindex directive is the word noindex, and the robots meta tag in the page head, or the X-Robots-Tag HTTP response header, is just how that rule is delivered. For PDFs and images, which have no head, the header is the only option.

Google, Bing and other well behaved crawlers honor the rule, but it is a request and not a lock. It never stops visitors, it is not a security control, and it covers one URL only. The full list of valid values is in Google's robots meta tag rules.

Meta robots tag versus X-Robots-Tag

A meta tag belongs in the HTML head and works for web pages only, although Google also reads one placed in the body. Your server sets the header, and it works for any file type. Both accept the same values, and when they disagree Google applies the more restrictive one.

Crawling is not indexing

Googlebot may fetch a thank you page because other pages link to it, but with noindex it never enters the index. The catch is that Googlebot has to fetch the page to read the rule, which is why blocking the URL in robots.txt defeats it. Read how Google crawls and indexes pages for the wider picture.

How SEO Site Checker tests your noindex tag

Our scanner opens your URL in headless Chromium and waits for the load event, with a limit of 20 seconds. It then takes the rendered HTML, so a robots tag that JavaScript adds during page load counts, while one it removed by then does not.

The exact rule: the item fails if the first meta element named robots has the text noindex anywhere in its content. Everything else passes.

Each scan produces only passed or failed. With no warning state, an intended noindex on a login page shows red just like an accident, beside a How to Fix button that opens the steps further down this page. The item counts as one check in the SEO score, and the same result also feeds the Indexable line in the AI readiness panel without changing the SEO score.

For the noindex rule, both our report and the Meta Tag Analyzer look at the robots meta tag and ignore response headers. The analyzer shows the raw HTML your server sends, without running JavaScript, so the two can disagree about a page that changes its tags in the browser. If a page carries two robots tags, the report uses the first and the analyzer shows the last, so they can differ there too. For headers and crawler specific tags, use the DevTools and curl steps in the next section.

What turns the check red

All of these fail: content="noindex", content="noindex, nofollow", content="noindex, follow", content="NOINDEX" and content="max-snippet:-1, noindex". The tag can sit anywhere in the rendered document, body included, which matches Google.

What the check does not see

Three kinds of noindex pass. An X-Robots-Tag noindex header is invisible, because only the meta tag is read. A googlebot meta tag or bingbot meta tag aimed at one crawler is ignored. The none value, which means noindex plus nofollow, lacks the text noindex, so it passes as well.

Only the first robots tag is read, and a name written as Robots is skipped even though Google treats names and values as case insensitive. A pass therefore proves one tag is absent. It does not prove the page can be indexed.

Be careful with JavaScript that removes a noindex. Google says it may skip rendering when it meets noindex in the original HTML, so a script that deletes the tag may never run for Googlebot. Our scanner renders in a browser, so it can show green for a page Google still treats as noindex.

Run the free SEO report

How to check if a page has a noindex tag

Our report reads only the robots meta tag, and response headers never show up in view source, so confirm a suspected noindex another way too.

1. Inspect the rendered HTML. Open DevTools, go to the Elements panel and search for robots. Plain view source shows only what the server first sent.
2. Read the response headers. In the Network tab, click the page request and look for X-Robots-Tag. Or run curl as shown below, and retry with a normal GET if your server answers HEAD requests differently.
3. Ask Google. The URL Inspection tool in Search Console shows whether indexing is allowed and the HTML Googlebot received. The Page indexing report lists affected URLs as Excluded by noindex tag, while Google's help page says URL marked noindex.
4. Use a free tool. Run the report and open the Meta Tag Analyzer to read the robots value on any URL.

If the report passes but Search Console says noindex, suspect a header, a googlebot tag, or a script that removes the tag after load. If the analyzer and report disagree, suspect JavaScript or a second robots tag.

Read the X-Robots-Tag header from the command line

The -I flag requests headers alone, and grep keeps the one line you care about.

curl -sIL https://example.com/page/ | grep -i x-robots-tag

The -L flag follows redirects, so you read the final page and not an http to https redirect. A clean page prints nothing. A header rule prints a line such as x-robots-tag: noindex, and the fix then lives in your server configuration.

How to remove a noindex tag and get your page indexed

If the tag has to go, work through these in order. The first two usually solve it.

1. Find the source. It comes from a template, a per page SEO plugin field, a site wide CMS switch or a server rule that adds a header.
2. Remove it. Delete the tag or the noindex value. Indexing is the default, so no replacement tag is needed. A header rule must be deleted from the Apache or Nginx configuration.
3. Check robots.txt. If the URL is disallowed, Google never fetches the page and never sees that the tag is gone.
4. Support the page. Keep the URL in your XML sitemap, which the free XML Sitemap Generator can build, and link to it from other pages.
5. Clear caches and confirm the live HTML. Purge your page cache and CDN, then check in DevTools or with the curl command above that no robots noindex or X-Robots-Tag is left, because a stale copy would send Googlebot back to the old tag.
6. Request indexing. Use URL Inspection once the live page is clean. Google notes that a revisit can take months for less important pages, so expect no fixed date and do not resubmit repeatedly. Then run the report again and wait for the green item.

Still not indexed? Check URL Inspection for a remaining noindex, then review the canonical and content quality.

WordPress

Open Settings, then Reading, and look for the checkbox labelled Discourage search engines from indexing this site, a staging leftover that the WordPress Reading settings guide describes. Then check the per post robots or advanced settings in your SEO plugin, such as Yoast SEO or Rank Math, and finally your theme's header.php or a snippet plugin for a hard coded robots tag.

Shopify and Wix

On Shopify, search your theme.liquid file, the product and collection templates and any SEO app for noindex. A store still behind its storefront password cannot be indexed at all. On Wix, open the page's SEO settings and switch on the option that lets search engines index it.

Squarespace and Webflow

On Squarespace, check that Hide page from search results is off in the page settings. On Webflow, switching off the Sitemap indexing toggle in the page settings adds a noindex robots tag.

How to add a noindex tag to a page or file

This is the supported way to deindex a page on purpose, and Google applies it at the next recrawl. Write the name and value in lower case. Google treats them as case insensitive, but our report only matches the lower case name robots. Test the result with the steps in the check section above.

Meta tag examples

Place the tag inside the head, as a real tag. These lines cover all crawlers, Google only, and two rules combined with a comma.

<meta name="robots" content="noindex">
<meta name="googlebot" content="noindex">
<meta name="robots" content="noindex, nofollow">

An unavailable_after rule drops a page after a date, which suits expired offers.

<meta name="robots" content="unavailable_after: 2026-12-31">

The date can be in a widely used format such as ISO 8601, an invalid date makes the rule get ignored, and after the date Googlebot sharply reduces how often it crawls the URL. Values such as noarchive, nosnippet, noimageindex and indexifembedded control other things and are not covered by this check. For more on where meta tags go, see meta tags for beginners.

Apache and Nginx header examples

Files without a head need the header form, documented in the MDN X-Robots-Tag reference. Apache mod_headers must be enabled, and a FilesMatch block limits the rule to PDFs.

# Apache, in .htaccess or the virtual host
<FilesMatch "\.pdf$">
  Header set X-Robots-Tag "noindex"
</FilesMatch>

# Nginx, inside the server block
location ~* \.pdf$ {
  add_header X-Robots-Tag "noindex" always;
}

Reload the server, then test one file with the curl command.

When should you use a noindex tag?

• Keep it on utility and low value pages. Thank you pages, order confirmations, internal search results, login, account and cart pages, printer friendly duplicates, expired campaigns and thin filter or tag archives all belong out of results.
• Protect staging properly. A staging site can use noindex, but a password is safer, because the tag is easy to forget at launch.
• Remove it from pages that should rank. Product, category and key landing pages must never carry it.
• Pick the right tool for duplicates. Noindex removes the page from results entirely, whereas a canonical tag or a redirect passes the signals to the preferred URL, so for duplicate content those usually fit better. The canonical tag guide covers how.
• Audit often. After every launch, migration or plugin change, filter the Page indexing report for noindex and compare it with the pages you meant to exclude.

Noindex tag vs robots.txt disallow, nofollow and canonical

These four controls each answer a different question.

Nofollow concerns links, not listing, and a canonical tag is a hint while noindex is a directive. See the Disallow directive guide and the nofollow tag guide.

Why Disallow and noindex cancel each other

Take a page blocked in robots.txt and tagged noindex. Googlebot obeys the block, never reads the tag, and the URL can still appear as a bare listing with no description. Fix it in order: allow crawling, wait for Google to read the noindex, and only then decide whether to block again.

Also, a Noindex line inside robots.txt does nothing, because Google stopped honoring it in September 2019. Use the meta tag or the header.

Common noindex tag mistakes that hide good pages

Most common cause: staging settings pushed live after a launch or migration. The new site inherits the noindex that protected the test copy, so every page fails at once.

• Leftover CMS settings. The WordPress search visibility box, or a plugin default that noindexes archives and attachment pages, stays on after launch.
• A header nobody sees. View source looks clean and our report passes, yet the server sends a noindex header.
• A blocked URL. If a disallow rule covers the page, the robots.txt basics guide shows how to find it, and the free Robots.txt Generator helps you rebuild a clean file.
• Mixed signals. Noindexed URLs left in the XML sitemap ask Google to index the very pages you told it to drop.