Technical SEO

Noindex Removed, but Search Console Still Excludes the Page

Compare the last indexed version with the live response, locate hidden noindex headers, and record a fix without promising indexing.

Published Sep 28, 20264 min readBy RankCrab Team

Removing a noindex tag changes your website immediately after publication, but it does not rewrite Google's last crawl. First establish whether the exclusion describes an old response or a directive still served today. The answer determines whether to change the site again.

Keep three pieces of evidence separate

Use this worksheet for one exact production URL. These entries are illustrative, not a real Search Console account or a measured recovery.

EvidenceExample observationWhat it establishes
Indexed inspectionCrawled September 20; indexing disallowedGoogle saw noindex on that crawl
Current responseSeptember 28; 200; no noindex in HTML or headersThis particular response no longer declares noindex
Live URL testFetch successful; indexing allowedGoogle's live test can access the current page

A successful live test is not proof that the page is already indexed. Conversely, an old exclusion is not proof that the current deployment failed. Google's Page indexing documentation recommends testing the live URL when diagnosing noindex.

Check the response that is actually being served

  1. Copy the affected URL from the report, preserving its hostname, path and query string.
  2. Open it while logged out. Record redirects and the final URL. A CMS preview can differ from the public page.
  3. Inspect the document response headers and HTML. Look for both robots and googlebot meta directives and X-Robots-Tag headers.
  4. Repeat on the canonical target if the URL points to another page.
  5. Run URL Inspection's live test and record its date separately from the indexed inspection.

This command saves a GET response; it does not authenticate as Googlebot:

curl -sS -L -D response-headers.txt -o response-body.html 'https://example.com/article'

-L follows redirects, so the header file can contain several responses. Read the final response and any redirect destinations. Do not assume a successful test from your computer proves that Google's network can fetch it.

The invisible header case

Suppose the HTML contains index, follow, but the server sends:

X-Robots-Tag: noindex

The page still has an exclusion directive. An HTML-only search missed it. Check the host/CDN configuration, an SEO plugin, or a staging header rule carried into production. Remove the rule only from URLs intended for search, publish, then capture the response again. Google's robots directive specification explains the interaction of page and response-header directives.

Choose the next action

Current findingNext actionAvoid
Live test still finds noindexFind the remaining source and republishRepeated indexing requests before fixing it
Current response is clear; Google cannot fetchInvestigate robots/access/status errorsAssuming the old noindex is the only issue
Live test is clear; report has an older crawlRequest indexing where appropriate and record the dateInventing a guaranteed processing deadline
URL redirects or canonicalizes elsewhereInspect the destination and intended URLForcing every duplicate into the index

Use the noindex directive tool to interpret copied directives; it is not a live Googlebot test. If the page is indexed despite a crawl block, use the separate robots-blocked workflow.

Completion checklist

  • The intended public URL and canonical target are recorded.
  • No unwanted directive remains in either HTML or response headers.
  • The live inspection result is saved with a date.
  • The original report date and any request date are preserved.
  • Later indexing or traffic observations are recorded without attributing them automatically to this change.

If noindex is gone but the page remains unindexed for a different reason, continue with crawled, currently not indexed diagnosis. RankCrab's paid workspace is prelaunch; launch notifications are optional.

Check the evidence before you change the site.

Review sitemap changes, redirect mappings and page directives with free browser tools.