Google Indexing in 2026: Fix & Keep Pages Indexed

Google processes an estimated 400 billion pages per day through its crawl pipeline — yet deliberately indexes only a fraction of the web (Google Search Central). Since 2022, Google has made selection more aggressive: its systems now weigh the "cost" of indexing against the likelihood a page will satisfy user queries. The result is that pages with thin content, poor signals, or structural problems are excluded — even when they are crawlable.

For site owners in 2026, the challenge is not getting Googlebot to visit a URL. It is passing the quality and relevance bar that determines whether a visited URL gets kept in the index.

1How Google Indexing Works in 2026

Indexing is a three-stage pipeline. Each stage can silently drop a URL.

Stage 1: Discovery

Googlebot discovers URLs through:

• Sitemaps submitted in Google Search Console (Google Search Central)
• Internal links crawled from already-indexed pages
• External links from other indexed sites pointing to yours
• URL inspection requests submitted manually via Search Console or the Indexing API

Discovery adds a URL to the crawl queue. It does not guarantee a visit — Googlebot prioritises the queue based on crawl budget signals, and low-authority or slow sites may wait days or weeks before a queued URL is actually fetched.

Stage 2: Crawling

When Googlebot fetches a URL, it downloads the HTML (and renders JavaScript for pages requiring it). At this point Googlebot reads:

• robots.txt directives (pre-fetch, not during the fetch itself)
• <meta name="robots"> tags
• X-Robots-Tag HTTP headers
• Canonical tags (<link rel="canonical">)
• HTTP status codes

A page blocked by robots.txt, noindex, or returning a non-200 status code (including soft 404s) will not proceed to indexing, even if crawled. See Google's robots.txt documentation for full rule syntax.

Stage 3: Indexing Decision

After crawling and rendering, Google evaluates whether the page is worth storing. This is where most invisibility problems originate. Google's systems ask:

1. Is this page unique? Duplicate or near-duplicate content is consolidated under the canonical URL.
2. Does this page provide value beyond what is already indexed? Thin pages, doorway pages, and over-optimised pages targeting the same intent as existing content are frequently excluded.
3. Is this page trustworthy? Signals including E-E-A-T, external links, and site-level authority all factor into the evaluation (Google Search Quality Evaluator Guidelines).

Google does not publish the exact threshold, but its guidance consistently points to content that "meaningfully serves users" (Google Search Central: Core updates).

For a deeper look at the crawling side of this pipeline, see Crawling.

---

2Reading Google Search Console Index Coverage

Search Console → Indexing → Pages gives you the definitive view of how Google sees every URL on your site. Understanding each status is essential before diagnosing a problem.

Index Coverage Status Reasons

Status	What It Means	Primary Cause	Priority
Indexed	URL is in the index and eligible to rank	—	Monitor
Crawled — currently not indexed	Googlebot visited the page but decided not to index it	Content quality, thin content, duplication	High
Discovered — currently not indexed	URL is known but Googlebot has not yet crawled it	Crawl budget exhausted, low internal link equity, slow site	High
Duplicate without user-selected canonical	Multiple URLs with same content; Google chose its own canonical	Missing or inconsistent canonical tags	Medium
Duplicate, submitted URL not selected as canonical	You submitted a URL in a sitemap but Google chose a different canonical	Canonical mismatch between sitemap and tags	Medium
Page with redirect	URL redirects to another URL	Expected for redirected pages	Low (expected)
Soft 404	Page returns 200 but Google treats it as empty/unhelpful	No-results search pages, thin user-generated content, empty category pages	High
Not found (404)	Page returns a 404 status	Deleted content, broken URL	Low (expected if intentional)
Blocked by robots.txt	robots.txt prevents crawling	Inadvertent or outdated robots.txt rules	Critical if unintentional
Excluded by 'noindex' tag	<meta name="robots" content="noindex"> or X-Robots-Tag present	Intentional or misconfigured noindex	Critical if unintentional
Alternate page with proper canonical tag	Non-canonical version of a page; canonical points elsewhere	Correct canonical implementation	Low (expected)
URL is unknown to Google	URL never discovered	No internal links, not in sitemap, not externally linked	Medium

Sources: Google Search Central: Index Coverage report, Google Search Central: Fix indexing problems

---

3Crawled — Currently Not Indexed: The Most Common Blocker

"Crawled — currently not indexed" is the status site owners encounter most frequently after Google's 2022–2024 quality tightening. It means Googlebot visited the page and decided it was not worth keeping.

Why Google Declines to Index a Crawled Page

1. Thin or low-value content. Pages with fewer than ~300 words of substantive text, pages that mostly repeat other pages on your site, or pages that list items without adding commentary or analysis are routinely excluded. Google's quality raters look for content that "demonstrates first-hand expertise and a depth of knowledge" (Google Search Quality Evaluator Guidelines).

2. Content that matches intent better on another URL. If a page targets a keyword already covered by a stronger page on your site or a competing site, Google may choose not to index the weaker candidate. This is different from duplication — the content may be unique but deemed redundant in intent.

3. Poor E-E-A-T signals. Pages on YMYL topics (health, finance, legal, safety) with no identified author, no credentials, and no external signals of authority are frequently excluded. The same pattern increasingly applies to comparative content after the December 2025 core update (Google Search Central: E-E-A-T).

4. Soft 404 behaviour. A page returning HTTP 200 but containing a message like "no results found" or "this product is out of stock" with no substantive content will be treated as a soft 404. Google's documentation explicitly identifies e-commerce out-of-stock pages, empty site-search result pages, and gateway pages with no real content as soft 404 examples (Google Search Central: Fix soft 404 errors).

5. Parasite or boilerplate content. Automatically generated text with minimal variation — common in faceted navigation, location pages, and programmatic SEO at scale — is frequently excluded when the variations are superficial.

How to Fix It

1. Run URL Inspection on a sample of affected URLs — the "Coverage" section shows last crawl date and detected issues.
2. Audit content depth. Does this page contain anything not already in the top 5 results for its target query? If no, expand substantively or consolidate into a stronger page via 301.
3. Add E-E-A-T signals. Name the author, add credentials and a bio, link to external sources, show publication and last-updated dates.
4. Fix soft 404s. Out-of-stock pages should return 404/410, redirect to a category page, or show genuinely useful alternatives.
5. Submit for reindexing. Use URL Inspection → "Request indexing" for priority pages. Update the sitemap's <lastmod> and resubmit via GSC for bulk signals (Google Search Central: Sitemaps).

---

✓Start Here

Run URL Inspection on affected URLs before auditing. GSC’s Coverage section reveals last crawl date and detected issues, helping pinpoint thin content or soft 404s.

4Discovered — Currently Not Indexed: Crawl Budget Problems

"Discovered — currently not indexed" means Google knows the URL exists but has not yet chosen to crawl it. For small sites (under ~1,000 indexed pages), this rarely indicates a crawl budget problem — the queue will eventually clear. For large sites, it signals that Googlebot is spending its crawl allocation elsewhere and deprioritising these URLs.

What Determines Crawl Budget

Google defines two factors (Google Search Central: Crawl Budget):

• Crawl capacity limit: How fast Googlebot can crawl without overloading your server — slow TTFB directly reduces the crawl rate.
• Crawl demand: How much Googlebot wants to crawl, based on PageRank, external links, and content freshness.

Sites with low link equity, stale content, and no traffic get fewer crawl slots — leaving large URL sets perpetually in the "discovered" queue.

Index Bloat: The Hidden Crawl Budget Killer

Index bloat occurs when Google is indexing (or attempting to index) thousands of low-value URLs that consume crawl budget without contributing ranking equity. Common sources:

• Faceted navigation URLs — /products?color=red&size=M&sort=price generates exponential URL combinations
• Duplicate parameter URLs — tracking parameters (?utm_source=email) creating duplicate pages
• Session ID URLs — user-session tokens appended to URLs
• Pagination beyond page 2–3 — paginated archives with no unique content
• Tag and category pages with minimal content, especially WordPress taxonomies
• Low-quality programmatic pages — location, template, or data-driven pages with thin variation

A site with 500 good pages and 50,000 bloat URLs will have its crawl budget consumed by the bloat, leaving good pages undiscovered or crawled infrequently (Google Search Central: URL structure).

How to Fix Discovered — Currently Not Indexed

1. Diagnose internal linking. Pages with few or no internal links pointing to them get low crawl priority. Use Screaming Frog or similar to map pages with zero or one internal link. Add contextual links from high-traffic pages.
2. Improve server response time. Target Time to First Byte (TTFB) under 200ms. Slow servers directly reduce Googlebot's crawl rate. (Google Search Central: Page Experience).
3. Manage faceted navigation. Use <meta name="robots" content="noindex"> on parameter combinations that don't deserve independent ranking. Configure robots.txt to disallow crawling of parameter patterns that generate duplicates. Implement canonical tags pointing to the clean URL.
4. Remove or noindex bloat. Audit your GSC Index Coverage report for pages that are indexed but generating no impressions or clicks over 12 months. Noindex or consolidate these pages to reclaim crawl budget for content that matters.
5. Submit a clean XML sitemap. Include only canonical URLs you want indexed. Remove redirects, noindex pages, and parameter URLs from the sitemap. Resubmit after cleanup (Google Search Central: Build a sitemap).

---

!Watch out

Index bloat can silently consume crawl budget. A site with 500 quality pages but 50,000 bloat URLs will see Googlebot burning resources on low-value pages instead of indexing important content.

5Canonical Tags and Duplicate Content

Duplicate content is the second-largest indexing problem after quality. Google handles duplicates by selecting a canonical URL from the duplicate set and indexing only that one. Your job is to ensure Google picks the URL you want.

Why Canonicalisation Fails

• Inconsistent internal linking — linking to both /page/ and /page (trailing slash mismatch) creates two candidate URLs.
• HTTP and HTTPS both accessible — ensure HTTP 301 redirects to HTTPS.
• www and non-www both accessible — pick one, redirect the other, set preferred domain in Search Console.
• Sitemap includes non-canonical URLs — contradicts the live canonical.
• Canonical tag conflicts with redirect — a redirect from /page-a to /page-b with a canonical on /page-b pointing back to /page-a creates an unresolvable loop.

Implementing Canonical Tags Correctly

Every page needs one <link rel="canonical" href="https://example.com/preferred-url"> in <head>. Self-referencing canonicals (a page pointing to itself) are correct — they block parameter-based duplication (Google Search Central: Consolidate duplicate URLs).

For e-commerce product variants: if they have distinct search demand, give each a self-referencing canonical with unique content. If not, point all variants to the base product page.

---

6The Google Indexing API

The Indexing API allows direct notification to Google when a page is published or removed, bypassing the normal crawl queue. It was designed for job postings and live-event pages and is documented with that scope in mind (Google Search Central: Indexing API).

What the Indexing API Actually Does

• Signals to Google that a URL should be crawled immediately
• Does not guarantee indexing — it prioritises the crawl, not the indexing decision
• Works best for pages with high freshness requirements (job listings, live sports, breaking news)

Indexing API Limits (2026)

The Indexing API has strict quotas per project:

• 200 requests per day by default
• Bursting is not allowed — requests are spread across the day
• Exceeding the limit returns a 429 error
• Limit increases require an application to Google, which is rarely granted without a documented justified use case (Google Search Central: Indexing API quotas)

For most sites, the Indexing API is useful for newly published content and for signalling content removal. It is not a substitute for fixing the underlying crawl and quality issues that cause pages to remain unindexed.

IndexNow Protocol

IndexNow is an open-source protocol (supported by Microsoft Bing, Yandex, and Naver) that allows immediate URL submission when content changes (IndexNow.org). Google does not participate in IndexNow — Googlebot's crawl decisions are not influenced by IndexNow submissions. Do not rely on IndexNow as a Google indexing accelerator.

---

iQuota Alert

The Indexing API has a daily quota of 200 requests. It’s designed for time-sensitive content like job postings; for most ecommerce sites, focus on fixing crawl and quality issues rather than relying on the API.

7Internal Linking and Crawl Equity

Internal links are the primary mechanism Googlebot uses to discover and reprioritise pages after the initial crawl. A page with no internal links pointing to it is effectively invisible to Googlebot's crawl path — even if it is in your sitemap.

Link Equity Principles

• PageRank flows through internal links. Pages with more inlinks are crawled more frequently and evaluated with higher authority (Brin & Page, 1998).
• Contextual links > navigation links. Body-copy links carry relevance signals that sidebar/footer links do not.
• Orphan pages are deprioritised. A URL accessible only through a sitemap but with no internal links is unlikely to be crawled regularly on large sites.

Internal Linking Audit for Indexing

1. Export all URLs from your sitemap.
2. Crawl the site with Screaming Frog and filter to pages with fewer than 3 inlinks.
3. For each, identify 3–5 related pages where a contextual link fits naturally.
4. Add links, prioritising pages that show "Discovered — currently not indexed" in GSC.

For more on how internal linking interacts with crawling and ranking, see Indexing.

---

8Diagnostic Workflow: From Problem to Fix

Use this workflow when pages are not appearing in Google search results.

Step 1 — Confirm the Page is Not Indexed

Search site:example.com/your-page-url in Google. No result = not indexed. A result = indexed but possibly not ranking (a different problem).

Also check Search Console → URL Inspection for the exact page — this gives the most accurate view including last crawl date and detected issues.

Step 2 — Check for Blockers

Run through this checklist in order:

• robots.txt — does it block the URL or its containing directory?
• <meta name="robots"> — is noindex present?
• X-Robots-Tag HTTP header — check with curl -I https://example.com/page
• HTTP status — does the page return 200? Check for soft 404 behaviour.
• Canonical tag — does it point to the correct URL (itself or the intended canonical)?
• JavaScript rendering — is the main content in the initial HTML, or does it require JS execution?

For JavaScript-rendered pages, use the URL Inspection Tool's "Test Live URL" → "View crawled page" to see what Googlebot actually rendered. If the main content is blank, Googlebot likely cannot render the JavaScript (Google Search Central: JavaScript SEO).

Step 3 — Identify the GSC Status

Check Search Console → Indexing → Pages. Find your URL's status from the table above and follow the fix path for that status.

Step 4 — Assess Content Quality

For "Crawled — currently not indexed" pages:

• Open the page alongside the top 3 Google results for its target query.
• If your page is materially thinner, expand it with original content, data, or expert perspective.
• If it near-duplicates another page on your site, consolidate via 301 to the stronger URL.

Step 5 — Fix, Wait, Re-check

1. Use URL Inspection → "Request indexing" for high-priority pages.
2. Update <lastmod> in your sitemap and resubmit via GSC → Sitemaps.
3. Wait 7–21 days before re-checking GSC status.
4. For large-scale bloat removals, allow 4–8 weeks for crawl budget reallocation.

---

9Index Bloat Management at Scale

Large sites (10,000+ pages) require ongoing index hygiene, not one-time fixes.

Regular Audit Process

Run this quarterly:

1. Export the indexed set from GSC → Pages → Indexed, sorted by impressions (ascending). Pages with zero impressions over 90 days are your primary bloat candidates.
2. Segment by URL pattern. Identify which templates generate the most zero-impression pages — faceted nav, tag pages, paginated archives, thin landing pages.
3. Decide per-segment: noindex, consolidate, 301, or delete/410.
4. Implement in batches. Remove 10–15% per month, not thousands at once, so Google can reassess crawl budget gradually.
5. Track crawl rate. A rising crawl rate in GSC → Settings → Crawl Stats confirms the cleanup is working.

Monitoring Crawl Stats

Search Console → Settings → Crawl Stats shows total daily crawl requests, download size per request, and response time trends. A spike in crawl errors or drop in pages crawled per day often precedes discovery or indexing problems. Check weekly for large sites (Google Search Central: Crawl Stats).

---

10Sitemaps: What They Do and Don't Do

Sitemaps communicate URL existence to Google — they do not compel crawling or indexing. A URL in a sitemap is a suggestion, not an instruction (Google Search Central: Sitemaps).

Sitemap Best Practices for 2026

• Include only canonical URLs — never noindex pages, redirect sources, or parameter URLs.
• Keep <lastmod> accurate. Google detects cosmetic date changes and stops trusting the field.
• Use sitemap index files for sites over 50,000 URLs or 50MB per file.
• Separate content types. A news sitemap alongside your main sitemap helps Google prioritise freshness-sensitive content.
• Submit via Search Console — the GSC submission generates status reporting that robots.txt alone does not.
• Maximum 50,000 URLs per sitemap file (Google Search Central: Sitemap file requirements).

---

Need Help Getting Pages Indexed?

EcomExperts can diagnose your crawl budget and quality blockers for free.

Book your free audit

---

Frequently Asked Questions

What does <code>Crawled — currently not indexed</code> mean?

Google visited the page but decided not to index it, usually due to thin content, duplication, or poor E-E-A-T signals.

Why does <code>Discovered — currently not indexed</code> appear on large sites?

It typically indicates a crawl budget issue: Googlebot has many URLs to crawl but limited resources, often due to index bloat from faceted navigation, parameter URLs, or low-value pages.

How can internal linking improve indexing?

Internal links signal crawl priority. Pages with few or no internal links are deprioritised. Adding contextual links from high-traffic pages boosts crawl demand.

What is the quickest fix for <code>Crawled — currently not indexed</code> status?

Enhance content depth to exceed what top-ranking pages offer, add author credentials and external citations, and fix any soft 404 behaviour before requesting reindexing.

Does the Indexing API guarantee indexing?

No. It prioritises crawling but Google still applies quality thresholds; it’s best used for time-sensitive content like job postings, not as a substitute for quality improvements.