Jump to the answers
SEO Talks

All clusters

What happens to SEO when you have 10,000 near-identical product pages?

  • Published November 6, 2026
  • Linked from 1 answer
  • In a cluster of 77

Google can index a fraction of them and leave the rest out, while it spends crawling on pages that will not rank. When thousands of pages differ only in a size, a color code or a part number, Google can group them as duplicates and pick a few to show. The others appear in Search Console as crawled but not indexed, or as duplicates of a URL Google chose. Nothing has to be penalized for this to cost you: a site can carry ten thousand URLs and get its search visibility from a few hundred.

What near-identical pages cost

Cost How it shows Who it affects
Pages left out of the index Page indexing report: large counts under "Crawled – currently not indexed" and the duplicate statuses The product pages themselves
Crawling spent on duplicates Crawl stats: requests concentrated on variant and parameter URLs; new products picked up late New and updated pages elsewhere on the site
Google choosing the wrong page The indexed version of a product is a variant URL, not the main one Shoppers landing on a size or color they did not want
A site that reads as repetitive overall Google's traffic-drop guidance asks owners to assess the whole site, not only single pages Pages with real value that share a site with the duplicates

The last row is the hardest to measure. Google’s page experience documentation says its core ranking systems generally evaluate content page by page but that it does have some site-wide assessments, so a catalog dominated by near-copies is not a neutral setting for the pages that are worth ranking.

Separating the two problems

Indexing and crawling need different fixes, and confusing them is where large catalogs go wrong.

  • To decide which page represents a product, start with how shoppers choose and buy, not with search demand alone. Variants that differ only in an option picked on the page, such as size or color, can share one product page. Google’s product variant documentation calls this a single-page setup: a URL with query parameters can open the page with one variant preselected, and the whole group has one canonical URL. Variants that need a page of their own, because of their own price, stock, images or search demand, can keep separate URLs. In that multi-page setup, Google asks for “full and self-contained markup” on each page for the entities defined there, so that nothing on another page is needed to understand it; its examples repeat the ProductGroup definition on every variant page alongside that page’s own variant. Either way, each variant needs a unique ID such as a SKU or GTIN, and ProductGroup describes how the variants relate.
  • To stop crawling of URLs that should never be fetched, such as parameter combinations, use robots.txt rules on those patterns.
  • To keep a crawlable page out of the index, use noindex, knowing it does not save crawling.

Working through the catalog

  1. Group pages by what makes them near-identical: variants, parameters, or separate products with copied descriptions.
  2. For each product, decide whether its variants belong on one page or on several, from how shoppers pick them, their price, stock and SKU, and their search demand; merge the ones that belong together.
  3. Block crawling only of parameter patterns that produce nothing but duplicates, such as sort orders or session IDs, after checking that none of them is a variant URL your product markup or product feed uses, or a variant people search for.
  4. For separate products that share copy, write for the ones with demand first.

To know whether your catalog is large enough for crawling to be the limit, check the size groups in at what site size crawl budget starts to matter. Fewer pages, each with its own reason to exist, stand a better chance of being indexed than ten thousand that repeat each other.

Related posts:

  1. Why does Google index one near-identical page but skip the other? Google treats the two pages as duplicates and shows one of them, so the skipped page is not being punished; it is being represented by its twin. Google's canonicalization documentation......
  2. Why does a 600-word page outrank a 3,000-word one on the same term? Likely because it fits the search better, since length is not what Google ranks; links, the site's standing and technical factors can also explain the order. Google's SEO Starter Guide......
  3. Noindex or canonical for near-duplicate pages, which when? Use a canonical when the near-duplicates are versions of the same page and you want one of them to represent the group in search. Use noindex when a page should......
  4. Why does identical content index on one domain and not another? Because when the same content exists on two sites, Google can treat the copies as duplicates and generally shows one of them for a given search, though in some searches......
  5. How do you keep category pages from cannibalizing product pages? Give each level a different kind of search. Category pages answer searches for a type of product, where the shopper wants to compare options: "women's waterproof hiking boots". Product pages......
  6. Should out-of-stock product pages be kept, redirected, or noindexed? It depends on whether the product is coming back. A product that is temporarily out of stock keeps its page, live and indexed, with the availability shown clearly. A product......
  7. Why do thin product pages with manufacturer copy fail to index? A likely reason is that they give Google nothing it does not already have, though it is a hypothesis to test, not a diagnosis the report hands you. The same......
  8. Does adding unique copy to every product page actually pay off? On the products people search for, it can; across every product in a large catalog, the hours are unlikely to come back. Unique copy is an investment per page, and......
  9. Does Product schema affect ranking or only the rich result? Its documented effect is on how and where the product can appear, not on its position. Google describes structured data as a way to understand a page's content and to......
  10. What happens to SEO when pagination is done with infinite scroll? Google sees the first batch of items and can miss the rest. Infinite scroll loads more content when a visitor reaches the bottom of the page, and Google does not......
  11. What happens to crawl rate when you suddenly add thousands of pages? Google does not crawl a new batch all at once. It works through the new URLs over time, within the limit your server can handle and in proportion to how......
  12. What happens to indexing when you publish 50 pages in one day? Google does not index all fifty at once. It discovers them through your links and sitemap, crawls them over the following days and weeks, and indexes the ones it judges......
  13. What happens when two of your pages compete for the same query? When the two pages answer the same need, they can split what one page would have had, and Google may show one of them, not always the one you would......
  14. What happens to rankings if you change a title that’s already ranking? The page can move up, down, or not at all, and nothing happens until Google recrawls it. Google's title link documentation notes that Google has to recrawl and reprocess a......
  15. What happens to your traffic when an AI Overview answers the query? It depends on how completely the Overview settles the searcher's need. For a simple question that the summary answers in full, fewer people need to click, so a page can......

Comments on this answer

Add a comment

Your email address will not be published. Required fields are marked *