What happens to crawl rate when you suddenly add thousands of pages?

Google does not crawl a new batch all at once. It works through the new URLs over time, within the limit your server can handle and in proportion to how much it wants to crawl them. Some new pages are fetched and indexed soon, others wait as Discovered – currently not indexed, and some are crawled but not indexed at all. How the batch fares depends less on its size than on your server’s capacity and on whether the pages look worth crawling.

The two limits a big batch runs into

Google’s crawl budget documentation describes crawl capacity and crawl demand as the two elements that set how much it crawls:

• Capacity. “Every site starts with the same default, conservative crawl capacity limit.” The limit rises when the server responds quickly and falls when it slows down or returns errors. A sudden batch that strains the server can lower the rate for the whole site, old pages included.
• Demand. Google weighs how many URLs it knows about, how popular they are, and how stale its copies are. Thousands of new URLs raise the inventory, but they do not come with popularity: few links point to them yet.

Site-wide events can raise demand. Google notes that a site move, for instance, “may trigger an increase in crawl demand in order to reprocess the content under the new URLs.” Google describes no such trigger for a batch of new pages on an existing site, though it may still adjust crawling as it discovers new URLs; your server logs show what actually happens.

Why quality decides how much of the batch lands

Crawling a page is not the same as indexing it. Google’s crawl budget guidance states that “not every page that is crawled will necessarily be indexed”, and pages that read as near-duplicates or add little can be crawled and left out. If a large share of the batch is thin or templated, the visible result is a growing count of Crawled – currently not indexed or Discovered – currently not indexed URLs in the Page indexing report. Whether many similar pages are worth publishing at all is a separate question, examined in what happens to SEO when you have 10,000 near-identical product pages.

How to publish a large batch

1. Check the server can handle more requests before launch; slow responses lower the crawl limit.
2. Link the new pages from existing pages that Google already crawls, such as category hubs, so they are discoverable through the site and not only through the sitemap.
3. Add them to the sitemap with accurate lastmod dates.
4. Release in groups if you want to read the results: publish one group, watch its indexing in the Page indexing report filtered to that sitemap, then decide whether the next group needs changes first.
5. Watch the Crawl Stats report for rising response times or server errors during the rollout.

A batch that is valuable, well linked, and served by a healthy server gets crawled steadily. A batch that is none of those can stay in the not-indexed statuses, however long you wait.