Should your sitemap include every URL or only the important ones?
- Published November 6, 2026
- Linked from 4 answers
- In a cluster of 51
Include every URL you want Google to crawl and show in search, and leave out the rest. That is not the same as “only the important ones”: a large site with thousands of useful pages should list thousands of URLs. What stays out are the addresses you do not want in results, such as redirects, noindexed pages, and duplicates whose canonical points elsewhere. Google’s crawl budget guidance puts the inclusion side simply: “Google reads your sitemap regularly, so be sure to include all the content that you want Google to crawl.”
What a sitemap entry tells Google
A sitemap is a list of URLs you are vouching for, and Google treats it as a hint. Its sitemap documentation says submitting a sitemap is “merely a hint: it doesn’t guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site.” For duplicates, listing a URL in a sitemap is a weak signal that it should be the canonical version. So every entry should be the version you want indexed.
The inclusion test
| URL | In the sitemap? |
|---|---|
| Returns 200, is its own canonical, no noindex | Yes |
| Redirects to another URL | No, list the destination instead |
| Carries a noindex rule | No |
| Has a canonical pointing to another URL | No, list the canonical |
| Returns 404 or 410 | No |
| Is blocked in robots.txt | No |
| A filter, sort, or tracking-parameter variant | No, unless you deliberately want that variant indexed |
Listing URLs that fail this test does not break anything outright, but it sends mixed messages: a noindexed URL in the sitemap asks Google to crawl a page that asks not to be indexed; that conflict is worked through in what happens when noindex and a sitemap entry conflict.
Practical limits and fields
- A single sitemap is limited to 50,000 URLs or 50MB uncompressed. Larger sites split their URLs across several sitemaps and list them in a sitemap index.
- Google ignores the
priorityandchangefreqfields, so they do not need maintaining. - Google can use
lastmodwhen it is consistently and verifiably accurate; for what that means in practice, read whether a last-modified date in the sitemap affects re-crawling. When the sitemap and the canonical tags disagree, follow how Google handles conflicting canonical and sitemap signals.
Keeping it clean over time
Generate the sitemap from your CMS rather than by hand, so new pages appear and retired ones disappear automatically. Then compare the sitemap’s URL count with the indexed count for submitted pages in Search Console’s Page indexing report, which can be filtered to sitemap URLs. A large gap between the two is a list of pages to investigate, one URL at a time.
Add a comment