Check this page with an assistantOpens a chat asking it to summarise this article and name the evidence behind each claim.
Claude opens with the prompt on your clipboard: Anthropic does not support prefilled prompts on the web, and we would rather copy it than ship a button that drops it.
A Sitemap Is an Inventory Hint, Not a Permission Slip
An XML sitemap tells search engines which canonical pages and files you consider important, when they materially changed, and, through extensions, how certain images, videos, news items, and localized versions relate. It helps discovery; it does not override page quality, indexing controls, status codes, or canonical selection.
A good sitemap is boring because it agrees with the site. Every listed URL returns a successful response, permits indexing, declares itself canonical, appears in a useful internal structure, and belongs in search. A bad sitemap becomes a landfill of historic URLs, filters, redirects, staging hosts, and changing dates.
Google says a well-linked site of roughly 500 pages or fewer may not need a sitemap for basic discovery. Most sites should still generate one because it is inexpensive, supports change discovery, and creates a useful submitted-versus-indexed diagnostic set. It is insurance, not architecture.
What Belongs in the File
| Include | Exclude | Reason |
|---|---|---|
| Canonical 200 pages intended for search | Duplicate parameter and tracking variants | The listed URL is a canonical preference |
| Current products, services, locations, articles | Redirects, 404s, 410s, soft 404s | Crawlers should reach a usable final destination |
| Useful pages permitted for crawling and indexing | Noindex or robots-blocked pages | Discovery and visibility signals should agree |
| Absolute production URLs on the intended host | Relative, staging, preview, and malformed URLs | Google attempts to crawl the exact listed address |
| Localized equivalents with valid annotations | Machine-translated or empty locale placeholders | The alternate set must contain real eligible pages |
Generate the list from the content or commerce database, not by crawling whatever the site happens to expose. A crawl can miss orphan pages and collect accidental URLs. The source system knows which records are published, canonical, available, and intended for search.
Use Lastmod Only When It Can Be Trusted
Google says it uses lastmod when the value is consistently and verifiably accurate. Set it to the date or timestamp of the last significant page update: changed main content, structured data, important links, product facts, or another meaningful user-facing element.
Do not update every URL because the sitemap regenerated, a copyright year changed, analytics code deployed, or a shared footer moved. False freshness teaches a crawler that the field has little information. If the publishing system cannot distinguish significant modification, omit lastmod until it can.
Google currently ignores priority andchangefreq. Remove them unless another supported consumer requires them. They do not turn a page into a crawl priority, and setting every URL to daily and 1.0 says nothing.
Choose a Structure That Improves Diagnosis
A single sitemap can contain up to 50,000 URLs or 50 MB uncompressed. A sitemap index can reference multiple child files. Those limits do not mean a 12,000-page site needs one opaque file. Split on boundaries the team can own and investigate.
| Useful split | Question it answers | Avoid |
|---|---|---|
| Products / categories / editorial | Which template has discovery or indexing drift? | Random numeric batches with no owner |
| Locations / services | Are local commercial destinations healthy? | Mixing test and production location sets |
| Current / recently retired | Did a migration cleanup finish? | Keeping retired URLs permanently submitted |
| Language or regional site sections | Which locale is under-discovered? | Splits that conflict with canonical ownership |
Submit One Clear Entry Point
Host sitemaps at a stable crawlable URL, usually near the site root. Submit the sitemap or index in Search Console so the team can see the last read time and processing errors. Add the fully qualified sitemap URL to robots.txt so compliant crawlers can discover it. Google allows multiple sitemap declarations.
Submission is still a hint. Do not repeatedly resubmit a clean, unchanged file expecting an indexing queue to move. Investigate the affected page set: internal discovery, response, robots rules, noindex, rendering, canonicalization, usefulness, and reporting delay.
Maintain production host discipline. A common release failure publishes a sitemap containing localhost, a preview domain, HTTP variants, or the wrong trailing-slash pattern. Validate the exact generated output during deployment.
Use Media and Localization Extensions When They Add Information
XML is the most versatile format because Google supports extensions for images, video, news, and localized alternates. Image entries can help expose assets that are otherwise difficult to discover. Video entries can include the watch page, thumbnail, title, description, and content or player URL. Hreflang annotations can also live in the sitemap.
Choose one maintainable hreflang implementation: HTML, HTTP headers, or sitemap. Google says the methods are equivalent and using all three adds complexity without a search benefit. Specialized metadata must match the visible page and remain accessible to crawlers.
The image SEO guide and video SEO guide explain when those discovery layers deserve the additional data.
Read Submitted Versus Indexed as a Triage Signal
Search Console can help compare sitemap-submitted pages with known indexing states, but a gap is not automatically an error. Some submitted pages may be recently published, duplicates, low-value, unavailable, or intentionally excluded after a stale sitemap.
- Confirm the sitemap was fetched and parsed successfully.
- Group excluded submitted URLs by template and stated reason.
- Sample live URLs instead of trying to “fix” the total percentage.
- Remove URLs that no longer meet the sitemap contract.
- Escalate only material canonical pages that fail the intended state.
The goal is not 100% indexing. It is accurate discovery and index selection for the canonical pages that matter. Use the Search Console guide to turn those states into a reviewable action.
Automate the Sitemap Contract
Test that every emitted URL uses the production origin, returns 200, permits indexing, declares the expected canonical, and belongs to the intended child file. Detect duplicate locations across sitemaps. Check count and file-size limits. Reject invalid XML and unescaped values before deployment.
Monitor sudden count changes. A drop can reveal a publishing outage; a spike can expose facet, parameter, or staging leakage. The generator should fail visibly when its data source is unavailable rather than silently publishing an empty or partial inventory.
Questions About XML Sitemaps
- Does submitting an XML sitemap guarantee indexing?
No. Google describes sitemap submission as a hint. A sitemap can improve discovery and communicate preferred canonical URLs, significant modification dates, and media or language relationships, but each page still must be crawlable, indexable, useful, and eligible for selection.
- Should noindex and redirected URLs be in a sitemap?
No. List the canonical URLs you want to see in search results. Remove redirects, errors, duplicates, blocked URLs, and intentional noindex pages. A sitemap that repeatedly advertises URLs the site says should not be indexed weakens its diagnostic value and sends conflicting signals.
- How often should an XML sitemap update?
Update it when canonical indexable URLs are added, removed, or significantly changed. Generate it automatically from the same governed source that controls page state. Do not change every lastmod date on every deployment when the underlying page content did not materially change.
- How many URLs can one sitemap contain?
Google’s documented limit is 50,000 URLs or 50 MB uncompressed per sitemap, whichever comes first. Larger sites should split URLs across child sitemaps and optionally submit one sitemap index. Smaller splits can still be useful when they align with meaningful templates or ownership.
Primary Sources Reviewed
- Google Search Central: What a sitemap does
- Google Search Central: Build and submit a sitemap
- Google Search Central: Image sitemaps
- Google Search Central: Video sitemaps
- Sitemaps.org: XML protocol
Reviewed July 28, 2026. Sitemap extensions and search-engine handling can change; validate optional fields against each engine’s current documentation.
