Most websites published today have an XML sitemap file without their owners being aware of it. Since WordPress 5.5, a native sitemap is automatically generated at /wp-sitemap.xml. Platforms like Framer or Shopify do the same upon publication. This silent generation poses a concrete problem: a sitemap file exists, but no one checks what it contains or what it signals to search engine crawlers.
Automatically generated XML sitemap: what CMSs include without warning
WordPress, in its default configuration, includes elements in the sitemap that the site owner never intended to have indexed. Test pages left online, unused custom post types, technical taxonomies created by plugins: everything ends up listed in the XML file submitted to Google.
You may also like : How to Succeed in Your Real Estate Project: Tips and Tricks for a Strong Start
The result is a sitemap that does not reflect the actual structure of the website. Indexing bots receive a distorted map, where low-value URLs coexist with the pages that one actually wants to rank. WPFormation recommends in its 2026 guide to disable or filter certain content via a plugin or code so that the sitemap serves navigation and SEO, not the other way around.
The same phenomenon affects other website builders. Shopify generates a sitemap structured by collections, products, and pages, but merchants who create temporary pages (promotions, A/B tests) see them persist in the file without manual intervention. On Adobe Experience Manager, sitemap configuration goes through specific components, and the default settings do not exclude internal admin pages.
You may also like : How to Manage Payroll for a Business?
Consulting the sitemap page of BestWeb allows you to visualize what a structured and filtered sitemap looks like, where only relevant URLs appear.
Silent degradation of the sitemap: an underestimated risk for SEO
A valid sitemap at the time of its creation can become problematic within a few weeks. Deleted or redirected URLs remain listed in the XML file. The sitemap then returns 404 error pages to Google’s bots, which dilutes the crawl budget and delays the indexing of recent content.
This type of degradation often goes unnoticed because no alert is triggered. The site owner only notices the drop in indexing and traffic several weeks after the problem arises. Services like Oh Dear have specialized in monitoring sitemaps to detect these anomalies:
- Sitemap file that has become invalid or returns a server error
- Listed URLs that return a 404 code or a redirect loop
- Unintentional modification of the number of referenced URLs (mass addition or deletion)
An unmonitored sitemap can be more harmful than a site without a sitemap. Search engines can discover pages through internal linking and external links. However, an XML file that points to dead ends sends a signal of poor technical maintenance.
HTML sitemap and XML sitemap: two files, two distinct functions
The confusion between XML sitemap and HTML sitemap remains common. The XML file is intended for crawlers. It lists URLs with technical metadata (last modified date, update frequency). The HTML sitemap is a page of the site intended for human visitors, presenting the structure in the form of clickable links.
A well-designed HTML sitemap improves navigation on sites with deep structures, where some pages are more than three clicks away from the homepage. For smaller sites with a clear menu, its usefulness remains marginal.
When the HTML sitemap provides real value
E-commerce sites with several hundred categories and subcategories benefit from an HTML sitemap. Information portals whose archives cover several years of publication do as well. In these cases, the HTML sitemap serves as a safety net for orphan pages, those that no internal link directly connects to the main structure.
For SEO, Google has indicated that the XML file remains the preferred format for communicating the list of pages to be crawled. The HTML sitemap has no direct effect on indexing, but it distributes internal link juice to deep pages.
Submitting and maintaining a sitemap in Google Search Console
Creating a sitemap file is not enough. Submitting it via Google Search Console allows you to verify that Google reads it correctly and to identify processing errors. The coverage report displays the number of submitted URLs, the number indexed, and the excluded URLs with their reasons.
Maintaining the sitemap involves some regular checks:
- Comparing the number of URLs in the sitemap with the number of pages actually indexed in Search Console
- Ensuring that URLs returning a 404 or 301 code are removed from the file
- Checking that newly published pages appear in the sitemap within 24 to 48 hours
- Controlling that the file does not exceed the limit of 50,000 URLs per sitemap (beyond that, a sitemap index is required)
The lastmod parameter must reflect the actual modification date of the page, not the date of the file’s regeneration. Google has specified that it ignores lastmod values when they do not correspond to an actual content change, making this field useless if the CMS updates it with every rebuild of the sitemap.
Segmenting the sitemap for large sites
Some SEOs split their sitemap files by content type (articles, categories, products, static pages). This segmentation facilitates diagnosis: if the indexing of product pages drops, the dedicated sitemap allows isolating the problem without reviewing the entire file.
A well-structured sitemap is not a file that you create once and forget. It is a permanent diagnostic tool, whose value entirely depends on its maintenance. The best sitemap is the one that contains only the URLs you want to see indexed, nothing more.