How (and why) to regularly update your sitemap for Google

Understanding the importance of sitemap updates

There are many types of sitemaps out there, but the one that Google and others are looking for site updates in is the XML sitemap.

Sitemaps organize your website effectively, facilitating better navigation and SEO by clarifying page hierarchy and relevance to visitors and search engines alike.

They’re like a tour guide for search engines, helping them understand your site’s structure, prioritizing pages based on importance and staying updated on how often and when they change or new content is added.

Maintaining an updated sitemap is like shining a spotlight on your website, enhancing its visibility and discoverability. This proactive approach can lead to better indexing and improvements in your website’s SEO.

Additionally, organizing content in a sitemap improves website structure, leading to clearer pathways for users, helping them navigate your site more effectively.

When to update your sitemap?

Think of your sitemap like a city map; it needs to keep pace with new “streets” and landmarks on your site. Update it whenever there’s a meaningful change, such as:

• Adding, removing or modifying content — new product pages, blog posts, retired sections or any other pages that get changed.
• Rebranding — refreshed structure, design, or messaging.
• Domain changes — migrations or URL restructures.
• Major redesigns — shifts in navigation or hierarchy; not a bad idea to use an AI sitemap generator to get started.
• Technical fixes — improved performance, server changes or internal linking updates.
• User-driven changes — insights from testing or feedback that alter the user journey.
• News or time-sensitive content — for sites publishing regularly, keep lastmod accurate so search engines catch updates quickly; Google News sitemaps should also update as stories are published.

The impact on search results

An updated sitemap is a compass guiding search engine crawlers to your most important web pages.

Metadata in your sitemap, like change frequency, signals how frequently a page might be updated, affecting its indexing. Even though Googlebot ignores changefreq and priority settings when doing a crawl, assigning accurate priority values can still be beneficial for other search engines in understanding your content structure.

Accordingly, high-quality content should be emphasized to give your site the best shot at being indexed to your liking, potentially improving your search results in the process.

Using Google Search Console and Bing for sitemap submission

When your sitemap is done with updates, the next step is its submission. Google Search Console is your go-to platform for this process, given that it’s the world’s most used search engine.

GSC accepts various sitemap formats:

• XML
• RSS feed
• Atom 1.0
• Plain text

The URLs should be UTF-8 encoded and fully qualified.

The Sitemaps report page is where the sitemap submission process happens; it’s also where you can track history and check parsing errors.

Google attempts to crawl newly submitted sitemaps promptly while maintaining an independent crawl schedule for the site, details of which are available in the Sitemaps report.

The report also offers insights into the last crawl request status for every sitemap and detailed error information if applicable. For troubleshooting submission issues, Google Search Console will continue trying to fetch and process the sitemap for several days if errors persist.

Submitting your updated sitemap

Submitting your updated sitemap to Google Search Console is a hassle-free process:

1. Go to GSC and click the “Sitemaps” tab
2. Add the complete URL of your sitemap file and click “Submit”
3. Ensure that the sitemap URL you submit does not redirect
4. Verify that the sitemap is submitted through the Sitemaps report or the Search Console Sitemaps API correctly

Note that you can submit up to 500 sitemap index files per site in a GSC account.

While submitting a sitemap index through the Search Console can expedite the indexing of web pages, it doesn’t ensure that all pages will be indexed.

Submitting to Bing via Webmaster Tools is another smart move and the process is similar:

1. Go to Bing Webmaster Tools and click the “Sitemaps” tab
2. Click “Submit sitemap” in the top right
3. Enter your sitemap URL and hit “Submit”
4. Look for the “Success” status code

Monitoring sitemap status

Post-submission, monitoring your sitemap’s status is crucial to confirm successful crawling and indexing.

Check for crawl errors such as 404 errors, parsing errors or URL structure issues in the Coverage report and promptly fix them.

Back in the Sitemaps report you can find details on the URL submitted, sitemap type, last submission date, latest crawl status and number of discovered pages.

You can identify the status of your latest sitemap crawl in the report, which can be ‘Success,’ ‘Has errors’ or ‘Couldn’t fetch’ and consult the report for troubleshooting fetch errors.

After updating your sitemap, verify its presence by navigating to your domain’s sitemap.xml and resubmit if necessary.

You can also use the URL Inspection Tool in Google Search Console to verify individual page indexation status within your sitemap.

For detailed reporting on index coverage and identifying any errors, click on a sitemap in the GSC Sitemaps report.

Automation tools for sitemap updates

The task of updating a sitemap manually, particularly for a large website or if you run multiple websites, can seem overwhelming.

Automated tools come to the rescue here, simplifying the process by generating a sitemap automatically.

Popular automation tools for generating and updating XML sitemaps include:

• Yoast SEO plugin for WordPress
• Google XML Sitemaps plugin for WordPress
• Screaming Frog SEO Spider
• XML-Sitemaps.com

Using these can save you time and ensure that your sitemap is always up to date and maintained.

Once updated, these tools also facilitate the submission of XML sitemaps to search engines.

There’s another way to have your sitemap automatically update itself; it’s called a dynamic sitemap but it requires coding expertise. Worth the effort for a site that gets updated often or for ecommerce sites with hundreds of changing product pages.

XML sitemap generators

These are an easy way to get an XML sitemap quickly. Popular options include:

• Slickplan’s XML sitemap generator: our free service for generating XML sitemaps for websites
• XML-Sitemaps: provides a free XML sitemap generation for small websites up to 500 pages

Enter your site’s address and they’ll create a well-structured, submission-ready sitemap, improving your website’s visibility and search engine rankings.

CMS integration

As mentioned above, sitemap updates are often handled inside CMSs. Most do this natively, while others require plugins; either way, it simplifies sitemap management.

Slickplan has CMS plugins for:

• WordPress
• Drupal
• ExpressionEngine
• concrete5
• Joomla

Managing multiple sitemaps for large sites

What exactly does a large site mean in this context?

50,000 pages or a sitemap file larger than 50MB.

If that’s your site, pay attention, this part is for you, because you’ll need to break down your sitemap into multiple sitemaps.

But don’t fret, in the next few sections we’ll take you through segmenting your sitemap and creating a sitemap index file that allows you to submit many sitemaps at once.

Creating a sitemap index

An index file is essentially a table of contents for your sitemaps.

What markup language is used for a sitemap index file?

It’s structured in XML format and begins with the <sitemapindex> root tag.

Each individual sitemap’s information is encapsulated using the <sitemap> tag. The URL location of each sitemap is specified within the <loc> tag in the sitemap index file.
As per Google, “you can submit up to 500 sitemap index files for each site in your Search Console account.”

Segmenting sitemaps

Look at segmenting sitemaps like dividing a large city map into smaller districts for easier navigation. This allows for more targeted crawling by search engines, which can be especially beneficial for bigger sites with diverse content.

Segmenting sitemaps can help identify specific areas of a site where indexing issues occur, allowing for quicker resolution and improved indexation.

Consistently updating the XML sitemap and assigning priority values to pages could assist in managing a limited crawl budget by search engine bots, allowing for efficient indexing of the most important pages.

For websites with content that grows over time, such as news publications, segmenting sitemaps can also involve using date naming conventions to ensure efficiency in crawling and indexing.

Creating a hierarchical structure of sitemaps can improve the organization of content and enhance a website’s topic authority.

For large websites, it is recommended to set up sitemap indexes for each parent theme to manage sitemaps efficiently. Structuring sitemaps hierarchically allows for granular reporting and increased sample data for troubleshooting.

Establishing rules for sitemap naming conventions, organization, hierarchy, sitemap generation and limits on URLs and indexes is important when briefing development teams on sitemap creation.

Ensure sitemap accuracy with manual checks

Automated tools and CMS integration simplify updates, but the value of manual checks cannot be understated.

Periodic manual checks of a sitemap ensure that key pages are accurately represented, which benefits both site management and indirectly aids SEO by highlighting pages that require attention. In some cases, it may even be necessary to manually create a sitemap to ensure accuracy and proper representation.

This is naturally much easier for smaller sitemaps.

Identifying errors manually

Like being a detective, manual checks for errors involve looking for broken links, unintended redirects and URLs that are no longer relevant or have been intentionally removed from the website.

It’s crucial to verify that all URLs in the sitemap respond with a 200 status code, indicating that the pages are accessible without 404 or 500 errors and the like.

Ensure included pages are not blocked by robots.txt, do not contain a no-index meta robots tag, have a canonical version and meet the XML file format standards.

Check for uniformity and completeness in sitemap URLs, including the presence of HTTP/HTTPS protocols and ‘www’ if applicable.

Be judicious when selecting pages to include in the sitemap, ensuring high-quality pages are indexed and avoiding overloading search engine resources which might neglect important content.

Updating your sitemap manually

Updating your sitemap manually might be tedious, but it’s worth the effort to ensure all important pages are included. CMS content can have sitemap updates automated through external programs that hit the CMS API, utilizing a date field for the <lastmod> tag.

For static pages without an easy last modification date, automating sitemap updates can involve using the ‘last full published’ date or omitting the date altogether.

However, manual sitemap updates are necessary for new or updated website content like blog posts and product pages not automatically captured by CMS-generated sitemaps.

Best practices for sitemap maintenance

Sitemap maintenance isn’t a one-off task; rather, it’s a continuous process. Conducting regular reviews of a sitemap ensures that it remains an accurate representation of the website’s structure.

The ‘priority’ attribute within a sitemap can be used to indicate the importance of pages, with values ranging from 0.1 to 1.0, guiding search engines to prioritize critical content.

To optimize crawl budget allocation and prevent confusion for search engines, follow these steps when creating a sitemap:

1. Include only URLs that you want search engines to index
2. Exclude ‘noindex’ URLs from the sitemap file
3. Place the sitemap in the root directory of the website, typically at /sitemap.xml, for easy access by search engines

By following these steps, you can ensure that your sitemap is effective in helping search engines understand and index your website.

Regular review schedules

When maintaining an effective sitemap, consistency is key.

Establish a regular review schedule for your sitemap to ensure that it remains updated with the most recent changes to your website’s content.

As a rule of thumb, regularly audit at least once a month, or more frequently if your website changes often, to maintain an accurate and effective sitemap.

Handling dynamic content

Sitemap maintenance poses a unique challenge when dealing with dynamic content. A sitemap model designed as a tree structure can streamline management of dynamic content across your site.

Sitemap trees can be rendered into site navigation menus, which also facilitate the creation of XML sitemaps.

Adherence to the standard sitemap protocol as outlined on sitemaps.org is crucial for the XML sitemap to be effectively crawled and indexed by search engines.

Make your sitemap work for you with regular updates

Your website needs a reliable map to guide search engines and users. A well-structured, updated and accurate sitemap serves that purpose.

Whether you’re updating manually, relying on automation, or integrating directly with your CMS, the key is consistency. Regular reviews and clean updates protect your SEO, increase website visibility and improve UX.

Keep those XML sitemaps updated and let them guide the way to your content.

Frequently asked questions