Everyone loves a well-organized website; however, it isn’t worth much when no one can find it. That’s when search engine optimization comes into play. Search engine optimization is much more complicated than we can explain in a single article, but there are some things that every web designer can do to improve their chances of showing up in search. In addition to creating user-friendly sites that keep people on your website longer, you’ll also need to let search engines know you are out there.
Submitting a website to search engines used to be the sole responsibility of SEO firms that help companies increase their online visibility. Now, anyone can do it by creating an XML sitemap.
What are XML sitemapsXML sitemaps are just one type of sitemap. In addition to XML, there are also HTML and visual sitemaps. Sitemaps provide an easy-to-read summary of any website, regardless of its size. XML sitemaps have the added ability of working seamlessly with search engines because they are formatted for reading by bots instead of people.
XML sitemaps can also provide metadata, making indexing of your site more accurate and useful for those using search engines. This type of information can tell search engines when and how often a page is updated as well as the relationship between the page and URLs linking to it.
Except for the recent trend of one-page websites, most web projects have multiple pages. Depending on the size of your site, these pages may or may not be easily discoverable by website visitors. That’s where good information architecture becomes important because it ensures that no one gets lost while trying to navigate your website. When a website wants a user to know which pages are relevant, it simply places a link on the homepage. Search engines do the same, except they also scan and consider large amounts of the internet (instead of a single site) and place links on their pages.
Search engines are dedicated to scanning the internet and collecting links to relevant data. When given an XML sitemap, their job is easier because they already have a summary of your site. Whereas search engines like Google can easily see your websites’ homepage or crawl the internet and stumble upon your top-level domain, they may not know about that one page that you have buried deep within your site. That is unless you tell them about it.
How XML sitemaps help websitesAn XML sitemap provides a map to Google and any other search engine that you’d like to be noticed by. It tells search engines which pages to crawl so that they can be considered for indexing. This can make a significant difference in the SEO of large websites, and as a result, the practice of using XML sitemaps is highly recommended by Google.
However, using XML sitemaps is optional. You don’t have to give search engines a sitemap. They are perfectly capable of deciding what information is important and which information is not (according to Google). But they don’t mind if you give them a little cheat sheet as to where to look.
In other words, an XML sitemap can help direct search engine crawlers to the information you want them to find and promote. It’s free and easy to do, but there’s no guarantee that it will sway the search bots in your favor. But, you have nothing to lose, because you will not be penalized for doing it. So why not make one?
Nothing to see here
Sometimes you don’t want search engines to see everything. For example, you may have created a page that is only accessible after paying a fee. If you give that page to Google and they find it interesting, they may list it for everyone to see (and access) without paying. Thankfully, there is a way to prevent this by using XML sitemaps.
The “no-index/no-follow” option tells search engine bots which information to ignore. No-index means the page should not be included in search results. No-follow means you do not want to be associated with the link that is leaving from your page. Both serve as ways to influence the type of information that is considered for inclusion in search engine results.
Why is this important? Well, the sad truth is, no one really knows exactly what search engines consider when they rank sites. Perhaps a few people know, but those people may get replaced periodically as far as we know, with the formula changing soon after that. However, over time, search engine specialists have learned that the overall content on a site matters just as much as the pages that will ultimately be listed. So, if your site is full of what they deem as ‘low-quality content,’ it may be categorized as low-quality as well; even if your home page is incredibly helpful, SEO-optimized and easy to navigate.
In that case, the no-index option could significantly improve your website’s chances of being ranked, because you could block pages that aren’t quality enough to be considered. Even if you have 100 pages on your site with more than half of them being low SEO quality, you could tell search engines to only consider those that you want it to. The XML sitemap would override the actual content on the site, allowing your site to be judged the way you’ve intended.
Better rankings with XML
Ranking on sites such as Google, becomes more difficult each year. As the number of websites grow, so do the number of searches. Increasing your site’s chance at being listed on the first page of a search is important, because many websites depend on search engine traffic.
XML sitemaps can help improve ranking, though not on their own. They need to be created strategically, with the proper use of no-follow and no-index protocols. You can also prioritize pages so that search engines do the same. If your site isn’t ranking high and you feel like it should be, create an XML sitemap and see if some pages should not be crawled.
Maybe you’ve done that already, but you aren’t sure which pages are being considered. If you submitted an XML sitemap to Google, you can see how many of your submitted pages have been indexed using the Google Search Console (a part of Google Webmaster Central). A sitemap that includes many submitted pages that do not get indexed should be reviewed for low quality pages and checked for spam comments. You may also think about optimizing any images.
XML sitemaps and Google Panda
Search engines change their algorithms as often as they deem necessary and Google Panda is one of many utilized by Google to determine the quality of a site (other algorithms include Hummingbird and Penguin). Panda focuses on the quality of the content within a site and it was first released in 2011.
Google Panda addressed a growing problem online – duplicate content. At the time, curated content was becoming quite popular, and many website owners found copies of their websites ranking higher than they were.
The best way to win the duplicate content war, is to make it clear to Google who published the content first. The most efficient way to do that is with an XML sitemap. With metadata that includes timestamps, there will be no question about who created it first. Unless you do not bother to create an XML sitemap, and in that case, the website that did may claim original ownership.
How to create an XML sitemap
Many website content management systems can create an XML sitemap for you with little effort. However, you should still double-check their work, because many times sitemaps may contain errors or include information that you would not liked crawled.
If you are creating an XML sitemap for Google, you’ll want to make sure that it fits their requirement. The current protocol is Sitemap Protocol 0.9 and you can read about it on this website.
Here are some of the key call-outs:
- XML sitemaps are written in code; begin with an opening tag and end with a closing tag
- Include the protocol within that tag
- Each URL is a parent tag and every parent has a child
- Use UTF-8 encoding
If none of this makes any sense, you may want to consider using a sitemap generator.
The beauty of XML sitemap generatorsSitemaps are not difficult to create, but they can definitely be a pain because the process can take so long. A better option (especially if you have a large site) is to use a sitemap generator. These generators create dynamic XML sitemaps that can be easily updated for new content as the need arises.
With dynamic sitemaps, you can make changes to existing sites and add new pages without worrying about creating a brand-new XML sitemap. Plus, with Slickplan’s XML sitemap generator you can drag and drop as needed to modify your sitemaps, audit content for quality, and keep track of your site assets in one place.
Plus, you don’t have to worry about using UTF-8 encoding – that has to be worth something.