Submit Sitemap to MSN, Google and Yahoo

In November 2006, Google, Yahoo! and MSN joined forces to support a new industry standard for sitemaps: Sitemaps 0.90 (see MSN announcement here). As long as webmasters follow the protocol, they can ensure their sites are fully and consistently indexed across all the major search engines (a real step forward). This article is important for all those with missing or poorly ranked pages.

The official site for the joint venture is at http://www.sitemaps.org/ and contains a lot of info about the new standard and it's syntax. What the site singularly fails to do is explain correctly how to submit your sitemap to the big three! The format suggested on the site of:

http://search-engine-url/ping?sitemap=your sitemap_url

does not currently work at any of the three sites! Until it does, We provide instructions for how to (a) create your sitemap and (b) how to submit to each of the three search engines...

Creating your Sitemap

Some hosting providers(for example 1and1) provide utilities via their web control panel, to create your sitemap, so you should always check with your provider first. If this service is not available, then make a visit to http://www.sitemapspal.com/ and enter your site URL into the generator box. Copy-and-paste the resulting sitemap into notepad, then save-and-upload to your site with the file name: sitemap.xml

If you want to validate the XML prior to uploading to the search engines (useful if you have made any manual amendments), pay a visit to the SmartIT XML Sitemap validator where you can put in the URL of your sitemap and check it against the standard.

Submit sitemap to MSN

MSN have yet to implement a formal interface for Sitemap submission (as at April 2007). To monitor the situation, please visit (from time to time) the following pages:

MSN Site Owner Help
Livesearch official blog (where future announcements are likely to be found)

Whilst MSN have yet to implement a front door, there is a recognised back door for submitting your sitemap to the MSN Search index; namely moreover.com! You should use the following syntax directly in your browser URL box:

http://api.moreover.com/ping?u=http://yourdomain.com/yoursitemap.xml

For example, to submit the sitemap for this site, the correct entry in the browser is as follows:

http://api.moreover.com/ping?u=http://www.seo-expert-services.co.uk/rss.xml

Since February 2005, moreover.com have been the official provider of RSS feeds to the myMSN portal (see press release) and reliable evidence suggests that submission to moreover.com will result in MSN spidering your pages within 2-3 weeks.

Update 18/07/07: MSN still do not support direct submission but now suggest on their blog that you add a reference to your Sitemap into your robots.txt file (something now supported by sitemaps.org). For example:

User-agent: *
Sitemap: http://www.yourdomain.com/sitemap.xml
Disallow: /cgi-bin/

This would tell MSN (and all other engines) to crawl your sitemap file but not to crawl your cgi-bin directory. For more info on how to implement a robots.txt file (in the root of your site webserver) please visit: http://www.robotstxt.org/

Submit sitemap to Google

Google originally developed the XML schema for sitemaps and have developed a dedicated portal for webmasters, from where you can submit your sitemap:

http://www.google.com/webmasters/

First, you need to tell Google all the sites you own, then verfiy that you indeed own them. The verifiaction is achieved by adding a metatag between the head tags on your site homepage. The syntax for the tag is as follows:

<meta content="unique code advised by google" name="verify-v1">

There are full instructions on how to do this on the Google site.

Submit sitemap to Yahoo

Yahoo follows a similar approach to Google. Again, there is a dedicated sevice for webmasters (Yahoo! Site Explorer) and a procedure for verifying your ownsership of the site. First go to:

http://siteexplorer.search.yahoo.com/

Add a site, then click on the verification button. You can then download a verfication key html file - which you will need to upload to the root directory of your webserver. Then you can return to Site Explorer and tell Yahoo to start authentication. This will take up to 24 hours. At the same time you can also add your sitemap by clicking on the manage button and then adding the sitemap as a feed.

Submit sitemap to Ask

Ask follows a simpler approach to the other three. To submit you sitemap, you simply enter a ping URL, followed by the full URL of where your sitemap is located:

http://submissions.ask.com/ping?sitemap=http%3A//www.yourdomain.com/sitemap.xml

After clicking return, you will get a reassuring message from Ask that they have received your submission. Very neat!