Index All Your Pages

Does your sitemap have too many links? That may prevent Google from indexing all your pages.

A sitemap provides a one-click access to all the pages in your site. Over time, however, your sitemap can have hundreds if not thousands of links, because your site can grow. Up to a certain point, search engine robots may have a hard time indexing all the pages in your web site.

Google recommends not having more than 100 links on each of your web pages. Some webmasters have had no problems with having 400 links on a page.

But why tempt fate, right?

That's why I've decided to break up my huge Secondthoughts Sitemap. The original long page is still available (Secondthoughts Archives), but to help search engines index all the pages here, I've put up a more manageable Secondthoughts Sitemap page.

Will this actually work? Let's give it some time and see. As of today, Google has only indexed about half of the posts here in Secondthoughts. It would be interesting to find out if this sitemap rationalization move will improve indexing.

(If it's true that Google spiderbot will only see 101kb or 100 links in a web page, then that probably explains why Google has indexed half of the pages posted in the 190kb Secondthoughts Archive page.)

[ First posted on 06/24/2006 by Manuel Viloria ]