Whenever Ajax enter the conversation for an SEO or internet marketer, chances are good there will always be a deep sigh or an “ugh” face. While it is true that search engines are getting better at indexing this type of content, we still aren’t at the point that you can realistically rely on them to index it properly or even at all. That doesn’t mean you can’t use it, it just means you need to take some extra steps to make sure that type of content is visible to crawlers and non Ajax users.
What you need to do is create specific unique URLs for each of those destinations. These URLs need to provide the information in way that ALL the search engine spiders can read and extract, not just the advanced experimental Ajax crawling spider from Google. This insures you will get traffic from Yahoo, Bing, Facebook, Twitter, Stumbleupon, and, heck, even services like Blekko* and Wolfram Alpha. Relying on just one search engine or source for your traffic is a dangerous strategy and not defensible in the whims of an algorithm update.(*Update Blekko notified me they have an Ajax Crawler)
Once you have each of those pages, you want to make sure the URL is as search engine friendly as possible: short with 3-5 keywords in the URL and without parameters. While it’s a bit of overkill, providing the rel=”canonical” tag is a good idea as well.
Where things get a little tricky is inbound linking, email, social media links, and user agent detection. Whether someone is viewing the Ajax version of the content or the static version of the content, you should provide a “link to this page,” “share this page,” or “email this page” functionality, and that should always go to the static URL.
While this may seem like a bit of double work, if you use Ajax properly, it’s probably not. You pull the same information from the same database–it’s only the method of rendering that changes. Flash, on the other hand, will be a bit more problematic, and would probably require a bit of double work. Therefore, it’s not a method I recommend. One of the primary reasons it’s a good idea to pull the data from same DB is it insures you don’t create a “bad cloaking” situation. Technically, cloaking is serving different content to the spiders and to the engines. If the actual content is the same, and it’s just the delivery technology and implementation that is the only difference, you have a low risk, highly defensible position. Especially if you use the canonical tag to nudge the spiders in the direction of the real URL.
Once you have the static URL in place, you need to provide a method for the search engines to see and access that content. You can use HTML sitemaps and XML sitemaps, but ideally you need to set up dedicated crawling paths. Unless your site is very small (less than a few hundred pages), I would suggest a limited test first. You should roll this out in phases on non mission critical sections of pages first. Use text browsers, text viewers, crawlers like Xenu link sleuth, or website auditor. Lastly, I would suggest setting up a monitoring page for use with services like change detection and/or Google alerts. It’s important that you know if something “breaks” or “jumps the rails” within 24 hours, not 30 days later when 70% of your content has dropped out of the index.
The last issue you want to consider is internal duplicate content. It’s not entirely unlikely that if the “Ajax crawling bot” finds its way to your pages, you don’t want them to be interested in it and index the content in that format. Using the rel=”canonical” tag that points to a static non-ajax URL will help, but I’ d also suggest the noindex, follow meta tags on the Ajax pages, just to be safe. Leaving things open to search engines to decide is where problems come from … sometimes BIG and EXPENSIVE problems …
So what are the takeaways from this post:
- Ajax isn’t evil, but the implementation is going to be more difficult and complex, so be smart about how you do it
- Province distinct static unique URLs that are accessible from the Ajax pages
- Use user detection to serve the best version OF THE SAME content
- Use spider simulators to insure you are calling the right version
- Use change detection and monitoring to detect problems with indexing quickly and correct them before your website falls off the map.