Browse like the Googlebot

I needed to browse a few sites I was managing, but I needed to see what the Googlebot actually saw. I'd used Lynx (http://lynx.browser.org) to see what the site looked like in text, but that wouldn't show me any responses that the server had specifically for Google.

Rather than play with my IE or Maxthon settings, it was faster to add the option to Firefox:

1. Start Firefox and open about:config
2. Right-click and add a New, String
3. Call the string: general.useragent.override
4. Create the value: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
5. Done!

You can now check your site's behaviour when the Googlebot comes a'crawlin'.

This is handy if your webdev team have promised that particular content is only visible to Google, or if a lazy webdev team have decided that someone who has hacked their useragent string ought to be able to see all content unrestricted... ;)

If you want more information on how user-agent parameters have evolved, as well as a great list of agent strings, head over to the beautiful Wikipedia: http://en.wikipedia.org/wiki/User_agent for more details.