Ever wonder why the results of a Google search lead you not to an article but a login page? How does the site get indexed by Google if it requires a login?

The answer is, sites recognize the "agent" known as Google Bot. They automatically allow Google Bot in so that their content may be included in searches. But when regular yackos like you and I try to surf to the referenced page, it wants us to login in order to see the content.

Here is a simple trick you can do on your computer to make your web browser report itself as the Google Bot. This will work for many website, except those that check your IP address (in which case they may ban you for spoofing!).

1) Using Firefox, get the User Agent Switcher add-on.

2) Once the add-on is installed, go to Tools > Add-ons, select Agent Switcher and click Preferences.

3) Click User Agents, click Add..., and for Description and User agent enter:
Googlebot/2.1 (http://www.googlebot.com/bot.html)

4) Click OK to save and close the preference.

5) Now go to Tools > User Agent Switcher, and select the Googlebot entry you just created. Try surfing some of those sites that wanted you to login and see if it works!

Make sure to switch back to Default when you resume normal surfing.