Choose Your Style
brianhelge.com [blog]

Robots and SEO

Aug 27, 2007
Lately at Big Stone, I've been doing a lot of work to try and boost our SEO for the two sites. I've been doing good lately as our news posts usually end up on the first page of Google within the first day that they are written.

Looking around at this, I've found a bunch of clever ways that are quick and easy to boost you to the top of Google. Within searching for different techniques to use, I found out how to find stuff that web sites try to hide from you, and the spiders, that give Google all of your information.

Almost every major site will have a robots.txt file that tells the spiders what not to allow when they browse through all of your information. So, if you want to find out what ESPN or Google is trying to hide from you when you google something that appears on their site, you can just check the robots.txt file and check out their directories.

Obviously, these sites will have their precious in-house data protected with CHMOD, but you can still have fun trying to see what they are trying to block from you.

Why you can't google your facebook profile, and why eBay hides a Disney folder.
0 Comments | Post URL (#)