How Search Engine Spiders Index Pages

1. The robot goes out and Grab Tags & Links (like hitting view source)
2. Robot brings back text and links. At this point, they cannot automatically inject page rank because of spam, which would give inaccurate results.
3. Robot then adds all collected text and links into a temporary table. At that point the robot's Job is done.
4. The search engine goes in and determines if you're a spammer. It goes from guilty until proven innocent. (Because 90+% of SE submissions are spam)
5. Step 1: SE checks against a black list
Step 2: SE checks for keyword stuffing
6. Once SE determines your not spam, it moves the collected information into a clean list or non-spam table
7. At this point, what is known as the SE Algorithm kicks in, which is a sequence of steps the SE takes to determine where your page is placed among all the other pages
8. Algorithm checks how many links are outgoing, how many links are incoming, how many links are relevant and other factors that come into play.
9. The next step is they analyze your title tag, keyword tag, description tag and content. (Does not apply to all SEs)
10. Next the SE scores your pages, then they have a nice compiled table of new scores, but you are not yet in the database.
11. Last step is the SE updates the new data, usually on a weekend. If you are already in the database, the old date is updated with this new data affecting the score (or ranking) of your page

Liked this Post? Help us by voting now! Thanks

Del.icio.us (0) | Digg | StumbleUpon | Reddit | Dzone