bots

[apples4u]

I do see what you are saying there. Double the visitors half the bw. half the bw double the visitors LOL

It is very interesting if you ask me. Some could be the size of the images, text, banners, etc...

Originally posted by Matt
Looking at it further, even more useless statistics.

MSNBot 28830 572.32 MB
Googlebot 13327 496.83 MB
Inktomi Slurp 12451 1.13 GB

It's clear that while googlebot eats twice the bandwidth as msnbot, Inktomi Slurp uses twice that of googlebot.

As you can see, I'm bored today

[apples4u]

as of a site I just put online this past week.

Robots/Spiders visitors (Top 25) - Full list - Last visit
7 different robots* Hits Bandwidth Last visit
Unknown robot (identified by 'spider') 50 426.06 KB 08 Aug 2008 - 11:56
Googlebot 30 212.73 KB 06 Aug 2008 - 15:59
Alexa (IA Archiver) 11 250.39 KB 08 Aug 2008 - 11:35
Yahoo Slurp 10 91.30 KB 07 Aug 2008 - 06:56
Unknown robot (identified by 'crawl') 4 144.88 KB 06 Aug 2008 - 00:09
Unknown robot (identified by 'robot') 1 56.41 KB 06 Aug 2008 - 13:18
Unknown robot (identified by 'bot/' or 'bot-') 1 9.80 KB 07 Aug 2008 - 03:56

[trader]

Hi Matt, I would advise you to not even pay attention to the bots because (unless it gets extremely excessive with bandwidth from bad bots, which is unlikely) there is not much that can be done or should be done due to lots of wasted time and frustration involved. Keep in mind the PPC firms, Adsense and AWstats ignore bot traffic anyway.

[saviourdlima]

If Googlebot comes to your sIte and cannot access It, your sIte may drop. we realized it was important to provide the ability for users to submit pages on their site and get real-time feedback on what Googlebot sees.

[michaljohn]

A 'bot' is a type of malware which allows an attacker to gain complete control over the affected computer. Computers that are infected with a 'bot' are generally referred to as 'zombies'. One of the best defenses against threats like these is to run anti-virus and anti-spyware software on your computer.

[shawnchin]

Really awesome but my question is that they are really helpful for SEO and if they are can we use artificial crawler?

[deadman]

Great counts n numbers using bots


Good everypne. Can i know how can be the bots used

[jdilegge]

IMO, waste of time. They don't use enough bandwidth to really hurt you unless you have a really bad host. Just write good content and don't worry about spiders, people tend to get paranoid when monitoring them too much.