stunthanger.com
General control line discussion => Open Forum => Topic started by: RC Storick on May 27, 2012, 12:33:38 PM
-
Seems SSW is having issues again.
-
I think he is trying to get rid of the "bot". Hope it works.
-
I think he is trying to get rid of the "bot". Hope it works.
Bot? Whats that?
-
Bot? Whats that?
In this case it seems to be something that is crawling his website and indexing it, for some reason. It's the same sort of thing that enables Google to see what is on a site and then make the words available to be searched. May be malicious or may be innocent but has bugs.
Brett
-
In this case it seems to be something that is crawling his website and indexing it, for some reason. It's the same sort of thing that enables Google to see what is on a site and then make the words available to be searched. May be malicious or may be innocent but has bugs.
Brett
The bot you speak of is on another server not his. I am not sure if your saying someone downloaded it to his site? Seems I have fixed the harvesting of email address problem we had here. They were trolling us and looking to harvest email addresses. That is why I had it closed for so long. It was hard to find the answer. I am sure a pro scripter could have fixed it quick but I am just a mechanic.
-
I think the problem is something was using up all his space. He pays for it himself. I read where he might have to shut down to get it to move on. Len has been there since "Isky's" ( and yes my Honda runs good). I am sure the archives the fine.
-
Let's get the Round-Up out and spray the "crawleis" and the "bots".....keep our Yard clean.
Dave.
ama 41041
-
I think the problem is something was using up all his space.....
Excessive number of "build thread" photos, perhaps?
-
Excessive number of "build thread" photos, perhaps?
Mike;
No, its not that. Wish it were that simple.
Search engines around the world are crawling sites to index what is on them. It has gotten so pervasive that total bandwidth has gone from about 1 GB/month to 9GB/month in the course of 12 months. And, Leonard has to pay for that.
Based on some info that Leonard got yesterday, we instituted some changes, and it looks as if the bandwidth usage has dropped off exponentially, albeit the outage today. The outage was actually my fault, as we had to make some changes that did not reflect properly in the configuration file. Simple fix, and all seems to be well. We will see if the changes limiting search engine bots gets the desired results.
Bob
-
Just exactly what did you do, Bob?
-
Just exactly what did you do, Bob?
Bill;
Bill,
Nothing fancy, just followed some of the guidance from these web site:
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449
http://www.robotstxt.org/robotstxt.html
Seems to be working, as the bandwidth trending from the last two days shows a huge drop in bandwidth, yet the logs show that user activity on the site to be about the same, save for an outage yesterday.
I never realized that these web crawling robots were as invasive on bandwidth as the are.
Bob
-
Too bad they can't catch these idiot nerds and hang them by their gonads. VD~
-
Too bad they can't catch these idiot nerds and hang them by their gonads. VD~
Yep....