Welcome, Guest.
Please login or register.

Welcome to the E-Blah Community!
We would like to welcome you to our community and invite you to register an account or login.
Being a registered member is important, as it gives you several advantages over the normal Guest status. After registering you will be able to download files and images, post messages, and access member-only portions of the forum - just to name a few. Registration is quick and simple, and only takes about a minute of your time.

E-Blah Community  ›  Technical Support  ›  Forum Support  ›  How do I disable bots from connecting .
Users Browsing Forum
No Members and 1 Guests

How do I disable bots from connecting .  This thread currently has 330 views. Print
1 Pages 1 Recommend Thread
Larry
August 5, 2008, 3:04pm Report to Moderator Report to Moderator

E-Blah Member
Posts: 26
Posts Per Day: 0.03
Time Online: 6 hours
My provider keeps disabling my CGI access because Yahoo and other bots are opening an alarming amount of connections, 500 in 2 days time!
My question is, how do I disable bots access to my forum? I thought I saw a setting in the Forum settings at one time but now with the latest version, I can not find anything relating to bot access.
Maybe I am dreaming?
Any help here will be appreciated!
Thanks
Logged Offline
Private Message Private message
iCONICA
August 5, 2008, 6:14pm Report to Moderator Report to Moderator

Forum Moderation
Posts: 826
Gender: Male
Posts Per Day: 2.32
Reputation: 97.30%
Reputation Score: +36 / -1
Time Online: 11 days 16 hours 56 minutes
Location: Manchester UK
You can restrict bots from accessing your site/forum by using a robots.txt file in your site root. using the following code in a robots.txt ascii uploaded plain text file.
Code
User-agent: *
Disallow: /


Should tell all bots "go away"... But 500 connection requests per two days from bots isn't a big strain on server resources I'd think...

I'd guess this forum gets that a minute at least...

Hope that helps!!!



The Best Free Online Technical Support Community! (Click Here)
i-CONICA's Reputation: Increase
Visit the "Transparent Glass" Tech Support Forum.

Revision History (1 edits)
iCONICA  -  August 5, 2008, 7:07pm
Logged Offline
Site Site Private Message Private message Windows Live Messenger WLM Reply: 1 - 5
Larry
August 5, 2008, 10:56pm Report to Moderator Report to Moderator

E-Blah Member
Posts: 26
Posts Per Day: 0.03
Time Online: 6 hours
I did put that line into my robots.txt file, however I still see yahoo and google connecting at the bottom along with myself and other users. Is that the norm?
Logged Offline
Private Message Private message Reply: 2 - 5
iCONICA
August 6, 2008, 2:00pm Report to Moderator Report to Moderator

Forum Moderation
Posts: 826
Gender: Male
Posts Per Day: 2.32
Reputation: 97.30%
Reputation Score: +36 / -1
Time Online: 11 days 16 hours 56 minutes
Location: Manchester UK
The bots will still visit, the second they arrive they are logged at the bottom along with you and other users, they'll then read the robots.txt file and go elsewhere without indexing your pages.

You really should speak to your host about this though, the TINY load that the bots pose on server resources shouldn't be an issue and this is meaning you can't have your site indexed by the search engines...


You can also put the following in the head of your template.html file as a secondary measure to the robots file...

Code
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">



The Best Free Online Technical Support Community! (Click Here)
i-CONICA's Reputation: Increase
Visit the "Transparent Glass" Tech Support Forum.
Logged Offline
Site Site Private Message Private message Windows Live Messenger WLM Reply: 3 - 5
Justin
August 6, 2008, 5:23pm Report to Moderator Report to Moderator

The E-Blah Developer
E-Blah Programmer
Posts: 15,080
Gender: Male
Posts Per Day: 6.42
Reputation: 93.42%
Reputation Score: +298 / -21
Time Online: 37 days 1 hours 13 minutes
Location: Tallassee, AL
Age: 22
I think it had to do with click logging, which CAN increase server load ...


I do installs for $25 and upgrades for $20.
Technical support is always free.

  Donate to E-Blah!  

My Websites: Revolution Reality (My Blog)  |  MinistryTalk.com  |  Portfolio

"But you, O Lord, are a compassionate and gracious God, slow to anger, abounding in love and faithfulness." — Psalm 86:15 NIV
Logged Offline
Site Site Private Message Private message Reply: 4 - 5
iCONICA
August 7, 2008, 8:37am Report to Moderator Report to Moderator

Forum Moderation
Posts: 826
Gender: Male
Posts Per Day: 2.32
Reputation: 97.30%
Reputation Score: +36 / -1
Time Online: 11 days 16 hours 56 minutes
Location: Manchester UK
Of course but not by any amount that should worry even a small shared host..  



The Best Free Online Technical Support Community! (Click Here)
i-CONICA's Reputation: Increase
Visit the "Transparent Glass" Tech Support Forum.
Logged Offline
Site Site Private Message Private message Windows Live Messenger WLM Reply: 5 - 5
1 Pages 1 Recommend Thread
Print

E-Blah Community  ›  Technical Support  ›  Forum Support  ›  How do I disable bots from connecting .

Thread Tags
My,  provider,  disabling,  CGI,  access,  Yahoo,  bots,  opening,  alarming,  amount,  connections,  ,  500
View Tag Cloud

© 2001-2008 E-Blah
About Us - Contact