Print Topic - Archive

E-Blah Community  /  Forum Updates  /  Making Your Forum Bot Friendly
Posted by: Justin, September 11, 2005, 1:02pm
This tuturial uses the Apache mod_rewrite modification to your forum in versions of E-Blah greater than Platinum 9.3.

This post is for users wishing to make their forums bot friendly.

First and formost, versions of E-Blah prior to Platinum 9.3 are NOT bot friendly, therefore this tuturial will not be benifical to you until you upgrade.

First thing you should do is create a Google Sitemap.  A good tuturial for doing this can be found here: The CLiX Network.

The next thing you should do is tell Google NOT to index the following directories (as they shall be called).  To do this, we must make a robots.txt file.  This file is simple, yet very effective.  Google likes to index all of your pages.  This is both good and bad.  For instance, you don't want all the quote pages indexed so when users search for "How to use Quote" they get a listing of 2,000 pages refrencing the actual quote page.  You also don't want your post page, period, to be index.  This is how your robots.txt file should look:


Code
User-agent: *
Disallow: /forum/v-post/
Disallow: /blahdocs/
Disallow: /forum/v-report/
Disallow: /forum/v-recommend/
Disallow: /forum/v-memberpanel/
Disallow: /forum/v-members/
Disallow: /forum/v-cal/
Disallow: /forum/v-search/
Disallow: /forum/v-register/
Disallow: /forum/v-login/
Disallow: /forum/v-print/a-1/



Of course if you are using, for instance, /bb/ for your directory use /bb/v-post/.

This code will be updated periodically so that you can use block various diffrent URL's which are not needed in the index.  All you really want in your index is the Board Index, Message Index, and Message Display pages.

Applying this simple scheme will help your forum be indexed by Google, and, hopefully, gain you new members and also help Google figure out with pages to index and not index.  Over the next few weeks I hope to get the eblah.com domain cleaned up of trash on Google and get this new scheme of things working.  I also plan to add a Google site search link so that users can search via Google and -- probably -- get more results than if using the E-Blah built in search engine.  Stay tuned for that.  :)

- Justin
Posted by: Tim Linden, September 11, 2005, 1:05pm; Reply: 1
Hey

Make sure you create the sitemap AFTER doing the mod_rewrite thing, so your sitemap contains only the new stuff ;-)

Tim
Posted by: Craig, September 14, 2005, 2:36pm; Reply: 2
...and of course Justin will have to realease 9.3 before any of this is useful ;D
Posted by: Justin, December 4, 2005, 1:22pm; Reply: 3
More info on bots:

http://www.eblah.com/forum/m-1133721011/s-0/
Print page generated: December 3, 2008, 3:01pm