Blog Documentation Try Vanilla Cloud
Please upgrade here. These earlier versions are no longer being updated and have security issues.
HackerOne users: Testing against this community violates our program's Terms of Service and will result in your bounty being denied.
Options

after july decrease my alexa rank and visiter count : about robots.txt

ChanuxChanux Admin of Geek.lk ✭✭
edited August 2012 in Vanilla 2.0 - 2.8

this is my robotes URL http://www.geek.lk/robots.txt

After july decrease my alexa rank and visiter count . but it is not true. after few days google says..

Over the last 24 hours, Googlebot encountered 301 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 15.8%.

If the site error rate is 100%:
Using a web browser, attempt to access http://www.geek.lk//robots.txt. If you are able to access it from your browser, then your site may be configured to deny access to googlebot. Check the configuration of your firewall and site to ensure that you are not denying access to googlebot.
If your robots.txt is a static page, verify that your web service has proper permissions to access the file.
If your robots.txt is dynamically generated, verify that the scripts that generate the robots.txt are properly configured and have permission to run. Check the logs for your website to see if your scripts are failing, and if so attempt to diagnose the cause of the failure.
If the site error rate is less than 100%:
Using Webmaster Tools, find a day with a high error rate and examine the logs for your web server for that day. Look for errors accessing robots.txt in the logs for that day and fix the causes of those errors.
The most likely explanation is that your site is overloaded. Contact your hosting provider and discuss reconfiguring your web server or adding more resources to your website.

what can i do.. please help...

Answers

Sign In or Register to comment.