A bot is an automated program that accesses webpages to read information. The Juggling Edge operates a bot named JugglingEdgeBot, which if you are reading this you have probably seen in your server logs (hello!).
If you are seeing a similar user agent string that is unique to your website, you will have already had details of what it is doing. If I have not been in touch then the traffic is not from the Juggling Edge.
Once per month
JugglingEdgeBot crawls all links posted to the forum and harvests the page title eg.
<title>This text here</title>
This text is used to provide a link on the monthly Green Eggs reports. One visit from JugglingEdgeBot equals one link from my site to yours.
If you would like to know what people have been saying about your website you can use the search page to query the URL that was crawled.
JugglingEdgeBot will check the root directory of your website for a robots.txt file (eg. https://www.yoursite.com/robots.txt) and will honour the instructions found there.
To explicity allow JugglingEdgeBot access to all pages (love you!) use:
User-agent: JugglingEdgeBot Allow: /
To explicity disallow JugglingEdgeBot access to all pages (sniff) use:
User-agent: JugglingEdgeBot Disallow: /
You can also allow/disallow access to specific pages. For example to allow JugglingEdgeBot access to your index page only use:
User-agent: JugglingEdgeBot Disallow: / Allow: /index.php
For more information about using a robots.txt file please see http://www.robotstxt.org/. To check the validity of your robots.txt file try the Motoricerca robots.txt checker.
If you have any problems with JugglingEdgeBot please let me know.