Managing Robots.txt with Recite CMS
When building your web site, it can be useful to control which content you want to appear in search engines such as Google or Bing. The robots.txt file does exactly this for you.
This file contains a number of directives which most search engines will honour, including which files or directories they can and can't access.
When using Recite CMS, you can easily create the robots.txt using the settings panel.
First, you must create the content of the file. You can use a tool such as Google Webmaster Tools to create the content of the robots.txt file. Once created, you can add to Recite CMS in the following manner:
- Log in to the Recite CMS Control Panel
- Click on Site Settings in the header
- Load the Global Settings settings bundle
- Enter the robots.txt content in the labelled text area
- Click Save Settings
Now when you visit the http://www.example.com/robots.txt file in your web browser you will see this data.