Main page
|
Register
|
robots.txt file
Disallow indexing of specific directories and pages
Create robots.txt file in your root directory and copy this into file:
...create as many as you need disallow rules or remove some if needed
Write full path to your file or directory from your root directory
* sign stands for all robots
Disallow indexing of specific directories and pages for specific bots
Create robots.txt file in your root directory and copy this into file:
Disallow indexing of all files
Create robots.txt file in your root directory and copy this into file:
Allow indexing of all files - Telling robots to visit all files
Create robots.txt file in your root directory and copy this into file: