How to edit Robots.txt file

Share this post!

Vote this!

Before starting let’s firs get an insight on what a Robot.txt file is -Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.These instructions may be to hide some information or to let bots to access some, totally depends on your requirement.

Following are a few examples of what you can type in a robots.txt file.                                         more...