Like This Site? 
RSS Feed Follow Us 

on Twitter! Be Our Fan!

How to edit Robots.txt file

Share this post!
 Vote this!

Before starting let’s firs get an insight on what a Robot.txt file is -Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.These instructions may be to hide some information or to let bots to access some, totally depends on your requirement.
Following are a few examples of what you can type in a robots.txt file.                                         more...


Post a Comment