# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: User-Agent: * # Disallow: / Disallow: /authentication/authenticate_fb Disallow: /authentication/authenticate Disallow: /blogs/edit/ Disallow: /blogs/new Disallow: /blogs/newBlogLinks/ Disallow: /blogs/editBlogLinks/ Disallow: /articles/