# robots.txt gets read by web crawlers. Created this to avoid 404s in the error logs # Syntax: # Use pound for comments # This would disallow all web crawlers: # User-agent: * # Disallow: /