User-agent: * Disallow: /WebResource.axd* # Do not index axd handlers (see FogBugz case 625) Crawl-delay: 5 # Wait 5 seconds between each request (non-standard extension) Request-rate: 1/5 # Only request 1 page every 5 seconds (extended standard) Visit-time: 2100-0900 # Only visit between 9pm and 9am GMT (extended standard)