User-agent: * # One Request every ten seconds (Extended syntax) Request-rate: 1/10 # Wait 10 seconds between requests (Yahoo - bah!) Crawl-delay: 10