# robots.txt, based on the file # /usr/share/doc/packages/apache2/robots.txt # # strictly configured to disallow everything User-agent: * Disallow: / #Disallow: /manual/ #Disallow: /doc/ #Disallow: /gif/ # but allow htdig to index our doc-tree #User-agent: susedig #Disallow: # disallow stress test #user-agent: stress-agent #Disallow: /