# Robots.txt file # # Contains instructions for search-engine robots coming to the site... # # Basically we want to allow access to all files except the /cgi-bin ones # User-agent: * Disallow : /cgi-bin/