# ALL robots will spider the domain
User-agent:*
Disallow:/cgi-bin/
Disallow:/script-folder/
Disallow:/download-folder/
Disallow:/visits.txt/
Disallow:/Google-robots/
Disallow:/Web-Posting-Information