# Robots.txt # For domain: http://www.mightydwarf.com # All robots will spider the domain User-agent: * Disallow: /cgi-bin/ Disallow: /webalizer/ Disallow: /cp/ Disallow: /widgets/ Disallow: /images/