# Disallow all crawlers access to certain pages User-agent: * Disallow: /admin/ Disallow: /cgi-bin/ Disallow: /docs/ Disallow: /images/ Disallow: /include/ Disallow: /lang/ Disallow: /media/ Disallow: /test/ Disallow: /themes/ Disallow: /checkout.php Disallow: /global.php Disallow: /libsecure.php