Вот хороший вариант файлика robots.txt для владельцев сайтов DataLife Engine:
User-Agent: * Disallow: /shop/CID_*.html?v=&f=2&s=1* Disallow: /shop/CID_*.html?v=&f=2&s=2* Disallow: /shop/CID_*.html?v=&f=2&s=3* Disallow: /shop/CID_*.html?v=&f=1&s=1* Disallow: /shop/CID_*.html?v=&f=1&s=2* Disallow: /shop/CID_*.html?v=&f=1&s=3* Disallow: /shop/CID_*.html?v=&f=2&s=6* Disallow: /shop/CID_*_ALL.html Disallow: /xls/ Disallow: /java/ Disallow: /newptip/ Disallow: /search/ Disallow: /install/ Disallow: /print/ Disallow: /pda/ Disallow: /rss/ Disallow: /pricemail/ Disallow: /spec/ Disallow: /users/ Disallow: /compare/ Disallow: /gbook/ Disallow: /?skin=red_classic Disallow: /?skin=phpshop_2 Disallow: /?skin=phpshop_3 Disallow: /?skin=phpshop_1 Disallow: /?skin=phpshop_5 Disallow: /??skin=aeroblue Disallow: /??skin=pblue_classic Disallow: /??skin=grass Disallow: /??skin=gray Disallow: /??skin=green_classic Allow: /shop/ Allow: /price/ Allow: /page/ Allow: /UserFiles/Image/ Allow: /map/ Allow: /news/ Allow: /order/ Sitemap: http://VASH_SITE.ru/sitemap.xml Host: VASH_SITE.ru
Читайте также:
• Делаем robots.txt
• Robots.txt для WordPress
• Robots.txt для PHPbb
• Robots.txt для DLE
• Robots.txt для IPB