This page has been robot translated, sorry for typos if any. Original content here.

The prohibition of indexing a subdomain or how to close the indexing of subdomains in robots.txt, in htaccess

Запрет индексации поддомена или как закрыть от индексации поддомен в robots.txt, в htaccess

The prohibition of indexing a subdomain in robots.txt, if it is in a subdirectory or subdirectory.

The prohibition of indexing a subdomain in robots.txt, if it is located in the subdirectory / dir /

 User-agent: *
 Disallow: / dir / 

The prohibition of indexing a subdomain in htaccess, if it is in the subdirectory / dir /

 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteRule ^ / dir /.*$ - [F]

or in /dir/.htaccess:

 RewriteEngine on
 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteRule ^. * $ - [F]

The prohibition of indexing a subdomain in robots.txt, if it is in the same directory with the site or other subdomains

 RewriteCond% {HTTP_HOST} pod1 \ .site \ .ru. * $
 RewriteRule ^ robots \ .txt $ robots1.txt

Create a robots1.txt it put in the root, and in it:

 User-agent: *
 Disallow: /

Thus, we will create several different robots.txt for each subdomain.

The prohibition of indexing a subdomain in htaccess, if it is in the same directory with the site or other subdomains

 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteCond% {HTTP_HOST} pod1 \ .site \ .ru. * $
 RewriteRule ^. * $ - [F]

You can also disallow all subdomains, except those specified.

This can be useful in the case of social networks or directories with personal data.

 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteCond% {HTTP_HOST} ^ ([a-z0-9 -] +) \. Site \ .ru. * $ [NC]
 RewriteCond% {HTTP_HOST}! Www \ .site \ .ru. * $ [NC]
 RewriteCond% {HTTP_HOST}! Pod1 \ .site \ .ru. * $ [NC]
 RewriteCond% {HTTP_HOST}! Pod2 \ .site \ .ru. * $ [NC]
 RewriteRule ^. * $ - [F]

Where: pod1, pod2 - subdomains, which you do not need to ban.