This page has been robot translated, sorry for typos if any. Original content here.

Banning indexing of a subdomain or how to close a subdomain from indexing in robots.txt, in htaccess

Запрет индексации поддомена или как закрыть от индексации поддомен в robots.txt, в htaccess

Prevent indexing of a subdomain in robots.txt if it is in a subdirectory or subdirectory.

Prevent indexing of a subdomain in robots.txt if it is in the / dir / subdirectory

 User-agent: *
 Disallow: / dir / 

Prevent indexing of a subdomain in htaccess if it is in the / dir / subdirectory

 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteRule ^ / dir /.*$ - [F]

or in /dir/.htaccess:

 RewriteEngine on
 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteRule ^. * $ - [F]

Prevent indexing of a subdomain in robots.txt if it is in the same directory as the site or other subdomains

 RewriteCond% {HTTP_HOST} pod1 \ .site \ .ru. * $
 RewriteRule ^ robots \ .txt $ robots1.txt

Create a robots1.txt put it in the root, and in it:

 User-agent: *
 Disallow: /

Thus we will create several different robots.txt for each subdomain.

Prevent indexing of subdomains in htaccess if it is in the same directory as the site or other subdomains

 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteCond% {HTTP_HOST} pod1 \ .site \ .ru. * $
 RewriteRule ^. * $ - [F]

You can also prohibit all subdomains, except those indicated.

This can be useful in the case of social networks or directories with personal data.

 RewriteCond% {HTTP_USER_AGENT} ^. * Google. * [NC]
 RewriteCond% {HTTP_USER_AGENT} ^. * Yandex. * [NC]
 RewriteCond% {HTTP_HOST} ^ ([a-z0-9 -] +) \. Site \ .ru. * $ [NC]
 RewriteCond% {HTTP_HOST}! Www \ .site \ .ru. * $ [NC]
 RewriteCond% {HTTP_HOST}! Pod1 \ .site \ .ru. * $ [NC]
 RewriteCond% {HTTP_HOST}! Pod2 \ .site \ .ru. * $ [NC]
 RewriteRule ^. * $ - [F]

Where: pod1, pod2 - subdomains that do not need to be prohibited.