Browse Source

Block moz.com dotbot from crawling the site

merge-requests/60/head
Deimos 6 years ago
parent
commit
f87d84759d
  1. 3
      tildes/static/robots.txt

3
tildes/static/robots.txt

@ -1,2 +1,5 @@
User-agent: BLEXBot User-agent: BLEXBot
Disallow: / Disallow: /
User-agent: dotbot
Disallow: /
Loading…
Cancel
Save