Browse Source

Block serpstatbot from crawling the site

merge-requests/86/head
Deimos 5 years ago
parent
commit
d021b38392
  1. 4
      tildes/static/robots.txt

4
tildes/static/robots.txt

@ -51,6 +51,10 @@ Disallow: /
User-agent: SemrushBot-BA
Disallow: /
# https://serpstatbot.com/
User-agent: serpstatbot
Disallow: /
# https://www.zoominfo.com/
User-agent: ZoominfoBot
Disallow: /
Loading…
Cancel
Save