Browse Source

robots.txt: Block OpenAI GPTBot

master
Deimos 10 months ago
parent
commit
f355bc6a80
  1. 4
      tildes/static/robots.txt

4
tildes/static/robots.txt

@ -30,6 +30,10 @@ Disallow: /
User-agent: dotbot
Disallow: /
# https://platform.openai.com/docs/gptbot
User-agent: GPTBot
Disallow: /
# Unknown/suspicious scraper - UA only contains a gmail address
User-agent: MauiBot
Disallow: /

Loading…
Cancel
Save