Browse Source

Add "http.rb" to bot user agent substrings

This will cover Mastodon as well as some other Ruby-based scrapers.
merge-requests/85/head
Deimos 5 years ago
parent
commit
4bfa896ecf
  1. 1
      tildes/tildes/request_methods.py

1
tildes/tildes/request_methods.py

@ -24,6 +24,7 @@ def is_bot(request: Request) -> bool:
bot_user_agent_substrings = (
"bingbot",
"Googlebot",
"http.rb",
"qotnews scraper",
"Prometheus",
"Python-urllib",

Loading…
Cancel
Save