mirror of https://gitlab.com/tildes/tildes.git
Browse Source
This seems to be a separate Semrush bot that doesn't listen to robots.txt for the main SemrushBot UA. Their site also says to block "SemrushBot-SA" and doesn't mention "-BA", but I don't know if that's a mistake on their end or if this won't work.merge-requests/85/head
Deimos
5 years ago
1 changed files with 4 additions and 0 deletions
Loading…
Reference in new issue