1.9 KiB
1.9 KiB
HTTP
User Agents
Clients blocked by user agent are given a 471 response code.
| Match | Added | Reason | Expires |
|---|---|---|---|
undefined |
1970-01-01 | NULL | Never |
~libwww-perl |
? | Library often used with scrapers that don't respect robots.txt. | Never |
~Go-http-client |
? | Most likely a Matrix server (DDoS). | Never |
~Synapse |
? | Matrix server AKA DDoSer. | Never |
~(Condu(wu)?it|Tuwunnel) |
2025-05-09 | Matrix server AKA DDoSer. | Never |
~Byte(spider|dance) |
? | Chinese. Doesn't respect robots.txt. DDoSer. | Never |
~AhrefsBot |
? | Doesn't respect robots.txt. | Never |
~PetalBot |
? | Chinese. Doesn't respect robots.txt. | Never |
~Kinza |
? | Doesn't respect robots.txt. | Never |
~Mb2345 |
? | Chinese. Doesn't respect robots.txt. | Never |
~LieBaoFast |
? | Chinese. Doesn't respect robots.txt. | Never |
~Semrush |
? | Scraper that doesn't respect robots.txt. | Never |
~serpstatbot |
? | Webstat bot. Abusive. | Never |
~ClaudeBot |
2025-05-09 | AI Scraper. | Never |
~AmazonBot |
2025-05-09 | AI Scraper. | Never |
~GPT |
2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never |
~OpenAI |
2025-05-09 | AI Scraper. | Never |
~DotBot |
2025-05-09 | Chinese scraper. | Never |
~AsyncHttpClient |
2025-05-09 | Scraper that doesn't respect robots.txt. | Never |
~Expanse |
2025-05-09 | Scraper with a really bad user agent. | Never |
~CheckHost |
2025-04-02 | Webstat bot. Abusive. | Never |
~MJ12 |
2025-04-02 | Scraper. Does not respect robots.txt | Never |
Referals
Clients blocked by referer are given a 472 response code.
| Match | Added | Reason | Expires |
|---|---|---|---|
~(soyjak.(st|party)) |
? | Attempted Brigading with CP material. | Never |
~gusfckschulz(.wixsite.com)? |
? | Attempted Brigading with CP material. | Never |
~onionfarms.com |
? | No reason given. | Never |