1.8 KiB
1.8 KiB
HTTP
User Agents
Clients blocked by user agent are given a 471 response code.
| Match | Added | Reason | Expires |
|---|---|---|---|
undefined |
1970-01-01 | NULL | Never |
~libwww-perl |
? | Library often used with scrapers that don't respect robots.txt. | Never |
~Go-http-client |
? | Most likely a Matrix server (DDoS). | Never |
~Synapse |
? | Matrix server AKA DDoSer. | Never |
| `~(Condu(wu)?it | Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. |
| `~Byte(spider | dance)~` | ? | Chinese. Doesn't respect robots.txt. DDoSer. |
~AhrefsBot |
? | Doesn't respect robots.txt. | Never |
~PetalBot |
? | Chinese. Doesn't respect robots.txt. | Never |
~Kinza |
? | Doesn't respect robots.txt. | Never |
~Mb2345 |
? | Chinese. Doesn't respect robots.txt. | Never |
~LieBaoFast |
? | Chinese. Doesn't respect robots.txt. | Never |
~Semrush |
? | Scraper that doesn't respect robots.txt. | Never |
~serpstatbot |
? | Webstat bot. Abusive. | Never |
~ClaudeBot |
2025-05-09 | AI Scraper. | Never |
~AmazonBot |
2025-05-09 | AI Scraper. | Never |
~GPTBot |
2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never |
~OpenAI |
2025-05-09 | AI Scraper. | Never |
~DotBot |
2025-05-09 | Chinese scraper. | Never |
~AsyncHttpClient |
2025-05-09 | Scraper that doesn't respect robots.txt. | Never |
~Expanse |
2025-05-09 | Scraper with a really bad user agent. | Never |
Referals
Clients blocked by referer are given a 472 response code.
| Match | Added | Reason | Expires |
|---|---|---|---|
| `~(soyjak.(st | party))` | ? | Attempted Brigading with CP material. |
~gusfckschulz(.wixsite.com)? |
? | Attempted Brigading with CP material. | Never |
~onionfarms.com |
? | No reason given. | Never |