47 lines
2.5 KiB
Markdown
47 lines
2.5 KiB
Markdown
# HTTP
|
|
|
|
## User Agents
|
|
Clients blocked by user agent are given a 471 response code.
|
|
| Match | Added | Reason | Expires |
|
|
| - | - | - | - |
|
|
| `undefined` | ? | Somebody forgot to define something. | Never |
|
|
| `~libwww-perl` | ? | Library often used with scrapers that don't respect robots.txt. | Never |
|
|
| `~Go-http-client` | ? | Most likely a Matrix server (DDoS). | Never |
|
|
| `~Synapse` | ? | Matrix server AKA DDoSer. | Never |
|
|
| `~Byte(spider\|dance)` | ? | Chinese. Doesn't respect robots.txt. DDoSer. | Never |
|
|
| `~AhrefsBot` | ? | Doesn't respect robots.txt. | Never |
|
|
| `~PetalBot` | ? | Chinese. Doesn't respect robots.txt. | Never |
|
|
| `~Kinza` | ? | Doesn't respect robots.txt. | Never |
|
|
| `~Mb2345` | ? | Chinese. Doesn't respect robots.txt. | Never |
|
|
| `~LieBaoFast` | ? | Chinese. Doesn't respect robots.txt. | Never |
|
|
| `~Semrush` | ? | Scraper that doesn't respect robots.txt. | Never |
|
|
| `~serpstatbot` | ? | Webstat bot. Abusive. | Never |
|
|
| `~CheckHost` | 2025-04-02 | Webstat bot. Abusive. | Never |
|
|
| `~MJ12` | 2025-04-02 | Scraper. Does not respect robots.txt | Never |
|
|
| `~ClaudeBot` | 2025-05-09 | AI Scraper. | Never |
|
|
| `~AmazonBot` | 2025-05-09 | AI Scraper. | Never |
|
|
| `~GPT` | 2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never |
|
|
| `~OpenAI` | 2025-05-09 | AI Scraper. | Never |
|
|
| `~DotBot` | 2025-05-09 | Chinese scraper. | Never |
|
|
| `~AsyncHttpClient` | 2025-05-09 | Scraper that doesn't respect robots.txt. | Never |
|
|
| `~Expanse` | 2025-05-09 | Scraper with a really bad user agent. | Never |
|
|
| `~(Condu(wu)?it\|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never |
|
|
|
|
## Referals
|
|
Clients blocked by referer are given a 472 response code.
|
|
| Match | Added | Reason | Expires |
|
|
| - | - | - | - |
|
|
| `~(soyjak.(st\|party))` | ? | Attempted Brigading with CP material. | Never |
|
|
| `~gusfckschulz(.wixsite.com)?` | ? | Attempted Brigading. | Never |
|
|
| `~onionfarms.com` | ? | No reason given. | Never |
|
|
| `~laitrcus.online` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~essociation.de` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~integsoft.com` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~quetek.com` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~postoru.com` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~twitoru.com` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~synology.me` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~spardhavijetha.in` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~totsufe.co.jp` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~365gangguan.cn` | 2025-05-12 | SEO bullshit. | Never |
|
|
| `~slashpet.com` | 2025-05-12 | SEO bullshit. | Never | |