From 7c8626afc255293703f91e7a3073d75f2b407234 Mon Sep 17 00:00:00 2001 From: Wirlaburla Date: Mon, 12 May 2025 20:02:17 -0400 Subject: [PATCH] Order user agent list --- http.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/http.md b/http.md index 0726280..19cef08 100644 --- a/http.md +++ b/http.md @@ -4,11 +4,10 @@ Clients blocked by user agent are given a 471 response code. | Match | Added | Reason | Expires | | - | - | - | - | -| `undefined` | 1970-01-01 | NULL | Never | +| `undefined` | ? | Somebody forgot to define something. | Never | | `~libwww-perl` | ? | Library often used with scrapers that don't respect robots.txt. | Never | | `~Go-http-client` | ? | Most likely a Matrix server (DDoS). | Never | | `~Synapse` | ? | Matrix server AKA DDoSer. | Never | -| `~(Condu(wu)?it\|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never | | `~Byte(spider\|dance)` | ? | Chinese. Doesn't respect robots.txt. DDoSer. | Never | | `~AhrefsBot` | ? | Doesn't respect robots.txt. | Never | | `~PetalBot` | ? | Chinese. Doesn't respect robots.txt. | Never | @@ -17,6 +16,8 @@ Clients blocked by user agent are given a 471 response code. | `~LieBaoFast` | ? | Chinese. Doesn't respect robots.txt. | Never | | `~Semrush` | ? | Scraper that doesn't respect robots.txt. | Never | | `~serpstatbot` | ? | Webstat bot. Abusive. | Never | +| `~CheckHost` | 2025-04-02 | Webstat bot. Abusive. | Never | +| `~MJ12` | 2025-04-02 | Scraper. Does not respect robots.txt | Never | | `~ClaudeBot` | 2025-05-09 | AI Scraper. | Never | | `~AmazonBot` | 2025-05-09 | AI Scraper. | Never | | `~GPT` | 2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never | @@ -24,8 +25,7 @@ Clients blocked by user agent are given a 471 response code. | `~DotBot` | 2025-05-09 | Chinese scraper. | Never | | `~AsyncHttpClient` | 2025-05-09 | Scraper that doesn't respect robots.txt. | Never | | `~Expanse` | 2025-05-09 | Scraper with a really bad user agent. | Never | -| `~CheckHost` | 2025-04-02 | Webstat bot. Abusive. | Never | -| `~MJ12` | 2025-04-02 | Scraper. Does not respect robots.txt | Never | +| `~(Condu(wu)?it\|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never | ## Referals Clients blocked by referer are given a 472 response code.