fetch: every outbound request carries one generic browser User-Agent

Stripping forwarding headers kept our IP and chain out of reach, but the
User-Agent went through untouched. Clients that fell back to their own name
("crawlkit-roles", "crawlkit-virk", ...) tied all ten exits together as one
actor, and one missing UA meant Go-http-client. The proxy now overwrites the
UA on every request (WEIRCON_USER_AGENT, default current Firefox) and the
egress prober uses the same string.

Also dropped: From, Sec-Ch-Ua* (would contradict a Firefox UA) and any
X-Weircon-* by prefix. New end-to-end test drives the whole handler and
checks what the target actually receives.

Claude-Session: https://claude.ai/code/session_01AUj1LSMmrowcomkn1ciAgj
This commit is contained in:
agw
2026-09-29 20:14:07 +02:00
parent 0aa56fc67c
commit 06bdd1423f
5 changed files with 116 additions and 3 deletions
+4
View File
@@ -105,6 +105,9 @@ type prober struct {
pool []*http.Transport
url string
timeout time.Duration
// same browser UA as proxied traffic, so the prober's own egress through
// the tunnels isn't a Go-http-client fingerprint on our exit IPs
userAgent string
}
// probe fetches the egress IP for a single slot. A non-2xx response or a body
@@ -116,6 +119,7 @@ func (p *prober) probe(id int) {
p.mgr.setEgress(id, "", 0, false)
return
}
req.Header.Set("User-Agent", p.userAgent)
start := time.Now()
resp, err := client.Do(req)
if err != nil {