rotate egress over a larger WG pool, with graceful drain and live status
Keep the N always-on tunnel slots fixed but let each slot's WireGuard config
rotate through a larger pool, so a 10-concurrent provider cap (e.g. Proton) can
still cycle 50-100 profiles.
- lxc/rotate.sh + weircon-rotate.{service,timer}: round-robin one slot at a
time through wg-pool/, repointing a symlink and restarting only that slot.
- service: proxyManager tracks per-slot in-flight + drain/undrain state; a
localhost admin server (WEIRCON_ADMIN_LISTEN) lets rotate.sh drain a slot
before teardown and warm it back in after, so no request is routed to a
tunnel mid-rotation. Slots self-heal if undrain never arrives.
- GET /status: poll-friendly JSON of per-slot egress IP/state plus inferred
next-rotation slot + ETA, fed by a background egress-IP prober.
- docs + env examples for all new knobs.
This commit is contained in:
Asger Weirsøe
@@ -56,6 +56,39 @@ Use it whenever you need an HTTP-facing API that returns results from rotating e
|
||||
|
||||
Tunnel count is configurable. Default deploy uses 10 tunnels; the WireGuard configs themselves can come from any provider that gives you a standard `[Interface] / [Peer]` config (ProtonVPN, Mullvad, AzireVPN, self-hosted, …). Drop the configs into `/etc/weircon-random-proxy/wg/proxy<N>.conf` and the rest is identical.
|
||||
|
||||
### Rotating over more configs than tunnels
|
||||
|
||||
A slot (netns `proxy<N>`, fixed IP, fixed SOCKS port) is decoupled from the WireGuard config it carries. `netns-up.sh` just reads `wg/proxy<N>.conf`, and that path can be a **symlink** into a larger pool. So if your provider caps concurrent connections (ProtonVPN allows 10) but you hold 50–100 configs, keep the 10 always-on slots and rotate *which config each slot points at*:
|
||||
|
||||
```
|
||||
/etc/weircon-random-proxy/
|
||||
wg-pool/ ← all your configs (proton-001.conf … proton-100.conf)
|
||||
wg/
|
||||
proxy0.conf -> ../wg-pool/proton-047.conf ← symlink, repointed over time
|
||||
…
|
||||
proxy9.conf -> ../wg-pool/proton-088.conf
|
||||
```
|
||||
|
||||
`weircon-rotate.timer` fires `rotate.sh`, which advances **one slot at a time** to the next unused config in the pool and restarts only that slot. The other 9 stay up, so there is always egress; over a full cycle every config gets used. Because `weircon-proxy@.service` tears the old tunnel down before raising the new one, a slot is never doubly-connected — you never exceed the provider's concurrent limit, even mid-rotation.
|
||||
|
||||
**Graceful drain.** So a request is never handed to a tunnel that's about to disappear, `rotate.sh` coordinates with the service over a localhost-only admin API before touching a slot:
|
||||
|
||||
1. `POST /drain?id=N` — the service stops routing **new** requests to slot N and blocks until that slot's in-flight requests finish (bounded by `WEIRCON_DRAIN_TIMEOUT_SEC`), plus a short `WEIRCON_DRAIN_SETTLE_SEC` quiet period.
|
||||
2. The symlink is repointed and the tunnel unit restarted.
|
||||
3. `POST /undrain?id=N` — slot N is returned to service, but held out of the candidate set for `WEIRCON_DRAIN_WARMUP_SEC` so the fresh WireGuard handshake can settle before traffic resumes.
|
||||
|
||||
Random picks skip draining/warming slots automatically; a request that *pins* a slot mid-rotation gets `503` + `Retry-After`. If `rotate.sh` ever dies between drain and undrain, the slot self-heals after `WEIRCON_DRAIN_MAX_HOLD_SEC` so the pool can't shrink permanently. The admin server binds `127.0.0.1:8081` by default — keep it off your reverse proxy. `GET /status` returns a per-slot snapshot (in-flight / draining / warming).
|
||||
|
||||
```sh
|
||||
cp proton-*.conf /etc/weircon-random-proxy/wg-pool/ # the pool (not in git)
|
||||
weircon-rotate init # seed 10 distinct symlinks
|
||||
systemctl restart weircon-proxy@{0..9}.service
|
||||
systemctl enable --now weircon-rotate.timer # auto-rotate (default every 5 min)
|
||||
systemctl edit weircon-rotate.timer # change interval (OnUnitActiveSec)
|
||||
```
|
||||
|
||||
The fetch service is unaware of any of this — it still sees 10 fixed SOCKS endpoints. The only observable change is that the egress IP behind a given `X-WEIRCON-PROXY-ID` drifts over time. (Trade-off: restarting a slot drops requests in flight on that one slot for the ~1–2 s WireGuard handshake; one-at-a-time rotation keeps the blast radius to a single slot.)
|
||||
|
||||
## Client API
|
||||
|
||||
Fetch a URL through a random tunnel:
|
||||
@@ -97,6 +130,37 @@ curl -H "X-WEIRCON-RANDOM-IP: $API_KEY" \
|
||||
|
||||
Status, headers, and body of the upstream response are returned directly. The service adds `X-WEIRCON-EGRESS-PROXY: <id>` so the caller can see which tunnel was used.
|
||||
|
||||
### Live status (`/status`)
|
||||
|
||||
`GET /status` returns a poll-friendly JSON snapshot for scraper-side dashboards — cheap to produce (no network I/O in the request path; egress IPs come from a background prober) and safe to hit every 1–5 s. It's behind the same API-key gate as `/`.
|
||||
|
||||
```bash
|
||||
curl -H "X-WEIRCON-RANDOM-IP: $API_KEY" https://random-proxy.example.com/status
|
||||
```
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"now": "2026-06-01T08:47:18Z",
|
||||
"proxies_total": 10,
|
||||
"available": 9, // slots currently selectable (not draining/warming)
|
||||
"proxies": [
|
||||
{ "id": 0, "state": "live", "inflight": 2, "egress_ip": "185.x.x.10", "reachable": true, "latency_ms": 42, "checked_age_sec": 7 },
|
||||
{ "id": 1, "state": "draining", "inflight": 0, "egress_ip": "185.x.x.11", "reachable": true, "latency_ms": 51, "checked_age_sec": 9 },
|
||||
// …
|
||||
],
|
||||
"rotation": {
|
||||
"next_slot": 2, // which slot rotates next (round-robin)
|
||||
"next_egress_ip": "185.x.x.12", // the egress IP about to be replaced
|
||||
"eta_sec": 137, // seconds until the next rotation
|
||||
"interval_sec": 300,
|
||||
"last_slot": 1, // slot that rotated most recently
|
||||
"last_rotated_age_sec": 163
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Per-slot `state` is `live` (selectable), `draining` (in-flight finishing, about to be torn down), or `warming` (new tunnel up, settling before traffic resumes) — drive your visual "rotating now" indicator off the latter two. The `rotation` block tells you *when* the next rotation lands and *which* egress IP it will replace, so you can show a countdown and flag the outgoing IP ahead of time. The `rotation` fields stay absent until enough rotations have happened to infer the cadence (≈ two timer ticks after boot); egress fields are absent when probing is disabled.
|
||||
|
||||
## Built-in API tester (`/ui`)
|
||||
|
||||
The fetch service ships with an embedded HTML page at `/ui` that documents the API and lets you exercise every endpoint interactively:
|
||||
@@ -119,10 +183,14 @@ The UI is enabled by default. Set `WEIRCON_UI_ENABLED=false` in `fetch.env` to d
|
||||
| `lxc/setup-container.sh` | Runs **inside** the LXC: apt deps, microsocks, helper scripts, systemd units. |
|
||||
| `lxc/netns-up.sh` | Brings one tunnel namespace up (`proxy<N>` with wg0 + veth + bridge). |
|
||||
| `lxc/netns-down.sh` | Tears it back down (invoked by systemd ExecStopPost). |
|
||||
| `lxc/rotate.sh` | Rotates one slot to the next pool config; `init` seeds the slots. |
|
||||
| `lxc/systemd/weircon-proxies.target`| Grouping target for all tunnels. |
|
||||
| `lxc/systemd/weircon-proxy@.service`| Templated unit. Enable with `weircon-proxy@{0..N-1}.service`. |
|
||||
| `lxc/systemd/weircon-rotate.service`| Oneshot that runs `rotate.sh` (one slot per invocation). |
|
||||
| `lxc/systemd/weircon-rotate.timer` | Fires the rotation on an interval (default 5 min; `systemctl edit` to change).|
|
||||
| `lxc/systemd/weircon-fetch.service` | The fetch service itself. Hardened (DynamicUser, no capabilities). |
|
||||
| `lxc/fetch.env.example` | Default env. Copy to `/etc/weircon-random-proxy/fetch.env`. |
|
||||
| `lxc/rotate.env.example` | Rotation env (`SLOTS`). Copy to `/etc/weircon-random-proxy/rotate.env`. |
|
||||
| `npm/advanced.conf` | Reverse-proxy snippet: header check, strip, forward. |
|
||||
| `wg-configs/` | Local placeholder. Actual `.conf` files live on the LXC, **not** in git. |
|
||||
|
||||
@@ -137,6 +205,14 @@ The UI is enabled by default. Set `WEIRCON_UI_ENABLED=false` in `fetch.env` to d
|
||||
| `WEIRCON_PROXY_COUNT` | `10` | Number of fallback endpoints. |
|
||||
| `WEIRCON_REQUEST_TIMEOUT_SEC` | `30` | Per-upstream-fetch ceiling. |
|
||||
| `WEIRCON_UI_ENABLED` | `true` | Toggle the embedded `/ui` page. |
|
||||
| `WEIRCON_ADMIN_LISTEN` | `127.0.0.1:8081` | Localhost drain/undrain API for rotation. Empty disables it. |
|
||||
| `WEIRCON_DRAIN_TIMEOUT_SEC` | `25` | Max wait for a slot's in-flight requests to clear during drain. |
|
||||
| `WEIRCON_DRAIN_SETTLE_SEC` | `2` | Quiet period after in-flight hits zero (the "couple secs before").|
|
||||
| `WEIRCON_DRAIN_WARMUP_SEC` | `3` | Slot held out of rotation after undrain (the "couple secs after").|
|
||||
| `WEIRCON_DRAIN_MAX_HOLD_SEC` | `60` | Auto-undrain safety if `rotate.sh` dies between drain and undrain.|
|
||||
| `WEIRCON_PROBE_URL` | `https://api.ipify.org` | IP-echo URL the prober fetches through each tunnel for `/status`. |
|
||||
| `WEIRCON_PROBE_INTERVAL_SEC` | `60` | Egress-IP probe cadence. `0` disables probing (`egress_ip` empty).|
|
||||
| `WEIRCON_PROBE_TIMEOUT_SEC` | `10` | Per-probe ceiling. |
|
||||
|
||||
If `WEIRCON_PROXY_ADDRS` is set it wins; otherwise the service computes `WEIRCON_PROXY_HOST:WEIRCON_PROXY_BASE_PORT + i` for `i` in `[0, WEIRCON_PROXY_COUNT)`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user