firegex/AGENTS.md

462 lines
159 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# AGENTS.md
Guidance for any coding agent working in this repository, and the one copy of it. Tool-specific
files — `CLAUDE.md` and anything like it — point here rather than repeating this, because two
copies of a document this long drift the first time one of them is edited, and the drift is
invisible: each agent reads its own file and neither is told the other now says something else.
That already happened once, and the stale copy went on describing rules that had been replaced.
## What this is
Firegex is a firewall built for CTF Attack-Defense competitions: it sits in front of a vulnerable service and blocks/mangles malicious traffic before it reaches it. Behind a single FastAPI backend and React frontend it ships **services** (a network layer plus an ordered chain of filters) and plain nftables rules (`firewall`). That is the whole product surface: `nfregex`, `nfproxy`, `tls` and `porthijack` were folded into `services` as choices rather than modules.
## Commands
### Running the whole app (Docker or standalone)
```bash
python3 run.py # build+start (Docker if available, else standalone chroot on Linux)
python3 run.py --standalone # force standalone mode (no Docker)
python3 run.py stop [--clear] # stop (optionally wipe the DB volume/rootfs)
python3 run.py restart # recreate the container WITHOUT rebuilding (stale image if source changed)
python3 run.py status
python3 run.py config --password [newpass] # change the running instance's password; omit the value to be prompted interactively
python3 run.py config --show # print the persisted settings (port, host, socket dir, auth)
python3 run.py start --unsafe-disable-auth # no authentication at all — see "Authentication" below
python3 run.py restart --env FGEX_PROXY_FILTER_TIMEOUT_MS=5000 # anything run.py has no opinion about
```
`--env KEY=VALUE` (repeatable, `KEY=` removes one) is the *only* supported way to set what the engine and the backend read straight out of their environment — `FGEX_PROXY_FILTER_TIMEOUT_MS`, `FGEX_STREAM_MAX_SIZE`. (`FGEX_PROXY_QUIC_ALPN` was the example here; it is gone — the engine reads each client's list off its first packet, see QUIC below.) It exists because `run.py` **rewrites** `.firegex-compose.yml` on every start, so a variable hand-edited into it is gone by the next one, which is what the QUIC documentation's "set it in the container's environment" silently ran into. It is stored in `.firegex-conf.json` like the other flags, reaches the standalone chroot as well as the container, and refuses the names `run.py` sets itself (`MANAGED_ENV`) rather than letting two writers of one variable decide by list order.
`run.py` has zero external dependencies (stdlib only) by design. `restart` runs `docker compose up -d --force-recreate`, so it re-reads the generated compose file and re-applies env vars (`ALLOWED_IPS`, `PROXY_IP_HEADER`, `UNSAFE_DISABLE_AUTH`) — a plain `docker compose restart` would keep the old container environment and silently ignore those. It still does **not** rebuild the image; after a source change use `stop` then the bare start command to force `docker compose up -d --build`.
Most `start`/`restart` flags are persisted to `.firegex-conf.json` and become the default for later runs, so a bare `python3 run.py start` inherits whatever was last passed.
### Frontend (`frontend/`)
```bash
bun install # bun.lock is canonical
bun run dev # vite dev server
bun run build # tsc && vite build
npx tsc --noEmit -p . # typecheck only (no separate lint script is configured)
```
### Backend (`backend/`)
Needs Linux with `NET_ADMIN` (nftables/nfqueue) and root, so it's normally only run via `run.py` (Docker or standalone). Direct iteration on a Linux box/VM:
```bash
cd backend && DEBUG=1 python3 app.py DEBUG
```
### Tests (`tests/`)
**One suite, one runner: `pytest`.** Everything drives a **live, already-running** firegex (`http://127.0.0.1:4444` by default), root/Linux, except `unit/` which needs nothing running at all.
```bash
cd tests
sudo ./run_tests.sh # installs deps, waits for the instance, runs pytest
sudo ./run_tests.sh --set-pass # against one still in its initial-setup state
sudo ./run_tests.sh unit # the ones needing no instance
pytest -k udp # one subject
pytest --layer proxy --no-ipv6 -x # narrow, stop at the first failure
pytest standalone # restarts the instance; opt-in, see below
```
**Run it as root, through the script.** Most of the suite only drives the API over HTTP and
works fine without, which is why it was run unprivileged for a long time — but what needs
root then *skips*, and a case that always skips proves nothing. `sudo ./run_tests.sh` is
what makes those run, and going through the script rather than `sudo pytest` is what makes
it work at all: `pip install --user` put every dependency in the invoking user's
site-packages, which a root process does not read, and the editable `firegex` is a `.pth`
there — a file only processed for *site* directories, so putting that path in `PYTHONPATH`
imports pytest and still leaves `pyworker.py --check` unable to find the filter library.
The script installs into the environment it is running in, which resolves all of it. Two
things that trip a root run were found by trying it, and both are fixed rather than worked
around: `tests/requirements.txt` was missing `fastapi` (see the file for why one unit test
needs the backend's own dependency, and why its absence stops the *whole* run rather than
one file), and `./run_tests.sh unit` — advertised in the script's own help — took `unit` as
a **password** and ran everything against an instance it could not authenticate to.
| Directory | Needs an instance | What it holds |
|---|---|---|
| `unit/` | no | `firegex.regex` against libhs, the `firegex.pyfilters` models, the address→nftables translation, and `fgex pyfilters` run as its own process in front of a local echo (`test_proxysim.py` — nothing else ran the simulator) |
| `integration/` | yes | the product, parametrised over the combinations |
| `standalone/` | yes, and restarts it | settings read once at process startup |
| `helpers/` | — | the API client, stand-in services, certificates, the traffic `Channel` |
| `bench/` | yes | measurement, not testing; `pytest` does not collect it |
**The combinations are parameters, not a shell loop.** `integration/conftest.py` holds `Layer(transport, ipv6, tls)` and the fixtures that fan out over it — `inspecting_layer`, `filtering_layer`, `proxy_layer`, `tls_layer`, `external_layer`, `any_layer`. Running both network layers with the same filters is the point: a filter that only works on one of them is exactly the failure the unified model exists to prevent. This was a shell script invoking one 1100-line program eight times, which meant the first failure ended the run and the other seven combinations were never reached, and a host without IPv6 on loopback failed rather than skipped.
Three things that have to stay true of this layout:
- **`helpers/`, never `utils/`.** The backend's own package is called `utils`, and this directory's used to be too — so whichever imported first won, and `unit/test_nftables_addresses.py` (which imports the backend) and everything importing `helpers.firegexapi` could not be collected in one pytest process. That name collision is why the suite *could not* be unified before.
- **The interception canary.** It is perfectly possible to reach the API of an instance whose datapath protects a different machine — a container without host networking, a VM with the port forwarded. Every call returns `ok` and not one packet goes near any of it, which without a check reads as dozens of "the filter did not block" failures rather than as a suite pointed somewhere it cannot reach. `datapath_in_our_path` asks once, plainly, and everything needing interception is skipped with the reason.
- **Skip with a reason rather than fail.** A laptop with no IPv6 loopback, a non-root run, an instance elsewhere — none of those are firegex bugs, and `-ra` is in `addopts` so the summary says which half of the suite the machine declined.
`Channel` (`helpers/traffic.py`) is the one place that knows whether TLS is in the way. The old suite branched on it at every traffic check, about twenty times.
`helpers/quicserver.py`'s `h3_is_blocked` counts **only a closed connection** as a block, because that is what a refusal on that layer does; a malformed request, an unreachable port and a timeout are failures with nothing to do with a filter, and catching them too let "the filter blocked it" pass for reasons that are not a block. It found one immediately: `authority()` brackets an IPv6 literal, and without that every IPv6 exchange was answered `H3_MESSAGE_ERROR` by the far end and read here as a refusal.
Run the suite **twice in a row** when changing anything about addresses or TLS: state left behind by the first pass only shows up on the second.
`standalone/` is opt-in because it drives `run.py stop`/`start` itself — the IP allowlist and the boot-time auth mode are read once at startup, so there is no way to test them against a running instance. It restores an unrestricted one at the end, which every other module depends on.
`tests/integration/test_grpc.py` and `tests/integration/test_http_versions.py` are the two that say whether HTTP/2 is really filtered. The second is **the** test of the whole rendering: one filter file, unchanged, in front of an HTTP/1.1 service, an HTTP/2 one and an HTTP/3 one, blocking the same request on all three — it can only pass because each version is shown to the chain as the same HTTP/1.1. `grpcio` is an optional dependency alongside `aioquic`, and everything needing it skips with a reason, because a gRPC service is the honest way to stand up a real HTTP/2 service with an HPACK-compressed path and a trailer-section status.
The Rust datapath has its own suite, which needs no running instance but does need the production toolchain (vectorscan and Python 3.14 — the filter library imports `compression.zstd`):
```bash
docker build --target compiler -t firegex-compiler .
docker run --rm -v "$PWD/backend:/execute" -w /execute/proxysrc firegex-compiler cargo test
cd backend/proxysrc && cargo test # on a host with libhs + fgex installed
cargo test --release --test throughput -- --ignored --nocapture # the matcher benchmark
```
Mount `backend/`, not `backend/proxysrc/`: `tests/pyworker.rs` launches `../modules/services/pyworker.py`. That file imports `firegex` and the library imports `brotli`, neither of which the compiler stage installs — without them on `PYTHONPATH` eight pyworker tests fail with a `ModuleNotFoundError` buried in captured output and read as real failures. The runtime image has both (`/usr/local/lib/python3.14/site-packages` and `/usr/local/lib64/...`), and the repo's own `fgex-lib` has to come *first* on the path or a stale image's `firegex` wins.
The suite runs against an instance with authentication turned off as well: `login()` reads a 403 there as "there is nothing to log in to" rather than as a failure, since every request is already allowed. `integration/test_auth.py` is the exception and skips itself — it is about passwords, and those endpoints answer 403 in that mode by design. Note `BearerSession.post` copies the header dict rather than writing into it: the form `Content-Type` belongs to the login request alone, and it used to stick, which went unnoticed only because a successful login replaces the whole dict a moment later.
CI (`.github/workflows/release.yml`) runs **only on `release: published`**, not on pushes or pull requests — nothing validates a PR automatically. Two jobs gate every publish job below them, both on `ubuntu-latest` and `ubuntu-24.04-arm`: `run_tests` does `python3 run.py start -P testpassword` then `cd tests && sudo ./run_tests.sh`, and `engine_tests` runs the Rust suite (`cargo test` in `proxysrc`) in the Dockerfile's `enginetest` stage — the compiler stage plus the filter library the pyworker tests start. That suite was not run before a release at all, though much of what it covers (HTTP/2, HTTP/3 and QUIC termination, the worker's framing, the UDP relay) no integration test reaches. `enginetest` sits **before** `final` on purpose: a build with no target, which is what compose does, takes the last stage. The tests' dependencies are pure python; `iperf3`, matplotlib and numpy belong to `bench/requirements.txt` and CI installs none of them.
## Architecture
### Three tiers
React (Vite + Mantine + react-router + TanStack Query + socket.io-client) → FastAPI backend → per-transport datapath binaries that do the actual interception. The backend mostly orchestrates: SQLite-backed config, spawning/talking to those processes, and deploying the built React app.
### The layers are compared in the interface, not only in the docs
`components/Services/LayerChoice.tsx` puts the trade in front of the operator where the choice is made: a table of what each layer costs, plus the paragraph for the one selected. `transportSummary()` carries the same sentence onto every service row and detail page. Keep them in step with `docs/services.md` — the point is that an operator never has to read documentation to find out that one layer reassembles in the kernel and the other has a kernel-guaranteed fail-open. There used to be a `losesClientAddress()` beside them, marking the one combination that gave the client's address up; source transparency is unconditional on every layer now, including UDP, so it answered `false` for every input and the badge it gated could not render. Do not reintroduce it without a combination that actually loses something.
### The two layers (`backend/modules/services/`)
The central idea, and the thing to preserve when changing anything here: **how traffic is intercepted and what is done with it are independent choices.**
- **Network layer** (`transports.py`): `nfqueue` — packets queued to userspace, verdict handed back, nothing terminated, kernel fail-open via `NFQA_CFG_F_FAIL_OPEN` plus `bypass` on the nft rule. `proxy` — the connection is terminated and reopened, which buys exact rewriting, kernel reassembly, backpressure and an ordered chain, and costs the kernel backstop (rebuilt by hand in `proxysrc/src/filter.rs`).
**The proxy is not the slow one, and the docs used to say it was.** Measured: 2.2× NFQUEUE's bulk throughput at one thread and 4.7× at eight, because NFQUEUE pays a userspace round trip *per packet* while the proxy pays once per connection and then the kernel moves the bytes. `NTHREADS` reaches the Rust engine too since 5.0.0 — `main.rs` builds its tokio runtime with that many workers rather than `#[tokio::main]` taking every core, so `run.py --threads` means one thing on both layers instead of one thing on one and nothing on the other. On short connections the two are indistinguishable — that claim cost three contradictory measurements before `tests/bench/connbench.py` started interleaving the layers within each pass instead of measuring one and then the other, which was handing the machine's drift to whichever went second. The trade between the layers is fail-open and transparency, not speed; keep `docs/services.md`, `LayerChoice.tsx` and `transportSummary()` saying that. `external` — a stateless destination rewrite that puts a proxy the operator wrote in the path (this is what `porthijack` used to be); no firegex process is involved, so `check()` refuses any filter, since a filter attached there would be one that never runs.
- **Filter layer**: `regex` (patterns compiled into one hyperscan database) or `pyfilter` (the user's Python). A service holds an ordered list of them.
`Transport.check()` is where an impossible combination is refused **before** anything starts: more than `MAX_CHAIN_POSITIONS` filters on `nfqueue`, any filter at all on `external`, and the `tls` protocol anywhere but the proxy layer. It is asked twice: at creation with an empty chain, for what the service alone makes impossible, and again whenever a chain is pushed. UDP was never removed from the datapath: `cppregex` runs in block mode for it and `cpproxy` handles it, and the only thing that had gone missing was the choice in the UI.
**Telling the Python worker about a connection never waits on it.** `connection_opened`/`connection_closed` are called from the relay itself, on the async runtime; they used to take the worker's lock there — held by whatever exchange was in flight, up to its whole deadline — and start the worker if it was not running, so one connection feeding a hanging filter held every new connection of the service on the threads carrying all the others. They queue their frames now (`PyWorkerRule::pending`, capped) and the next exchange, which runs on a blocking thread, writes them ahead of its chunk. A ruleset reload prepares its filters on a blocking thread for the same reason: starting a worker is an interpreter start-up.
**UDP on the proxy layer (`proxysrc/src/udp.rs`).** The TCP path recovers where a connection was headed with `SO_ORIGINAL_DST`, which the kernel implements **for TCP and SCTP only** (`ENOPROTOOPT` otherwise). So UDP is relayed with **one socket per protected address**, each with its upstream fixed — nothing has to be recovered because nothing was lost. The engine prints one `UDP <upstream>|<onward> <port>` line per relay after `PORT`, and `_add_proxy` points each address's rule at its own port.
**One spelling of an upstream, and one owner of the map.** `udp_relay_key()` in `nftables.py` is the only place the `host:port` / `[host]:port` token is built, and `udp_relay_host()` the only place an interface name is resolved to something a relay can bind — IPv4 first and link-local last, because an interface almost always carries an `fe80::` address it was never configured with and binding there puts the service where nobody dials. Both were written out by hand in four and three places respectively, and the engine's map, the rule that points at a relay and the manager adding an address to a running service all have to agree on the token exactly; when they stop, the symptom is a service whose UDP silently goes nowhere. For the same reason `ProxyTransport.udp_ports` is the *only* map of relays: `start()` returns that dict as `_steer["udp_ports"]`, so it is the same object the rules read, and `ServiceManager.address_added` asks `add_udp_target()` for a relay rather than keeping a second copy of the answer beside it.
**A relay is named by where it sends *and* what it speaks there** — `udp_relay_slot()`, `<host:port>|<onward>`, and the engine's `QuicManager` keys the same pair. Keyed on the upstream alone, two addresses reaching one service port with different answers (HTTP/3 relayed as QUIC to `udp/443` beside HTTP/3 turned into HTTPS on `tcp/443`, both real) got one relay between them, and an address whose `upstream` was **edited** on a running service was handed the relay it already had: the edit was saved, shown back as a tag, and reached nothing until a restart. Now the edit opens the relay for the new answer and repoints the rule; the old relay stays bound and idle until the service restarts, the same as a removed address's.
Client address transparency is preserved via `transparent::connect_as_udp` using `IP_TRANSPARENT` with mark `0x1339` and policy routing `table 1339`. New UDP addresses can be added to running services dynamically via `ADD_UDP` over the control channel without restarting the proxy.
What UDP keeps: per-flow filter state keyed by client address and released after `IDLE` — UDP has no close to observe, so a timeout is the only thing that ends a flow — and both directions inspected. `Verdict::Reject` drops the datagram rather than closing anything, because there is nothing to close. Replies leave through the **listener** socket so conntrack rewrites them to appear from the address the client dialled; sending from the upstream socket would reach a client not expecting that source. The relay's own dial carries `SELF_MARK`, or the redirect rule would catch it and loop. A flow outlives its service restarting: a datagram sent while the service was down comes back as ICMP "port unreachable", which the flow's connected socket reports once as `ConnectionRefused`, and `replies` carries on past it. It used to end the task reading the service's answers while the flow stayed in the table, and a client that kept talking kept the flow from ever going idle — so it never heard from the service again, while a new client did (`tests/udp.rs::a_flow_outlives_its_service_restarting`).
`ConnectionMeta.l4` carries the transport to the filter, so the library's stream and HTTP models key off what the engine says rather than a hardcoded `True` in the worker. It is an enum and not a boolean because QUIC broke the question in two: a QUIC stream is ordered and reliable, so every model built on a stream applies to it, and it rides on UDP, so every rule that matches the wire matches UDP — one flag answering both got one of them wrong whichever way it was set. The worker is sent both words (`is_tcp` for the wire, `l4` for what can be built on it) and `RawPacket` exposes `l4` and `is_stream` beside `is_tcp`; `is_stream` is what `models/tcp.py` and `models/http.py` gate on. Absent, `l4` derives from `is_tcp`, because the NFQUEUE binaries carry TCP and UDP and have nothing else to say.
### A service is a list of addresses
`service_addresses` holds `(ip, port, proto)` rows; the service itself holds the name, transport, protocol and TLS material. One daemon routinely answers on more than one address — a v4 and a v6 one, a public and an internal one — and all of them deserve the same chain. Two services with hand-copied chains is how one of them silently stops being protected.
Consequences worth keeping:
- **Adding an address to a running service drops nothing.** `ServiceManager.address_added` installs only that address's rules against the datapath already running (`self._steer` holds what to point at). On TCP and TLS the proxy listener is dual-stack from the start, and on UDP new relays are bound dynamically on the fly, so adding any address (IPv4, IPv6, UDP, TCP, TLS) never drops connections or restarts the service.
- **`address_removed` drops the address from `self.srv.addresses` itself** rather than re-reading the table, because the caller takes the rules back *before* deleting the row. Re-reading would put it straight back, and the next restart would re-protect an address that no longer exists — with TLS, by claiming a port for a stream nobody owns, which then refuses the next service that wants it.
- **`proto` is denormalised onto each address** and rewritten when the service's changes. That is what makes `(ip, port, proto)` a usable uniqueness key, so a TCP service and a UDP one can share an address exactly as the kernel allows.
- **The `external` proxy endpoint belongs to the address, not the service**, and a partial unique index forbids two addresses sharing one: the return rule recognises the operator's proxy by address and port to put the original port back, so two addresses behind one endpoint could not be told apart on the way out.
- **The proxy listener is dual-stack whenever the host supports IPv6** (`[::]:0`), and `transparent::unmap` turns the v4-mapped peer and local addresses back into IPv4 before anything else in the engine touches them — conntrack, the source-preserving dial and the metadata a filter reads all want the address the packet actually had.
- **Addresses accept network interface names as well as IPs.** An entry in `service_addresses` can be an IP/CIDR or a network interface name (1-15 chars, e.g. `eth0`, `wg0`, `tun0`, `lo`), checked via `utils.is_interface_name` and marked by `Address.is_interface`. The interface is not a second spelling of an address, and the interface is what the operator is choosing when the address is somebody else's to hand out. The frontend says which of the two it understood *while it is being typed* — `addressKind` in `components/InterfaceInput.tsx` puts it on a badge in the field, because `eth-0` is a perfectly good interface name and a perfectly wrong address, and the two are matched by completely different rules. That rule is written once on the frontend too, as `isInterfaceName`/`isIpAddress` in `js/utils.tsx`, reading the same charset the backend does — it used to be pasted into `AddEditService.tsx` and `AddressList.tsx`, two copies of a rule the picker beside them did not consult at all, which is how the hand-off layer came to offer interface names in the list and refuse them on submit. `InterfaceInput` lists interfaces and addresses as two groups, each interface carrying what it holds right now (one question decides between them: "and what is on it"), and stays a combobox rather than a select because an address a service will answer on need not exist on this host yet. **Link-local is not offered** — every interface carries an `fe80::` it was never configured with, which doubled the list with addresses that would put a service where no client looks for it, the same reason `udp_relay_host()` takes them last; one typed by hand is still accepted, and an interface holding nothing else says "link-local only" rather than listing it. A row is **the badge as a column, and everything else in the one beside it** — value, then the note, wrapping inside that column and never under the badge. Neither of those two is given a fixed share, because which one is long changes with the group (the addresses a link carries, or the address itself): fixing it took the whole width from a full-length IPv6 and left the other wrapping a character per line, a row a screen tall beside an address cut off mid-digit; and three items in one wrapping row put that same address at the container's left edge, under the badge, three lines tall. The list's own width is **computed from the longest value in it**, between a floor and a ceiling, and three simpler rules were wrong first: inherited from the content it collapsed to the field's width, because content allowed to wrap asks for less than it needs to stay on one line; pinned to the widest row any host could produce, every list was that wide beside fields half the size; and a floor alone put the one long address back over three lines. A host answering on `10.60.3.1` gets the floor, one carrying a full-length IPv6 gets the room to show it with its note beside it, and monospace is what makes the arithmetic honest. **The row beside them is laid out by the same arithmetic, and got it wrong the same way.** Its note (`now 10.10.0.3`, `on eth0`) is a flex item with a basis wide enough for a full-length IPv6, because a basis is what decides whether an item shares a line or drops to one of its own — at 80px it shared, and the first row with a couple of tags on it squeezed the note into the gap that was left and broke an IPv6 across three lines mid-digit. A min-width beside that basis is not the fix and was tried: on a phone the column is narrower than any floor worth setting, so the floor pushed the address past the card's edge, and it buys nothing anyway — an item only shares a line when its basis fitted, and on a line of its own it is already as wide as it can be. The row's own container grows to what the buttons leave rather than to its contents, or each row is as wide as its own longest line and two rows of one list wrap opposite ways: the one carrying a full IPv6 kept its note inline while the one saying `now 127.0.0.1` pushed it down. `overflowWrap: anywhere` stays as the last resort, which is what a 375px screen needs and nothing wider should reach. **`AddressList` reads in the same words it was chosen in** — the same badge at the same `BADGE_WIDTH`, then the value, then `on eth0` for an address and `now 10.10.0.3` for an interface — because a row an operator comes back to check, described differently from the list it came out of, is a reason to wonder whether it is the same thing. What is *shown* is `bareAddress`: the backend stores what it parsed, so `127.0.0.1` comes back as `127.0.0.1/32`, a spelling nobody types that reads like a range beside real ones; a host-length prefix is dropped for display and a real one is kept, which is also what tells `addressKind` apart. A real range survives an edit for the same reason — the modal seeded itself by cutting at the first slash, so opening "move this address" on a `/16` and saving narrowed a service to one host of it. Where interfaces are not an answer — the `external` proxy endpoint, and the service address on that layer — they are not listed at all and a name typed there is badged red, rather than offered and then refused on submit. Nftables matches use `meta iifname <iface>` for inbound/prerouting and `meta oifname <iface>` for outbound/postrouting/route. On `proxy` (TCP/TLS), redirect rules match `iifname`; the return path needs no per-interface rule, because it follows the conntrack mark (below). **The output hook cannot match either of those, and matches the interface's addresses instead.** `iifname` means nothing for a packet this host generates, and `oifname` is the wrong question twice over: `oifname eth0 dport 80` says *we* are calling somebody else's port 80, which is the host's own outbound traffic and none of firegex's business — and a connection from this host to one of its **own** addresses is routed through `lo` anyway, so it would never match. So `_add_proxy` resolves what the interface carries (`interface_addresses`, link-local dropped for the reason `udp_relay_host` takes it last) and writes one `daddr` rule per address. For a while it wrote nothing there at all, on the correct observation that `oifname` would hijack the wrong traffic, and the consequence was a service intercepted for the outside world and silently not for anything on the box — reported by an operator curling their own service and watching a blocked payload sail through. The cost is that an address the interface gains later is not covered *for traffic this host generates* until the service restarts; inbound keeps following the interface, and `udp_relay_host` pays the same price for the same reason. Each of those rules carries the interface's name as a **comment**, because nothing else in it does: `delete()` finds rules by what they match, and without it every one would outlive the service that installed it as a redirect to a port nobody is listening on. On UDP `proxy`, the primary IP assigned to the interface is resolved dynamically via `psutil.net_if_addrs()` to bind the relay. On `nfqueue`, interface traffic is queued natively in prerouting/postrouting without needing an IP. On `external`, interfaces are disallowed because return rewrites require a concrete IP.
**An address says what is spoken at it and where the service is, and those are two different questions from the one the rules ask.** `service_addresses` carries `edge` — `tcp`, `tls`, `udp` or `quic`, the same words a service speaks minus `http`, which is the one that means "ask the addresses" — and `target_port`, which is where the traffic goes when that is not the port it arrived on. The two are scoped by different things, and confusing the two scopes is what the first version of this did. `edge` is only ever *chosen* on an `http` service, because that is the only protocol whose addresses are not all the same; elsewhere it is the service's own and a value disagreeing with it is refused, while `proto` (what the kernel matches) stays `L4.l4_of(edge)`. `target_port` is a question for the **layer**: publishing is being reachable one more way than the service is, and what that takes is something that *dials*. The proxy terminates the connection and opens the one to the service, so it can open it on another port, whatever the protocol — a plain `tcp` service published on a second port is exactly as real as an HTTPS one, and `_address_target()` was keyed on `http` only because `http` was the protocol the case was first noticed on. NFQUEUE hands the kernel a verdict on packets already on their way and opens nothing; `external` gives the traffic to the operator's own proxy, which decides for itself where the service is. On those two it is **refused** rather than stored, because a port nothing reads would tell an operator their service is published while it is merely intercepted. **And every one of them is editable after the fact.** `edit_address` rewrites `ip_int`, `port`, `proto`, `edge`, `target_port` and `upstream` — it took the first two and the proxy endpoint alone for a while, which made the rest add-only and left an operator who had set one while creating the service with no way back to it but deleting the address. The edit already takes that address's rules back and reinstalls them, which is the whole cost of changing any of it, and only that address stops being steered meanwhile. Two properties hold it together: everything the row will say is worked out **before** the old rules come back, so a refusal changes nothing; and a field the caller did not mention **keeps what the row had**, because this is also the endpoint that merely moves an address, and a move sending only an address and a port would otherwise quietly unpublish it. Taking one off is a value (`0`, `same`), not an absence. The interface asks for all of it in one place, `AddressOptions.tsx`, used by the ⚙ on each row of the creation form and by the modal on the service's page: the capability rules and the wording were written twice before, which is how the form came to offer what the backend then refused, and how one setting came to be described in two sets of words on two screens. The wording rule there took two corrections to settle. It began as field names — "The service is on port", "The service behind speaks" — which were true and left the operator to work out the effect. Rewritten as pure effect, "Send it to another port" was **read backwards**: with a port at each end, an operator with a cleartext service on `:80` who wanted it reached over TLS on `:443` typed `80` as the address and `443` here, and got a service intercepted where it already listened and published nowhere. So the label names the **fact the operator supplies** and the description says **what firegex does with it**, naming both ends: "The service is on another port" over "Clients arrive on :443 — firegex dials the service on :80". And because the misreading is really about the model rather than the sentence, the empty state says where a second way in comes from: another address, not another port in this box.
`target_port` is what makes firegex a **publisher** rather than only a transparent interceptor, on the addresses that use it: one daemon answering HTTP/1.1 in the clear on `:80` is reached over TLS on `:443` and over HTTP/3 on a UDP port, all three pointed back at `:80`, with one chain over them. It is per address and not per service because that is the shape of the thing — the service has one place it listens and several places it is published.
**`resolve_target()` and `service_at()` are the two halves and must not be merged again.** The first is where the world knocks, which is what a *rule matches*; the second is where the service is, which is what the engine *dials*. They were one function for as long as those were one answer, and making it return the target was tried first: the rules then redirected the service's own port twice and left the published port unprotected, which is a service that looks configured and answers nothing. Everything that matches asks the first (`FiregexTables.add`, `Rule.matches`); everything that dials asks the second (`_udp_targets`, the relay key in `add`, `_published`).
The TCP side needs telling because one listener fronts every address and recovers each connection's destination from conntrack: `FGEX_PROXY_TARGETS` at startup and `PUBLISH`/`WITHDRAW` on the control channel while it runs, both carrying `<address>|<edge>[=<service>]`. UDP and QUIC need none of it — a relay's upstream is fixed when it is bound, so `ADD_UDP` already answered both questions. **Only `tls` is sent as an edge**, because only it is a promise the engine keeps (a cleartext client on that port is refused); "in the clear" is the permissive default that still terminates a client bringing TLS, and making it strict was tried and broke every existing `http` service, whose single TCP address has always carried both and decided per connection.
**The service form asks three questions and the address row asks one, and everything else is behind a button.** It had grown into a wall: a five-answer protocol picker, a service-wide upstream control, a layer picker greyed out to one option, and two ports plus an edge on every address row — all of it in front of an operator who wanted a TCP service on one port. The shape now is **is it encrypted → which protocol → its certificate**, then the addresses, then the layer; and `AddressRow.tsx` puts a service published elsewhere, a service answering in the clear, and the hand-off endpoint behind one ⚙ per address, offering only what applies to the service being configured. What is set comes back as a **tag under the address that opens the panel it came from** — folded is not hidden, and an option an operator cannot see they set is one that will surprise them at the worst moment. **The layer picker is not shown at all for a service that is decrypted**, because decrypting means terminating and one layer does that: a control with two disabled options and one answer is a question nobody is being asked. It is replaced by the sentence, not dropped, since the layer is a real property of the service — and an effect pins `transport` to `proxy`, because a form that hides a control still submits its value.
**An ordered chain works on every layer, and gets there differently.** The proxy walks a list inside one process. NFQUEUE cannot — its binaries fuse transport and filter — so there a chain is a chain of *processes*: one binary per filter, each with its own queue and its own pair of base chains, at `-307 + position` (prerouting) and `107 + position` (postrouting). A packet a filter accepts carries on to the next base chain in the same hook, so **filter order is base-chain priority order**. This was established by experiment, not assumed: two `cppregex` processes at adjacent priorities, where the second refuses traffic the first passed. The cost is a process and a reassembly pass per filter, which is why `MAX_CHAIN_POSITIONS` exists at all.
Changing the *shape* of an NFQUEUE chain (adding, removing, reordering) cannot be pushed: the shape is the arrangement of processes and rules. `NfqueueTransport.reload` raises `ChainShapeChanged` and `ServiceManager.update_chain` rebuilds, logging that it did. Editing a pattern inside an existing filter still costs nobody their connection. A service with no active filter runs one stand-in stage (`_EmptyFilter`), and "nothing active, only the stand-in running" is not a change of shape — compared as ids it always looked like one, so every edit to a service whose filters were all switched off rebuilt its datapath.
### One engine per concern, never two
Two implementations of the same concept is the mistake this module was restructured to remove, and it is worth restating because it keeps being tempting:
- **There is one HTTP/1.1 rendering, and every terminated HTTP protocol is shown it.** `proxysrc/src/http1.rs` holds it; `h2.rs` and `h3.rs` are adapters to hyperium's `h2` and `h3` crates and nothing more. Two copies of the rendering would be two things a filter can be shown, and the day they drift the symptom is a pattern that matches over one version and not another with nothing to say so. For the same reason neither protocol is reimplemented here: HPACK, QPACK, multiplexing and flow control are the crates' problem.
- **Regex matching is hyperscan, everywhere — there is no second regex engine in the tree.** `binsrc/regex/` on the NFQUEUE side, `proxysrc/src/hyperscan.rs` on the proxy side, a binding rather than a reimplementation. Adding `regex` back to `Cargo.toml` would undo this.
- **A pattern blocks. There is no second action, and `rewrite` is not coming back.** It existed, and was withdrawn because it could not keep the promise it made: rewriting scanned one chunk at a time, since bytes already forwarded cannot be taken back, so a match straddling two chunks was never rewritten. Measured — a pattern sent whole arrived redacted, the same pattern split across two TCP segments arrived intact, and blocking (which scans in stream mode) caught both. The miss was not the problem; its silence was — no block, no log, no counter, just a rule the operator trusted that had quietly not applied. The removal has to be complete to mean anything, and it was half-done for a while: an `action` column that was gone from the datapath still had a `rewritten` field travelling from `debug.rs` through the backend model to a panel in `RegexDebugger.tsx` that could never render, a single-member `Action` enum in `firegex.regex`, and a `replace_with` on the frontend `Regex` type. If any of those spellings reappears, it is residue, not a feature.
- **A pattern is validated against the mode it will run in**, which is streaming, because that is how a pattern follows a connection across chunk boundaries. hyperscan does not accept quite the same patterns in every mode (`a{1,1000}b` streams but does not block-scan), and judging by the wrong one refuses rules that work — the same class of lie as a tester that disagrees with the engine. Block-mode scanning survives for one job only: reporting *where* a match starts, so the tester can highlight it. `debug.rs` reports `unscannable` for patterns that are valid, will run, and cannot be highlighted.
- **The C++ binaries swap their configuration through `std::atomic<shared_ptr<…>>`** (`regex_config`, `config`). The updater thread replaces it while every queue thread copies it per packet or per stream, and a plain `shared_ptr` read and reset from two threads at once is a data race: the copy can take a reference on a control block the reset has just freed, so an operator editing a rule under load could crash the binary with it. Neither binary defines `DEBUG` any more — `cpproxy` did, which printed the operator's whole filter source to its stderr, and so into the service log, on every reload.
- **A new configuration reaches the connections already open, on every engine.** The proxy layer hands a connection's next chunk to the new chain, `cppregex` resets its matchers when the version changes, and `cpproxy` now does the same with its filter contexts (`PyCodeConfig::version`, `follow_config`, `stream_ctx::clean_filters`, which leaves the TCP sequence bookkeeping alone). It kept `latest_config_ver` and never read it, so a stream ran the code it started with for as long as it stayed open: a check added against the attack in progress did not reach the connection carrying it. `cpproxy` also no longer runs a full `PyGC_Collect()` per packet, and runs one per `COLLECT_EVERY` (64) dropped contexts rather than per context. A full pass cost ~0.9 ms with the library loaded, and the subinterpreter's own collector is on (`before_loop`), so the per-packet one bought nothing. The per-context one did buy something, which is why it is batched rather than dropped: a context is a cycle (the module's functions hold its globals), and left to the automatic schedule a promoted one waits for a full pass triggered by object counts rather than bytes. Measured with a filter building a table at module level, 1500 short connections: ~8 s with a pass per context and memory flat at 95 MB; ~0.8 s with none, but memory settling around 300 MB; ~0.8 s and 135–160 MB batched.
- **A connection the NFQUEUE binary did not see open is still inspected.** Both binaries call `follower.follow_partial_streams(true)` in `before_loop`. libtins' default drops any stream whose SYN it missed, and on this layer that is the ordinary case, not an edge one: a new filter is a new process, a chain that changes shape is rebuilt, and a service can be started on a port that already has connections. Each time, every open connection went uninspected until it closed, and nothing said so. It was found by holding a connection open across adding a filter and watching the pattern go through (`test_a_filter_added_to_a_running_service_reaches_a_connection_already_open`, and `test_a_connection_open_before_the_service_started_is_inspected`). A stream met halfway has no handshake to say which end is the client, so `cpproxy` takes the direction from the packet (`pkt->is_input`, from the queue mark) instead of from which libtins callback fired; `cppregex` already did. A stream met halfway can be met mid-message, and so can one whose filter state was replaced (below): see the next bullet for what the HTTP model does about it. The proxy layer has nothing equivalent to fix: a connection made before its redirect existed was never NATed to it.
- **A filter that takes over a connection does not judge a message by its second half.** Three things give a Python filter a context that begins after its connection did: a queued stage meeting a stream halfway (above), `cpproxy` replacing its contexts for new code (`clean_filters`, which records the TCP streams in `stream_ctx::taken_over`), and a proxy-layer worker that never got the connection's open frame (new code, or respawned after a crash). Each sets `__firegex_joined_late` in that context's globals, and the HTTP model then treats a parse failure *before the direction has parsed anything cleanly* as having come in mid-message: that direction is carried without its HTTP filters until the connection closes, with one `[info]` line per process (`_met_mid_message`). It used to go to `invalid_encoding_action`: with REJECT a valid upload in flight was refused whenever a filter changed, and without it a working client was logged as "not HTTP". Measured on both layers before the fix, pinned by `test_a_request_in_flight_when_the_filter_changes_is_not_judged_by_its_second_half`. A connection met between two messages parses, is in step (`<parser>_in_step`), and answers to the usual rules from then on; a connection seen from its first byte never gets the benefit. The trade is deliberate — never break a working client over firegex's own change of state: an attacker holding a message open across a filter change keeps that direction out of the HTTP filters for the connection's life, where before the partial-stream fix every open connection was out of every filter.
- **An edit reaches only the filter it was made to.** Every edit reloads the whole chain, and a reload used to reset every Python filter's per-connection state — module globals, the HTTP parser's place, on the proxy the addresses told at open — over a pattern changed in some other filter. `_QueueStage.reload` no longer sends a stage the payload it last acknowledged (`_in_force`), which also keeps `cppregex`'s matchers mid-stream; the proxy carries a `PyWorkerRule` over into the new chain when `Filter::reuse_key` is unchanged (`parse_ruleset_reusing`: id, command, enabled functions, deadline and a hash of the code itself, since saving rewrites the same path). Pinned by `an_unchanged_worker_is_carried_into_the_next_ruleset_with_its_state` and `test_an_edit_to_another_filter_leaves_a_python_filters_connections_alone`.
- **The Python API is `firegex.pyfilters`, everywhere.** `cpproxy` embeds it; `modules/services/pyworker.py` runs it out of process. An earlier version of that worker defined a look-alike `@pyfilter` taking `(data, direction)`, which meant a filter written from the docs loaded on one transport and failed on the other.
- **A filter file's protocol is read off the file, never declared.** The library already decides when to call a filter from what its parameters are annotated with, so `infer_proto` in `firegex/pyfilters/internals` reads the same table back the other way: asking for an `HttpRequest` makes a file an HTTP filter, asking only for a `RawPacket` makes one run on anything, and asking for two *different* application protocols is refused naming both functions. `type_annotations_associations` is the single place a protocol is defined; a second list of names would be one more thing to keep in step, and the day it fell behind the symptom would be a filter that silently never runs. The backend stores the detected value on the row so the interface can show it, and `PUT .../code` is what recomputes it. Passing one in from outside is exactly what used to make an HTTP filter on a TCP service a 500: the *service's* protocol was sent to the datapath, so the filter was refused for asking for an `HttpRequest`, which is the only thing an HTTP filter does.
- **A filter file's functions run in the order the file defines them**, and the first to answer anything but ACCEPT ends the packet. `@pyfilter` sets a mark on the function and hands it back; `collect_pyfilters` reads a module's globals, which are in definition order. What that replaced was a *set of names* on a module-global attribute of the decorator, so the order was whatever hashing produced — which decided which function got credited with a block and whether an earlier `UNSTABLE_MANGLE` survived, with nothing in the file able to tell you which. The `position` column on `pyfilters` carries the same order to the interface, which used to list them `ORDER BY name`. Three consequences of that registry are also gone with it: `get_filters_info` had to clear it before and after (and reached it by `exec`ing import strings), two compilations could race, and a function registered from somewhere the module globals could not reach was registered and then not found.
- **A pyfilter file's functions are switched on and off individually, and the code is never edited to do it.** `pyfilters` holds one row per `@pyfilter` the code defines, reconciled with the file on every `PUT .../code`: functions that have gone are dropped, new ones arrive active, survivors keep the operator's `active` and `blocked`. Only the enabled names reach `__firegex_pyfilter_enabled`, which is already what decides whether the library ever calls a function. Three states are distinct all the way down (`Option<Vec<String>>` in the spec, `*` / `=a,b` / `=` on the worker's argv): **absent** means every function the file defines — the case of a filter whose list was never derived, where defaulting to "none" would silently stop every filter running — while an **empty list** means the operator switched them all off. `ChainLink.enabled_functions` returns `None` for no rows and `[]` for all-inactive, which is the same distinction.
- **gRPC is a body format inside HTTP, not a protocol beside it, and that is what keeps it free.** `GrpcMessage`/`GrpcRequest`/`GrpcResponse` live under `"http"` in `type_annotations_associations` and nowhere else. The alternative — a `"grpc"` value in `filters.proto`'s CHECK and in `PROTO` — would have been a schema change, and changing that dict **deletes and recreates `db/services.db`**, so the price of a model would have been every service on the instance. Everything *around* a call already arrives through `HttpRequest`: the method is the path, the metadata are headers, the status is a trailer section. What did not was the body, which is a sequence of `[flag][4-byte length][protobuf]`.
- **The parser learned to release body chunks, and only the gRPC models turn it on.** A server-streaming or bidirectional RPC has a body that does not finish until the stream closes, so a model shown only finished bodies is called once, at the end — a log entry rather than a block. `release_body_chunks` on `InternalCallbackHandler` hands the message over as its body grows; it is off by default, and each gRPC model has its own `_parser_class()`, so `HttpRequest` keeps its contract exactly (twice: once with the headers, once with the whole message). Two details are load-bearing: the partial body is **undecoded**, because `content-encoding` is undone once at message-complete and half a gzip stream is not half a body (the models decline while one is in the way, which for real gRPC never happens — it compresses per message instead); and the cursor lives on the `InternalHTTPMessage`, not beside it, because a keep-alive connection carries many messages through one parser and a per-connection cursor would start the second one part-way in. Separate parser keys are not tidiness either: taking a frame *consumes* it, so two models sharing one parser would leave the second silently never running.
- **`RawPacket` is the only model that survives a datagram**, and that fact is stated once, as `DATAGRAM_SAFE_MODELS` in `routers/services.py`. Every other model declines to be built when `is_tcp` is false, so a filter asking for one on a UDP service would sit in the chain and never be called. `--check` reports the models a file asks for, saving refuses the combination naming them, and `edit_service` asks the same of every Python filter of a service being switched to UDP (`_filters_needing_a_stream`) — `Transport.check()` can only see the stored protocol, which says whether a file parses HTTP and not whether it wants a TCP stream. Python filters *do* work on UDP on both layers: `cpproxy` gained `filter_action_udp` (no stream follower, no ack fixing — a datagram is complete in itself), and its per-flow contexts are bounded because a datagram has no close to observe and a spoofed-source flood would otherwise accumulate one set of module globals per forged address: a UDP flow's context goes after `UDP_IDLE_SECONDS` (60) of silence, and past the service's own `max_connections` (`FIREGEX_MAX_FLOWS`; 0, the default, is no limit) the least recently used one goes to make room. It used to be a fixed 4096 applied to **every** context, TCP included, in flow-id order — so a burst of datagrams could take the state of a TCP stream in the middle of a request, whose next packet was then parsed from a clean slate. Only UDP flows are counted and evicted now (`udp_recent`/`udp_where`). `handle_next_packet` used to throw for anything but TCP, with a message claiming UDP was supported.
- **A filter file is validated by the process that will run it.** `pyworker.py --check <file>` builds the module exactly as the worker does, compiles it, and prints one JSON object: `{ok, proto, filters}` or an error carrying **type, message, line, column, the source line and the traceback**. `validate_code()` runs it in a **subprocess with a timeout** — user code used to be `exec`ed inside the backend's own event loop by `detect_proto`, so a `while True:` at module level would have taken the interface down. `POST .../check` answers 200 even when the answer is no: a refusal is the result, and the editor wants a position to mark, not a string to print. Saving goes through the same check, so the operator is never handed "the filter code did not load: worker exited" and left to find the traceback in the service log. **Nothing else runs the operator's module in the backend either**: `_reconcile_functions` takes the function names from the check's answer rather than `exec`ing the file a second time, and the NFQUEUE payload (`_python_payload`) hands the binary the enabled names — `None` for all of them — and lets `firegex.pyfilters.internals.compile()` resolve them against the module it has just compiled, passing over a name the file no longer defines. And the check is awaited, not waited for: `validate_code`, `check_pattern` and `_ask_engine` run their helper processes through `_run_helper` (asyncio). They used `subprocess.run` inside `async` endpoints, which blocks the one event loop the whole backend runs on — for a Python start per keystroke-pause in the editor, and up to `CHECK_TIMEOUT` for slow module code — while no ACK and no `BLOCKED` line was read, until an engine's stdout pipe filled and its synchronous `println!` stalled the datapath itself.
- `_where_in_source` finds the line three ways: a `SyntaxError`'s own position; a traceback frame belonging to the file (`<string>` counts as theirs — the library execs the source to find its filters, and everything the library runs from has a real path); otherwise any identifier in the message looked back up among the file's `def`s, **last definition wins**, because that is the one Python is left holding.
- `Filters` compiles once under the operator's file name instead of re-parsing per connection, so runtime tracebacks in the log name their file and their line too.
- A filter is called directly. `context_call` used to write the arguments into the module globals, `eval` a call expression built from them, and delete the three keys again — per filter, per packet — which is exactly `func(*args)`, because the operator's code was `exec`ed into those globals and so the function's `__globals__` *is* that dict.
- **`FGEX_STREAM_MAX_SIZE` bounds what a stream costs in memory, and three things had got past it.** A compressed body was decoded whole (`on_message_complete`): a megabyte of gzip under the cap is a gigabyte after it, so every step of the decoding is bounded by the cap now, and a body that would go past it meets `FGEX_FULL_STREAM_ACTION` like any other stream past the cap (with `FLUSH`, handed over still encoded). A WebSocket message is bounded the same way (`max_size` to `Frame.parse`). The flush itself cleared the *view* of a body and subtracted it from the count while the buffer it is read from kept growing — so on a gRPC stream, whose body is handed over as it grows, memory grew past every flush and the count went negative (`_flush_body`); framed gRPC bytes are also dropped as they are handed over (`_forget_framed`), and a frame past the cap is skipped to its end so the next one is still found where it starts. And `HttpHistory` counted messages but not bytes, a hundred bodies of a megabyte per direction per connection; each direction's history is held under the cap too (`_trim_history`). Separately, the flush raised on a message still in its headers (`len(None)`), so headers past the cap were an exception and the packet failed open.
- **Traffic the HTTP parser cannot read is carried by default, and said so** (`invalid_encoding_action` defaults to `ACCEPT`). It was refused, so a request the service would have taken and llhttp would not — one client's quirk, a protocol that only resembles HTTP — closed the connection with nobody having chosen that, credited to the filter function as if it had decided it. Refusing is one line in the filter file for an operator who wants it against request smuggling; when it refuses, the block is credited to `@INVALID_ENCODING`, and the warning is one line per connection and direction rather than a traceback per packet.
- The default for `FGEX_STREAM_MAX_SIZE` is 1 MiB, as documented. It read `1*8e20` under a comment saying "1MB", so `FGEX_FULL_STREAM_ACTION` never fired unless a size was set by hand, and the float it produced would have been refused by the setter beside it.
- **The editor's hints are introspected, never written down.** `modules/services/pyapi.py` reads the models out of `type_annotations_associations` and their members out of the classes — including `writable`, from `property.fset`, which is how the interface can say whether a member can be assigned to — none can, since the payload stopped being writable with `UNSTABLE_MANGLE`. `GET /api/services/pyfilter-api` serves it and `pyfilterHints.ts` feeds Monaco's completion and hover from it. A second, hand-written description would be wrong the first time a model gained a property, and autocompletion that lies is worse than none. The route is declared **before** `/{service_id}`, or that one matches it and looks for a service by that name.
- **A filter that raises says so the same way on both layers.** `_on_exception` writes one sentence — the traffic was forwarded unfiltered — and it has to be reachable from either. NFQUEUE gets there directly: the interpreter is embedded in the binary, which sends `EXCEPTION`. The proxy cannot, because `pyworker.py` catches the exception in a separate process and answers with an ordinary ACCEPT, so nothing crosses the frame boundary that the engine could tell apart from a filter agreeing. It marks stderr with `EXCEPTION_MARK` instead and `Transport._pump_stderr` turns that back into the same call. The traceback is printed *beside* it, not instead of it: the traceback says which filter and which line, and the sentence says what happened to the connection — for a while the proxy printed only the first, which tells an operator a filter broke and not that traffic went through unchecked. The marker is spelled in both files because the worker is launched as a script and imports nothing from the backend, which is what keeps the user's code out of this process's imports; `tests/integration/test_resilience.py` asks for the sentence on both layers and so keeps them in step.
- **A block names the function, not just the file.** Both layers report `<filter_id>/<function>`: the NFQUEUE binaries already reported the function name, and the proxy worker now returns it in the verdict frame (`[verdict][name len][name][payload]`). `ServiceManager._on_block` splits that one token in one place. A file with ten functions blocking 900 connections is not an answer to "which of them is doing it".
- **Nothing below the application layer crosses into a filter.** `RawPacket` exposes addresses, ports, family and direction as read-only metadata, and `data` — the payload — read-only as well: a filter answers with a verdict, and nothing comes back but that verdict and the function that gave it. The payload was writable for `UNSTABLE_MANGLE`, and stayed writable after it was removed, which made `packet.data = ...` a silent no-op on both layers while the editor still offered a "rewrite" snippet ending in `return UNSTABLE_MANGLE` — a name that no longer exists, so the snippet raised on its first match and the traffic went through unfiltered. The setter, the snippet, the hover text and the docs' example are gone together, and so is the C++ path that applied a rewrite: `cpproxy` treats action `3` (the retired `MANGLE`) as `INVALID`, which fails open and is reported as an exception. The number stays unused rather than being given to anything new. `FilterAction::MANGLE` in `nfqueue.cpp` is **not** that: it is how `reject()` sends the RST with the payload stripped, and it stays. That boundary exists because the two layers cannot honestly offer the same thing underneath it: NFQUEUE has a real header whose rewriting desynchronises the stream, while the proxy terminated the connection and writes its own. The old model papered over that by handing the proxy a literal `FAKE:IP:TCP:HEADERS:` prefix as `raw_packet`, so the same filter did different things depending on where it was attached. On the proxy the metadata is *better* than a header would be — the engine knows both ends and sends them in a `KIND_OPEN` frame (`ConnectionMeta` → `Filter::connection_opened`), once per connection rather than on every chunk.
- **The order the operator set is the order that runs.** `parse_ruleset` groups patterns by their `filter` id and keeps the positions the backend sent. An earlier version grouped every block, then the Python — which turned the chain into a set, so a filter placed after another could run before it, and nothing said so. Within one filter the operator expressed no order, so blocking goes before rewriting there; across filters, never reorder. `tests/rules.rs` pins both.
- **The regex tester runs the engine.** `POST /api/services/debug-regex` shells out to `fgex-proxy --debug-regex`, so what it says is what will happen. A `re`-based approximation would bless patterns hyperscan rejects.
### Per-module pattern
Each module (`services`, `firewall`) follows the same shape:
- `backend/routers/<mod>.py` — FastAPI endpoints + its own SQLite DB, via `utils.sqlite.SQLite`.
- `backend/modules/<mod>/models.py` — plain model classes, `from_dict`.
- `backend/modules/<mod>/firewall.py` — `FirewallManager`/`ServiceManager` pair: the manager tracks one `ServiceManager` per DB row, which starts/stops that one service.
- `backend/modules/<mod>/nftables.py` — builds/tears down the module's nftables rules; extends `utils.NFTableManager` (a `Singleton`), so there is exactly one rule-set owner per module.
**Every nftables object firegex creates is named `fgex_`, and lives in a table firegex owns.** The services module holds `table inet fgex` with `fgex_queue_in_<n>`, `fgex_queue_out_<n>`, `fgex_hijack_*`, `fgex_divert`, `fgex_nat`, `fgex_nat_out`, `fgex_route`, `fgex_guard` and the set `fgex_datapath` — and its `reset()` deletes **the whole table**, because flushing it chain by chain was one batch that nft refuses outright if any chain named in it is missing, so a table left by a version with other chains (an upgrade after a crash) was never cleared and went on steering traffic at dead queues and ports; the firewall module holds `fgex_filter` and `fgex_mangle` per family, with base chains `fgex_input`/`fgex_forward`/`fgex_output`/`fgex_prerouting`/`fgex_postrouting` jumping to `fgex_rules_in`/`_out`/`_fwd`. One `grep fgex_` over the ruleset is the whole footprint.
**Something else removing those tables is noticed, said and undone, within `TABLE_WATCH` (3 s).** `nft -f /etc/nftables.conf` begins with `flush ruleset` on Debian, and so does restarting its unit; before this, every service went on reading ACTIVE with nothing reaching a filter, and a default-deny firewall became no firewall, with nothing said. Each module's `FirewallManager` runs a `_watch_table` task from `init()` to `close()`/`stop_watching()`, asking `nft.intact()` — one small chain that `init()` always leaves rules in, so a deleted table and an emptied one both fail it. The services module rebuilds the table and calls `ServiceManager.put_rules_back()`, which is the second half of `enable()` alone: the datapath never stopped, so nothing restarts and no connection is dropped. It runs under the service's lock, like `disable()`, or rules put back while a stop is waiting on its datapath would outlive the stop (`test_rules_are_not_put_back_behind_a_stop`). Each service logs it as an error, at most once per `TABLE_TOLD_QUIET`, because traffic went through unfiltered in between. Partial surgery — one rule of ours deleted by hand — is not detected; that is somebody editing firegex's table, not somebody else's tool wiping the ruleset.
**An interface rule says where traffic *arrived*, not where it was going, and that is deliberate.** `iifname docker0 tcp dport 443` matches every container's **outbound** HTTPS as well, because that traffic arrives on `docker0` too, and a `redirect` then takes it into the engine, which terminates it with this service's certificate — measured: `pip install` inside any container failed with `CERTIFICATE_VERIFY_FAILED: hostname mismatch` while such a service was running, and worked the moment it stopped. Pinning the destination to the addresses the interface carries was written, tested and **reverted**, because it removes the one case interfaces exist for: an operator names a link when **the address is not theirs to know**, which includes firegex in front of a service on another machine reached through it, and a rule pinned to this host's own addresses silently stops protecting that. Protecting a link protects what crosses it. Naming one that carries other people's traffic is a choice, and the answer to not wanting it is a narrower address, not a narrower rule — `docs/services.md` says so where the choice is made, because it is surprising the first time. Do not re-narrow this without a way to tell the two deployments apart.
**An interface stands for the addresses it carries, and every place that needs an address has to resolve it — three of them gave up instead.** `udp_relay_host()` got this right from the start; the others did not, and each failure was silent in its own way. `_add_proxy` wrote no output-hook rule at all, so a service protected on `lo` filtered the outside world and not a client on the box. `_published()` and `ServiceManager.address_added` skipped the announcement, because the engine keys that map on what `SO_ORIGINAL_DST` hands back and an interface name is not an address — so an HTTPS edge declared on `lo:443` was never announced, the engine met the connection knowing nothing, terminated it as an ordinary one and dialled `:443` where nothing listens. HTTPS on that address simply did not answer while the same address written `127.0.0.1:443` worked, which is how it was found and is as confusing a symptom as this module has produced. All three now go through `interface_addresses()`, which returns the concrete addresses (link-local dropped) and hands back a single-element list for an address, so the two spellings take one path. The cost is the one `udp_relay_host` already pays: the answer is resolved when the rules are installed, so an address the interface gains later is not covered until the service restarts. When adding anything that needs *an address* rather than *a match*, the question to ask is what it does when handed `eth0` — and the answer is never "skip it".
**The proxy layer's redirect runs before anybody else's destination rewrite, and that is not tuning.** `NAT_PRIORITY` is -120: below `dstnat` (-100), above conntrack (-200). Docker and podman publish a container's port by appending their own chain to `nat PREROUTING` and `nat OUTPUT`, which `iptables-nft` registers at exactly `dstnat` — the priority these chains used to share. Two base chains at one priority in two tables are ordered by neither of them, and it was measured going the wrong way: the DNAT rewrote the destination to the container's address, firegex's rule (which matches the *published* address) stopped matching, its counter stayed at zero, and a request carrying a blocked pattern was answered by the container. The service still read `ACTIVE`. Running first, the published port is captured before anyone can rewrite it — the property the NFQUEUE layer always had for free by living at `raw`, three hundred below all of this, which is why that layer filtered a containerised service correctly the whole time. Nothing downstream is lost: the engine's own dial carries `SELF_MARK` and is skipped by the rule it would otherwise match, so the container runtime's DNAT still does its job on the way out. `tests/integration/test_addresses.py::test_a_container_runtimes_rewrite_does_not_take_the_traffic` stages a competing rewrite at `dstnat` and asks for the block; it was confirmed to fail with the old priority before being kept. A CTF service is almost always in a container, so this is not an edge case — it is the deployment.
The firewall module is the one that had to move to get there, and the reason is worth keeping. It used to write into the tables called `filter` and `mangle` — the ones `iptables-nft` claims — adding base chains named INPUT and FORWARD beside iptables' own and driving their policy. That broke two things, both reproduced before being fixed:
- **`iptables` stopped working on the host.** `iptables-nft` refuses an entire table as soon as one rule in it cannot be expressed in iptables' own format, and it stores conntrack matches as an `xt` blob where nft writes a native `ct state`. Bisecting the rules firegex writes, `ct state` was the only one that tripped it — which is to say `allow_established` and `drop_invalid`, on in any sane configuration. From then on `iptables -L` and `iptables-save` answer ``table `filter' is incompatible, use 'nft' tool``, on a box where Docker, ufw or the organisers' scripts are the ones asking.
- **Stopping firegex opened the host.** `reset()` re-added `filter INPUT` with `policy accept`, and in nftables `add chain` on an existing chain *updates* its policy. That chain was the administrator's: a host carrying default-deny had it silently removed when the firegex firewall was switched off.
A base chain carries its own policy wherever it lives, so default-deny is expressed exactly as before and a `drop` is terminal whichever table's chain reached it. **One thing about what gets blocked did change, and `allow_dnat` puts it back**: in iptables' own `FORWARD` chain, Docker's or podman's accept for a published port came before or after firegex's jump in the same chain, and either way traffic to a published container port never met firegex's policy; a base chain of our own is evaluated on its own, so a drop policy dropped every connection to a published container port — how every service on a CTF box is reached. `dnat_rules()` appends `ct status dnat accept` to `fgex_forward` after the jump (on by default, a switch in the firewall settings): the operator's forward rules still apply first, and the runtime's own chain still decides for itself. **And what the containers send**, which the first version of it left out: with the policy at drop, a web container could not reach its database on the same bridge and no container reached the network — measured with two containers on a Docker network — because Docker accepts that in `FORWARD` too, beside where firegex's rules used to be. So traffic arriving from a container runtime's bridge (`CONTAINER_BRIDGES`: `docker0`, `br-*`, `docker_gwbridge`, `podman*`, `cni-podman*`; br_netfilter shows bridged packets with the bridge as their interface) is left to the runtime the same way. `get_rules` also had the family of every rule narrowed by the first one naming an address (`families` set once, outside the loop): an "accept port 80" listed below an IPv4 rule did not exist for IPv6. What changed is that firegex now adds a layer instead of taking one over: an `accept` in `fgex_rules_in` means "firegex does not block this", never "nothing else will", which is all it ever honestly meant. `reset()` is now deleting two tables per family (`add` before `delete`, because a batch is atomic and deleting a table that is not there fails all of it).
`Rule.table` is still `filter` or `mangle` in the database, the API and the interface — it names the hook a rule is evaluated at. `NFT_TABLES` maps it to the table it lands in; the two were one string only because that string used to be somebody else's table.<>
A refused edit must change nothing: `_apply_chain(service_id, undo)` in `routers/services.py` rolls the write back when the transport rejects the resulting chain. Without it an operator is told their filter was rejected and then finds it in the list, and the next start fails for the reason they thought they had avoided.
`edit_service` holds itself to the same rule, and did not. It now checks the service it would become **before writing** — `Transport.check()` against the chain, whether or not the service is running (a stopped one used to take any combination and fail on its next start), a published port refused on a layer that does not dial, and the addresses' `edge`/`proto`/`upstream` recomputed for the new protocol (`_address_under`) — then writes the service and its addresses in **one transaction** (`db.queries`), because a protocol change that collided with another service's address used to leave the service rewritten and its addresses not. A running service whose new definition will not start has the old one written back and is started again on it, rather than being left stopped on the definition that failed. What an address can still be given afterwards — the port of the operator's own proxy on a service moved to the hand-off layer — is not checked there, exactly as at creation: `ExternalTransport.check` names it at start. **Only what differs from the row counts as an edit.** The form sends every setting on every save, and each save used to stop and start a running service — resetting every connection it carried for opening the dialog and pressing save. A change of name alone is written and nothing is restarted, because nothing running reads it.
Order is the load-bearing detail in `services/firewall.py`, because getting it wrong is not an error anybody sees — it is a window during which traffic goes unfiltered. Starting: the capture interface if the service decrypts (the engine opens its socket to it at startup, so one that appears afterwards is one this process will not find), then the datapath, then the nft rules that point at it. Stopping is the exact reverse.
### Block statistics (`modules/services/stats.py`)
A cumulative counter per filter, per pattern and per `@pyfilter` function lives in the database; the bucketed history answers the other question — *when* did this start. Blocks are counted per filter into one-minute buckets, kept for `HISTORY_BUCKETS`, and flushed at most every `FLUSH_INTERVAL` seconds. Both properties are load-bearing for the same reason the live log's are: a service refusing thousands of connections a minute must not be able to take the backend down through its own bookkeeping, which is what one row per block would do. There is deliberately **no background task** — flushing happens on the next write once the interval has passed, and always on a read — so nothing has to be started, cancelled or waited for at shutdown. The frontend draws it as hand-written SVG (`StatsPanel.tsx`) rather than pulling in a charting library several times the size of the page it sits on.
**Per-block counters are written once a second, in one transaction** (`ServiceManager.flush_blocks`, `BLOCK_FLUSH`), and the names a block is logged under are looked up once per rule until the chain changes. They were three commits per refused connection, about 17 ms a block, so the backend fell behind any flood of refusals — and that was not only its own problem: the engine wrote `BLOCKED` to a pipe the backend was not reading fast enough, the write blocked the runtime thread that made it, and the flood slowed every other client of the service (measured: 5000 refusals took 74 s and a legitimate client waited a median 4.5 s, failing half the time; now 4 s and 1 ms). Every services API request writes what is waiting first (the router's `_counters_written` dependency), so a read is exact and an edit resetting a pattern's counter cannot have the old pattern's last blocks written onto it afterwards. On the engine's side **nothing on the datapath writes to stdout** (`report.rs`): lines go through a bounded queue and a thread of their own; `BLOCKED` and `STATS` are dropped and counted when the backend is that far behind, and replies the backend waits for (`PORT`, `UDP`, `ACK`) wait for room, in order.
`share` is computed in the backend, not the browser, so a chart and the table beside it cannot disagree about what a rule accounts for.
**History is stored per *rule*, not per filter, and that is what makes a time range mean one thing.** `block_history` keys on the token the datapath reported — a pattern id, `<filter>/<function>`, or a filter id — so `GET .../stats?range_from=&range_to=` can answer the chart, the per-filter totals, the per-pattern totals and the shares over the *same* window. Storing only per filter would have left the chart honouring the range and the tables beside it reporting all of time. A filter's total is the **sum of its rules**, not a lookup of its own id: that token appears only when nothing named itself.
`bucket_width` takes either a `target` bar count (the default, which is what makes the bar widen with the range) or an explicit `want` in seconds from the operator, and both are floored at `MAX_STEPS` bars — a step fine enough to melt the chart is widened, never refused, and `bucket_seconds` reports what was used rather than what was asked. A range never grows the number of buckets, only their width (`bucket_width`): everything is an aggregate of the one-minute rows, so twelve hours costs the same rows as one and is drawn in the same handful of bars. Ranges are clamped to **two** floors — `retention_start()` and the service's `filtering_since` — and the served range is reported back, so the interface can say what it actually showed. `filtering_since` is written by `_note_filtering()`, which is called from `start_service` and from `_apply_chain` (the two places the two halves of "running, with an active filter" change) and once per service at `startup()`, which is what covers services restored from a backup or predating the column. It is written once and never moved: a service stopped and started again has a real gap in its chart, and moving the mark would hide it. Note the clamp happens *after* the caller's ends are put in order, and the `filtering_since` floor applies **only to a window that reaches into the service's life**: a window that ended before it began is answered exactly as asked and comes back empty, because it is. Dragging its start forward instead would have reported the current minute's blocks under a range an hour old — buckets are minute-aligned, so a zero-width range is not an empty one. `all_time` is reported alongside `total` so a narrow window showing nothing is not read as a contradiction against the cards.
`StatsCharts.tsx` draws bars, lines, stacked area and 100%-share from the same series — four views, one dataset, still hand-written SVG for the reason the first chart was. Hover is tracked once on the container and turned into a step index, not per shape: a line and an area have nothing under the cursor to attach a handler to, and one handler means the four charts cannot disagree about which step is being pointed at. The readout itself is HTML positioned over the plot, for the same reason the axis labels are — `preserveAspectRatio="none"` stretches rectangles correctly and distorts glyphs. `clock()` is exported from there and used by the axis labels, so a bar's tick and its readout cannot disagree about what a step is called.
**Editing a pattern keeps the row, resets its counters, and hands its history to the filter.** `PUT .../regexes/{id}` validates the *resulting* row through the engine (changing the action alone can invalidate a stored pattern, since blocking compiles for streaming and rewriting for block scanning), and when the pattern text itself changed it resets `blocked` and calls `stats.disown_rule` — counts belong to the text that produced them, so the new text must not inherit them. That call happens **after** `_apply_chain` accepts, never before: a refused edit has to change nothing, and history moved ahead of the verdict cannot be put back by `undo()`. The rule's *pending* batch moves with it, or the last few blocks of the old pattern would be flushed under the new one.
What `disown_rule` must not do is **delete** those rows, which is what it used to do. Those connections were refused, by that filter, in those minutes, and the chart is answering *when this service was under attack* — a question a later typo fix has nothing to say about. Deleting took a bite out of the timeline on every correction, and on a filter holding one pattern it emptied the chart. So the rows are re-keyed to the filter's own id, already the token the datapath reports when nothing finer named itself: `window()` and `totals_by_filter()` group by `filter_id` and do not change at all, while `totals()` groups by `rule_id` and correctly credits the new text with nothing.
The cost is that some blocks in a range then belong to no rule still present — which was already true of a *deleted* pattern, whose rows are left behind on purpose. The stats route closes that by adding one **residual** entry per filter for the difference between what the filter was credited with and what its current rules account for (`residual: true`, prose rather than a pattern, so `StatsPanel` prints it as text and not in a `Code` box). Without it the per-pattern list quietly adds up to less than the chart above it, and the operator is left hunting for the difference. Note `check_pattern` coerces `case_sensitive`: a caller checking a row it read back from SQLite hands over `1`, and the engine's request format refuses an integer where a boolean belongs.
**Traffic totals are reported in the unit each layer can honestly produce, and the two are never divided by one another.** This is the part to preserve:
- **Packets and bytes** come from `counter` statements on the intercept rules — free, kernel-side, and unskewable because no userspace filter is involved in producing them. `FiregexTables.traffic()` reads them back, from **one inbound chain per transport**: the nfqueue layer installs a rule pair per filter position all seeing the same packets, so summing them would multiply the answer by the length of the chain.
- **Proxy services report no packet count at all.** Their rule lives in a `nat` chain, and conntrack translates a connection once — every packet after the first is handled without the rule being walked again. That counter is *new connections wearing a packet label*, so it is dropped rather than shown.
- **Connections** come from the engine's periodic `STATS seen=… refused=…` line. They live on `ProxyStats`, not on `ChainStats`: a chain is replaced wholesale on every ruleset push, and counters that reset on a rule edit would make the ratio jump for a reason nobody could see. Because both numbers are connections, `refused_share` there is exact.
- NFQUEUE has no connection to count, so it reports `null` for the share rather than dividing refused connections by a packet count. **The asymmetry is the difference between the layers showing through; do not "fix" it by inventing a denominator.** The engine's counters arrive on a timer, so a test that reads them has to wait for them.
### The live log (`modules/services/logs.py`)
A bounded ring per service plus a coalescing flush over a `log` Socket.IO event (distinct from `update`, which only invalidates queries). **A warning or an error does not wait for the service's page to be opened**: `ProblemNotifier` (mounted in `MainLayout`) turns `warn`/`error` lines into a notification wherever the operator is — one per service per 20 s, the rest counted into it, except that an error is never folded into a warning ("restarting it" followed by "could not restart it" hid the second until a third came, which it need not) — and the service list carries the newest one (`ServiceLog.last_problem`, `problem` on `ServiceModel`) as a badge for an hour; clearing the log dismisses it. The log panel was the only place these went, so an engine that died and came back, a filter raising on every packet, or a fallback that cost the service its clients' addresses waited for somebody to look. `cpproxy`'s merged stdout/stderr goes through `_on_engine` like every other datapath's (`_QueueStage._forward_output`, whole lines only): it went in as the user's output, so its `[error]` lines and the traceback of a filter that raised were grey text among the prints. Two properties are load-bearing and easy to lose: the ring is capped, and the flush is on a timer with a per-batch cap that reports a dropped count rather than carrying it — a service refusing thousands of connections a minute must not be able to take the backend or the browser down through its own logging.
Sources: block attribution (resolved to the filter name and pattern, one query per block, on a path that only runs when something was already refused), the user's `print()`, and the datapath's own stderr. That last one is why `pyworker.py` reassigns `sys.stdout = sys.stderr` before running user code: stdout is the length-prefixed protocol channel, and a `print()` landing in the middle of a frame gets the worker killed on every packet with nothing to explain it. `tests/pyworker.rs` has a regression test for exactly that.
### The proxy datapath (`backend/proxysrc/`)
A Rust binary, built in the Dockerfile's `compiler` stage rather than a `rust` image on purpose: it links the same `libhs` the C++ binaries do, and a binary built against one distro's vectorscan and run against another's is a bad afternoon.
- `filter.rs` — the chain, and everything that keeps a filter from taking the datapath down: `catch_unwind`, a deadline on `spawn_blocking`, and a filter that misbehaves losing its say rather than the traffic being held. `panic = "unwind"` in `Cargo.toml` is load-bearing, not tuning. **Losing its say for missing the deadline is a pause, not a sentence** (`REARM_AFTER`, 30 s): it used to last for the life of the chain, and a Python filter misses its deadline when exchanges queue behind one another — which is what a flood does, the one moment it is wanted — so the protection stayed off until somebody next edited a rule. A filter that is still slow loses its say again after the same few misses, which bounds what a genuinely hung one strands to a few threads per pause. A panic is broken code and stays out until the chain is replaced; the warning says how.
- Filters that keep state do so **per connection, per direction** (`ChainSessions`), because one client's bytes deciding another client's verdict is both a false positive and a way to smuggle a pattern past a filter by splitting it across two connections.
- `pyworker.rs` frames carry a connection id and a close frame, so the Python side can honour the documented promise that each stream gets its own module globals.
- Source preservation is unconditional: the engine always dials the service from the client's address. `ct status dnat` is what separates the engine answering a client from the service answering the engine, which is why redirect (not tproxy) works even when the service is on this host. A transparent dial the service **refuses** is not retried as ourselves: nothing is listening, and the retry used to blame source preservation for a service that was simply down.
- **A service behind a container runtime's rewrite works from anywhere, on both layers** — it is the ordinary CTF deployment. On the proxy layer the engine dials the published address as the client, the runtime's DNAT sends it to the container, and the container answers **from its own address**: `socket transparent` in `fgex_divert` found no socket talking to that address, so the answer went on to the real client, which reset it, and every connection from another host waited out the dial and was retried as this host — slow, and the client's address lost. `fgex_divert` now also marks home replies carrying the engine's conntrack mark (the one `fgex_route` copies onto every connection the engine opens). On NFQUEUE, a client **on this host** was never inspected: the runtime rewrites that connection in `nat OUTPUT`, so it leaves towards the container without passing prerouting, and the container's answer arrives for a local socket without passing postrouting. `fgex_queue_lreq_<n>` (output) and `fgex_queue_lrep_<n>` (input), at `LOCAL_QUEUE_PRIORITY` (-90: after the rewrite on the way out, before it is undone on the way in, so both halves show the container's address and a stream follower sees one stream), match such connections by conntrack's **original** destination with `ct status dnat`, and are found again by `get()` through their `address <l4> <port> <ip>` comment. Measured with a real Docker container; `tests/integration/test_remote_clients.py` stages the same rewrite into a network namespace so no Docker is needed.
- **The service's answers are brought home by who opened the connection, not by the port they come from.** `fgex_route` copies the engine's `SELF_MARK` onto the conntrack entry of every connection it opens and marks whatever answers one of those for local delivery — and that is all it holds. The per-address rule that used to sit there matched the service by the address it is protected on, which is where the engine dials it only while the address *is* the service: one published on another port, or an HTTP/3 edge in front of an HTTP/1.1 service (dialled over TCP), answered from a port nothing matched, and the answer was routed out to the client instead of home. A client on this host never noticed — the reply is local either way — so every test passed while a client on another host timed out. `tests/integration/test_remote_clients.py` puts the client in a network namespace to ask. Matching every reply *from* the service's port instead was not the fix: that port can be reachable directly, and its own clients' answers would have been diverted. The per-address rule was exactly that, for the protected port itself, and it was removed for exactly that reason: a connection open before the service started is never redirected (conntrack translates a connection at its first packet), so its answers belong to the client, and a client on another host stopped hearing back the moment protection began. Measured with a connection held open from a network namespace across the start, TCP and UDP (`test_a_connection_already_open_survives_the_service_starting`); NFQUEUE never had the problem.
- **The engine's own ports answer only what was redirected to them** (`fgex_guard`). The TCP listener and every UDP/QUIC relay are bound to the wildcard, because a redirect delivers to whichever address the packet arrived on — so a relay, which forwards to its service whoever sent the datagram, put a service protected on `127.0.0.1:53` on the network. `fgex_datapath` holds `l4proto . port` of every one of them and `fgex_guard` (input hook) drops what reaches one without conntrack's DNAT status; `fgex_divert`'s mark exempts replies to the engine's transparent sockets, whose destination is a client's port and can be any number. A port is added when its redirect is installed and released only when the **whole service** comes down (`FiregexTables._guarded`), because a relay stays bound after its address is removed and would otherwise be reachable directly again — and an element left behind after a stop would drop traffic for whatever process is given that port next.
- **A connection nothing redirected is refused** (`dialled_at_ourselves`), behind the guard as well as in front of it. The listener is on an ephemeral port of every address (`[::]:0`), so anybody can dial it straight; `SO_ORIGINAL_DST` then answers with the address the connection was accepted on, and forwarding there is the engine dialling itself — measured, one such client became as many connections as the limit allowed in two seconds, and with no limit it grows until the process runs out of descriptors. The old guard compared with the configured listen address, which is the wildcard and never matches.
- **Stopping or restarting a proxied service does not cut what the engine carries.** The engine terminates every connection, so `stop()` — killing it — cut them all, measured: a connection through the engine read EOF on the next write after a stop and after a restart, while the same test on NFQUEUE carried on. `ServiceManager.disable` takes the rules off and calls `Transport.retire` instead: conntrack keeps delivering what it already translated to the old engine's ports, new connections go to the service (a stop) or to the new engine (a restart — `handing_over`, from `restart()` and `refresh()`), and the old engine exits once a `STATS` line counted after it was taken out (`DRAIN_FRESH_AFTER`) says `live=0` — which counts TCP connections, UDP flows and QUIC connections alike. A stop pushes it an empty chain first, because stopping is how an operator takes a hurting filter out of the way; a restart keeps the old chain. Bounded by `DRAIN_LIMIT` (10 min) and `MAX_DRAINING` engines per service, both logged when they close something. Its ports stay guarded until it is gone (`nft.delete(keep_guards=True)` → `release_guards`, which leaves alone a port another service has since been given), and its TLS files are removed at once — it read them at startup, and removed later they would be the new engine's copies. `FirewallManager.close()` still kills: the table that brings the answers home goes with it (`stop_draining`). Pinned by `test_stopping_a_proxied_service_does_not_cut_its_connections`, `test_a_restart_hands_over_without_cutting_connections` and `unit/test_proxy_drain.py`.
- **The backend watches the engine** (`ProxyTransport._watch`, the same `on_died` → `_recover_datapath` the NFQUEUE stages always had). The nft rules outlive the process, so a dead engine used to leave every connection redirected at a closed port — a total outage while the service read `ACTIVE`, and nothing ever restarted it. `reload()` on a dead engine raises rather than returning quietly, so an edit is not reported applied to nothing. **A datapath that does not acknowledge a command is killed, and one that refuses it is left running** (`Transport._abandon`). A refusal leaves the previous configuration in force on all three binaries; the NFQUEUE stages used to be stopped on one anyway — and stopped *on purpose*, so the watchdog said nothing and the service read active with no process behind its queue. A missing answer is different: what was applied cannot be known, and the answer arriving late was read as the answer to the next command, putting every command after it one answer out of step. So the process is killed without being marked stopped, and the watchdog rebuilds the service from the saved configuration. The wait for a configuration is `LOAD_TIMEOUT` per Python filter rather than `ACK_TIMEOUT`, because loading one runs its module body and the code check allows that eight seconds.
- **The client's SNI goes on to a TLS upstream**, beside its ALPN list (and on the QUIC → HTTPS path once the client's handshake has said it). A service choosing its certificate or virtual host by SNI was handed the bare address — which TLS does not even send — and answered as its default host. A QUIC upstream is dialled before the client's handshake and cannot be told.
### The `external` hand-off
Three nft chains, not two: prerouting for traffic from outside, **output for traffic this host generates** (which never reaches prerouting — the same reason the proxy transport has `fgex_nat_out`), and postrouting to undo the rewrite on the way back. A `mangle` target is one host, so `one_address()` strips the prefix that service addresses carry;
Note for tests: Linux refuses a loopback connection to a port with no listener *without emitting a packet*, so no rule can rescue it. A hand-off test has to keep the real service listening and tell the two apart by their replies, not by which one answers.
### Holding up under load
**A service carries at most `max_connections` at once, and the operator says what happens to the rest.** It exists because the alternative was measured, not argued: silent connections cost two descriptors each — one from the client, one to the service, since the upstream is dialled on accept — and ~505 of them from one host exhausted the container's 1024 and took **every** service down. UDP was cheaper still: 600 datagrams from 600 forged sources in 0.03 s took 400 descriptors and held them a minute after the sender left. A cap on the same shape existed on the NFQUEUE side (a fixed `MAX_UDP_FLOWS` of 4096 in `pyproxy.cpp`) and had never been carried to the Rust engine; TCP had none anywhere. That fixed cap is gone: on NFQUEUE the same `max_connections`, set in the form's advanced settings for a UDP service, bounds how many UDP flows keep their Python state, and idle flows are let go after a minute whether or not one is set.
Say what it buys, and no more. **A cap does not save the service being attacked** — it cannot tell a connection that is silent because it is an attack from one that is silent because the client is slow, so an attacker who fills the limit still fills it. What it buys is that one service's attacker stops being everyone's: measured, 400 silent connections against a limited service left the service beside it answering. Do not let the interface or the docs imply otherwise.
- One number for both halves, because TCP connections and UDP flows spend the same descriptors and two limits for that would be two things to keep in step. `0` is no limit, kept as a value rather than an absence, because "unlimited" is a choice. It **is** one number: `udp.rs` counts its flows in the same `stats.live` the TCP and QUIC paths do, with a `Slot` held by the flow. It used to compare each relay's own flow count with the limit, so a service with several UDP addresses held several times its limit beside a full complement of TCP connections — and since UDP flows were never counted in `accepted` or `closed_by_filter` either, a UDP service reported no connections and a refused share of zero whatever it refused. A refusal is counted once per flow, because the number it goes into is compared with flows seen.
- `over_limit_forwards` is the same trade as fail-open at a different moment, and off by default for the same reason: a filter that quietly stops filtering is the worse surprise. On UDP a flow past the limit is carried as a flow of its own with an empty chain, answers included. It used to be a datagram sent from a socket closed straight after, so the service's answer reached a port nobody held: forwarded, and useless to every request/response protocol UDP carries.
- Folded into the form's one **Advanced settings** accordion, not dropped: these are settings an operator reaches for once — after something went wrong, or before a competition — and never again, and left in the open they are more things to read past on every service. One fold rather than one per topic, and its summary is built from what *differs from the defaults* (`changed`), so an untuned service says "Everything at its default" and a tuned one says exactly how. Folded is not hidden. What is inside depends on the layer, because the settings belong to different ones: the connection limits are the proxy's, fail-open is NFQUEUE's — with, for a UDP service only, the same number read as how many UDP flows keep their Python state (`limitsQueuedFlows`) — and the hand-off layer has none of them, so it gets no accordion at all.
- **Proxy layer only**, and absent from the form elsewhere. NFQUEUE hands the kernel a verdict on packets in flight; it accepts nothing and dials nothing. The form also resets it to `0` when the layer changes, because a stored setting that does nothing is the trap this whole file keeps arguing against.
- `Slot` is a guard, not a decrement at the end of `handle_connection`: that function has a dozen ways out — a refused chain, a failed dial, a timed-out handshake, a panic — and a counter that leaks on any one of them is a limit that tightens until it refuses everything.
- The accept loop **backs off** after a failed accept. That one is not a policy question: out of descriptors, the loop spun on `accept` → `EMFILE` → print → `accept`, measured at ~40% of a core with four clients knocking, writing a log line per attempt into the pipe the backend reads.
**`first_byte_timeout` is what the cap cannot be.** A cap contains a phantom flood; a deadline ends it — measured, with the limit full and the attacker still holding every socket, clients got back in once it passed. Two properties are load-bearing and easy to break:
- **Either direction counts.** The flag is set by whichever pump moves a byte first, so a service that greets its client (SMTP, SSH, most game protocols) satisfies it with its banner. Requiring the *client* to speak would hang every server-speaks-first protocol, and the failure would look like firegex breaking them at random.
- **Only until the first byte, never again.** A connection that has spoken and gone quiet is a session, and sessions think. This is not an idle timeout; making it one would close long-lived connections that are legitimately waiting.
The flag is set *before* the chain runs, because a chunk a filter goes on to refuse is still a connection that said something. The watchdog fires through the same `stop_tx` a refusal uses, so both directions shut down cleanly rather than being dropped — a dropped peer reads a truncation, and this connection has done nothing to deserve one. Zero is off, and there is no default worth picking: how long a legitimate client may stay silent is a fact about the service, not about firegex.
**The trace is deliberately in two places.** `_on_over_limit` writes `over_limit_hits`/`_first`/`_last` to the database *and* logs at warning level, rate-limited by `OVER_LIMIT_QUIET`. The log answers "why are clients failing right now" and is a bounded ring, so a burst at three in the morning is gone by breakfast; the columns answer "did we ever hit the wall", which is asked the next day. The engine's counter is cumulative *per process*, so the transport hands up only the **increase** — passing the absolute number would reset the durable count on every restart of the service.
### TLS
**The two controls that constrain each other read one rule.** `carries(transport, proto)` in `components/Services/utils.ts` is the only place the constraint lives: the protocol picker hides what the chosen layer cannot carry, the layer picker disables what cannot carry the chosen protocol, and `Transport.check()` refuses it on the server. Written twice, the two halves drift and the form ends up offering a combination the backend then refuses — being told no after filling in a form is a worse way to learn it than not being offered the option. Both pickers say no the same way: the option stays where it is, **disabled**, with a sentence underneath saying why and what would make it available. An option that vanishes leaves nothing to ask about — the operator is left wondering whether it exists at all or whether they misremembered seeing it — and two controls side by side that refuse in two different ways are a confusion of their own.
**`ProtocolChoice.tsx` is the other half of that pair, and it exists for a confusion that was reported rather than guessed at.** The picker used to be a bare row of five buttons headed "Transport protocol" — five alternatives on one axis, which they are not. An operator with a plain HTTP/1.1 service in the clear had nothing to tell them whether `TCP` or `HTTP` was meant for them (both are true sentences about their service), and picking `HTTP` then asked for a certificate for something with nothing encrypted about it. It is now two questions, the first being the one that actually divides them and the one the certificate hangs off — **is the traffic encrypted?** — with the second asked *inside* that answer and offering only protocols of that kind: `tcp`/`udp` in the clear, `tls`/`quic`/`https` encrypted. Every option is a protocol under its own name, which matters: an intermediate version asked "on what transport?" and computed the value from the pair, showing the name only in a badge underneath, and that reads as the form having decided something on your behalf. `http` is labelled **HTTPS** in the interface — `protoLabel()` in `components/Services/utils.ts`, the one place a protocol is named, used by the picker, the service row and the detail page alike, because being offered `HTTPS` and then shown `HTTP` on the row reads as two different settings. It is still `http` in the database, the API and the docs; the rename is only about where it is chosen, since beside TLS and QUIC an option called "HTTP" reads as the cleartext web protocol. Two more things are load-bearing: the lock on the encrypted answer, which says before the click which branch will want a certificate, and the certificate fields sitting directly under the choice that asks for them rather than below the advanced settings — cause and request a screenful apart is most of why the request looked unmotivated. **And the inline text is one clause per option, deliberately.** What is needed while choosing is which option is yours; what a protocol costs and what filters attach to it is documentation, and it lives in the popover and in `docs/services.md`. Three successive versions of this component grew paragraphs explaining the trade inline and each one had to be cut back — the same lesson `LayerChoice` already records one section above.
**TLS is one of `L4`'s values, beside `tcp`, `udp` and `quic` — not a boolean on top of one.** It is what the service speaks, and as a flag it made three combinations expressible that are not real: TLS on a UDP service, on NFQUEUE, and on a hand-off, each caught by a check written by hand. Two of those are now unrepresentable, and the third (`NfqueueTransport.check()`) is asked at **creation** as well as at start — `add_service` calls `transports.build_class(...).check(srv, [])` with an empty chain, which catches what the *service* makes impossible while leaving what the *chain* makes impossible to the moment there is one. A row that is created happily and then refuses to start every time is a trap.
The second answer that split off from it: an address carries the transport **the kernel matches on** (`L4.l4_of`), not the service's own, because `tls` is TCP on the wire and `quic` is UDP. Without that, `(ip, port, proto)` would let a TCP service and a TLS one claim one `ip:port` between them, which is the same port twice. `l4_proto()` in `nftables.py` is gone with it — the address already holds the answer it used to compute.
It belongs to the **proxy** layer: decrypting means terminating the connection, and terminating is what that layer does. That is not a lost feature so much as an honest one — TLS on NFQUEUE worked by putting nginx in front to terminate and re-encrypt, so the connection *was* terminated and the layer's one distinguishing property was already gone.
**The engine terminates it, and a TLS service therefore occupies no port at all.** `proxysrc/src/tls.rs` decrypts from the client and re-encrypts towards the service inside the process that filters, so `resolve_target()` points the rules at the address the world dials, exactly as for a plain service. `ProxyTransport._tls_env()` hands the engine `FGEX_PROXY_TLS_CERT`/`_KEY` — **paths**, because a certificate on a command line or in an environment dump is a certificate in a log — written `0600` under `TLS_DIR` and removed after the engine is gone, never before.
What that replaced: nginx terminated on one derived loopback port and re-encrypted from a second, with the filters attached to the leg in between. Protecting one address cost a pair of ports produced by hashing `ip:port` — *chosen* ports, which could collide with something real — plus a generated config, a `tls_streams` table keyed by address id, an nft chain of its own, and a failure mode where one unloadable certificate stopped nginx and took **every** TLS service on the instance down with it. All of that is gone: `modules/tls/` no longer exists, and the blast radius is now structural, since each service carries its certificate into its own engine process.
**ALPN is mirrored, and the ordering that makes it possible is the design.** The client's handshake is *started* and held at the ClientHello (`tokio_rustls::LazyConfigAcceptor`), because that message carries the protocols the client will speak; the upstream handshake goes next offering exactly that list, and whatever the service picked is the single protocol the client is then told (`tls::with_alpn` / `tls::answering_with`, both cloning the config per connection — the list comes from the client, so there is no fixed set to precompute, and a config clone is nothing beside a handshake). Answering something the service did not say is how a proxy breaks a protocol it was only carrying: a client told `h2` against an HTTP/1.1 service sends frames nothing can read. When the service picks nothing the client is told nothing — no agreement is invented. `tests/integration/test_tls.py` pins all five cases, including both mismatches and a client that offers only one protocol. Before this, ALPN was dropped in both directions and every client silently fell back to HTTP/1.1.
**Mirroring ALPN means `h2` can be agreed, and `h2.rs` renders it.** For a while nothing did, and the symptom is worth keeping because it is the failure this whole file is arranged against: HTTP/2 compresses its headers with HPACK exactly as HTTP/3 does with QPACK, so on an `h2` connection a pattern written against a request line matched nothing and a filter asking for an `HttpRequest` was handed the `PRI * HTTP/2.0` preface — parsed as one fictitious request — and then frames it could not read, so it was never called again. No block, no log, no counter. Only a body travelled in the clear, and since gRPC *is* HTTP/2, gRPC could not be filtered at all. The engine named it at `warn` once per process (`warned_h2`) and the docs said to turn HTTP/2 off on the service; both are gone, because the real fix is done. See the HTTP/2 section below.
What cannot be carried, and is a limit rather than a missing setting: **TLS 1.0 and 1.1**, which rustls does not implement at all; the cipher list, which is fixed and forward-secret only; and **client certificates**, since `with_no_client_auth()` is the only honest answer when the certificate a client would present is one only the service can check. Say these are impossible rather than unconfigured — an operator who needs TLS 1.0 needs the `external` layer, not a flag.
The key must be one `ring` will sign with — RSA ≥ 2048 bits, ECDSA, or Ed25519. nginx could be told to take a 1024-bit key with `@SECLEVEL=1`; rustls has no equivalent, so a smaller key is refused when the service starts. That refusal is worth having land well: `_pump_stderr` keeps the engine's last `[fatal]` line and `_died_because` puts it in the exception, because "the proxy engine did not report a listening port" is the symptom and the engine already printed the reason.
**What the service behind speaks is its own question, and it belongs to the *address*.** `upstream` on `service_addresses` — `same`, `tcp`, `tls` — because that is where it has an answer: one daemon reached over TLS on one port and in the clear on another is re-encrypted for the first and handed the plaintext for the second, and what leaves is a property of the way in. It was a service column first, which forced one answer on every edge of an `http` service. It reaches the engine **with the address**: `FGEX_PROXY_TARGETS` and `PUBLISH` for a TCP one (`<address>|<edge>|<onward>[=<service>]`), `FGEX_PROXY_UDP` and `ADD_UDP` for a relay — never as a variable of its own, since there is no longer a per-process answer. `Onward` in `proxy.rs` is the one spelling of the three values (a second enum, `UpstreamProtocol`, existed in `quic.rs` for an hour and is exactly the drift this file forbids); `TlsSetup.upstream` is built whenever the engine terminates anything and *used* per address. **It is only asked where firegex decrypts** (`upstream_applies`): the cleartext address of an `http` service is carried as it arrived — the engine's plaintext path, h2c included, never reads `published.upstream` — so a choice there is refused rather than stored, the form does not offer it (`addressCapabilities` takes the edge), and moving an address to the clear without mentioning the upstream resets it to `same` instead of refusing the move. It used to be offered, saved and shown back as a tag there, and read by nothing.
`same` is what every address did before there was a choice: a connection this engine terminated goes back out encrypted, one it never terminated was never encrypted to begin with, so a plain TCP service is untouched by this and always was. The other two say the service speaks HTTP/1.1 — in the clear or under its own TLS — whatever arrived at the front.
On the TCP edge that is nothing but a config: `build_tls` hands back no upstream client configuration and the dial is an ordinary socket. **On the QUIC edge it is the one place in this engine where what leaves is not the version that arrived**, and it was the point of the whole exercise: most of the web is a service that speaks HTTP/1.1 and never will speak QUIC, so firegex terminates HTTP/3 in front of it. What makes that honest rather than a translation invented here is that the exchange is *already* rendered as HTTP/1.1 for the chain — `judge` sees those bytes, `firegex0` carries those bytes, and now the service receives those same bytes. Three things are load-bearing:
- **`http1.rs` grew the sending mirror of `Incoming`** — `Outbound`, `OutboundBody`, `Answer` — and `h3.rs`'s `exchange`/`carry_request`/`carry_response` are generic over it. The alternative was a second copy of the orchestration for the HTTP/1.1 case, which is the drift the `Incoming` trait already exists to prevent one level down. `h1up.rs` implements them over hyper's HTTP/1.1 client (client+http1 features only): the request direction is free because the rendering already built it, and the *answer* is what needed a crate, since HTTP/1.1 is the one version this engine has never parsed — an HTTP/1.1 connection elsewhere is a byte pump. Writing a response parser beside hyper's is what "one engine per concern" forbids.
- **The request target has to be reduced to origin form.** HTTP/2 and HTTP/3 carry it split across `:scheme`, `:authority` and `:path`, which `http::Uri` holds as one absolute URI; written to a socket unchanged that is `GET https://host/path`, the *proxy* form, which an origin server may refuse and which every service reading the path back gets wrong. `h1up` reduces it and moves the authority to `host` — the same decision `render_request` already makes for the chain, so what the filters saw and what the service receives stay the same message. The tests caught this immediately; it is not theoretical.
- **One connection per exchange**, because a QUIC stream is one request and HTTP/1.1 has no multiplexing. Pooling would put two clients' requests on one socket and with it the thing `ChainSessions` exists to prevent, at a layer nobody would think to look at. Its body is a channel into hyper, and `finish()` **drops the sender** — waiting on the channel instead (`Sender::closed()`) waits for the service to answer a request this end has not finished sending, which is a deadlock the chunked test found at once.
One combination cannot work, and it is impossible rather than unimplemented: a QUIC edge carrying something that is **not HTTP/3** to a service that speaks HTTP/1.1, because opaque stream bytes have no HTTP/1.1 form. Nothing refuses it at configuration time any more — there is no list to check it against, since the protocol is each client's (see below) — so on such an address the engine answers `h3` itself and a client offering only something else is refused by that handshake. `upstream_refusal()` used to refuse it up front against a configured list, and went with the list.
**The decrypted traffic is written to an interface of its own, and those are reconstructed packets.** Terminating in-process means the plaintext is never a packet, so `proxysrc/src/capture.rs` frames it as the TCP stream it was and sends it to `firegex0` over `AF_PACKET`: one interface carrying every decrypted service's plaintext — TLS and QUIC both, see the QUIC section for what a stream costs there — and nothing else, which is what a capture tool wants pointed at it. The bytes are real — what the filters saw and what was forwarded — but the framing is invented, because the framing on the wire was encrypted. Sequence numbers start at zero, there are no retransmissions, and the segmentation is the engine's read sizes; a SYN and a FIN are emitted so Wireshark has a stream to follow rather than orphan segments. Say so wherever it is offered: a capture from here is not evidence of what was on the wire.
Three things keep it from being able to hurt anything. `modules/services/mirror.py` creates the device in `FirewallManager.init()`, before any engine starts (a socket opened before the interface exists finds nothing), and `release_capture()` removes it from the process's `shutdown()` hook and nowhere else — deliberately not from `close()`, which `reset()` also calls, or the device would vanish and reappear underneath whatever was capturing from it. It is the **instance's** device, not each TLS service's: it used to be created and released per service, on the argument that an interface present while nothing is decrypting is one somebody points a capture at and watches stay empty — but a capture tool is attached once, for a round, and Zeek and Suricata exit rather than wait when the interface under them disappears, which restarting a single TLS service was enough to cause. Watching it stay empty is the better failure. `release_capture_if_idle()` is gone; do not add a per-service release back. Every failure is silent and local — no interface, no socket, a failed send — and the traffic goes on exactly as it would have. And the `AF_PACKET` socket needs `CAP_NET_RAW`, which `docker-entrypoint.sh` asks for **and starts without**: it tries the full capability set, falls back to the set without `cap_net_raw` while saying what that costs, and only then runs as root. A host that will not grant it loses the capture, not the firewall.
Note for tests: a client now dials the service's own address, because there is no second port to be pointed at. The old note about the DNAT skipping loopback belonged to nginx's public-port redirect and went with it.
### QUIC
**QUIC is terminated, and unlike TLS there was never a second option.** TLS over TCP can be carried past unopened — the bytes are framed by a transport the kernel understands, so filtering the ciphertext is a thing an operator can choose. Past its Initial packet QUIC encrypts the frames, the stream boundaries and the packet number along with the payload, so a packet queued to NFQUEUE is a datagram of noise. `NfqueueTransport.check()` and `ExternalTransport.check()` refuse it saying that, rather than the sentence they say about TLS.
It is UDP to the kernel, and that is the whole of what the backend had to learn: `L4.l4_of(quic) == udp`, so the addresses, the uniqueness key, the nft rules, `udp_relay_key`/`udp_relay_host`, `ProxyTransport.udp_ports` and `ADD_UDP` are the code that was already there. What changes is what binds the port — `FGEX_PROXY_QUIC=1` beside the certificate makes the engine build `quic::QuicManager` instead of `udp::UdpManager`, and `relays::Relays` is the two-armed enum the control channel and the startup list both go through. `Service.decrypts` (once `terminates_tls`) is what gates the certificate, on both protocols.
**The ALPN is mirrored, as on the TLS path, and nothing is configured.** The ClientHello travels inside an Initial packet, and that was taken for a reason it could not be read before answering — so the engine offered the service a candidate list instead, `FGEX_PROXY_QUIC_ALPN` for the whole instance and then a per-service field, and a QUIC service speaking anything but HTTP/3 was unreachable until somebody found the setting. But Initial keys are derived from the Destination Connection ID the client sends in the clear (RFC 9001 §5.2), exactly so a middlebox can read this. `quic_hello.rs` does: `SniffingSocket` wraps the endpoint's socket, every long-header Initial is decrypted with `rustls::quic::Suite::keys`, CRYPTO frames are reassembled by offset across datagrams (a ClientHello with post-quantum key shares does not fit one packet), and the ALPN list is parked per client address for `QuicRelay::offered`, which waits `HELLO_WAIT` for the rest of a split hello. `dial_upstream` then offers the service exactly that list and `QuicSetup::answering_with` tells the client the one protocol the service chose — both built per connection, as the TLS path does. The map is bounded (`MAX_PENDING`, `KEPT_FOR`) because forged Initials cost the sender nothing. A hello that cannot be read falls back to `h3`, said once per relay. The parser is tested against packets protected with rustls's own client-side keys, and `tests/quic.rs` drives real quinn clients through it, including one whose hello spans packets.
**One stream is one connection to the chain** (`Carrier::open`), with its own `ChainSessions` and its own module globals, released when the stream ends. A filter's state follows a stream of bytes from start to end, which is what a QUIC stream is and what a QUIC connection is not; on HTTP/3 it also lands where an operator expects, one request to a stream. The raw path reuses `proxy.rs`'s pumps unchanged — `spawn_pumps`, `pump` and `join_pumps` are `pub(crate)` for exactly this, and `spawn_pumps` takes the `spoken` flag rather than making one, because the first-byte deadline is the *connection's* question and a flag per stream would never answer it for a connection that opened a hundred and spoke on none.
**A refusal closes the connection, not the stream.** Resetting one stream would leave the client free to ask again on the next, which is not what a block means anywhere else; it is what keep-alive already does on HTTP/1.1. Both ends are closed with the same reason through `Carrier::refuse`.
**HTTP/3 is terminated too, and rendered to the chain as the HTTP/1.1 it would have been** (`proxysrc/src/h3.rs`, on hyperium's `h3` — a binding, not a second implementation). This is the point of the whole exercise: HTTP/3 puts the method, path and headers in a QPACK-compressed HEADERS frame, so a filter shown the stream would be shown a compression format and every pattern written for the TCP service beside it would silently stop matching. Rendering it means `HttpRequest`, `HttpResponse` and a hyperscan pattern mean one thing on both. The same bargain the capture interface makes — the bytes are real, the framing is reconstructed — and the details are load-bearing: the request line says `HTTP/1.1`, `Host` comes from `:authority`, a declared length is passed through, and **the head is held back until it is known whether a body is coming**, so a bodiless request is never shown a `Transfer-Encoding: chunked` it never carried. A filter looking for that header is looking for smuggling, and one invented by the proxy in front of it is the worst possible answer; `tests/h3.rs` pins both halves of that. What goes on to the service is HTTP/3 again, re-encoded — the rendering is a view, never a translation the service sees.
Two things about that hold, both of which were wrong first and are pinned now. **It is bounded** (`HEAD_HOLD`, 100 ms), because holding it indefinitely deadlocks every exchange the service is supposed to speak first in — a bidirectional RPC sends a head and waits to be answered, and a proxy that will not forward the head until a body arrives is waiting for something that is waiting for it. `head_framing` is what makes the hold cheap enough to bound: a declared `content-length` frames the body by itself and waits for nothing at all, a message with no body has already ended its stream, and what is left over is a sender holding the stream open — which is a message of undeclared length, which is what chunked means, so the timeout renders rather than refuses. **And a trailer section is always shown to the chain**, which is what decides the framing of a message carrying one and no body: in HTTP/1.1 trailers belong to a chunked message and nowhere else, so that message is rendered chunked with an empty body — the shape of a gRPC trailers-only answer, which used to reach the far side with nothing having read it. The one rendering that cannot carry trailers is a declared length, whose body ends where the header said; there `carried()` **drops** them with a log line, because forwarding a piece of a message no filter saw is the thing nothing else here does. `Incoming` exists so both directions decide all of this in one place rather than in two copies that can drift.
**The plaintext is written to `firegex0`, one TCP stream per QUIC stream.** A QUIC stream is ordered, reliable and starts at zero, which is exactly what a `Tap` rebuilds; what it has not got is an identity a capture tool can use, because every stream of one connection shares the connection's four-tuple and writing them out as they are would interleave a hundred of them into one conversation whose bytes decode as nothing. So `Tap::open_stream` gives each a **synthetic client port** from a cycling counter — a counter and not a hash of the stream id, since ids restart at zero on every connection and two clients' first streams would land in one conversation. That is one more piece of invented framing on top of the invented framing this file already is, and it has to be said wherever the capture is offered, which `AddressList`'s panel now does: the port names a stream, not a socket. On HTTP/3 what is written is the **rendering** — the same HTTP/1.1 the filters were shown, since `judge` is the funnel every rendered byte already passes through — which is the only way `firegex0` can keep the promise it makes everywhere else, that what is on it is what the filters saw. It is not what left this process (that went out as HTTP/3 again), and the two cannot disagree because they are the same bytes. Written only once the chain has agreed, as the TCP pumps do it: a refused piece is one the traffic never got past this process, so the conversation ends where it did.
**QUIC datagrams are carried, and one connection's are one flow.** `L4::QuicDatagram` is its own value rather than `Quic` with a flag, because the two answer the stream question differently and `is_stream` is what every model built on a stream turns on — a datagram gets `RawPacket` and nothing else, which is the same answer the plain UDP relay gives. `Carrier::carry_datagrams` opens the flow **on the first datagram**, not on the connection, so a connection that negotiates the extension and never uses it costs no filter state; both directions share the one id, and the sessions persist across datagrams exactly as the UDP relay's do — a pattern split over two datagrams sent a millisecond apart is otherwise a way straight through, and `tests/quic.rs` pins it. A refused datagram is **dropped, and the connection carries on**: that is a block meaning something different from what it means one function up, and the difference belongs to the traffic rather than to a preference — a stream is a conversation that can be ended, a datagram is complete by the time it is judged. `DATAGRAM_BUFFER` bounds what one connection may have waiting, because unbounded it is memory bought in this process by whoever sends fastest.
**They are not carried on HTTP/3, and the handshake says so rather than the peer finding out.** An HTTP/3 datagram names the stream it belongs to, and the request streams this proxy opens towards the service are not the ones the client opened, so the name would point somewhere else on the far side; `QuicSetup::build` therefore gives the `h3` accept configuration a transport that does not advertise the extension, and the upstream edge — dialled before the service has said what it speaks — is decided by the candidate list instead. **WebTransport is not carried for a further reason and is a bigger question than its datagrams**: its streams are bound to a session by a varint every one of them carries, which breaks both of the rules the rest of this rests on — "one stream is one connection to the chain" would hand a filter a routing header it does not know to skip, and the HTTP/1.1 rendering has nothing to say for "this stream belongs to that request". It needs a session model, not a switch.
Two things are refused rather than left unconfigured, and should be said as limits: **0-RTT** (`max_early_data_size = 0`), because early data is replayable and a refused request cannot be un-delivered; and a **Retry for every unvalidated client**, so that nothing is opened towards the service on the word of a source address nobody has checked. `over_limit_forwards` is honoured but is not free here: a QUIC connection past the limit is still terminated, decrypted and re-encrypted, because there is no such thing as forwarding a QUIC connection unopened.
### HTTP/2, and one rendering for every version
**`src/http1.rs` is the rendering, and `h2.rs`/`h3.rs` are what is left of each protocol once it is taken out.** The HTTP/1.1 view a filter is shown — `Framing`, `HEAD_HOLD`, `head_framing`, `carried`, `render_request`/`render_response`, `push_body`/`close_body`, `judge` — lived inside `h3.rs` and had nothing QUIC about it. Copying it into `h2.rs` would have been two renderings that can drift, which is the argument the `Incoming` trait already makes one level down, and the day they drift the symptom is a pattern that matches over HTTP/3 and not over HTTP/2, silently. So it lives in one place and both protocols are adapters to somebody else's parser: hyperium's `h3`, and `h2` for HTTP/2 — a binding, never a second implementation, for the reason stated under "one engine per concern".
**An HTTP/2 client can be put in front of a service that speaks HTTP/1.1, and for a while it could not.** `h2.rs` takes its upstream as an `Outbound` — `H2Upstream` where the service speaks HTTP/2, `h1up::H1Upstream` where the address said `upstream = plaintext` — so the rendering the chain is shown is also what the service receives, exactly as on the QUIC edge. Before that, `h2::carry` opened an HTTP/2 client handshake and nothing else, which is why the ALPN could only ever be *mirrored*: with a cleartext service there was no upstream handshake to ask, so the client was told nothing and fell back to HTTP/1.1. The asymmetry is what gave it away — the same cleartext service was reachable over HTTP/3 and not over HTTP/2, one rendering underneath, opposite answers. `proxy.rs` now answers `h2` itself when the address says the service is cleartext, **and only while the chain will render it**: a bypassed chain is a byte pump, and a byte pump between an HTTP/2 client and an HTTP/1.1 service carries frames nothing at the far end can read.
Two things the trait had to grow for it, both because HTTP/2 states on the head what HTTP/1.1 frames for itself. `Outbound::open` takes **`ends_it`**: a head that is the whole message has to go out as one, or a trailers-only gRPC answer becomes an answer with a body, which is the thing `head_is_the_message` exists to prevent — HTTP/1.1 just closes the body early and HTTP/3 ignores it. And `Incoming` grew **`release`** and **`ended`**: flow control is HTTP/2's alone, and releasing the window only *after* a piece has been forwarded is what turns the far end's window into backpressure on the near one, so it could not stay inside a loop that no longer knows which protocol it is reading.
**Terminating HTTP/2 was never forced the way terminating QUIC is.** The bytes are framed by a transport the kernel understands, so firegex could have gone on forwarding them. What forces it is the promise: a filter written once has to work everywhere, and there is no way to keep that over HPACK without decompressing it, which is what parsing HTTP/2 means.
Three things differ from h3 and each is a way to get this wrong:
- **Flow control is ours to run.** `h2` hands received bytes over and waits for them to be released. Releasing on read means no backpressure and this process buying memory for whichever side is faster; never releasing stalls the stream after the first window. So capacity is released **after the piece has been forwarded**, and outbound writes reserve capacity before writing rather than queueing in the connection.
- **The end of a message rides on its last frame.** A head that *is* the whole message has to be sent with `end_of_stream` on the head; sending it open and closing with an empty DATA frame turns a **trailers-only** answer into one with a body — which is exactly a gRPC status-only reply, and gRPC clients refuse the mangled version. `head_is_the_message` is the one place that is decided, and it asks the sender (`RecvStream::is_end_stream`) rather than inferring, which is also what keeps a `content-length: 0` request headers-only through the proxy.
- **One stream is one connection to the chain**, as on h3 — and unlike HTTP/1.1, where a keep-alive connection carries many requests through one set of filter state. HTTP/2 interleaves its streams, and sharing state between two of them is "one client's bytes deciding another client's verdict", which is what `ChainSessions` exists to prevent and also a way to smuggle a pattern past a filter by splitting it across two streams. Say this in the docs: it is a real difference an operator writing a stateful filter can be caught by.
**One authority, or the request is malformed** (`conflicting_authority`, asked by `h2.rs` and `h3.rs` before anything is rendered). HTTP/2 and HTTP/3 carry the authority in `:authority`, a client may add a `host` header beside it, and the rendering shows the chain one `Host` line while the request goes on to the service with both — where an HTTP/2 or HTTP/3 service routes on `:authority` (RFC 9113 §8.3.1). Two that disagree showed every filter one host and delivered the request to the other, so a rule written against an admin vhost was walked past with `host: public` beside `:authority: admin`. They are reset (`PROTOCOL_ERROR` / `H3_MESSAGE_ERROR`) with a log line, as are two `host` headers.
**What belongs to a connection does not cross into another version** (`http1::connection_specific`, `http1::drop_connection_headers`). HTTP/2 and HTTP/3 have no hop-by-hop headers: the framing and the connection's lifetime are the protocol's own. An HTTP/1.1 service says things about its connection in almost every answer — `transfer-encoding: chunked` for a body of unknown length, `connection: keep-alive` — and those were passed on as they came: the h2 crate refused to send such a response at all, so an HTTP/2 client in front of an ordinary HTTP/1.1 service got a stream error for any chunked answer, and an HTTP/3 client is required to reject one. `h1up` takes them out, with whatever `connection` names, before the answer is rendered or sent. The other direction: h2 refuses a request carrying one itself, but the h3 crate does not, so `h3.rs` stops such a stream as malformed (RFC 9114 §4.2) rather than hand an HTTP/1.1 service a `connection` naming headers to drop or a `te` that becomes a transfer coding; `te: trailers` is the one allowed. The test stand-ins had never sent either header, which is how the first half went unnoticed.
**One client connection is bounded like one connection.** `max_connections` counts a TCP or QUIC connection once, and on the HTTP edges each stream is an exchange of its own — towards an HTTP/1.1 service, a connection of its own. The h2 crate sets no limit on concurrent streams by default, so one HTTP/2 client could open the protected service as many times as it liked; `h2.rs` advertises `MAX_CONCURRENT_STREAMS` (100, what quinn already allows per QUIC connection). A client's head is bounded the same way on both (`http1::MAX_HEAD_BYTES`, 1 MiB — the budget the filter library holds a stream to by default, and far past what any service accepts, so a large cookie or heavy gRPC metadata is never what it refuses): h2 allowed 16 MiB and h3 had no limit at all. Both crates answer an oversized head with `431` themselves. And the connection dialled before the client's protocol is known is dropped on the HTTP/2 → HTTP/1.1 path, where every exchange dials its own; held, it sat idle towards the service for as long as the client stayed.
**Server push is refused in the handshake, not by dropping what arrives** (`enable_push(false)` towards the service) — the same courtesy the QUIC edge pays about datagrams, and for the same reason: a peer that learns from SETTINGS is better off than one that watches its pushes vanish. **Extended CONNECT is not advertised**, and a plain `CONNECT` stream is reset with a log line rather than rendered: a tunnel is opaque bytes with no HTTP/1.1 message in it, and inventing one would be the fabrication everything here avoids. Both are the WebTransport answer from the QUIC section, arriving at the same place.
**A refusal closes the connection, not the stream**, with `Reason::CANCEL` — deliberately not `REFUSED_STREAM`, which tells a client the request was never acted on and may be retried, and a block a client retries past is not a block. The GOAWAY is polled onto the wire before the socket goes (`GOAWAY_GRACE`), for the same reason the byte pumps shut down rather than abort.
**A bypassed chain is not terminated.** Rendering for a chain with nothing to say is work paid for no answer, so a service with no filters, one whose filters have all been disabled, and a connection admitted past the limit with `over_limit_forwards` all take the byte pump. **What the client opened with is still asked on an optional-TLS listener** (an `http` service), because there the answer decides more than rendering: whether the connection is decrypted, which is what keeps an `Edge::Tls` address's promise and what lets a service answering in the clear be reached over TLS at all. Skipping the peek for a bypassed chain made `opening` always `Other`, and every connection to a TLS address was refused as "not TLS" — a new HTTPS service answered nobody until its first filter, and one went dark the moment its filters were all switched off or had all lost their say, which is the opposite of failing open. `tests/tls.rs` pins it with an empty chain. The cost is that a connection admitted while the chain was empty stays a byte pump for its whole life even if a filter is pushed a moment later — the same trade `over_limit_forwards` itself makes. Note what this does to tests: `tests/h2.rs` spells its no-op chain `LIVE` rather than `""`, because an empty one passed every assertion while exercising none of this, which is the class of false green that found itself immediately (a trailer section that should have been dropped was carried, because nothing had been terminated).
**`h2c` — HTTP/2 in the clear — is recognised from the connection preface, not from a setting.** Most gRPC is deployed without TLS behind a load balancer, so leaving it out would have made "gRPC is filtered" half true. A prior-knowledge client opens with a fixed 24 bytes that no other protocol begins with, and `sniff()` in `proxy.rs` **peeks** rather than reads — nothing is consumed, so a connection that turns out to be something else is handed to the byte pumps exactly as it arrived and there is no buffer to replay. Two properties are load-bearing: it **races the service**, because a protocol where the server speaks first (SMTP, SSH, most game protocols) would otherwise be held waiting for a client that is correctly waiting for a banner, and firegex would look like it broke them at random; and it is **bounded** (`SNIFF_WAIT`), so two silent peers cannot hold the two descriptors open. The same peek answers the TLS question for an `http` service, which is why there is one `Opening` and not two sniffs.
What is *not* carried and is a limit rather than an unconfigured setting: the **HTTP/1.1 `Upgrade: h2c`** handshake. The upgrade request itself is filtered as the HTTP/1.1 request it is, but if the service answers `101` everything after it is HTTP/2 nobody rendered. RFC 9113 deprecated the mechanism and nginx dropped it in 1.25.1, so in practice no service accepts it; say so rather than implying it is handled.
### A service can speak every version of HTTP at once (`http`)
`L4.HTTP` is the odd value in that enum and deliberately so: the other four say what is on the wire, and this one says what the *service* is, letting each address say how it is reached — `:80` in the clear, `:443` under TLS, `:443/udp` under QUIC. One service, one chain, one certificate.
It exists because the alternative was measured against the same rule that put several addresses under one service: HTTP/1.1 and HTTP/2 live on `tls` (TCP) and HTTP/3 on `quic` (UDP), so covering the three took two or three services with the same filter chain copied between them by hand — and a chain kept in step by hand is a chain that stops protecting one of them silently.
**Nothing about the kernel became ambiguous**, because nothing here is asked of the kernel. Each address still carries its own `tcp` or `udp` in `service_addresses.proto`, which is what the rules match and what `(ip, port, proto)` is keyed on; `nftables.py` needed no change at all, because it was already asking the address. What changed is that the questions which used to be asked of the *service* — "does this need a UDP relay", "does the TCP listener need to accept IPv6" — now go through `Service.carries(l4)` and `Service.udp_addresses`, since `Service.l4` no longer has one answer. `L4.l4_of("http")` returns `tcp`, which is the right default for an address that did not say and never a claim about all of them.
**What the connection turns out to be is decided per connection, from what the client sent.** `TlsSetup.optional` is the whole of the mechanism: the TCP listener terminates TLS only for the connections that start a handshake, so the cleartext port and the encrypted one are the same listener, the same chain and the same certificate. The upstream leg mirrors it — a connection that arrived in the clear is forwarded in the clear, one that arrived under TLS is re-encrypted — because the service is being dialled on the port the client chose, and that port's edge is the one the client has just demonstrated. This is the ALPN mirroring rule applied one layer down: firegex carries what the two ends are doing rather than deciding it for them.
Two consequences worth keeping:
- **TLS on TCP and QUIC on UDP now live in one engine process**, which they could not before: `main.rs` built one or the other on the reasoning that a service speaks one or the other. `FGEX_PROXY_TLS` says so explicitly now, and the old implicit rule (a certificate means terminate, unless this is a QUIC service) still applies to everything that does not.
- **A certificate is required**, even for the cleartext addresses beside it. A service that only answers in the clear is a `tcp` one — where HTTP/1.1 is filtered as it always was and HTTP/2 is now recognised from its preface — so the TLS and HTTP/3 edges are the whole of what `http` adds, and neither exists without one.
`http` is **proxy-layer only**, refused at creation by `NfqueueTransport.check` and `ExternalTransport.check` with its own sentence rather than the TLS one or the QUIC one: the reason is neither, it is that an `http` service is reached over TLS *and* QUIC *and* in the clear and that layer can read none of it — not even the cleartext edge, because HTTP/2 in the clear is HPACK and rendering it means terminating the connection.
### Authentication (`backend/app.py`)
Every module router is mounted onto `api = APIRouter(prefix="/api", dependencies=[Depends(is_loggined)])`, so a new file in `routers/` is authenticated by construction. The whole auth surface is four places: that dependency, the Socket.IO `connect` handler (reads the token out of `auth`), and the three routes declared on `app` directly rather than on `api` — `/api/status`, `/api/login`, `/api/set-password`. Adding an unauthenticated route means declaring it on `app`, which should stay a deliberate, rare act.
`--unsafe-disable-auth` (env `UNSAFE_DISABLE_AUTH`) hands access control to a reverse proxy: `check_login` returns `True`, the socket.io handler skips its check, and `APP_STATUS()` reports `run` even with no password stored. The password endpoints (`/api/login`, `/api/set-password`, `/api/change-password`) answer `403` in this mode on purpose — otherwise an anonymous caller could plant a credential that keeps working once authentication is turned back on. `run.py config --password` still works, writing the hash straight into the DB from the host, and an explicit `-P` at first start is kept so that re-enabling authentication lands on a known password instead of the open initial-setup state.
**The flag is state, not a constant.** It lives in `keys_values["auth_disabled"]` and every check reads it (`auth_disabled()`); the environment is only the seed, decided once per boot by `seed_auth_mode`. **Which boot matters.** A container's environment is fixed when it is created, and Docker starting it again by itself — a reboot, a daemon restart, a crash under `restart: unless-stopped` — brings back that environment, older than anything `run.py config` said since: authentication re-enabled with `run.py config --password` was off again after the next reboot, while `config --show` said it was on. So `run.py config` also writes `auth_disabled_host`, and a boot that is not fresh takes it over the environment; `docker-entrypoint.sh` marks the first boot of each container (`/execute/.firegex-booted`, in the container's own layer, gone from a container `run.py` recreates) and exports `FIREGEX_FRESH_BOOT=1`, which `run.py` also sets for every standalone start. A fresh boot takes the environment — which already says what `.firegex-conf.json` says — and clears the key. What `/api/auth-mode` sets writes neither, so it lasts as long as the process, as the interface says. It used to be a module-level constant read at import, which is why `run.py config --password` could print "it will take effect immediately" at a process that had already decided every caller was an administrator — the hash landed in the database and nothing ever asked for it. `run.py`'s `put_runtime_settings()`/`get_runtime_setting()` are the one way in and out of a running instance's `keys_values` (Docker exec or the standalone rootfs, same function), used by both `--password` and `--[no-]unsafe-disable-auth`, and `config --show` says when the running instance disagrees with the persisted config. **`config --password` re-enables authentication when it was off**, live and persisted: setting a password is asking for one to be asked for, and a stored-but-unchecked password is worse than none because it reads like one. It stands down when the same command line said otherwise — an explicit `--unsafe-disable-auth`, or `--keep-auth-disabled` for an instance deliberately held open behind a proxy. Note the password block runs *after* the auth block in `handle_config_command`, which is why the auth block's "there is no password set" warning checks whether one is about to be set.
`POST /api/auth-mode` turns it off and back on at runtime. **The two directions are not symmetrical, and that asymmetry is the design:** off is an administrator's deployment choice, but on, asked while it is off, would let any passer-by set a password and keep the real operator out for good — a lasting foothold, unlike the vandalism the mode already permits. So re-enabling takes a token this instance signed *before* it was turned off. Nothing new is signed while it is off (login and set-password both 403), so holding one is exactly that proof, and the operator who turned it off still has theirs. `token_is_valid()` is split out of `check_login` for this single caller: it needs the question without the "authentication is off, so yes" that `check_login` answers first. Either direction lasts only as long as the process, because the environment is what firegex comes up with — `run.py config` writes the live database, `auth_disabled_host` and `.firegex-conf.json`.
Frontend note: a 401 used to call `window.location.reload()`. A token left over from a database that has since been recreated (a new `secret`) 401s on the first query, the page reloads itself, and what comes back is whatever `APP_STATUS()` calls for — "choose a password", if none is set. That is the "sometimes a password screen appears on refresh" report. The token is now dropped and the app re-renders from the next `/api/status`, and no `Authorization: Bearer null` is sent when there is no token at all.
Independently of that, `IPFilterMiddleware` restricts access by CIDR (`--allowed-ips`, optionally reading the client IP from `--proxy-ip-header`). It **fails closed**: a missing, unparseable, or non-matching IP is denied. Note the header variant trusts a client-supplied value, so it only holds if the proxy overwrites it.
### SQLite wrapper (`backend/utils/sqlite.py`)
Each DB file's schema is versioned by hashing the Python schema dict; on mismatch the **entire file is deleted and recreated** (not migrated) — any schema dict change wipes that DB file on next startup, including `db/firegex.db`'s `keys_values` table (password/JWT secret). `dump()`/`load()` back the export/import backup feature; `dump()` deliberately excludes `password`, `secret`, **`auth_disabled` and `auth_disabled_host`** from exported `keys_values` rows, so `/api/import` explicitly re-preserves all four around the import. The last two are the easiest to lose: they are not credentials, they are *whether credentials are asked for*, which belongs to the deployment and not to the configuration being restored — a backup able to carry them is a backup able to turn a firewall's authentication off. The import restarts the application state with `startup_main(seed_auth=False)`: re-deciding authentication from the environment is what a boot does, and doing it during an import opened a window — the length of every service's start — in which a container booted without authentication answered everybody as an administrator before the preserved value was written back. `tests/integration/test_backup.py` turns authentication off *around* the import on purpose — asserting it is still on afterwards passes either way on a container whose environment agrees, and the bug only shows where the running state and the boot-time environment differ.
### Real-time frontend updates
One Socket.IO event, `"update"`, whose payload is a list of strings treated as a react-query `queryKey` prefix (e.g. `["services"]`, `["firewall"]`). `App.tsx` does `queryClient.invalidateQueries({ queryKey: data })` — prefix matching invalidates any query whose key starts with that array. Convention: each module's mutating endpoints emit the same tag string that module's frontend queries use as the first `queryKey` element. `staleTime: Infinity` is set globally, so this event is the **only** automatic refresh mechanism — a missing/mismatched tag means silent staleness for other clients/tabs.
### Frontend structure
`pages/<Module>/` (routes) + `components/<Module>/` (ServiceRow, AddEditService, `utils.ts` with query key + API wrapper) per module, mirroring the backend split. Detail drill-down lives at `pages/<Module>/ServiceDetails.tsx` under a nested `:srv` route (see `App.tsx`'s `PageRouting`). Theme tokens are CSS variables in `src/index.css`; Mantine `Card` defaults are set globally in `index.tsx`. Prefer Mantine `Group`/`Stack` with `gap` over the legacy `Box className="center-flex"` utility classes still present in older parts of the codebase.
### `fgex-lib/`
Separate, independently-published PyPI package (`pip install fgex`/`firegex`) — the library users write Python filters against (`@pyfilter`, `RawPacket`/`HttpRequest`/etc.), plus the `fgex` CLI, a local `proxysim` for testing filters, and `firegex.regex` for testing *pattern* rulesets, without a real firegex instance.
The library module is **`firegex.pyfilters`**, and the CLI command is `fgex pyfilters`. Both were `nfproxy` when that was one of several modules. A compatibility alias carried the old spellings through 4.x and was **removed in 5.0.0**, which is a breaking change and the reason the version is a major one: `from firegex.nfproxy import pyfilter` is the first line of every filter written before it, including the ones already sitting in a running firegex's database.
Two things about that removal are worth keeping. The directory has to go, not just its contents — a `firegex/nfproxy/` left behind with only a `__pycache__` in it is a *namespace package*, so the import succeeds and hands back something empty, and `from firegex.nfproxy import pyfilter` then fails with an `ImportError` about a name rather than a `ModuleNotFoundError` about a module. And the failure is made to explain itself where the operator meets it: `pyworker.py --check` recognises this one import and says what to write instead, because the alternative is a filter that stops loading at the first start after an upgrade, which at a competition is the start of a round.
Do not reintroduce the alias as a file of re-exports if it ever comes back. `type_annotations_associations` maps model **classes** to protocols, and both the library and the datapath decide when to call a filter by looking an annotation up in that table, so a shim building second copies of `HttpRequest` would leave a filter annotating a class the table has never heard of — importing cleanly and then never being called. The old one aliased submodules into `sys.modules` for exactly that reason: one class per model, two spellings of one package.
**Deploying is `run.py`, and only `run.py`.** The package used to carry a `fgex deploy` passthrough that exec'd a copy of it; it is gone, along with the build-time copy `setup.py` made. The package is for writing and testing filters — `firegex.pyfilters`, `firegex.regex`, the two simulators — and starting a firegex is a separate job done from a clone. Do not reintroduce a deployment command here: the reason the old one was a verbatim passthrough was that any second description of `run.py`'s argument surface drifts, and not shipping one at all is the same argument taken one step further.
**The HTTP parser ships as source, and nobody upstream will bump it.** `fgex-lib/llhttp/` carries llhttp itself (`lib/`, verbatim from a `release/*` tag) and our binding beside it, because `pyllhttp` is archived, every binding with prebuilt wheels fails to load in the subinterpreters `cpproxy` runs filters in, and a ctypes binding against the system `libllhttp` would make `pip install firegex` assume a package installed beforehand. `PROVENANCE.md` there is what makes that maintainable: which tag, which checksums, what is ours and what is theirs, and how to move it. Each release changes what the parser *accepts* — going from 9.2.1 to 9.4.3, an empty `Transfer-Encoding` stopped being accepted and tabs around `Content-Length` started being — so `tests/unit/test_http_parsing.py` pins both through the public models: a change there is a decision about what filters are shown, never a test to fix. A parser more lenient than the service behind it is how a filter gets bypassed while still looking switched on. **The binding names itself and nothing else**: `pyllhttp` is where it was forked from, not something it uses, and the error path that imported it at runtime raised `SystemError: ... returned NULL without setting an exception` on every malformed request anywhere that archived package was not installed — which is everywhere except the machines it was written on. `wheel_smoke.py` is the check that could have caught it, and the reason it is a file rather than a `-c` string: `cibuildwheel` runs it against each wheel in an environment with nothing else in it, parsing something malformed as well as something valid. **A method is named by a switch over `HTTP_ALL_METHOD_MAP`** (`method_name`), never by indexing a table: the table the binding inherited was built from `HTTP_METHOD_MAP`, which runs 0–33 and jumps to QUERY at 46 without PRI or the RTSP methods — all of which a request parser accepts — so `QUERY` and `PLAY` read past its end and `FLUSH rtsp://x/ RTSP/1.0`, one line from any client, crashed the process parsing it (on NFQUEUE that is `cpproxy`, and after five crashes a minute the service is stopped). `tests/unit/test_llhttp_binding.py` asks for every such method in a process of its own. The module's per-interpreter-GIL slot is declared only from 3.12, where it exists: unguarded, the binding did not compile for the 3.10 and 3.11 wheels the release builds, and the publish job needed all of them.
`firegex/regex/` binds `libhs` through ctypes rather than depending on a wheel: no build step, nothing to go stale, and by construction the same library the product loads. It is a third binding to one engine, not a third engine — the distinction that matters is that there is still only one thing deciding what matches. Where libhs is absent the tools raise `HyperscanMissing` and stop; adding an `re` fallback would reintroduce exactly the lie the in-app tester was built to avoid. `tests/unit/test_regex_library.py` pins the agreement: validity in the mode a pattern will run in, direction, the shared file format, and that a ruleset asking to `rewrite` is refused rather than quietly treated as a block. Not part of the backend/frontend build. `fgex-lib/README.md` is a short index (PyPI-facing) linking out to `docs/*.md`; it isn't rendered anywhere itself.
### Module documentation (`docs/*.md`)
Each module's user-facing docs (`docs/services.md`, `docs/pyfilter.md`, `docs/firewall.md`) are single markdown sources, each imported directly with Vite's `?raw` suffix by a matching `<Module>Docs.tsx` component (e.g. `components/Services/ServicesDocs.tsx`) and rendered through the shared `components/MarkdownDocs.tsx` (React Markdown + `remark-gfm`, code fences routed through `CodeHighlight`, relative `.md` cross-links rewritten to the equivalent GitHub blob URL since they'd otherwise resolve against the app's own origin). `DocsButton.tsx`'s `DocType` enum wires each module to its component. Every language a fence uses has to be in `index.tsx`'s `createHighlighter({ langs })` — Shiki throws for one it was not given, and that throw is uncaught, so it takes the whole page down rather than the block it could not colour. Editing a module's docs means editing its one `docs/*.md` file — never the `*Docs.tsx` component directly. Reading these files across the `frontend/`↔repo-root boundary requires `vite.config.ts`'s `server.fs.allow: ['..']` (dev) and the `Dockerfile` frontend build stage's `COPY ./docs/*.md /docs/` (prod/standalone rootfs) to stay in sync with any path changes.