traefik.md (6266B)
1 --- 2 title: "Traefik HTTP/3 Slow-Body Timeout Testing" 3 section: "Network Services" 4 sectionSlug: "network-services-pentesting" 5 sourcePath: "src/network-services-pentesting/pentesting-web/traefik.md" 6 sourceUrl: "https://github.com/HackTricks-wiki/hacktricks/blob/188de82beb54e70956b2952367a0af91d26758b8/src/network-services-pentesting/pentesting-web/traefik.md" 7 sha: "188de82beb54e70956b2952367a0af91d26758b8" 8 isIndex: false 9 modified: true 10 license: "CC-BY-NC-4.0" 11 --- 12 13 # Traefik HTTP/3 Slow-Body Timeout Testing 14 15 ## HTTP/3 slow-body resource exhaustion 16 17 Traefik normally applies `entryPoints.<name>.transport.respondingTimeouts.readTimeout` to the TCP connection used by HTTP/1.1 and HTTP/2. HTTP/3 uses QUIC instead: in the affected implementation, Traefik copied the HTTPS `Handler` into a separate quic-go `http3.Server` but did not carry over an equivalent request-read deadline. The timeout could therefore appear in parsed configuration and DEBUG output while remaining ineffective for HTTP/3.<sup>[[1]](#references)[[3]](#references)</sup> 18 19 An attacker can start a valid HTTP/3 upload and then send body bytes very slowly, or stop sending without closing the stream. If the reverse proxy has already opened a connection to an application that reads the complete body before responding, every incomplete request can pin one upstream connection. Many low-bandwidth requests can consequently exhaust a bounded backend connection pool without requiring a traffic-volume flood.<sup>[[1]](#references)[[3]](#references)</sup> 20 21 This protocol-differential bug affected Traefik `v2.8.2` through `v2.11.55` and `v3.0.0` through `v3.7.11`; the maintained fixes are `v2.11.56` and `v3.7.12`.<sup>[[1]](#references)</sup> 22 23 ## Differential validation 24 25 Test only an authorized environment and begin with one request per protocol. The backend used for validation **must consume the complete request body before responding** and log when its accepted connection is released; otherwise Traefik may release the upstream immediately and hide whether a resource was retained. Client-side waiting by itself proves only that the client has not received a response.<sup>[[1]](#references)[[3]](#references)</sup> 26 27 For a short lab run, enable HTTP/3 and reduce the incoming read timeout on the same HTTPS entrypoint:<sup>[[1]](#references)</sup> 28 29 ```yaml 30 entryPoints: 31 websecure: 32 address: ":8443" 33 http3: 34 advertisedPort: 8443 35 transport: 36 respondingTimeouts: 37 readTimeout: 5s 38 ``` 39 40 Then send identical uploads over HTTP/1.1 and HTTP/3. Ensure that the total body duration exceeds the configured timeout; a curl build with HTTP/3 support is required for `--http3-only`.<sup>[[1]](#references)[[3]](#references)</sup> 41 42 ```bash 43 URL='https://localhost:8443/' 44 slow_body() { for _ in $(seq 1 8); do printf x; sleep 4; done; } 45 46 slow_body | curl -vk -T - --http1.1 "$URL" 47 slow_body | curl -vk -T - --http3-only "$URL" 48 ``` 49 50 Interpret the test using both proxy output and backend connection-lifetime logs:<sup>[[1]](#references)[[3]](#references)</sup> 51 52 | Observation | Meaning | 53 | --- | --- | 54 | HTTP/1.1 releases the upstream near 5 seconds, but HTTP/3 retains it for the complete upload | The control proves that the setting is active while the QUIC request path bypasses it. | 55 | Both protocols release the upstream near 5 seconds | Equivalent request-read enforcement is probably present. | 56 | Both protocols outlive 5 seconds | The control failed; recheck the active entrypoint, configuration, route, and backend behavior. | 57 | `--http3-only` cannot connect | HTTP/3 was not demonstrated on that endpoint; this does not test the condition. | 58 59 For the default 60-second setting, increase the upload window beyond one minute. The original reproducer used 23 bytes separated by four-second pauses and observed the upstream leg rather than inferring retention from curl alone.<sup>[[1]](#references)[[3]](#references)</sup> 60 61 ## White-box review checklist 62 63 Treat timeout configuration as a claim to verify, not proof of enforcement. Review each protocol-specific server construction and follow the value to the primitive that interrupts body reads:<sup>[[1]](#references)[[3]](#references)</sup> 64 65 - A `SetReadDeadline` call on a TCP socket cannot constrain a QUIC stream. 66 - Reusing an `http.Handler` does not automatically inherit the source `http.Server` timeouts, limits, or cancellation behavior. 67 - A deadline implementation must interrupt a `Read` that is **already blocked** waiting for the next body byte. Merely checking elapsed time after `Read` returns remains bypassable when the peer stops transmitting. 68 - With a custom quic-go body wrapper, expiry must close or cancel the HTTP/3 body/stream; closing quic-go's HTTP/3 body cancels the stream read and unblocks the handler. 69 70 The Traefik fix wraps the HTTP/3 handler and uses `http.NewResponseController(rw).SetReadDeadline(...)` for requests that may carry a body. It also propagates `IdleTimeout` and `MaxHeaderBytes` to `http3.Server`, providing a useful review pattern for protocol-parity regressions.<sup>[[2]](#references)</sup> 71 72 ## Operational signals and remediation 73 74 Slow bodies are syntactically valid and transfer little data, so byte-rate or request-rate alerts alone may miss them. Correlate protocol (`h3`), unusually long request-body duration, active upstream connections, pool wait time, and requests terminated with deadline errors. A growing backend pool with long-lived HTTP/3 uploads is more useful evidence than a client that simply remains connected.<sup>[[1]](#references)[[3]](#references)</sup> 75 76 Upgrade affected installations to Traefik `v2.11.56`, `v3.7.12`, or a later maintained release. Unsupported affected branches require migration rather than waiting for a branch-specific patch.<sup>[[1]](#references)</sup> 77 78 ## References 79 80 - [1] [Traefik security advisory GHSA-7ghq-v6jf-g56c](https://github.com/traefik/traefik/security/advisories/GHSA-7ghq-v6jf-g56c) 81 - [2] [Traefik patch: apply read timeout, idle timeout, and maximum header size to HTTP/3](https://github.com/traefik/traefik/commit/a8d0bc425859dde7481a6c9a324e610b81d754d0) 82 - [3] [Bishop Fox - Traefik HTTP/3 Request Read Timeout Bypass Through Version 3.7.11](https://bishopfox.com/blog/traefik-version-through-3-7-11)