# Nginx Upstream Pools in Production: Consistent Hashing, Passive Health Checks, and the 1.29.7 Keepalive Shift

> A Juejin engineering deep dive measures how consistent hashing limits key migration, why max_fails=0 never ejects a dead node, and how nginx 1.29.7 changed upstream keepalive defaults.

- Publication: Thoai Dispatch
- Canonical URL: https://vokythoai.dev/article/nginx-upstream-consistent-hashing-health-checks-keepalive
- Section: News
- Author: Juejin Tech (CN)
- Published: 2026-11-21
- Tags: nginx, load-balancing, consistent-hashing, health-checks, keepalive, upstream, devops, infrastructure

Juejin Tech's seventh installment of its 15-part Nginx series moves beyond three-machine round-robin to production-grade upstream groups: algorithm selection, session stickiness, passive ejection with `proxy_next_upstream`, DNS self-healing in containers, and Layer 4 forwarding via `stream`. All figures in the article were measured on nginx 1.30.6 (stable), per [the original post](https://juejin.cn/post/7710084602304446507).

## Why it matters

Most Nginx tutorials stop at round-robin. The operational cost appears later: a cache shard rebalance that invalidates the whole cache, a dead upstream that keeps receiving traffic, or a keepalive configuration copied from an old guide that silently opens a new TCP connection per request. The article quantifies each of these failure modes.

## Technical details

### Consistent hashing limits migration

With plain `hash $key`, the article measured that shrinking a pool from three nodes to two remapped 178 of 300 uids, or 59.33%. Adding `consistent` (the ketama ring) moved only 102 of 300, or 34.00%, close to the theoretical 1/3. The practical rule: cache sharding must enable `consistent`, or a single scale-out event invalidates the entire cache at once. The key choice also matters. `hash $arg_uid` is convenient for reproducible load tests, but production should use `hash $cookie_uid` or `hash $http_x_user_id`, since a dropped URL parameter degrades silently to random distribution while requests still return 200.

### Passive health checks have sharp edges

Open source Nginx offers only passive checks. `max_fails` defaults to 1 and `fail_timeout` to 10s, which is both the counting window and the ejection duration; what counts as a failure is defined by `proxy_next_upstream` (default `error timeout`). Two measured behaviors stand out. First, failover is not free: with one of three nodes stopped and `max_fails=3`, the first three requests routed to the dead node each paid roughly 1s of `proxy_connect_timeout` before switching, so the article recommends 1s or 2s rather than the 60s default. Second, the failure counter only resets on success, so a node returned to the pool after `fail_timeout` is ejected again immediately if its first request fails. Setting `max_fails=0` disables counting entirely: the article measured requests hashed to a stopped node waiting about 1s every time, with only `upstream timed out` in the error log and no `temporarily disabled` entry.

Active checks (`health_check`) and the smoothing parameters `slow_start` and `queue` are Nginx Plus only; the article reports `unknown directive` and `invalid parameter` errors when they are used in open source builds. Alternatives listed are external probes with template-driven reloads, OpenResty/Lua shared-memory probing, or DNS-based service discovery.

### Keepalive defaults changed at 1.29.7

Using a probe backend that reports the source port of each upstream TCP connection, the article counted distinct ports across 20 requests. On nginx 1.30.6 with no explicit configuration, all 20 reused a single connection. On 1.24-alpine with no configuration, 20 requests used 20 connections, because requests went out as HTTP/1.0. Since 1.29.7, Nginx defaults to HTTP/1.1 with keep-alive to upstreams, and the documented `keepalive` default is `keepalive 32 local`, a 32-slot pool not shared across locations. The historical `proxy_http_version 1.1` plus empty `Connection` header pair is still required on 1.29.6 and earlier. The article also notes that `keepalive N` caps idle connections per worker, and that `proxy_socket_keepalive` only sets TCP `SO_KEEPALIVE`, unrelated to the pool.

### Session stickiness

The article's decision table favors `sticky cookie` (available from 1.29.6) for stateful sessions, `hash $key consistent` for backend cache affinity, and `ip_hash` only as a fallback, since it collapses behind NAT/CGNAT and fails with IPv6. In a measured three-node test, the first response set `srv_id=a` and the next four cookie-bearing requests all landed on app1.

## What to watch

Audit upstream blocks for `max_fails=0`, which hides dead nodes behind repeated timeouts. Lower `proxy_connect_timeout` so failover cost stays bounded. Verify whether keepalive settings were copied from pre-1.29.7 guides. Choose the hashing key deliberately, and treat `ip_hash` as a last resort.

## Sources

- [Juejin Tech (CN)](https://juejin.cn/post/7710084602304446507) (2026-11-20)
