Hi,
We use haproxy in an auto-scaling environment. On an auto-scaling event, the haproxy configuration is rewritten to list all existing servers for each proxied service. A graceful reload is then performed.
The issue is that by default haproxy assumes a server is UP (going down) until the first healthcheck tells it otherwise. If one of the servers for a proxy/backend is not yet actually healthy (e.g. in the initial moments after a new instance is booted), then some requests will be forwarded to the new instance, resulting in 503 responses.
This was a must before 1.6’s server-state-file feature as we’d not want everything to be marked down for a few seconds after a reload. However when using a server-state-file we know the state of every server other than any new ones. We’d like these new ones to be marked DOWN (going up) so they do not receive requests until the first healthcheck is passed.
We’re currently testing a patch which adds an “initial-state up/down” option to each server (and the default-server option) - the default behaviour remains unchanged:
https://github.com/beamly/haproxy-1.6/commit/9e7ad68a0c6582a38591eb27626fdb31bb5f8c18
I’m wondering if this is something that could be considered for a future haproxy release?
Many thanks,
Chris
We use haproxy in an auto-scaling environment. On an auto-scaling event, the haproxy configuration is rewritten to list all existing servers for each proxied service. A graceful reload is then performed.
The issue is that by default haproxy assumes a server is UP (going down) until the first healthcheck tells it otherwise. If one of the servers for a proxy/backend is not yet actually healthy (e.g. in the initial moments after a new instance is booted), then some requests will be forwarded to the new instance, resulting in 503 responses.
This was a must before 1.6’s server-state-file feature as we’d not want everything to be marked down for a few seconds after a reload. However when using a server-state-file we know the state of every server other than any new ones. We’d like these new ones to be marked DOWN (going up) so they do not receive requests until the first healthcheck is passed.
We’re currently testing a patch which adds an “initial-state up/down” option to each server (and the default-server option) - the default behaviour remains unchanged:
https://github.com/beamly/haproxy-1.6/commit/9e7ad68a0c6582a38591eb27626fdb31bb5f8c18
I’m wondering if this is something that could be considered for a future haproxy release?
Many thanks,
Chris
