Hello
We have been trying to test a shared IP/port ECC/RSA SSL
implementation that is available in HAProxy, and largely followed this
basic setup guide
http://blog.haproxy.com/2015/07/15/serving-ecc-and-rsa-certificates-on-same-ip-with-haproxy/
and adapted it to our setup.
It works fine using a localhost as a listener (see commented lines in
the config below), however with a unix@ socket setup I can not seem to
get the setup to work.
Log messages while accessing the site using sockets are:
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61393
[05/Jan/2017:15:35:16.502] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 2/-1/1 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61394
[05/Jan/2017:15:35:16.512] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61395
[05/Jan/2017:15:35:16.732] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61396
[05/Jan/2017:15:35:16.760] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:21 localhost haproxy[12713]: Z.Z.Z.Z:5909
[05/Jan/2017:15:35:21.779] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:21 localhost haproxy[12713]: Z.Z.Z.Z:61400
[05/Jan/2017:15:35:21.786] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 2/-1/1 0 SC 0/0/0/0/5 0/0
After doing some research before posting to this list, suggestions
here pointed to either a chroot or permissions issue with the socket.
I have tested removing (commenting out) chroot as well as setting
permissions on the socket to 777 as well as ensuring that the socket
is set to haproxy uid/gid via bind-unix global statement, and neither
seem to make any difference.
Does anyone have any additional suggestions to try or sees an obvious
thing I may have missed?
Thanks in advance!
HAProxy compile string:
haproxy -vv
HA-Proxy version 1.7.1 2016/12/13
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Running on zlib version : 1.2.3
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.1.0c 10 Nov 2016
Running on OpenSSL version : OpenSSL 1.1.0c 10 Nov 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 7.8 2008-09-05
Running on PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built without Lua support
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[TRACE] trace
[COMP] compression
Relevant HAProxy config:
global
log 127.0.0.1 local2 notice
pidfile /var/run/haproxy.pid
maxconn 400000
user haproxy
group haproxy
#chroot /var/lib/haproxy
unix-bind prefix /var/run/ mode 600 user haproxy group haproxy
#unix-bind prefix /var/run/ mode 777 user haproxy group haproxy
daemon
nbproc 23
tune.maxrewrite 1024
tune.bufsize 16384
cpu-map 1 1
cpu-map 2 2
cpu-map 3 3
cpu-map 4 4
cpu-map 5 5
cpu-map 6 6
cpu-map 7 7
cpu-map 8 8
cpu-map 9 9
cpu-map 10 10
cpu-map 11 11
cpu-map 12 12
cpu-map 13 13
cpu-map 14 14
cpu-map 15 15
cpu-map 16 16
cpu-map 17 17
cpu-map 18 18
cpu-map 19 19
cpu-map 20 20
cpu-map 21 21
cpu-map 22 22
cpu-map 23 23
stats socket /var/run/haproxy01.stat process 1
stats socket /var/run/haproxy02.stat process 2
stats socket /var/run/haproxy03.stat process 3
stats socket /var/run/haproxy04.stat process 4
stats socket /var/run/haproxy05.stat process 5
stats socket /var/run/haproxy06.stat process 6
stats socket /var/run/haproxy07.stat process 7
stats socket /var/run/haproxy08.stat process 8
stats socket /var/run/haproxy09.stat process 9
stats socket /var/run/haproxy10.stat process 10
stats socket /var/run/haproxy11.stat process 11
stats socket /var/run/haproxy12.stat process 12
stats socket /var/run/haproxy13.stat process 13
stats socket /var/run/haproxy14.stat process 14
stats socket /var/run/haproxy15.stat process 15
stats socket /var/run/haproxy16.stat process 16
stats socket /var/run/haproxy17.stat process 17
stats socket /var/run/haproxy18.stat process 18
stats socket /var/run/haproxy19.stat process 19
stats socket /var/run/haproxy20.stat process 20
stats socket /var/run/haproxy21.stat process 21
stats socket /var/run/haproxy22.stat process 22
stats socket /var/run/haproxy23.stat process 23
ca-base /etc/ssl/certs
crt-base /etc/ssl/certs
tune.ssl.cachesize 256000
tune.ssl.lifetime 900
ssl-default-bind-ciphers
ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
ssl-default-bind-options no-sslv3 no-tls-tickets
tune.ssl.default-dh-param 2048
defaults
log global
option redispatch
retries 5
timeout http-request 30s
timeout queue 60s
timeout connect 30s
timeout client 60s
timeout server 60s
timeout http-keep-alive 30s
timeout check 30s
timeout tarpit 15s
maxconn 400000
errorfile 408 /dev/null
frontend example-ecc-rsa-relay
bind-process 12-13
bind X.X.X.X:443
mode tcp
option tcplog
option dontlog-normal
tcp-request inspect-delay 5s
log 127.0.0.1 local2 debug
acl ip_blacklist src -f /etc/haproxy/ip_blacklist.lst
tcp-request connection reject if ip_blacklist
tcp-request content accept if { req.ssl_ec_ext 0 }
use_backend example-ecc-relay if { req.ssl_ec_ext 1 }
default_backend example-rsa-relay
backend example-ecc-relay
bind-process 12-13
mode tcp
option tcplog
log 127.0.0.1 local2 debug
server example-ecc-rsa-in unix@haproxy_example_ecc_in.sock send-proxy
#server example_ecc_in 127.1.1.67:80 send-proxy
backend example-rsa-relay
bind-process 12-13
mode tcp
option tcplog
log 127.0.0.1 local2 debug
server example-ecc-rsa-in unix@haproxy_example_rsa_in.sock send-proxy
#server example_rsa_in 127.1.1.67:81 send-proxy
frontend example-ecc-rsa-in
bind unix@haproxy_example_ecc_in.sock ssl crt
../ecc_letsencrypt/example.com.ecc.san.dh2048.bundle.pem accept-proxy
bind unix@haproxy_example_rsa_in.sock ssl crt
../rsa_letsencrypt/example.com.rsa.san.dh2048.bundle.pem accept-proxy
#bind 127.1.1.67:80 accept-proxy name example_ecc_in ssl crt
../ecc_letsencrypt/example.com.ecc.san.dh2048.bundle.pem
#bind 127.1.1.67:81 accept-proxy name example_rsa_in ssl crt
../rsa_letsencrypt/example.com.rsa.san.dh2048.bundle.pem
bind-process 12-13
mode tcp
option tcplog
log 127.0.0.1 local2 debug
option dontlog-normal
default_backend example-ssl-out
backend example-ssl-out
bind-process 11
mode http
option httplog
log 127.0.0.1 local2 debug
http-request set-header X-Forced-Ssl on
http-request set-header X-Forwarded-Proto https
http-request set-header X-Forwarded-Ssl on
rspadd Strict-Transport-Security:\ max-age=15552000
server example-ssl-proxy-in-070 127.1.1.70:80 send-proxy
frontend example-http-in
bind-process 11
bind 127.1.1.70:80 accept-proxy name example_ssl_proxy_in_070
bind 1X.X.X.Y:80 name example_http_in
bind 1X.X.X.X:80 name ecc_http_in
mode http
option httplog
option dontlog-normal
log 127.0.0.1 local2 debug
option forwardfor if-none
option http-server-close
tcp-request inspect-delay 5s
default_backend example-http-out
backend example-http-out
bind-process 11
mode http
option httplog
log 127.0.0.1 local2 debug
option http-server-close
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ example.com
balance roundrobin
http-request allow
server www001 10.2.51.1:80 check inter 5s rise 1 fall 1
Socket permissions (with chmod 777):
srwxrwxrwx 1 haproxy haproxy 0 Jan 5 15:46 haproxy_example_ecc_in.sock
srwxrwxrwx 1 haproxy haproxy 0 Jan 5 15:46 haproxy_example_rsa_in.sock
We have been trying to test a shared IP/port ECC/RSA SSL
implementation that is available in HAProxy, and largely followed this
basic setup guide
http://blog.haproxy.com/2015/07/15/serving-ecc-and-rsa-certificates-on-same-ip-with-haproxy/
and adapted it to our setup.
It works fine using a localhost as a listener (see commented lines in
the config below), however with a unix@ socket setup I can not seem to
get the setup to work.
Log messages while accessing the site using sockets are:
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61393
[05/Jan/2017:15:35:16.502] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 2/-1/1 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61394
[05/Jan/2017:15:35:16.512] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61395
[05/Jan/2017:15:35:16.732] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:16 localhost haproxy[12713]: Z.Z.Z.Z:61396
[05/Jan/2017:15:35:16.760] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:21 localhost haproxy[12713]: Z.Z.Z.Z:5909
[05/Jan/2017:15:35:21.779] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 1/-1/0 0 SC 0/0/0/0/5 0/0
Jan 5 15:35:21 localhost haproxy[12713]: Z.Z.Z.Z:61400
[05/Jan/2017:15:35:21.786] example-ecc-rsa-relay
example-ecc-relay/example-ecc-rsa-in 2/-1/1 0 SC 0/0/0/0/5 0/0
After doing some research before posting to this list, suggestions
here pointed to either a chroot or permissions issue with the socket.
I have tested removing (commenting out) chroot as well as setting
permissions on the socket to 777 as well as ensuring that the socket
is set to haproxy uid/gid via bind-unix global statement, and neither
seem to make any difference.
Does anyone have any additional suggestions to try or sees an obvious
thing I may have missed?
Thanks in advance!
HAProxy compile string:
haproxy -vv
HA-Proxy version 1.7.1 2016/12/13
Copyright 2000-2016 Willy Tarreau <willy@haproxy.org>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.3
Running on zlib version : 1.2.3
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.1.0c 10 Nov 2016
Running on OpenSSL version : OpenSSL 1.1.0c 10 Nov 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 7.8 2008-09-05
Running on PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built without Lua support
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available filters :
[SPOE] spoe
[TRACE] trace
[COMP] compression
Relevant HAProxy config:
global
log 127.0.0.1 local2 notice
pidfile /var/run/haproxy.pid
maxconn 400000
user haproxy
group haproxy
#chroot /var/lib/haproxy
unix-bind prefix /var/run/ mode 600 user haproxy group haproxy
#unix-bind prefix /var/run/ mode 777 user haproxy group haproxy
daemon
nbproc 23
tune.maxrewrite 1024
tune.bufsize 16384
cpu-map 1 1
cpu-map 2 2
cpu-map 3 3
cpu-map 4 4
cpu-map 5 5
cpu-map 6 6
cpu-map 7 7
cpu-map 8 8
cpu-map 9 9
cpu-map 10 10
cpu-map 11 11
cpu-map 12 12
cpu-map 13 13
cpu-map 14 14
cpu-map 15 15
cpu-map 16 16
cpu-map 17 17
cpu-map 18 18
cpu-map 19 19
cpu-map 20 20
cpu-map 21 21
cpu-map 22 22
cpu-map 23 23
stats socket /var/run/haproxy01.stat process 1
stats socket /var/run/haproxy02.stat process 2
stats socket /var/run/haproxy03.stat process 3
stats socket /var/run/haproxy04.stat process 4
stats socket /var/run/haproxy05.stat process 5
stats socket /var/run/haproxy06.stat process 6
stats socket /var/run/haproxy07.stat process 7
stats socket /var/run/haproxy08.stat process 8
stats socket /var/run/haproxy09.stat process 9
stats socket /var/run/haproxy10.stat process 10
stats socket /var/run/haproxy11.stat process 11
stats socket /var/run/haproxy12.stat process 12
stats socket /var/run/haproxy13.stat process 13
stats socket /var/run/haproxy14.stat process 14
stats socket /var/run/haproxy15.stat process 15
stats socket /var/run/haproxy16.stat process 16
stats socket /var/run/haproxy17.stat process 17
stats socket /var/run/haproxy18.stat process 18
stats socket /var/run/haproxy19.stat process 19
stats socket /var/run/haproxy20.stat process 20
stats socket /var/run/haproxy21.stat process 21
stats socket /var/run/haproxy22.stat process 22
stats socket /var/run/haproxy23.stat process 23
ca-base /etc/ssl/certs
crt-base /etc/ssl/certs
tune.ssl.cachesize 256000
tune.ssl.lifetime 900
ssl-default-bind-ciphers
ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
ssl-default-bind-options no-sslv3 no-tls-tickets
tune.ssl.default-dh-param 2048
defaults
log global
option redispatch
retries 5
timeout http-request 30s
timeout queue 60s
timeout connect 30s
timeout client 60s
timeout server 60s
timeout http-keep-alive 30s
timeout check 30s
timeout tarpit 15s
maxconn 400000
errorfile 408 /dev/null
frontend example-ecc-rsa-relay
bind-process 12-13
bind X.X.X.X:443
mode tcp
option tcplog
option dontlog-normal
tcp-request inspect-delay 5s
log 127.0.0.1 local2 debug
acl ip_blacklist src -f /etc/haproxy/ip_blacklist.lst
tcp-request connection reject if ip_blacklist
tcp-request content accept if { req.ssl_ec_ext 0 }
use_backend example-ecc-relay if { req.ssl_ec_ext 1 }
default_backend example-rsa-relay
backend example-ecc-relay
bind-process 12-13
mode tcp
option tcplog
log 127.0.0.1 local2 debug
server example-ecc-rsa-in unix@haproxy_example_ecc_in.sock send-proxy
#server example_ecc_in 127.1.1.67:80 send-proxy
backend example-rsa-relay
bind-process 12-13
mode tcp
option tcplog
log 127.0.0.1 local2 debug
server example-ecc-rsa-in unix@haproxy_example_rsa_in.sock send-proxy
#server example_rsa_in 127.1.1.67:81 send-proxy
frontend example-ecc-rsa-in
bind unix@haproxy_example_ecc_in.sock ssl crt
../ecc_letsencrypt/example.com.ecc.san.dh2048.bundle.pem accept-proxy
bind unix@haproxy_example_rsa_in.sock ssl crt
../rsa_letsencrypt/example.com.rsa.san.dh2048.bundle.pem accept-proxy
#bind 127.1.1.67:80 accept-proxy name example_ecc_in ssl crt
../ecc_letsencrypt/example.com.ecc.san.dh2048.bundle.pem
#bind 127.1.1.67:81 accept-proxy name example_rsa_in ssl crt
../rsa_letsencrypt/example.com.rsa.san.dh2048.bundle.pem
bind-process 12-13
mode tcp
option tcplog
log 127.0.0.1 local2 debug
option dontlog-normal
default_backend example-ssl-out
backend example-ssl-out
bind-process 11
mode http
option httplog
log 127.0.0.1 local2 debug
http-request set-header X-Forced-Ssl on
http-request set-header X-Forwarded-Proto https
http-request set-header X-Forwarded-Ssl on
rspadd Strict-Transport-Security:\ max-age=15552000
server example-ssl-proxy-in-070 127.1.1.70:80 send-proxy
frontend example-http-in
bind-process 11
bind 127.1.1.70:80 accept-proxy name example_ssl_proxy_in_070
bind 1X.X.X.Y:80 name example_http_in
bind 1X.X.X.X:80 name ecc_http_in
mode http
option httplog
option dontlog-normal
log 127.0.0.1 local2 debug
option forwardfor if-none
option http-server-close
tcp-request inspect-delay 5s
default_backend example-http-out
backend example-http-out
bind-process 11
mode http
option httplog
log 127.0.0.1 local2 debug
option http-server-close
option httpchk OPTIONS * HTTP/1.1\r\nHost:\ example.com
balance roundrobin
http-request allow
server www001 10.2.51.1:80 check inter 5s rise 1 fall 1
Socket permissions (with chmod 777):
srwxrwxrwx 1 haproxy haproxy 0 Jan 5 15:46 haproxy_example_ecc_in.sock
srwxrwxrwx 1 haproxy haproxy 0 Jan 5 15:46 haproxy_example_rsa_in.sock