Hello again all,
I have a configuration problem, and I have a workaround that I don't like,
so I'm hoping someone here might have a better solution -
I have a number of capture fields such as ssl_c_s_dn, request header
referrer, request header X-Forwarded-For, etc that get logged using %hr,
among another 12 or so fields such as %t, %H, %B, %ci and so on.
The problem is the client has an established log format which goes to an
enterprise centralized log db. I'd love to be able to write out this
format directly with haproxy, but since all the captured fields in haproxy
get appended together by %hr and %hs, I don't know of a way to put one
captured field in spot #4 of a log line, and then another captured field in
spot #7, etc.
In order to work around this, I have haproxy instead outputting all this
data to an "intermediate" log format, and then have another process running
which picks these up and rearranges them into the arbitrary log format that
the enterprise accepts.
I'd love to get rid of having to maintain this separate process long term.
When I first saw the discussion of defined captures, I thought maybe
there'd be a way to use those definitions in the log format directly, so
you could have a log line that was in a certain sequence where the captures
were mixed in: time, client ip, client subject dn, HTTP request uri, http
status code, X-Forwarded-For IP, bytes transferred, etc (not the actual
format, but you get the point). However, I don't see a way to do that.
Then I saw the LUA logging calls, and so I was reading the API, trying to
figure out if I could write the whole log statement using LUA, but without
having tried it yet, is it a) possible to do so for every request, and b )
is there a performance penalty for doing so? Is there another way to
handle this?
How have other people dealt with this problem?
Thanks!
Phillip
PS - That was an amazing find last week on the pipelining bug; thanks for
your hard work! It's appreciated.
I have a configuration problem, and I have a workaround that I don't like,
so I'm hoping someone here might have a better solution -
I have a number of capture fields such as ssl_c_s_dn, request header
referrer, request header X-Forwarded-For, etc that get logged using %hr,
among another 12 or so fields such as %t, %H, %B, %ci and so on.
The problem is the client has an established log format which goes to an
enterprise centralized log db. I'd love to be able to write out this
format directly with haproxy, but since all the captured fields in haproxy
get appended together by %hr and %hs, I don't know of a way to put one
captured field in spot #4 of a log line, and then another captured field in
spot #7, etc.
In order to work around this, I have haproxy instead outputting all this
data to an "intermediate" log format, and then have another process running
which picks these up and rearranges them into the arbitrary log format that
the enterprise accepts.
I'd love to get rid of having to maintain this separate process long term.
When I first saw the discussion of defined captures, I thought maybe
there'd be a way to use those definitions in the log format directly, so
you could have a log line that was in a certain sequence where the captures
were mixed in: time, client ip, client subject dn, HTTP request uri, http
status code, X-Forwarded-For IP, bytes transferred, etc (not the actual
format, but you get the point). However, I don't see a way to do that.
Then I saw the LUA logging calls, and so I was reading the API, trying to
figure out if I could write the whole log statement using LUA, but without
having tried it yet, is it a) possible to do so for every request, and b )
is there a performance penalty for doing so? Is there another way to
handle this?
How have other people dealt with this problem?
Thanks!
Phillip
PS - That was an amazing find last week on the pipelining bug; thanks for
your hard work! It's appreciated.