Busk at 01:12, 24 December 2022

2022-12-24T01:12:16Z

Busk at 01:09, 24 December 2022

2022-12-24T01:09:51Z

Busk: Created page with "# sumo - https://www.sumologic.com/blog/haproxy-log-format/"

2022-12-24T00:53:16Z

Created page with "# sumo - https://www.sumologic.com/blog/haproxy-log-format/"

New page

# sumo
- https://www.sumologic.com/blog/haproxy-log-format/

← Older revision		Revision as of 01:12, 24 December 2022
Line 3:		Line 3:

	https://www.haproxy.com/blog/haproxy-log-customization/		https://www.haproxy.com/blog/haproxy-log-customization/
		+
		+
		+	```
		+	HAProxy Log Customization
		+	Baptiste Assmann \| Oct 29, 2012 \| BASICS, TECH \| 10 comments
		+
		+	One of HAProxy’s greatest strengths has got to be its logging system. The amount of information it provides can be invaluable when diagnosing unexpected behavior, studying your website’s traffic, or just getting a feel for how HAProxy works.
		+
		+	HAProxy comes with a few standard log formats that define which fields will be captured. Common wisdom says that you should add the option httplog configuration directive to your frontend or defaults section when using HAProxy as a Layer 7 proxy. Or, use option tcplog when using it as a Layer 4 proxy. Either one will give your logs an instant power-up, making them more comprehensive.
		+
		+	Sometimes, though, these premade settings are not exactly what you need. In that case, you can configure a custom log format. In this blog post, you’ll see how.
		+
		+
		+
		+	Find out more in our blog post “Introduction to HAProxy Logging” or by registering for our webinar: “Deep Dive Into HAProxy Logging“
		+
		+	Customizing the HAProxy Logs
		+	There are plenty of reasons why you might want to customize the fields captured by the HAProxy logs. For example:
		+
		+	the default log format is giving you more information that you need
		+	you’re missing an important piece of information with the default log format
		+	you need to structure the fields in a way that an external tool can read them
		+	you rely on a standard log format and HAProxy must also comply
		+	Luckily, it’s easy to add a template that says which fields to log and in which order. You just need to use the log-format directive. Let’s dive into its structure.
		+
		+	Adding a New HAProxy Log Format
		+	The log-format directive goes into your defaults, frontend or listen section. It specifies a string that contains variables referring to the fields that you’d like to capture.
		+
		+	defaults
		+	log-format "<format-string>"
		+	view raw
		+	When you place this into your defaults section, it affects all of the proxy sections (frontend and listen) that follow. If you need a different format for a particular proxy, adding another log-format to that section will override the default.
		+
		+	In the next section, you’ll learn about the variables that are available.
		+
		+	Variables
		+	A large part of your custom log format will likely be made up of variables that are pre-defined by HAProxy. Others you will define yourself.
		+
		+	In the broadest terms, all variables follows the rules below when added to a format string:
		+
		+	It is preceded by a percent character: %
		+	It can take flags in braces {}
		+	If multiple flags, then they are separated by commas within the braces
		+	Flags may be added or removed by prefixing them with a + or a – sign
		+	Spaces, other than those between variables, must be escaped by a backslash
		+	Currently, there are three flags:
		+
		+	Q: quotes a string
		+	X: hexadecimal representation
		+	E: escapes characters (“, \, ]) with backslashes
		+	Here are a few examples:
		+
		+	log-format "%{+Q}r"
		+	# Outputs: "GET / HTTP/1.1"
		+
		+	log-format "%{+X}Ts"
		+	# Outputs: 5C5342A0
		+
		+	log-format "%{+E}HQ"
		+	# Outputs: ?myparam=[some_brackets\]
		+
		+	log-format "%{+E,+Q}HQ"
		+	# Outputs: "?myparam=[some_brackets\]"
		+	view raw
		+	The pre-defined variables (also available in the documentation) are:
		+
		+	Variable Meaning
		+	%B bytes read
		+	%CC captured request cookie
		+	%CS captured response cookie
		+	%H hostname
		+	%HM HTTP method (e.g. POST)
		+	%HP HTTP request URI without query string
		+	%HQ HTTP request URI query string
		+	%HU HTTP request URI
		+	%HV HTTP version (e.g. HTTP/1.1)
		+	%ID unique ID
		+	%ST status code
		+	%T GMT datetime
		+	%Ta active time of requests (from %TR to end)
		+	%Tc time to establish TCP connection to server
		+	%Td Tt – (Tq + Tw + Tc + Tr)
		+	%Tl local datetime
		+	%Th connection handshake time (SSL, PROXY protocol)
		+	%Ti idle time before the HTTP request
		+	%Tq time to get the client request
		+	%TR time to receive the full request from first byte
		+	%Tr response time
		+	%Ts timestamp
		+	%Tt total session duration time
		+	%U bytes uploaded
		+	%ac process concurrent connections
		+	%b backend name
		+	%bc backend concurrent connections
		+	%bi backend source IP (HAProxy connects with)
		+	%bp backend source port (HAProxy connects with)
		+	%bq backend queue
		+	%ci client IP
		+	%cp client port
		+	%f frontend name
		+	%fc frontend concurrent connections
		+	%fi frontend IP
		+	%fp frontend port
		+	%ft frontend name transport (‘~’ suffix for SSL)
		+	%lc frontend log counter
		+	%hr captured request headers
		+	%hrl captured request headers (CLF style)
		+	%hs captured response headers
		+	%hsl captured response headers (CLF style)
		+	%ms accept date (milliseconds, left-padded with 0)
		+	%pid HAProxy process ID
		+	%r HTTP request
		+	%rc number of retries
		+	%rt request counter
		+	%s server name
		+	%sc server concurrent connections
		+	%si server IP
		+	%sp server port
		+	%sq server queue
		+	%sslc SSL cipher used (e.g. AES-SHA)
		+	%sslv SSL version (e.g. TSLv1)
		+	%t datetime (with millisecond resolution)
		+	%tr datetime of start of HTTP request
		+	%trg GMT datetime of start of HTTP request
		+	%trl local datetime of start of HTTP request
		+	%ts termination state
		+	%tsc termination state with cookie status
		+
		+
		+	The default log formats use these variables in the following ways:
		+
		+	TCP log format:
		+
		+	log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq"
		+	view raw
		+	HTTP log format:
		+
		+	log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r"
		+	view raw
		+	CLF log format:
		+
		+	log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B \"\" \"\" %cp %ms %ft %b %s %TR %Tw %Tc %Tr %Ta %tsc %ac %fc %bc %sc %rc %sq %bq %CC %CS %hrl %hsl"
		+	view raw
		+	You can create your own format by using any of the pre-defined variables in the order that you wish. The following logs the client’s IP address and port, the server’s IP address and port, the path, and the response status:
		+
		+	log-format "%ci:%cp %si:%sp %HU %ST"
		+	# Outputs: 192.168.50.1:56428 127.0.0.1:80 / 304
		+	view raw
		+	If you prefer, you can add text to give the fields more context. Placing %{+Q}o at the beginning will apply the flag, in this case quotes, to all of the following strings.
		+
		+	log-format "%{+Q}o\ client_address = %ci, client_port = %cp, server_address = %si, server_port = %sp path, = %HU, status = %ST"
		+	# Outputs: client_address = "192.168.50.1", client_port = 56479, server_address = "127.0.0.1", server_port = 8080, path = "/", status = 304
		+	view raw
		+	You can also capture specific pieces of data not represented by the pre-defined variables. For example, you can log request headers. You’ll need to capture them first, though, and then refer to them by number. The numbers are incremented in the order in which the field was captured. Here are a few examples:
		+
		+	http-request capture req.hdr(Host) len 10
		+	http-request capture req.hdr(Cookie) len 20
		+
		+	log-format "%[capture.req.hdr(0)] %{+Q}[capture.req.hdr(1)]"
		+	# Outputs: example.com "MyCookie=abc123"
		+	view raw
		+	Custom variables can be logged by using the var fetch method. Note that you must prefix your variable names with txn.
		+
		+	http-request set-var(txn.MyVar) str("My Value")
		+	log-format "%{+Q}[var(txn.MyVar)]"
		+	# Outputs: "My Value"
		+	view raw
		+	Fetch methods can be included by placing them inside square brackets. In this example, the http_first_req fetch shows a 1 if this is the first request from the client, or a 0 if not.
		+
		+	log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %[http_first_req]"
		+	# Outputs: 192.168.50.1:53547 [16/Oct/2019:13:52:09.226] fe_main be_servers/s1 0/0/1/2/3 200 465 - - ---- 2/1/0/0/0 0/0 "GET / HTTP/1.1" 1
		+	```

← Older revision		Revision as of 01:09, 24 December 2022
Line 1:		Line 1:
	# sumo		# sumo
	- https://www.sumologic.com/blog/haproxy-log-format/		- https://www.sumologic.com/blog/haproxy-log-format/
		+
		+	https://www.haproxy.com/blog/haproxy-log-customization/

Haproxy logs - Revision history

Busk at 01:12, 24 December 2022

Busk at 01:09, 24 December 2022

Busk: Created page with "# sumo - https://www.sumologic.com/blog/haproxy-log-format/"