They use it!
10GbE load-balancing (updated)
Web Based User Interface
HATop: Ncurses Interface
Thanks for your support !
August, 14th, 2016 : 1.6.8 and 1.7-dev4
HAProxy 1.6.8 was released on 2016/08/14. It added 19 new commits after version 1.6.7, and fixes all the annoying issues that were still pending in 1.6. So this is the first 1.6 version for which we're not aware of any strange problem.
Among the issues fixed, we can cite the occasional segfault hitting a few users of zlib (it was the result of a misunderstanding of the API apparently), a segfault when using sc_trackers with a different table, a potential memory corruption when using "sni" on the server line, a few crashes in Lua's txn_done() called by actions and fetchers, a risk to freeze some random file descriptors after an attempt to pass data to a file descriptor being experiencing a connection retry explaining some observed zombie connections, some peers protocol encoding issues explaining some abnormal values and synchro losses, a problem with incorrect sample duplication during processing which could lead certain fetchers to fail to work as a stick table key, and I think that's about all for this time. Code and changelog are available here as usual.
For 1.7-dev4, we had quite some new stuff. One change that may affect some users is that we removed the magic consisting in assigning a server's check port to the same port as the first port of the first "bind" directive in the listener if any. It doesn't make sense at all, is not documented, doesn't work in many situations (eg: unix sockets) and makes it impossible to improve the configuration. Normally nobody uses this anymore since 1.6 due to the fact that it is not allowed anymore to specify a port on the "listen" line.
Developers may notice that now everything is rebuilt when they modify a ".h" file. Just do like me, append "DEP=" to your make command line and it will continue to work as before. Non-developer users are protected against easy mistakes and we are not bothered by a dependency hell.
A number of build fixes for OpenBSD were merged. In fact it would not build anymore since 1.6 due to various missing includes (or include order). It's now OK. I'm surprized that we didn't receive any complaint in one year, in the past people would report OpenBSD breakage. Maybe these users are now on FreeBSD which seems to work very well.
There were other updates like "set-src-port", "set-dst", "set-dst-port" actions, to force the incoming src/dst address/port to be replaced by the one in argument (useful for logging and also to force a connection to go to a server configured as 0.0.0.0). Another new action is the "track-sc" for http-response. This is nice to for counting certain response events. The "show tls-keys" CLI command can now display the current secrets. There were some filter changes. The SNI filters now support multicerts (rsa/ecdsa). We can also decode the Netscaler's CIP protocol which is an alternative to haproxy's PROXY protocol. We now have a few new sample fetch functions reporting various TCP-level information on Linux, FreeBSD and NetBSD such as RTT, number of retransmits, etc. It can make logs more usable during troubleshooting. And finally the command-line "-f" argument now supports directories in addition to file names. Files are loaded in alphabetical order. It is convenient for certain users, but beware of the orderning, use at your own risk!
Code and changelog are available here as usual.
April, 13th, 2016 : 1.5.17
I've just released HAProxy 1.5.17 which fixes the CPU usage regression introduced in 1.5.16 after improperly fixing the buffer space calculation code. It also fixes another bug impacting header captures when there are exactly MAX_HDR_HISTORY headers captured (10 by default), in which case the process may crash by dereferencing a pointer at a negative position in the pointers history. The remaining patches are minor bugs and documentation. All users of 1.5 should upgrade to 1.5.17, there are too many bugs impacting older versions. Code and changelog are available here as usual.
March, 14th, 2016 : 1.7-dev2, 1.6.4, 1.5.16, 1.4.27, 1.3.28 (EOL)
After 3 months for the most recent versions and one year for the oldest ones, a new
version was released in each supported branch. This also marks the last 1.3 release
as this branch has now reached end-of-life almost 10 years after it was created.
A number of important bugs were fixed since last releases. Some of them impact 1.6
when using http-reuse (orphaned connections). A few Lua bugs were fixed as well, one
of them causing a segfault and another one dead connections. Sample fetch functions
were protected against misuse of layer 7 in tcp connection rules causing a segfault.
And session variables could also be improperly used in connection rules with the same
effect. For the less important fixes, some race conditions were addressed in the systemd
wrapper possibly causing the orphaned processes some people were experiencing. In 1.5
there were two issues with idle keep-alive timeout handling on the server side, sometimes
causing some short-time busy polling loops and sometimes causing a new incoming request
to be aborted on a persistent connection (that browsers had to resend).
In 1.7-dev we've got the filters which introduce a number of hooks to plug some code in
a more flexible way than analysers. The compression code was already adapted to use them.
More to come later, possibly traffic shaping. The stats have been improved : now everything
available in HTML is also available in the CSV output, and there is a new "typed" output
format that is more friendly to aggregators. It is now possible to manipulate environment
variables from within the config files, this will solve the problem people are facing when
migrating to systemd since it doesn't allow reloaded processes to see changes in environment
As it's been a long time for all versions, users are encouraged to upgrade. Code and
changelogs are available here as usual.
November, 1st, 2015 : 1.5.15
With all the recent activity on 1.6, 1.5 was a little bit left behind. For the last 4 months, a few fixes have accumulated there, including the annoying one striking again on http-send-name-header. Another one may cause the old process to die during a soft reload when a proxy references a disabled peers section. The next annoying one affects those who set memory limits to their processes, as the memory size computation was accidently performed on 32-bits which is limited by todays standards (4GB max) so a typical 5 GB allocation would result in 1 GB only due to integer overflow. The remaining patches are for minor bugs, cleanups and doc updates. For the vast majority of users there's no emergency to update. However if you're deploying now, please consider using this version in order to avoid these bugs later. Code and changelog are available here as usual.
October, 20th, 2015 : 1.6.1
HAProxy 1.6.1 has been released with the fixes for the pending bugs introduced in 1.6.0, including the SSL crash when two certs were present on a line, the issue with impossibility to bind frontends when namespaces were enabled but not used, the incorrect use of ANY type DNS queries, and a few doc and build issues. The small number of bugs (3) in one week is much smaller than what we had in 1.5 (7) in less time, which is encouraging, and matches the quality we were reaching in the last -dev versions. As usual, code and changelog are available here.
October, 13th, 2015 : HAProxy 1.6.0 now released!
HAProxy 1.6.0 has been released. It includes a lot of new features gathered from many contributors during 16 months of development and stabilization. There are too many features to list here. More than 1150 commits were merged from 59 people, and 2/3 came from HAProxy Technologies, meaning that the remaining 1/3 came from the rest of the community, explaining the faster development rate. Among the most user-visible changes, we can cite the simpler handling of multiple configuration files, the support for quotes and environment variables in the configuration, a significant reduction of the memory usage thanks to a new dynamic buffer allocator, notifications over e-mail, server state keeping across reloads, dynamic DNS-based server address resolution, new scripting capabilities thanks to the embedded Lua interpreter, use of variables in the configuration to manipulate samples, request body buffering and analysis, support for two third-party device identification products (DeviceAtlas and 51Degrees), a lot of new sample converters including arithmetic operators and table lookups, TLS ticket secret sharing between nodes, TLS SNI to the server, full tables replication between peers, ability to instruct the kernel to quickly kill dead connections, support for Linux namespaces, and a number of other less visible goodies. The performance has also been improved a lot with support for server connection multiplexing, much faster and cheaper HTTP compression via libslz, and the addition of a pattern cache to speed up certain expensive ACLs. The great flexibility offered by this version will allow many users to significantly simplify their configurations. Some users will notice a huge performance boost after they enable the features designed for them. This release also marks the opening of the 1.6-stable branch and the 1.7-dev branch which is where new development will be done.
The next release date for 1.7.0 is set to end of September 2016, or approximately one year. This time, in order to satisfy more contributors, we'll have a 3-phase development cycle. The first phase ending in March 2016 will merge the most sensitive changes, possibly causing a lot of breakage. It is only for developers. A second phase, ending in June, will be dedicated to fixing the breakage and will still allow small improvements to be made as long as they are not expected to cause regressions. It is possibly where we will decide to revert some of the early breakage if some features are too broken. Enthousiasts may start to test during this phase and report issues. The last phase ending in September will be dedicated to the final polishing, portability issues and doc updates, and should be acceptable for most early adopters. So let's get back to the whiteboards now.
HAProxy is a free, very fast and reliable solution offering
load balancing, and
proxying for TCP and HTTP-based applications. It is particularly suited for very
high traffic web sites and powers quite a number of the world's most visited ones.
Over the years it has become the de-facto standard opensource load balancer, is
now shipped with most mainstream Linux distributions, and is often deployed by
default in cloud platforms. Since it does not advertise itself, we only know it's
used when the admins report it :-)
Its mode of operation makes its integration into existing architectures very easy
and riskless, while still offering the possibility not to expose fragile web servers
to the net, such as below :
We always support at least two active versions in parallel and an extra old
one in critical fixes mode only. The currently supported versions are :
- version 1.5 : the most featureful version, supports SSL, IPv6, keep-alive, DDoS protection, etc...
- version 1.4 : the most stable version for people who don't need SSL. Still provides client-side keep-alive
- version 1.3 : the old stable version for companies who cannot upgrade for internal policy reasons.
Each version brought its set of features on top of the previous one.
Upwards compatibility is a very important aspect of HAProxy, and even
the lastest stable version (1.5) is able to run with configurations made
for version 1.0 13 years ago. The most differenciating features of each
version are listed below :
- version 1.5, released in 2014
This version further expands 1.4 with 4 years of hard work :
native SSL support on both sides with SNI/NPN/ALPN and OCSP stapling,
IPv6 and UNIX sockets are supported everywhere,
full HTTP keep-alive for better support of NTLM and improved efficiency in static farms,
HTTP/1.1 compression (deflate, gzip) to save bandwidth,
PROXY protocol versions 1 and 2 on both sides,
data sampling on everything in request or response, including payload,
ACLs can use any matching method with any input sample
maps and dynamic ACLs updatable from the CLI
stick-tables support counters to track activity on any input sample
custom format for logs, unique-id, header rewriting, and redirects,
improved health checks (SSL, scripted TCP, check agent, ...),
much more scalable configuration supports hundreds of thousands of backends and certificates without sweating
- version 1.4, released in 2010
This version has brought its share of new features over 1.3, most of which were long awaited :
client-side keep-alive to reduce the time to load heavy pages for clients over the net,
TCP speedups to help the TCP stack save a few packets per connection,
response buffering for an even lower number of concurrent connections on the servers,
RDP protocol support with server stickiness and user filtering,
source-based stickiness to attach a source address to a server,
a much better stats interface reporting tons of useful information,
more verbose health checks reporting precise statuses and responses in stats and logs,
traffic-based health to fast-fail a server above a certain error threshold,
support for HTTP authentication for any request including stats, with support for password encryption,
server management from the CLI to enable/disable and change a server's weight without restarting haproxy,
ACL-based persistence to maintain or disable persistence based on ACLs, regardless of the server's state,
log analyzer to generate fast reports from logs parsed at 1 Gbyte/s,
- version 1.3, released in 2006
This version has brought a lot of new features and improvements over 1.2, among which
content switching to select a server pool based on any request criteria,
ACL to write content switching rules, wider choice of
load-balancing algorithms for better integration,
content inspection allowing to block unexpected protocols,
transparent proxy under Linux, which allows to directly connect to
the server using the client's IP address, kernel TCP splicing to forward
data between the two sides without copy in order to reach multi-gigabit data rates,
layered design separating sockets, TCP and HTTP processing for more
robust and faster processing and easier evolutions, fast and fair scheduler
allowing better QoS by assigning priorities to some tasks, session rate limiting
for colocated environments, etc...
Version 1.2 has been in production use since 2006 and provided an improved performance level
on top of 1.1. It is not maintained anymore, as most of its users have switched to 1.3 a long
time ago. Version 1.1, which has been maintaining critical sites online since 2002, is not
maintained anymore either. Users should upgrade to 1.4 or 1.5.
HAProxy is known to reliably run on the following OS/Platforms :
- Linux 2.4 on x86, x86_64, Alpha, Sparc, MIPS, PARISC
- Linux 2.6 / 3.x on x86, x86_64, ARM, Sparc, PPC64
- Solaris 8/9 on UltraSPARC 2 and 3
- Solaris 10 on Opteron and UltraSPARC
- FreeBSD 4.10 - 10 on x86
- OpenBSD 3.1 to -current on i386, amd64, macppc, alpha, sparc64 and VAX (check the ports)
- AIX 5.1 - 5.3 on Power™ architecture
Highest performance is achieved with modern operating systems supporting scalable polling mechanisms such as
epoll on Linux 2.6/3.x or kqueue
on FreeBSD and OpenBSD. This requires haproxy version newer than 1.2.5. Fast data transfers are made possible
on Linux 3.x using TCP splicing and haproxy 1.4 or 1.5. Forwarding rates of up to 40 Gbps have already been
achieved on such platforms after a very careful tuning. While Solaris and AIX are supported, they should not
be used if extreme performance is required.
Current typical 1U servers equipped with a dual-core Opteron or Xeon generally
achieve between 15000 and 40000 hits/s and have no trouble saturating 2 Gbps
Well, since a user's testimony is better than a long demonstration, please take a look at
Chris Knight's experience
with haproxy saturating a gigabit fiber in 2007 on a video download site. Since then,
the performance has significantly increased and the hardware has become much more capable, as
my experiments with
Myricom's 10-Gig NICs have shown two years later. Now as of
2014, 10-Gig NICs are too limited and are hardly suited for 1U servers since they do rarely
provide enough port density to reach speeds above 40-60 Gbps in a 1U server. 100-Gig NICs
are coming and I expect to run new series of tests when they are available.
HAProxy involves several techniques commonly found in Operating Systems
architectures to achieve the absolute maximal performance :
- a single-process,
event-driven model considerably reduces the cost of
and the memory usage. Processing several hundreds of tasks in a millisecond is
possible, and the memory usage is in the order of a few kilobytes per session
while memory consumed in preforked or threaded servers is more in the order of
megabytes per process.
- O(1) event checker on systems that allow it (Linux and FreeBSD)
allowing instantaneous detection of any event on any connection among tens of
- Delayed updates to the event checker using a lazy event cache ensures
that we never update an event unless absolutely required. This saves a lot of
- Single-buffering without any data copy between reads and writes whenever
possible. This saves a lot of CPU cycles and useful memory bandwidth. Often,
the bottleneck will be the I/O busses between the CPU and the network
interfaces. At 10-100 Gbps, the memory bandwidth can become a bottleneck too.
- Zero-copy forwarding is possible using the splice() system
call under Linux, and results in real zero-copy starting with Linux 3.5. This
allows a small sub-3 Watt device such as a Seagate Dockstar to forward HTTP
traffic at one gigabit/s.
memory allocator using fixed size memory pools for immediate memory
allocation favoring hot cache regions over cold cache ones. This dramatically
reduces the time needed to create a new session.
- Work factoring, such as multiple accept() at once, and
the ability to limit the number of accept() per iteration when
running in multi-process mode, so that the load is evenly distributed among
- CPU-affinity is supported when running in multi-process mode, or simply
to adapt to the hardware and be the closest possible to the CPU core managing the
NICs while not conflicting with it.
- Tree-based storage, making heavy use of the Elastic Binary tree I have
been developping for several years. This is used to keep timers ordered, to keep
the runqueue ordered, to manage round-robin and least-conn queues, to look up ACLs
or keys in tables, with only an O(log(N)) cost.
- Optimized timer queue : timers are not moved in the tree if they are
postponed, because the likeliness that they are met is close to zero since they're
mostly used for timeout handling. This further optimizes the ebtree usage.
- optimized HTTP header analysis : headers are parsed an interpreted on
the fly, and the parsing is optimized to avoid an re-reading of any previously
read memory area. Checkpointing is used when an end of buffer is reached with
an incomplete header, so that the parsing does not start again from the
beginning when more data is read. Parsing an average HTTP request typically
takes half a microsecond on a fast Xeon E5.
- careful reduction of the number of expensive system calls. Most of the
work is done in user-space by default, such as time reading, buffer aggregation,
- Content analysis is optimized to carry only pointers to original data and
never copy unless the data needs to be transformed. This ensures that very
small structures are carried over and that contents are never replicated when
not absolutely necessary.
All these micro-optimizations result in very low CPU usage even on moderate
loads. And even at very high loads, when the CPU is saturated, it is quite common
to note figures like 5% user and 95% system, which means that the
HAProxy process consumes about 20 times less than its system counterpart. This
explains why the tuning of the Operating System is very important. This
is the reason why we ended up building
our own appliances,
in order to save that complex and critical task from the end-user.
In production, HAProxy has been installed several times as an emergency solution
when very expensive, high-end hardware load balancers suddenly failed on Layer 7
processing. Some hardware load balancers still do not use proxies and process requests
at the packet level and have a great difficulty at supporting
requests across multiple packets and high response
times because they do no buffering at all. On the
other side, software load balancers use TCP buffering
and are insensible to long requests and high response times. A
nice side effect of HTTP buffering is that it
increases the server's connection acceptance by reducing the
session duration, which leaves room for new requests.
There are 3 important factors used to measure a load balancer's performance :
- The session rate
This factor is very important, because it directly determines when the load
balancer will not be able to distribute all the requests it receives. It is
mostly dependant on the CPU.
Sometimes, you will hear about requests/s or hits/s, and they are the same as
sessions/s in HTTP/1.0 or HTTP/1.1 with
keep-alive disabled. Requests/s with keep-alive enabled is generally much
higher (since it significantly reduces system-side work) but is often meaningless
for internet-facing deployments since clients often open a large amount of connections
and do not send many requests per connection on avertage. This factor is
measured with varying object sizes, the fastest results generally coming from
empty objects (eg: HTTP 302, 304 or 404 response codes).
Session rates around 100,000 sessions/s can be achieved on Xeon E5
systems in 2014.
- The session concurrency
This factor is tied to the previous one. Generally, the session rate
will drop when the number of concurrent sessions increases (except with the
epoll or kqueue polling mechanisms). The slower
the servers, the higher the number of concurrent sessions for a same session rate.
If a load balancer receives 10000 sessions per second and the servers respond in
100 ms, then the load balancer will have 1000 concurrent sessions. This number is
limited by the amount of memory and the amount of file-descriptors the system can
handle. With 16 kB buffers, HAProxy will need about 34 kB per session, which
results in around 30000 sessions per GB of RAM. In practise, socket
buffers in the system also need some memory and 20000 sessions per GB of RAM is
more reasonable. Layer 4 load balancers generally announce millions of
simultaneous sessions because they need to deal with the TIME_WAIT sockets
that the system handles for free in a proxy. Also they don't process any data
so they don't need any buffer. Moreover, they are sometimes designed to be used
in Direct Server Return mode, in which the load balancer only sees forward
traffic, and which forces it to keep the sessions for a long time after their end
to avoid cutting sessions before they are closed.
- The data forwarding rate
This factor generally is at the opposite of the session rate. It is measured
in Megabytes/s (MB/s), or sometimes in Gigabits/s (Gbps). Highest data rates
are achieved with large objects to minimise the overhead caused by session
setup and teardown. Large objects generally increase session concurrency, and
high session concurrency with high data rate requires large amounts of memory
to support large windows. High data rates burn a lot of CPU and bus cycles on
software load balancers because the data has to be copied from the input
interface to memory and then back to the output device. Hardware load balancers
tend to directly switch packets from input port to output port for higher data
rate, but cannot process them and sometimes fail to touch a header or a cookie.
Haproxy on a typical Xeon E5 of 2014 can forward data up to about 40 Gbps.
A fanless 1.6 GHz Atom CPU is slightly above 1 Gbps.
A load balancer's performance related to these factors is generally announced for
the best case (eg: empty objects for session rate, large objects for data rate).
This is not because of lack of honnesty from the vendors, but because it is not
possible to tell exactly how it will behave in every combination. So when those 3
limits are known, the customer should be aware that it will generally perform below
all of them. A good rule of thumb on software load balancers is to consider an
average practical performance of half of maximal session and data rates for
average sized objects.
You might be interested in checking the 10-Gigabit/s page.
Being obsessed with reliability, I tried to do my best to ensure a total
continuity of service by design. It's more difficult to design something
reliable from the ground up in the short term, but in the long term it reveals
easier to maintain than broken code which tries to hide its own bugs behind
respawning processes and tricks like this.
In single-process programs, you have no right to fail : the smallest bug
will either crash your program, make it spin like mad or freeze. There has not
been any such bug found in stable versions for the last 13 years, though
it happened a few times with development code running in production.
HAProxy has been installed on Linux 2.4 systems serving millions of pages
and which have only known one reboot in 3 years for a complete OS upgrade.
Obviously, they were not directly exposed to the Internet because they did not receive
any patch at all. The kernel was a heavily patched 2.4 with Robert Love's
jiffies64 patches to support time wrap-around at 497 days (which
happened twice). On such systems, the software cannot fail without being
immediately noticed !
Right now, it's being used in many Fortune 500 companies around the world to
reliably serve billions of pages per day or relay huge amounts of money. Some
people even trust it so much that they use it as the default solution to solve
simple problems (and I often tell them that they do it the dirty way). Such
people sometimes still use versions 1.1 or 1.2 which sees very limited evolutions
and which targets mission-critical usages. HAProxy is really suited for such environments
because the indicators it returns provide a lot of valuable information about the application's
health, behaviour and defects, which are used to make it even more reliable.
Version 1.3 has now received far more testing than 1.1 and 1.2 combined, so
users are strongly encouraged to migrate to a stable 1.3 or 1.4 for mission-critical
As previously explained, most of the work is executed by the Operating System.
For this reason, a large part of the reliability involves the OS itself. Latest
versions of Linux 2.4 have been known for offering the highest level of stability
ever. However, it requires a bunch of patches to achieve a high level of performance,
and this kernel is really outdated now so running it on recent hardware will often
be difficult (though some people still do). Linux 2.6 and 3.x include the features
needed to achieve this level of performance, but old LTS versions only should be
considered for really stable operations without upgrading more than once a year.
Some people prefer to run it on Solaris (or do not have the choice). Solaris 8 and
9 are known to be really stable right now, offering a level of performance comparable
to legacy Linux 2.4 (without the epoll patch). Solaris 10 might show performances
closer to early Linux 2.6. FreeBSD shows good performance but pf (the firewall)
eats half of it and needs to be disabled to come close to Linux. OpenBSD sometimes
shows socket allocation failures due to sockets staying in FIN_WAIT2 state
when client suddenly disappears. Also, I've noticed that hot reconfiguration does
not work under OpenBSD.
The reliability can significantly decrease when the system is pushed to its
limits. This is why finely tuning the sysctls is important. There is no
general rule, every system and every application will be specific. However, it is
important to ensure that the system will never run out of memory and
that it will never swap. A correctly tuned system must be able to run for
years at full load without slowing down nor crashing.
Security is an important concern when deploying a software load balancer. It is
possible to harden the OS, to limit the number of open ports and accessible
services, but the load balancer itself stays exposed. For this reason, I have been
very careful about programming style. Vulnerabilities are very rarely encountered
on haproxy, and its architecture significantly limits their impact and often allows
easy workarounds. Its remotely unpredictable even processing makes it very hard to
reliably exploit any bug, and if the process ever crashes, the bug is discovered.
All of them were discovered by reverse-analysis of an accidental crash BTW.
Anyway, much care is taken when writing code to manipulate headers. Impossible
state combinations are checked and returned, and errors are processed from the
creation to the death of a session. A few people around the world have reviewed
the code and suggested cleanups for better clarity to ease auditing. By the way,
I'm used to refuse patches that introduce suspect processing or in which not
enough care is taken for abnormal conditions.
I generally suggest starting HAProxy as root because it
can then jail itself in a chroot and drop all of its privileges
before starting the instances. This is not possible if it is not started as
root because only root can execute chroot(),
contrary to what some admins believe.
Logs provide a lot of information to help maintain a satisfying security
level. They are commonly sent over UDP because once chrooted, the
/dev/log UNIX socket is unreachable, and it must not be possible to
write to a file. The following information are particularly useful :
- source IP and port of requestor make it possible to find their origin
in firewall logs ;
- session set up date generally matches firewall logs, while tear
down date often matches proxies dates ;
- proper request encoding ensures the requestor cannot hide
non-printable characters, nor fool a terminal.
- arbitrary request and response header and cookie capture help to
detect scan attacks, proxies and infected hosts.
- timers help to differentiate hand-typed requests from browsers's.
HAProxy also provides regex-based header control. Parts of the request, as
well as request and response headers can be denied, allowed, removed, rewritten, or
added. This is commonly used to block dangerous requests or encodings (eg: the
Apache Chunk exploit),
and to prevent accidental information leak from the server to the client.
Other features such as Cache-control checking ensure that no sensible
information gets accidentely cached by an upstream proxy consecutively to a bug in
the application server for example.
The source code is covered by GPL v2. Source code and pre-compiled binaries for
Linux/x86 and Solaris/Sparc can be downloaded right here for some old versions :
- Development version (1.7) :
- Latest version (1.6) :
- Previous version (1.5) :
- Older version (1.4) :
- Oldest version (1.3) :
- Unmaintained branch (1.2) :
- Various Patches :
- Browsable directory for other files (not only patches)
There are three types of documentation now : the Reference Manual which explains
how to configure HAProxy but which is outdated, the Architecture Guide which will
guide you through various typical setups, and the new Configuration Manual which
replaces the Reference Manual with more a explicit configuration language explanation. The
official documentation is the pure-text one provided with the sources. However, Cyril
Bonté's automated conversion to HTML is much easier to use and constantly up to date,
so it is the preferred one when available.
- Reference Manual for version 1.7 (devel) :
- Reference Manual for version 1.6 (stable) :
- Reference Manual for version 1.5 (stable) :
- Reference Manual for version 1.4 (stable) :
- Reference Manual for version 1.3 (stable) :
- Reference Manual for version 1.2 (old stable) :
- Reference Manual for version 1.1 (unmaintained) :
- architecture.txt : Architecture Guide
- Article on Load Balancing (HTML version) : worth reading for people who don't know what type of load balancer they need
In addition to Cyril's HTML converter above, an automated format converter is being developed by Pavel Lang. At the time of writing these lines, it is able to produce a PDF from the documentation, and some heavy work is ongoing to support other output formats. Please consult the
project's page for more information.
Here's an example
of what it is able to do on version 1.5 configuration manual.
If you think you don't have the time and skills to setup and maintain a free load
balancer, or if you're seeking for commercial support to satisfy your customers or
your boss, you have the following options :
I also find it important to credit Loadbalancer.org. I am
not affiliated with them at all but like us, they have contributed a fair amount of time and money to the
project to add new features and they help users on the mailing list, so I have some respect for what they
do. They're a UK-based company and their load balancer also employs HAProxy, though it is somewhat different
from the ALOHA.
- contact HAProxy Technologies
to hire some professional services or subscribe a support contract ;
- install HAProxy Enterprise Edition (HAPEE),
which is a long-term maintained HAProxy package accompanied by a well-polished collection of software, scripts,
configuration files and documentation which significantly simplifies the setup and maintenance of a completely
operational solution ; it is particularly suited to Cloud environments where deployments must be fast.
- try an ALOHA appliance
(hardware or virtual), which will even save you from having to worry about the system, hardware and from managing a Unix-like
Some happy users have contributed code which may or may not be included. Others
spent a long time analysing the code, and there are some who maintain ports up to
date. The most difficult internal changes have been contributed in the form of
paid time by some big customers who can afford to pay a developer for several
months working on an opensource project. Unfortunately some of them do not want
to be listed, which is the case for the largest of them.
Some contributions were developped and not merged, most often by lack of sign of
interest from the users or simply because they overlap with some pending changes
in a way that could make it harder to maintain future compatibility.
- Geolocation support
Quite some time ago now, Cyril Bonté contacted me about a very interesting
feature he has developped, initially for 1.4, and which now supports both 1.4
and 1.5. This feature is Geolocation, which many users have been asking for
for a long time, and this one does not require to split the IP files by country
codes. In fact it's extremely easy and convenient to configure.
The feature was not merged yet because it does for a specific purpose (GeoIP)
what we wanted to have for a more general use (map converters, session variables,
and use of variables in the redirect URLs), which will allow the same features to
be implemented with more flexibility (eg: extract the IP from a header, or pass
the country code and/or AS number to a backend server, etc...). Cyril was very
receptive to these arguments and accepted to maintain his patchset out of tree
waiting for the features to be implemented (Update: 1.5-dev20 with
maps now make this possible). Cyril's code is well maintained and used in
production so there is no risk in using it on 1.4, except the fact that the
configuration statements will change a bit once you upgrade to 1.5.
The code and documentation are available here : https://github.com/cbonte/haproxy-patches/wiki/Geolocation
- sFlow support
Neil Mckee posted a patch to the list in early 2013, and unfortunately this patch
did not receive any sign of interest nor feedback, which is sad considering the
amount of work that was done. I personally am clueless about sFlow and expressed
my skepticism to Neil about the benefits of sampling some HTTP traffic when you
can get much more detailed informations for free with existing logs.
Neil kindly responded with the following elements :
I agree that the logging you already have in haproxy is more flexible and detailed,
and I acknowledge that the benefit of exporting sFlow-HTTP records is not immediately
The value that sFlow brings is that the measurements are standard, and are designed to
integrate seamlessly with sFlow feeds from switches, routers, servers and applications to
provide a comprehensive end to end picture of the performance of large scale multi-tier
systems. So the purpose is not so much to troubleshoot haproxy in isolation, but to
analyze the performance of the whole system that haproxy is part of.
Perhaps the best illustration of this is the 1-in-N sampling feature.
If you configure sampling.http to be, say, 1-in-400 then you might
only see a handful of sFlow records per second from an haproxy
instance, but that is enough to tell you a great deal about what is
going on -- in real time. And the data will not bury you even if you
have a bank of load-balancers, hundreds of web-servers, a huge
memcache-cluster and a fast network interconnect all contributing
their own sFlow feeds to the same analyzer.
Even after that explanation, no discussion emerged on the subject on the list, so
I guess there is little interest among users for now. I suspect that sFlow is
probably more deployed among network equipments than application layer equipments,
which could explain this situation. The code is large (not huge though) and I am not
convinced about the benefits of merging it and maintaining it if nobody shows even
a little bit of interest. Thus for now I prefer to leave it out of tree. Neil has
posted it on GitHub here :
Please, if you do use this patch, report your feedback to the mailing list, and invest
some time helping with the code review and testing.
This table enumerates all known significant contributions that
led to version 1.4, as well as proposed fundings and features yet to be developped but
waiting for spare time. It is not more up to date though.
Some older code contributions which possibly do not appear in the table above are still listed here.
- Application Cookies
Aleksandar Lazic and Klaus Wagner implemented this feature which
was merged in 1.2. It allows the proxy to learn cookies sent by the server
to the client, and to find it back in the URL to direct the client to the right
server. The learned cookies are automatically purged after some inactive time.
- Least Connections load balancing algorithm
This patch for haproxy-1.2.14 was submitted by Oleksandr Krailo. It implements
a basic least connection algorithm. I've not merged this version into 1.3 because
of scalability concerns, but I'm leaving it here for people who are tempted to
include it into version 1.2, and the patch is really clean.
- Soft Server-Stop
Aleksandar Lazic sent me this patch against 1.1.28 which in fact does two things.
The first interesting part allows one to write a file enumerating servers which
will have to be stopped, and then sending a signal to the running proxy to tell
it to re-read the file and stop using these servers. This will not be merged into
mainline because it has indirect implications on security since the running
process will have to access a file on the file-system, while current version can
run in a chrooted, empty, read-only directory. What is really needed is a way to
send commands to the running process. However, I understand that some people
might need this feature, so it is provided here. The second part of the patch has
been merged. It allowed both an active and a backup server to share a same
cookie. This may sound obvious but it was not possible earlier.
Usage: Aleks says that you just have to write the server names that you
want to stop in the file, then kill -USR2 the running process. I have
not tested it though.
- Server Weight
Sébastien Brize sent me this patch against 1.1.27 which adds the
'weight' option to a server to provide smoother balancing between fast and slow
servers. It is available here because there may be other people looking for this
feature in version 1.1.
I did not include this change because it has a side effect that with
high or unequal weights, some servers might receive lots of consecutive
requests. A different concept to provide a smooth and fair
balancing has been implemented in 1.2.12, which also supports
weighted hash load balancing.
Usage: specify "weight X" on a server line.
Note: configurations written with this patch applied will normally still
work with future 1.2 versions.
- IPv6 support for 1.1.27
I implemented IPv6 support on client side for 1.1.27, and merged it into
haproxy-1.2. Anyway, the patch is still provided here for people who want to
experiment with IPv6 on HAProxy-1.1.
- Other patches
Please browse the directory for other useful
If you don't need all of HAProxy's features and are looking for a simpler solution,
you may find what you need here :
Linux Virtual Servers (LVS)
Very fast layer 3/4 load balancing merged in Linux 2.4 and 2.6 kernels. Should
be coupled with Keepalived to monitor
servers. This generally is the solution embedded by default in most
IP-based load balancers.
Nginx ("engine X")
Nginx is an excellent piece of software. Initially it's a very fast and reliable
web server, but it has grown into a full-featured proxy which can also offer
load-balancing capabilities. Nginx's load balancing features are less advanced
than haproxy's but it can do extra things (eg: caching, running FCGI apps), which
explains why they are very commonly found together. I strongly recommend it to
whoever needs a fast, reliable and flexible web server !
Pound is very small and reasonably good. It aims at remaining small and auditable
prior to being fast. It used to support SSL and keep-alive before HAProxy. Its
configuration file is small and simple. It's thread-based, but can be a simpler
alternative to HAProxy for a small site when the flexibility and performance of
HAProxy are not required.
Pen is a very simple load balancer for TCP protocols. It supports source IP-based
persistence for up to 2048 clients. Supports IP-based ACLs. Uses select()
and supports higher loads than Pound but will not scale very well to thousands of
simultaneous connections. It's more versatile however, and could be considered as
the missing link between HAProxy and socat.
Feel free to contact us for any questions or comments :
Some people regularly ask if it is possible to send donations, so I have set up a Paypal account for this.
Click here if you want to donate.
An IRC channel for haproxy has been opened on FreeNode (but don't seek me there, I'm not) :
Here are some links to possibly useful external contents I gathered on the net.
I have found most of them due to their link to haproxy's site ;-)