2020/07/07 - HAProxy coding style - Willy Tarreau ------------------------------------------------------------ A number of contributors are often embarrassed with coding style issues, they don't always know if they're doing it right, especially since the coding style has elvoved along the years. What is explained here is not necessarily what is applied in the code, but new code should as much as possible conform to this style. Coding style fixes happen when code is replaced. It is useless to send patches to fix coding style only, they will be rejected, unless they belong to a patch series which needs these fixes prior to get code changes. Also, please avoid fixing coding style in the same patches as functional changes, they make code review harder. A good way to quickly validate your patch before submitting it is to pass it through the Linux kernel's checkpatch.pl utility which can be downloaded here : http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/scripts/checkpatch.pl Running it with the following options relaxes its checks to accommodate to the extra degree of freedom that is tolerated in HAProxy's coding style compared to the stricter style used in the kernel : checkpatch.pl -q --max-line-length=160 --no-tree --no-signoff \ --ignore=LEADING_SPACE,CODE_INDENT,DEEP_INDENTATION \ --ignore=ELSE_AFTER_BRACE < patch You can take its output as hints instead of strict rules, but in general its output will be accurate and it may even spot some real bugs. When modifying a file, you must accept the terms of the license of this file which is recalled at the top of the file, or is explained in the LICENSE file, or if not stated, defaults to LGPL version 2.1 or later for files in the 'include' directory, and GPL version 2 or later for all other files. When adding a new file, you must add a copyright banner at the top of the file with your real name, e-mail address and a reminder of the license. Contributions under incompatible licenses or too restrictive licenses might get rejected. If in doubt, please apply the principle above for existing files. All code examples below will intentionally be prefixed with " | " to mark where the code aligns with the first column, and tabs in this document will be represented as a series of 8 spaces so that it displays the same everywhere. 1) Indentation and alignment ---------------------------- 1.1) Indentation ---------------- Indentation and alignment are two completely different things that people often get wrong. Indentation is used to mark a sub-level in the code. A sub-level means that a block is executed in the context of another block (eg: a function or a condition) : | main(int argc, char **argv) | { | int i; | | if (argc < 2) | exit(1); | } In the example above, the code belongs to the main() function and the exit() call belongs to the if statement. Indentation is made with tabs (\t, ASCII 9), which allows any developer to configure their preferred editor to use their own tab size and to still get the text properly indented. Exactly one tab is used per sub-level. Tabs may only appear at the beginning of a line or after another tab. It is illegal to put a tab after some text, as it mangles displays in a different manner for different users (particularly when used to align comments or values after a #define). If you're tempted to put a tab after some text, then you're doing it wrong and you need alignment instead (see below). Note that there are places where the code was not properly indented in the past. In order to view it correctly, you may have to set your tab size to 8 characters. 1.2) Alignment -------------- Alignment is used to continue a line in a way to makes things easier to group together. By definition, alignment is character-based, so it uses spaces. Tabs would not work because for one tab there would not be as many characters on all displays. For instance, the arguments in a function declaration may be broken into multiple lines using alignment spaces : | int http_header_match2(const char *hdr, const char *end, | const char *name, int len) | { | ... | } In this example, the "const char *name" part is aligned with the first character of the group it belongs to (list of function arguments). Placing it here makes it obvious that it's one of the function's arguments. Multiple lines are easy to handle this way. This is very common with long conditions too : | if ((len < eol - sol) && | (sol[len] == ':') && | (strncasecmp(sol, name, len) == 0)) { | ctx->del = len; | } If we take again the example above marking tabs with "[-Tabs-]" and spaces with "#", we get this : | [-Tabs-]if ((len < eol - sol) && | [-Tabs-]####(sol[len] == ':') && | [-Tabs-]####(strncasecmp(sol, name, len) == 0)) { | [-Tabs-][-Tabs-]ctx->del = len; | [-Tabs-]} It is worth noting that some editors tend to confuse indentations and alignment. Emacs is notoriously known for this brokenness, and is responsible for almost all of the alignment mess. The reason is that Emacs only counts spaces, tries to fill as many as possible with tabs and completes with spaces. Once you know it, you just have to be careful, as alignment is not used much, so generally it is just a matter of replacing the last tab with 8 spaces when this happens. Indentation should be used everywhere there is a block or an opening brace. It is not possible to have two consecutive closing braces on the same column, it means that the innermost was not indented. Right : | main(int argc, char **argv) | { | if (argc > 1) { | printf("Hello\n"); | } | exit(0); | } Wrong : | main(int argc, char **argv) | { | if (argc > 1) { | printf("Hello\n"); | } | exit(0); | } A special case applies to switch/case statements. Due to my editor's settings, I've been used to align "case" with "switch" and to find it somewhat logical since each of the "case" statements opens a sublevel belonging to the "switch" statement. But indenting "case" after "switch" is accepted too. However in any case, whatever follows the "case" statement must be indented, whether or not it contains braces : | switch (*arg) { | case 'A': { | int i; | for (i = 0; i < 10; i++) | printf("Please stop pressing 'A'!\n"); | break; | } | case 'B': | printf("You pressed 'B'\n"); | break; | case 'C': | case 'D': | printf("You pressed 'C' or 'D'\n"); | break; | default: | printf("I don't know what you pressed\n"); | } 2) Braces --------- Braces are used to delimit multiple-instruction blocks. In general it is preferred to avoid braces around single-instruction blocks as it reduces the number of lines : Right : | if (argc >= 2) | exit(0); Wrong : | if (argc >= 2) { | exit(0); | } But it is not that strict, it really depends on the context. It happens from time to time that single-instruction blocks are enclosed within braces because it makes the code more symmetrical, or more readable. Example : | if (argc < 2) { | printf("Missing argument\n"); | exit(1); | } else { | exit(0); | } Braces are always needed to declare a function. A function's opening brace must be placed at the beginning of the next line : Right : | int main(int argc, char **argv) | { | exit(0); | } Wrong : | int main(int argc, char **argv) { | exit(0); | } Note that a large portion of the code still does not conforms to this rule, as it took years to get all authors to adapt to this more common standard which is now preferred, as it avoids visual confusion when function declarations are broken on multiple lines : Right : | int foo(const char *hdr, const char *end, | const char *name, const char *err, | int len) | { | int i; Wrong : | int foo(const char *hdr, const char *end, | const char *name, const char *err, | int len) { | int i; Braces should always be used where there might be an ambiguity with the code later. The most common example is the stacked "if" statement where an "else" may be added later at the wrong place breaking the code, but it also happens with comments or long arguments in function calls. In general, if a block is more than one line long, it should use braces. Dangerous code waiting of a victim : | if (argc < 2) | /* ret must not be negative here */ | if (ret < 0) | return -1; Wrong change : | if (argc < 2) | /* ret must not be negative here */ | if (ret < 0) | return -1; | else | return 0; It will do this instead of what your eye seems to tell you : | if (argc < 2) | /* ret must not be negative here */ | if (ret < 0) | return -1; | else | return 0; Right : | if (argc < 2) { | /* ret must not be negative here */ | if (ret < 0) | return -1; | } | else | return 0; Similarly dangerous example : | if (ret < 0) | /* ret must not be negative here */ | complain(); | init(); Wrong change to silent the annoying message : | if (ret < 0) | /* ret must not be negative here */ | //complain(); | init(); ... which in fact means : | if (ret < 0) | init(); 3) Breaking lines ----------------- There is no strict rule for line breaking. Some files try to stick to the 80 column limit, but given that various people use various tab sizes, it does not make much sense. Also, code is sometimes easier to read with less lines, as it represents less surface on the screen (since each new line adds its tabs and spaces). The rule is to stick to the average line length of other lines. If you are working in a file which fits in 80 columns, try to keep this goal in mind. If you're in a function with 120-chars lines, there is no reason to add many short lines, so you can make longer lines. In general, opening a new block should lead to a new line. Similarly, multiple instructions should be avoided on the same line. But some constructs make it more readable when those are perfectly aligned : A copy-paste bug in the following construct will be easier to spot : | if (omult % idiv == 0) { omult /= idiv; idiv = 1; } | if (idiv % omult == 0) { idiv /= omult; omult = 1; } | if (imult % odiv == 0) { imult /= odiv; odiv = 1; } | if (odiv % imult == 0) { odiv /= imult; imult = 1; } than in this one : | if (omult % idiv == 0) { | omult /= idiv; | idiv = 1; | } | if (idiv % omult == 0) { | idiv /= omult; | omult = 1; | } | if (imult % odiv == 0) { | imult /= odiv; | odiv = 1; | } | if (odiv % imult == 0) { | odiv /= imult; | imult = 1; | } What is important is not to mix styles. For instance there is nothing wrong with having many one-line "case" statements as long as most of them are this short like below : | switch (*arg) { | case 'A': ret = 1; break; | case 'B': ret = 2; break; | case 'C': ret = 4; break; | case 'D': ret = 8; break; | default : ret = 0; break; | } Otherwise, prefer to have the "case" statement on its own line as in the example in section 1.2 about alignment. In any case, avoid to stack multiple control statements on the same line, so that it will never be the needed to add two tab levels at once : Right : | switch (*arg) { | case 'A': | if (ret < 0) | ret = 1; | break; | default : ret = 0; break; | } Wrong : | switch (*arg) { | case 'A': if (ret < 0) | ret = 1; | break; | default : ret = 0; break; | } Right : | if (argc < 2) | if (ret < 0) | return -1; or Right : | if (argc < 2) | if (ret < 0) return -1; but Wrong : | if (argc < 2) if (ret < 0) return -1; When complex conditions or expressions are broken into multiple lines, please do ensure that alignment is perfectly appropriate, and group all main operators on the same side (which you're free to choose as long as it does not change for every block. Putting binary operators on the right side is preferred as it does not mangle with alignment but various people have their preferences. Right : | if ((txn->flags & TX_NOT_FIRST) && | ((req->flags & BF_FULL) || | req->r < req->lr || | req->r > req->data + req->size - global.tune.maxrewrite)) { | return 0; | } Right : | if ((txn->flags & TX_NOT_FIRST) | && ((req->flags & BF_FULL) | || req->r < req->lr | || req->r > req->data + req->size - global.tune.maxrewrite)) { | return 0; | } Wrong : | if ((txn->flags & TX_NOT_FIRST) && | ((req->flags & BF_FULL) || | req->r < req->lr | || req->r > req->data + req->size - global.tune.maxrewrite)) { | return 0; | } If it makes the result more readable, parenthesis may even be closed on their own line in order to align with the opening one. Note that should normally not be needed because such code would be too complex to be digged into. The "else" statement may either be merged with the closing "if" brace or lie on its own line. The later is preferred but it adds one extra line to each control block which is annoying in short ones. However, if the "else" is followed by an "if", then it should really be on its own line and the rest of the if/else blocks must follow the same style. Right : | if (a < b) { | return a; | } | else { | return b; | } Right : | if (a < b) { | return a; | } else { | return b; | } Right : | if (a < b) { | return a; | } | else if (a != b) { | return b; | } | else { | return 0; | } Wrong : | if (a < b) { | return a; | } else if (a != b) { | return b; | } else { | return 0; | } Wrong : | if (a < b) { | return a; | } | else if (a != b) { | return b; | } else { | return 0; | } 4) Spacing ---------- Correctly spacing code is very important. When you have to spot a bug at 3am, you need it to be clear. When you expect other people to review your code, you want it to be clear and don't want them to get nervous when trying to find what you did. Always place spaces around all binary or ternary operators, commas, as well as after semi-colons and opening braces if the line continues : Right : | int ret = 0; | /* if (x >> 4) { x >>= 4; ret += 4; } */ | ret += (x >> 4) ? (x >>= 4, 4) : 0; | val = ret + ((0xFFFFAA50U >> (x << 1)) & 3) + 1; Wrong : | int ret=0; | /* if (x>>4) {x>>=4;ret+=4;} */ | ret+=(x>>4)?(x>>=4,4):0; | val=ret+((0xFFFFAA50U>>(x<<1))&3)+1; Never place spaces after unary operators (&, *, -, !, ~, ++, --) nor cast, as they might be confused with they binary counterpart, nor before commas or semicolons : Right : | bit = !!(~len++ ^ -(unsigned char)*x); Wrong : | bit = ! ! (~len++ ^ - (unsigned char) * x) ; Note that "sizeof" is a unary operator which is sometimes considered as a language keyword, but in no case it is a function. It does not require parenthesis so it is sometimes followed by spaces and sometimes not when there are no parenthesis. Most people do not really care as long as what is written is unambiguous. Braces opening a block must be preceded by one space unless the brace is placed on the first column : Right : | if (argc < 2) { | } Wrong : | if (argc < 2){ | } Do not add unneeded spaces inside parenthesis, they just make the code less readable. Right : | if (x < 4 && (!y || !z)) | break; Wrong : | if ( x < 4 && ( !y || !z ) ) | break; Language keywords must all be followed by a space. This is true for control statements (do, for, while, if, else, return, switch, case), and for types (int, char, unsigned). As an exception, the last type in a cast does not take a space before the closing parenthesis). The "default" statement in a "switch" construct is generally just followed by the colon. However the colon after a "case" or "default" statement must be followed by a space. Right : | if (nbargs < 2) { | printf("Missing arg at %c\n", *(char *)ptr); | for (i = 0; i < 10; i++) beep(); | return 0; | } | switch (*arg) { Wrong : | if(nbargs < 2){ | printf("Missing arg at %c\n", *(char*)ptr); | for(i = 0; i < 10; i++)beep(); | return 0; | } | switch(*arg) { Function calls are different, the opening parenthesis is always coupled to the function name without any space. But spaces are still needed after commas : Right : | if (!init(argc, argv)) | exit(1); Wrong : | if (!init (argc,argv)) | exit(1); 5) Excess or lack of parenthesis -------------------------------- Sometimes there are too many parenthesis in some formulas, sometimes there are too few. There are a few rules of thumb for this. The first one is to respect the compiler's advice. If it emits a warning and asks for more parenthesis to avoid confusion, follow the advice at least to shut the warning. For instance, the code below is quite ambiguous due to its alignment : | if (var1 < 2 || var2 < 2 && | var3 != var4) { | /* fail */ | return -3; | } Note that this code does : | if (var1 < 2 || (var2 < 2 && var3 != var4)) { | /* fail */ | return -3; | } But maybe the author meant : | if ((var1 < 2 || var2 < 2) && var3 != var4) { | /* fail */ | return -3; | } A second rule to put parenthesis is that people don't always know operators precedence too well. Most often they have no issue with operators of the same category (eg: booleans, integers, bit manipulation, assignment) but once these operators are mixed, it causes them all sort of issues. In this case, it is wise to use parenthesis to avoid errors. One common error concerns the bit shift operators because they're used to replace multiplies and divides but don't have the same precedence : The expression : | x = y * 16 + 5; becomes : | x = y << 4 + 5; which is wrong because it is equivalent to : | x = y << (4 + 5); while the following was desired instead : | x = (y << 4) + 5; It is generally fine to write boolean expressions based on comparisons without any parenthesis. But on top of that, integer expressions and assignments should then be protected. For instance, there is an error in the expression below which should be safely rewritten : Wrong : | if (var1 > 2 && var1 < 10 || | var1 > 2 + 256 && var2 < 10 + 256 || | var1 > 2 + 1 << 16 && var2 < 10 + 2 << 16) | return 1; Right (may remove a few parenthesis depending on taste) : | if ((var1 > 2 && var1 < 10) || | (var1 > (2 + 256) && var2 < (10 + 256)) || | (var1 > (2 + (1 << 16)) && var2 < (10 + (1 << 16)))) | return 1; The "return" statement is not a function, so it takes no argument. It is a control statement which is followed by the expression to be returned. It does not need to be followed by parenthesis : Wrong : | int ret0() | { | return(0); | } Right : | int ret0() | { | return 0; | } Parenthesisis are also found in type casts. Type casting should be avoided as much as possible, especially when it concerns pointer types. Casting a pointer disables the compiler's type checking and is the best way to get caught doing wrong things with data not the size you expect. If you need to manipulate multiple data types, you can use a union instead. If the union is really not convenient and casts are easier, then try to isolate them as much as possible, for instance when initializing function arguments or in another function. Not proceeding this way causes huge risks of not using the proper pointer without any notification, which is especially true during copy-pastes. Wrong : | void *check_private_data(void *arg1, void *arg2) | { | char *area; | | if (*(int *)arg1 > 1000) | return NULL; | if (memcmp(*(const char *)arg2, "send(", 5) != 0)) | return NULL; | area = malloc(*(int *)arg1); | if (!area) | return NULL; | memcpy(area, *(const char *)arg2 + 5, *(int *)arg1); | return area; | } Right : | void *check_private_data(void *arg1, void *arg2) | { | char *area; | int len = *(int *)arg1; | const char *msg = arg2; | | if (len > 1000) | return NULL; | if (memcmp(msg, "send(", 5) != 0) | return NULL; | area = malloc(len); | if (!area) | return NULL; | memcpy(area, msg + 5, len); | return area; | } 6) Ambiguous comparisons with zero or NULL ------------------------------------------ In C, '0' has no type, or it has the type of the variable it is assigned to. Comparing a variable or a return value with zero means comparing with the representation of zero for this variable's type. For a boolean, zero is false. For a pointer, zero is NULL. Very often, to make things shorter, it is fine to use the '!' unary operator to compare with zero, as it is shorter and easier to remind or understand than a plain '0'. Since the '!' operator is read "not", it helps read code faster when what follows it makes sense as a boolean, and it is often much more appropriate than a comparison with zero which makes an equal sign appear at an undesirable place. For instance : | if (!isdigit(*c) && !isspace(*c)) | break; is easier to understand than : | if (isdigit(*c) == 0 && isspace(*c) == 0) | break; For a char this "not" operator can be reminded as "no remaining char", and the absence of comparison to zero implies existence of the tested entity, hence the simple strcpy() implementation below which automatically stops once the last zero is copied : | void my_strcpy(char *d, const char *s) | { | while ((*d++ = *s++)); | } Note the double parenthesis in order to avoid the compiler telling us it looks like an equality test. For a string or more generally any pointer, this test may be understood as an existence test or a validity test, as the only pointer which will fail to validate equality is the NULL pointer : | area = malloc(1000); | if (!area) | return -1; However sometimes it can fool the reader. For instance, strcmp() precisely is one of such functions whose return value can make one think the opposite due to its name which may be understood as "if strings compare...". Thus it is strongly recommended to perform an explicit comparison with zero in such a case, and it makes sense considering that the comparison's operator is the same that is wanted to compare the strings (note that current config parser lacks a lot in this regards) : strcmp(a, b) == 0 <=> a == b strcmp(a, b) != 0 <=> a != b strcmp(a, b) < 0 <=> a < b strcmp(a, b) > 0 <=> a > b Avoid this : | if (strcmp(arg, "test")) | printf("this is not a test\n"); | | if (!strcmp(arg, "test")) | printf("this is a test\n"); Prefer this : | if (strcmp(arg, "test") != 0) | printf("this is not a test\n"); | | if (strcmp(arg, "test") == 0) | printf("this is a test\n"); 7) System call returns ---------------------- This is not directly a matter of coding style but more of bad habits. It is important to check for the correct value upon return of syscalls. The proper return code indicating an error is described in its man page. There is no reason to consider wider ranges than what is indicated. For instance, it is common to see such a thing : | if ((fd = open(file, O_RDONLY)) < 0) | return -1; This is wrong. The man page says that -1 is returned if an error occurred. It does not suggest that any other negative value will be an error. It is possible that a few such issues have been left in existing code. They are bugs for which fixes are accepted, even though they're currently harmless since open() is not known for returning negative values at the moment. 8) Declaring new types, names and values ---------------------------------------- Please refrain from using "typedef" to declare new types, they only obfuscate the code. The reader never knows whether he's manipulating a scalar type or a struct. For instance it is not obvious why the following code fails to build : | int delay_expired(timer_t exp, timer_us_t now) | { | return now >= exp; | } With the types declared in another file this way : | typedef unsigned int timer_t; | typedef struct timeval timer_us_t; This cannot work because we're comparing a scalar with a struct, which does not make sense. Without a typedef, the function would have been written this way without any ambiguity and would not have failed : | int delay_expired(unsigned int exp, struct timeval *now) | { | return now >= exp->tv_sec; | } Declaring special values may be done using enums. Enums are a way to define structured integer values which are related to each other. They are perfectly suited for state machines. While the first element is always assigned the zero value, not everybody knows that, especially people working with multiple languages all the day. For this reason it is recommended to explicitly force the first value even if it's zero. The last element should be followed by a comma if it is planned that new elements might later be added, this will make later patches shorter. Conversely, if the last element is placed in order to get the number of possible values, it must not be followed by a comma and must be preceded by a comment : | enum { | first = 0, | second, | third, | fourth, | }; | enum { | first = 0, | second, | third, | fourth, | /* nbvalues must always be placed last */ | nbvalues | }; Structure names should be short enough not to mangle function declarations, and explicit enough to avoid confusion (which is the most important thing). Wrong : | struct request_args { /* arguments on the query string */ | char *name; | char *value; | struct misc_args *next; | }; Right : | struct qs_args { /* arguments on the query string */ | char *name; | char *value; | struct qs_args *next; | } When declaring new functions or structures, please do not use CamelCase, which is a style where upper and lower case are mixed in a single word. It causes a lot of confusion when words are composed from acronyms, because it's hard to stick to a rule. For instance, a function designed to generate an ISN (initial sequence number) for a TCP/IP connection could be called : - generateTcpipIsn() - generateTcpIpIsn() - generateTcpIpISN() - generateTCPIPISN() etc... None is right, none is wrong, these are just preferences which might change along the code. Instead, please use an underscore to separate words. Lowercase is preferred for the words, but if acronyms are upcased it's not dramatic. The real advantage of this method is that it creates unambiguous levels even for short names. Valid examples : - generate_tcpip_isn() - generate_tcp_ip_isn() - generate_TCPIP_ISN() - generate_TCP_IP_ISN() Another example is easy to understand when 3 arguments are involved in naming the function : Wrong (naming conflict) : | /* returns A + B * C */ | int mulABC(int a, int b, int c) | { | return a + b * c; | } | | /* returns (A + B) * C */ | int mulABC(int a, int b, int c) | { | return (a + b) * c; | } Right (unambiguous naming) : | /* returns A + B * C */ | int mul_a_bc(int a, int b, int c) | { | return a + b * c; | } | | /* returns (A + B) * C */ | int mul_ab_c(int a, int b, int c) | { | return (a + b) * c; | } Whenever you manipulate pointers, try to declare them as "const", as it will save you from many accidental misuses and will only cause warnings to be emitted when there is a real risk. In the examples below, it is possible to call my_strcpy() with a const string only in the first declaration. Note that people who ignore "const" are often the ones who cast a lot and who complain from segfaults when using strtok() ! Right : | void my_strcpy(char *d, const char *s) | { | while ((*d++ = *s++)); | } | | void say_hello(char *dest) | { | my_strcpy(dest, "hello\n"); | } Wrong : | void my_strcpy(char *d, char *s) | { | while ((*d++ = *s++)); | } | | void say_hello(char *dest) | { | my_strcpy(dest, "hello\n"); | } 9) Getting macros right ----------------------- It is very common for macros to do the wrong thing when used in a way their author did not have in mind. For this reason, macros must always be named with uppercase letters only. This is the only way to catch the developer's eye when using them, so that they double-check whether they are taking a risk or not. First, macros must never ever be terminated by a semi-colon, or they will close the wrong block once in a while. For instance, the following will cause a build error before the "else" due to the double semi-colon : Wrong : | #define WARN printf("warning\n"); | ... | if (a < 0) | WARN; | else | a--; Right : | #define WARN printf("warning\n") If multiple instructions are needed, then use a do { } while (0) block, which is the only construct which respects *exactly* the semantics of a single instruction : | #define WARN do { printf("warning\n"); log("warning\n"); } while (0) | ... | | if (a < 0) | WARN; | else | a--; Second, do not put unprotected control statements in macros, they will definitely cause bugs : Wrong : | #define WARN if (verbose) printf("warning\n") | ... | if (a < 0) | WARN; | else | a--; Which is equivalent to the undesired form below : | if (a < 0) | if (verbose) | printf("warning\n"); | else | a--; Right way to do it : | #define WARN do { if (verbose) printf("warning\n"); } while (0) | ... | if (a < 0) | WARN; | else | a--; Which is equivalent to : | if (a < 0) | do { if (verbose) printf("warning\n"); } while (0); | else | a--; Macro parameters must always be surrounded by parenthesis, and must never be duplicated in the same macro unless explicitly stated. Also, macros must not be defined with operators without surrounding parenthesis. The MIN/MAX macros are a pretty common example of multiple misuses, but this happens as early as when using bit masks. Most often, in case of any doubt, try to use inline functions instead. Wrong : | #define MIN(a, b) a < b ? a : b | | /* returns 2 * min(a,b) + 1 */ | int double_min_p1(int a, int b) | { | return 2 * MIN(a, b) + 1; | } What this will do : | int double_min_p1(int a, int b) | { | return 2 * a < b ? a : b + 1; | } Which is equivalent to : | int double_min_p1(int a, int b) | { | return (2 * a) < b ? a : (b + 1); | } The first thing to fix is to surround the macro definition with parenthesis to avoid this mistake : | #define MIN(a, b) (a < b ? a : b) But this is still not enough, as can be seen in this example : | /* compares either a or b with c */ | int min_ab_c(int a, int b, int c) | { | return MIN(a ? a : b, c); | } Which is equivalent to : | int min_ab_c(int a, int b, int c) | { | return (a ? a : b < c ? a ? a : b : c); | } Which in turn means a totally different thing due to precedence : | int min_ab_c(int a, int b, int c) | { | return (a ? a : ((b < c) ? (a ? a : b) : c)); | } This can be fixed by surrounding *each* argument in the macro with parenthesis: | #define MIN(a, b) ((a) < (b) ? (a) : (b)) But this is still not enough, as can be seen in this example : | int min_ap1_b(int a, int b) | { | return MIN(++a, b); | } Which is equivalent to : | int min_ap1_b(int a, int b) | { | return ((++a) < (b) ? (++a) : (b)); | } Again, this is wrong because "a" is incremented twice if below b. The only way to fix this is to use a compound statement and to assign each argument exactly once to a local variable of the same type : | #define MIN(a, b) ({ typeof(a) __a = (a); typeof(b) __b = (b); \ | ((__a) < (__b) ? (__a) : (__b)); \ | }) At this point, using static inline functions is much cleaner if a single type is to be used : | static inline int min(int a, int b) | { | return a < b ? a : b; | } 10) Includes ------------ Includes are as much as possible listed in alphabetically ordered groups : - the includes more or less system-specific (sys/*, netinet/*, ...) - the libc-standard includes (those without any path component) - includes from the local "import" subdirectory - includes from the local "haproxy" subdirectory Each section is just visually delimited from the other ones using an empty line. The two first ones above may be merged into a single section depending on developer's preference. Please do not copy-paste include statements from other files. Having too many includes significantly increases build time and makes it hard to find which ones are needed later. Just include what you need and if possible in alphabetical order so that when something is missing, it becomes obvious where to look for it and where to add it. All files should include because this is where build options are prepared. HAProxy header files are split in two, those exporting the types only (named with a trailing "-t") and those exporting variables, functions and inline functions. Types, structures, enums and #defines must go into the types files which are the only ones that may be included by othertype files. Function prototypes and inlined functions must go into the main files. This split is because of inlined functions which cross-reference types from other files, which cause a chicken-and-egg problem if the functions and types are declared at the same place. Include files must be protected against multiple inclusion using the common #ifndef/#define/#endif trick with a tag derived from the include file and its location. 11) Comments ------------ Comments are preferably of the standard 'C' form using /* */. The C++ form "//" are tolerated for very short comments (eg: a word or two) but should be avoided as much as possible. Multi-line comments are made with each intermediate line starting with a star aligned with the first one, as in this example : | /* | * This is a multi-line | * comment. | */ If multiple code lines need a short comment, try to align them so that you can have multi-line sentences. This is rarely needed, only for really complex constructs. Do not tell what you're doing in comments, but explain why you're doing it if it seems not to be obvious. Also *do* indicate at the top of function what they accept and what they don't accept. For instance, strcpy() only accepts output buffers at least as large as the input buffer, and does not support any NULL pointer. There is nothing wrong with that if the caller knows it. Wrong use of comments : | int flsnz8(unsigned int x) | { | int ret = 0; /* initialize ret */ | if (x >> 4) { x >>= 4; ret += 4; } /* add 4 to ret if needed */ | return ret + ((0xFFFFAA50U >> (x << 1)) & 3) + 1; /* add ??? */ | } | ... | bit = ~len + (skip << 3) + 9; /* update bit */ Right use of comments : | /* This function returns the position of the highest bit set in the lowest | * byte of , between 0 and 7. It only works if is non-null. It uses | * a 32-bit value as a lookup table to return one of 4 values for the | * highest 16 possible 4-bit values. | */ | int flsnz8(unsigned int x) | { | int ret = 0; | if (x >> 4) { x >>= 4; ret += 4; } | return ret + ((0xFFFFAA50U >> (x << 1)) & 3) + 1; | } | ... | bit = ~len + (skip << 3) + 9; /* (skip << 3) + (8 - len), saves 1 cycle */ 12) Use of assembly ------------------- There are many projects where use of assembly code is not welcome. There is no problem with use of assembly in haproxy, provided that : a) an alternate C-form is provided for architectures not covered b) the code is small enough and well commented enough to be maintained It is important to take care of various incompatibilities between compiler versions, for instance regarding output and cloberred registers. There are a number of documentations on the subject on the net. Anyway if you are fiddling with assembly, you probably know that already. Example : | /* gcc does not know when it can safely divide 64 bits by 32 bits. Use this | * function when you know for sure that the result fits in 32 bits, because | * it is optimal on x86 and on 64bit processors. | */ | static inline unsigned int div64_32(unsigned long long o1, unsigned int o2) | { | unsigned int result; | #ifdef __i386__ | asm("divl %2" | : "=a" (result) | : "A"(o1), "rm"(o2)); | #else | result = o1 / o2; | #endif | return result; | } 13) Pointers ------------ A lot could be said about pointers, there's enough to fill entire books. Misuse of pointers is one of the primary reasons for bugs in haproxy, and this rate has significantly increased with the use of threads. Moreover, bogus pointers cause the hardest to analyse bugs, because usually they result in modifications to reassigned areas or accesses to unmapped areas, and in each case, bugs that strike very far away from where they were located. Some bugs have already taken up to 3 weeks of full time analysis, which has a severe impact on the project's ability to make forward progress on important features. For this reason, code that doesn't look robust enough or that doesn't follow some of the rules below will be rejected, and may even be reverted after being merged if the trouble is detected late! 13.1) No test before freeing ---------------------------- All platforms where haproxy is supported have a well-defined and documented behavior for free(NULL), which is to do nothing at all. In other words, free() does test for the pointer's nullity. As such, there is no point in testing if a pointer is NULL or not before calling free(). And further, you must not do it, because it adds some confusion to the reader during debugging sessions, making one think that the code's authors weren't very sure about what they were doing. This will not cause a bug but will result in your code to get rejected. Wrong call to free : | static inline int blah_free(struct blah *blah) | { | if (blah->str1) | free(blah->str1); | if (blah->str2) | free(blah->str2); | free(blah); | } Correct call to free : | static inline int blah_free(struct blah *blah) | { | free(blah->str1); | free(blah->str2); | free(blah); | } 13.2) No dangling pointers -------------------------- Pointers are very commonly used as booleans: if they're not NULL, then the area they point to is valid and may be used. This is convenient for many things and is even emphasized with threads where they can atomically be swapped with another value (even NULL), and as such provide guaranteed atomic resource allocation and sharing. The problem with this is when someone forgets to delete a pointer when an area is no longer valid, because this may result in the pointer being accessed later and pointing to a wrong location, one that was reallocated for something else and causing all sort of nastiness like crashes or memory corruption. Moreover, thanks to the memory pools, it is extremely likely that a just released pointer will be reassigned to a similar object with comparable values (flags etc) at the same positions, making tests apparently succeed for a while. Some such bugs have gone undetected for several years. The rule is pretty simple: +-----------------------------------------------------------------+ | NO REACHABLE POINTER MAY EVER POINT TO AN UNREACHABLE LOCATION. | +-----------------------------------------------------------------+ By "reachable pointer", here we mean a pointer that is accessible from a reachable structure or a global variable. This means that any pointer found anywhere in any structure in the code may always be dereferenced. This can seem obvious but this is not always enforced. This means that when freeing an area, the pointer that was used to find that area must be overwritten with NULL, and all other such pointers must as well if any. It is one case where one can find more convenient to write the NULL on the same line as the call to free() to make things easier to check. Be careful about any potential "if" when doing this. Wrong use of free : | static inline int blah_recycle(struct blah *blah) | { | free(blah->str1); | free(blah->str2); | } Correct use of free : | static inline int blah_recycle(struct blah *blah) | { | free(blah->str1); blah->str1 = NULL; | free(blah->str2); blah->str2 = NULL; | } Sometimes the code doesn't permit this to be done. It is not a matter of code but a matter of architecture. Example: Initialization: | static struct foo *foo_init() | { | struct foo *foo; | struct bar *bar; | | foo = pool_alloc(foo_head); | bar = pool_alloc(bar_head); | if (!foo || !bar) | goto fail; | foo->bar = bar; | ... | } Scheduled task 1: | static inline int foo_timeout(struct foo *foo) | { | free(foo->bar); | free(foo); | } Scheduled task 2: | static inline int bar_timeout(struct bar *bar) | { | free(bar); | } Here it's obvious that if "bar" times out, it will be freed but its pointer in "foo" will remain here, and if foo times out just after, it will lead to a double free. Or worse, if another instance allocates a pointer and receives bar again, when foo times out, it will release the old bar pointer which now points to a new object, and the code using that new object will crash much later, or even worse, will share the same area as yet another instance having inherited that pointer again. Here this simply means that the data model is wrong. If bar may be freed alone, it MUST have a pointer to foo so that bar->foo->bar is set to NULL to let foo finish its life peacefully. This also means that the code dealing with foo must be written in a way to support bar's leaving. 13.3) Don't abuse pointers as booleans -------------------------------------- Given the common use of a pointer to know if the area it points to is valid, there is a big incentive in using such pointers as booleans to describe something a bit higher level, like "is the user authenticated". This must not be done. The reason stems from the points above. Initially this perfectly matches and the code is simple. Then later some extra options need to be added, and more pointers are needed, all allocated together. At this point they all start to become their own booleans, supposedly always equivalent, but if that were true, they would be a single area with a single pointer. And things start to fall apart with some code areas relying on one pointer for the condition and other ones relying on other pointers. Pointers may be substituted with "flags" or "present in list" etc here. And from this point, things quickly degrade with pointers needing to remain set even if pointing to wrong areas, just for the sake of not being NULL and not breaking some assumptions. At this point the bugs are already there and the code is not trustable anymore. The only way to avoid this is to strictly respect this rule: pointers do not represent a functionality but a storage area. Of course it is very frequent to consider that if an optional string is not set, a feature is not enabled. This can be fine to some extents. But as soon as any slightest condition is added anywhere into the mux, the code relying on the pointer must be replaced with something else so that the pointer may live its own life and be released (and reset) earlier if needed. 13.4) Mixing const and non-const -------------------------------- Something often encountered, especially when assembling error messages, is functions that collect strings, assemble them into larger messages and free everything. The problem here is that if strings are defined as variables, there will rightfully be build warnings when reporting string constants such as bare keywords or messages, and if strings are defined as constants, it is not possible to free them. The temptation is sometimes huge to force some free() calls on casted strings. Do not do that! It will inevitably lead to someone getting caught passing a constant string that will make the process crash (if lucky). Document the expectations, indicate that all arguments must be freeable and that the caller must be capable of strdup(), and make your function support NULLs and document it (so that callers can deal with a failing strdup() on allocation error). One valid alternative is to use a secondary channel to indicate whether the message may be freed or not. A flag in a complex structure can be used for this purpose, for example. If you are certain that your strings are aligned to a certain number of bytes, it can be possible to instrument the code to use the lowest bit to indicate the need to free (e.g. by always adding one to every const string). But such a solution will require good enough instrumentation so that it doesn't constitute a new set of traps. 13.5) No pointer casts ---------------------- Except in rare occasions caused by legacy APIs (e.g. sockaddr) or special cases which explicitly require a form of aliasing, there is no valid reason for casting pointers, and usually this is used to hide other problems that will strike later. The only suitable type of cast is the cast from the generic void* used to store a context for example. But in C, there is no need to cast to nor from void*, so this is not required. However those coming from C++ tend to be used to this practice, and others argue that it makes the intent more visible. As a corollary, do not abuse void*. Placing void* everywhere to avoid casting is a bad practice as well. The use of void* is only for generic functions or structures which do not have a limited set of types supported. When only a few types are supported, generally their type can be passed using a side channel, and the void* can be turned into a union that makes the code more readable and more verifiable. An alternative in haproxy is to use a pointer to an obj_type enum. Usually it is placed at the beginning of a structure. It works like a void* except that the type is read directly from the object. This is convenient when a small set of remote objects may be attached to another one because a single of them will match a non-null pointer (e.g. a connection or an applet). Example: | static inline int blah_free(struct blah *blah) | { | /* only one of them (at most) will not be null */ | pool_free(pool_head_connection, objt_conn(blah->target)); | pool_free(pool_head_appctx, objt_appctx(blah->target)); | pool_free(pool_head_stream, objt_stream(blah->target)); | blah->target = NULL; | } 13.6) Extreme caution when using non-canonical pointers ------------------------------------------------------- It can be particularly convenient to embed some logic in the unused bits or code points of a pointer. Indeed, when it is known by design that a given pointer will always follow a certain alignment, a few lower bits will always remain zero, and as such may be used as optional flags. For example, the ebtree code uses the lowest bit to differentiate left/right attachments to the parent and node/leaf in branches. It is also known that values very close to NULL will never represent a valid pointer, and the thread-safe MT_LIST code uses this to lock visited pointers. There are a few rules to respect in order to do this: - the deviations from the canonical pointers must be exhaustively documented where the pointer type is defined, and the whole control logic with its implications and possible and impossible cases must be enumerated as well ; - make sure that the operations will work on every supported platform, which includes 32-bit platforms where structures may be aligned on as little as 32-bit. 32-bit alignment leaves only two LSB available. When doing so, make sure the target structures are not labelled with the "packed" attribute, or that they're always perfectly aligned. All platforms where haproxy runs have their NULL pointer mapped at address zero, and use page sizes at least 4096 bytes large, leaving all values form 1 to 4095 unused. Anything outside of this is unsafe. In particular, never use negative numbers to represent a supposedly invalid address. On 32-bits platforms it will often correspond to a system address or a special page. Always try a variety of platforms when doing such a thing. - the code must not use such pointers as booleans anymore even if it is known that "it works" because that keeps a doubt open for the reviewer. Only the canonical pointer may be tested. There can be a rare exception which is if this is on a critical path where severe performance degradation may result from this. In this case, *each* of the checks must be duly documented and the equivalent BUG_ON() instances must be placed to prove the claim. - some inline functions (or macros) must be used to turn the pointers to/from their canonical form so that the regular code doesn't have to see the operations, and so that the representation may be easily adjusted in the future. A few comments indicating to a human how to turn a pointer back and forth from inside a debugger will be appreciated, as macros often end up not being trivially readable nor directly usable. - do not use int types to cast the pointers, this will only work on 32-bit platforms. While "long" is usually fine, it is not recommended anymore due to the Windows platform being LLP64 and having it set to 32 bits. And "long long" isn't good either for always being 64 bits. More suitable types are ptrdiff_t or size_t. Note that while those were not available everywhere in the early days of hparoxy, size_t is now heavily used and known to work everywhere. And do not perform the operations on the pointers, only on the integer types (and cast back again). Some compilers such as gcc are extremely picky about this and will often emit wrong code when they see equality conditions they believe is impossible and decide to optimize them away. 13.7) Pointers in unions ------------------------ Before placing multiple aliasing pointers inside a same union, there MUST be a SINGLE well-defined way to figure them out from each other. It may be thanks to a side-channel information (as done in the samples with a defined type), it may be based on in-area information (as done using obj_types), or any other trusted solution. In any case, if pointers are mixed with any other type (integer or float) in a union, there must be a very simple way to distinguish them, and not a platform-dependent nor compiler-dependent one.