:experimental: [[sect-Defensive_Coding-C-Allocators]] == Memory Allocators === `malloc` and Related Functions The C library interfaces for memory allocation are provided by `malloc`, `free` and `realloc`, and the `calloc` function. In addition to these generic functions, there are derived functions such as `strdup` which perform allocation using `malloc` internally, but do not return untyped heap memory (which could be used for any object). The C compiler knows about these functions and can use their expected behavior for optimizations. For instance, the compiler assumes that an existing pointer (or a pointer derived from an existing pointer by arithmetic) will not point into the memory area returned by `malloc`. If the allocation fails, `realloc` does not free the old pointer. Therefore, the idiom `ptr = realloc(ptr, size);` is wrong because the memory pointed to by `ptr` leaks in case of an error. [[sect-Defensive_Coding-C-Use-After-Free]] ==== Use-after-free errors After `free`, the pointer is invalid. Further pointer dereferences are not allowed (and are usually detected by [application]*valgrind*). Less obvious is that any *use* of the old pointer value is not allowed, either. In particular, comparisons with any other pointer (or the null pointer) are undefined according to the C standard. The same rules apply to `realloc` if the memory area cannot be enlarged in-place. For instance, the compiler may assume that a comparison between the old and new pointer will always return false, so it is impossible to detect movement this way. On a related note, `realloc` frees the memory area if the new size is zero. If the size unintentionally becomes zero, as a result of unsigned integer wrap-around for instance, the following idiom causes a double-free. [source,c] ---- new_size = size + x; /* 'x' is a very large value and the result wraps around to zero */ new_ptr = realloc(ptr, new_size); if (!new_ptr) { free(ptr); } ---- ==== Handling Memory Allocation Errors Recovering from out-of-memory errors is often difficult or even impossible. In these cases, `malloc` and other allocation functions return a null pointer. Dereferencing this pointer lead to a crash. Such dereferences can even be exploitable for code execution if the dereference is combined with an array subscript. In general, if you cannot check all allocation calls and handle failure, you should abort the program on allocation failure, and not rely on the null pointer dereference to terminate the process. See <> for related memory allocation concerns. [[sect-Defensive_Coding-C-Allocators-alloca]] === `alloca` and Other Forms of Stack-based Allocation Allocation on the stack is risky because stack overflow checking is implicit. There is a guard page at the end of the memory area reserved for the stack. If the program attempts to read from or write to this guard page, a `SIGSEGV` signal is generated and the program typically terminates. This is sufficient for detecting typical stack overflow situations such as unbounded recursion, but it fails when the stack grows in increments larger than the size of the guard page. In this case, it is possible that the stack pointer ends up pointing into a memory area which has been allocated for a different purposes. Such misbehavior can be exploitable. A common source for large stack growth are calls to `alloca` and related functions such as `strdupa`. These functions should be avoided because of the lack of error checking. (They can be used safely if the allocated size is less than the page size (typically, 4096 bytes), but this case is relatively rare.) Additionally, relying on `alloca` makes it more difficult to reorganize the code because it is not allowed to use the pointer after the function calling `alloca` has returned, even if this function has been inlined into its caller. Similar concerns apply to *variable-length arrays* (VLAs), a feature of the C99 standard which started as a GNU extension. For large objects exceeding the page size, there is no error checking, either. In both cases, negative or very large sizes can trigger a stack-pointer wraparound, and the stack pointer and end up pointing into caller stack frames, which is fatal and can be exploitable. If you want to use `alloca` or VLAs for performance reasons, consider using a small on-stack array (less than the page size, large enough to fulfill most requests). If the requested size is small enough, use the on-stack array. Otherwise, call `malloc`. When exiting the function, check if `malloc` had been called, and free the buffer as needed. If portability is not important in your program, an alternative way of automatic memory management is to leverage the `cleanup` attribute supported by the recent versions of GCC and Clang. If a local variable is declared with the attribute, the specified cleanup function will be called when the variable goes out of scope. [source,c] ---- static inline void freep(void *p) { free(*(void**) p); } void somefunction(const char *param) { if (strcmp(param, "do_something_complex") == 0) { __attribute__((cleanup(freep))) char *ptr = NULL; /* Allocate a temporary buffer */ ptr = malloc(size); /* Do something on it, but do not need to manually call free() */ } } ---- [[sect-Defensive_Coding-C-Allocators-Arrays]] === Array Allocation When allocating arrays, it is important to check for overflows. The `calloc` function performs such checks. If `malloc` or `realloc` is used, the size check must be written manually. For instance, to allocate an array of `n` elements of type `T`, check that the requested size is not greater than `((size_t) -1) / sizeof(T)`. See <>. GNU libc provides a dedicated function `reallocarray` that allocates an array with those checks performed internally. However, care must be taken if portability is important: while the interface originated in OpenBSD and has been adopted in many other platforms, NetBSD exposes an incompatible behavior with the same interface. [[sect-Defensive_Coding-C-Allocators-Custom]] === Custom Memory Allocators Custom memory allocates come in two forms: replacements for `malloc`, and completely different interfaces for memory management. Both approaches can reduce the effectiveness of [application]*valgrind* and similar tools, and the heap corruption detection provided by GNU libc, so they should be avoided. Memory allocators are difficult to write and contain many performance and security pitfalls. * When computing array sizes or rounding up allocation requests (to the next allocation granularity, or for alignment purposes), checks for arithmetic overflow are required. * Size computations for array allocations need overflow checking. See <>. * It can be difficult to beat well-tuned general-purpose allocators. In micro benchmarks, pool allocators can show huge wins, and size-specific pools can reduce internal fragmentation. But often, utilization of individual pools is poor, and external fragmentation increases the overall memory usage. === Conservative Garbage Collection Garbage collection can be an alternative to explicit memory management using `malloc` and `free`. The Boehm-Dehmers-Weiser allocator can be used from C programs, with minimal type annotations. Performance is competitive with `malloc` on 64-bit architectures, especially for multi-threaded programs. The stop-the-world pauses may be problematic for some real-time applications, though. However, using a conservative garbage collector may reduce opportunities for code reduce because once one library in a program uses garbage collection, the whole process memory needs to be subject to it, so that no pointers are missed. The Boehm-Dehmers-Weiser collector also reserves certain signals for internal use, so it is not fully transparent to the rest of the program.