first pass at an Antora conversion

modules/ROOT/pages/programming-languages/C-Allocators.adoc

@@ -0,0 +1,166 @@
+
+:experimental:
+
+[[sect-Defensive_Coding-C-Allocators]]
+== Memory Allocators
+
+=== `malloc` and Related Functions
+
+The C library interfaces for memory allocation are provided by
+`malloc`, `free` and
+`realloc`, and the
+`calloc` function. In addition to these
+generic functions, there are derived functions such as
+`strdup` which perform allocation using
+`malloc` internally, but do not return
+untyped heap memory (which could be used for any object).
+
+The C compiler knows about these functions and can use their
+expected behavior for optimizations. For instance, the compiler
+assumes that an existing pointer (or a pointer derived from an
+existing pointer by arithmetic) will not point into the memory
+area returned by `malloc`.
+
+If the allocation fails, `realloc` does not
+free the old pointer. Therefore, the idiom `ptr =
+realloc(ptr, size);` is wrong because the memory
+pointed to by `ptr` leaks in case of an error.
+
+[[sect-Defensive_Coding-C-Use-After-Free]]
+==== Use-after-free errors
+
+After `free`, the pointer is invalid.
+Further pointer dereferences are not allowed (and are usually
+detected by [application]*valgrind*). Less obvious
+is that any *use* of the old pointer value is
+not allowed, either. In particular, comparisons with any other
+pointer (or the null pointer) are undefined according to the C
+standard.
+
+The same rules apply to `realloc` if the
+memory area cannot be enlarged in-place. For instance, the
+compiler may assume that a comparison between the old and new
+pointer will always return false, so it is impossible to detect
+movement this way.
+
+==== Handling Memory Allocation Errors
+
+Recovering from out-of-memory errors is often difficult or even
+impossible. In these cases, `malloc` and
+other allocation functions return a null pointer. Dereferencing
+this pointer lead to a crash. Such dereferences can even be
+exploitable for code execution if the dereference is combined
+with an array subscript.
+
+In general, if you cannot check all allocation calls and
+handle failure, you should abort the program on allocation
+failure, and not rely on the null pointer dereference to
+terminate the process. See
+<<sect-Defensive_Coding-Tasks-Serialization-Decoders>>
+for related memory allocation concerns.
+
+[[sect-Defensive_Coding-C-Allocators-alloca]]
+=== `alloca` and Other Forms of Stack-based Allocation
+
+Allocation on the stack is risky because stack overflow checking
+is implicit. There is a guard page at the end of the memory
+area reserved for the stack. If the program attempts to read
+from or write to this guard page, a `SIGSEGV`
+signal is generated and the program typically terminates.
+
+This is sufficient for detecting typical stack overflow
+situations such as unbounded recursion, but it fails when the
+stack grows in increments larger than the size of the guard
+page. In this case, it is possible that the stack pointer ends
+up pointing into a memory area which has been allocated for a
+different purposes. Such misbehavior can be exploitable.
+
+A common source for large stack growth are calls to
+`alloca` and related functions such as
+`strdupa`. These functions should be avoided
+because of the lack of error checking. (They can be used safely
+if the allocated size is less than the page size (typically,
+4096 bytes), but this case is relatively rare.) Additionally,
+relying on `alloca` makes it more difficult
+to reorganize the code because it is not allowed to use the
+pointer after the function calling `alloca`
+has returned, even if this function has been inlined into its
+caller.
+
+Similar concerns apply to *variable-length
+arrays* (VLAs), a feature of the C99 standard which
+started as a GNU extension. For large objects exceeding the
+page size, there is no error checking, either.
+
+In both cases, negative or very large sizes can trigger a
+stack-pointer wraparound, and the stack pointer and end up
+pointing into caller stack frames, which is fatal and can be
+exploitable.
+
+If you want to use `alloca` or VLAs for
+performance reasons, consider using a small on-stack array (less
+than the page size, large enough to fulfill most requests). If
+the requested size is small enough, use the on-stack array.
+Otherwise, call `malloc`. When exiting the
+function, check if `malloc` had been called,
+and free the buffer as needed.
+
+[[sect-Defensive_Coding-C-Allocators-Arrays]]
+=== Array Allocation
+
+When allocating arrays, it is important to check for overflows.
+The `calloc` function performs such checks.
+
+If `malloc` or `realloc`
+is used, the size check must be written manually. For instance,
+to allocate an array of `n` elements of type
+`T`, check that the requested size is not
+greater than `((size_t) -1) / sizeof(T)`. See
+<<sect-Defensive_Coding-C-Arithmetic>>.
+
+[[sect-Defensive_Coding-C-Allocators-Custom]]
+=== Custom Memory Allocators
+
+Custom memory allocates come in two forms: replacements for
+`malloc`, and completely different interfaces
+for memory management. Both approaches can reduce the
+effectiveness of [application]*valgrind* and similar
+tools, and the heap corruption detection provided by GNU libc, so
+they should be avoided.
+
+Memory allocators are difficult to write and contain many
+performance and security pitfalls.
+
+* When computing array sizes or rounding up allocation
+requests (to the next allocation granularity, or for
+alignment purposes), checks for arithmetic overflow are
+required.
+
+* Size computations for array allocations need overflow
+checking. See <<sect-Defensive_Coding-C-Allocators-Arrays>>.
+
+* It can be difficult to beat well-tuned general-purpose
+allocators. In micro benchmarks, pool allocators can show
+huge wins, and size-specific pools can reduce internal
+fragmentation. But often, utilization of individual pools
+is poor, and external fragmentation increases the overall
+memory usage.
+
+=== Conservative Garbage Collection
+
+Garbage collection can be an alternative to explicit memory
+management using `malloc` and
+`free`. The Boehm-Dehmers-Weiser allocator
+can be used from C programs, with minimal type annotations.
+Performance is competitive with `malloc` on
+64-bit architectures, especially for multi-threaded programs.
+The stop-the-world pauses may be problematic for some real-time
+applications, though.
+
+However, using a conservative garbage collector may reduce
+opportunities for code reduce because once one library in a
+program uses garbage collection, the whole process memory needs
+to be subject to it, so that no pointers are missed. The
+Boehm-Dehmers-Weiser collector also reserves certain signals for
+internal use, so it is not fully transparent to the rest of the
+program.