defensive-coding-guide/en-US/programming-languages/C-Allocators.adoc

:experimental:

[[sect-Defensive_Coding-C-Allocators]]
== Memory Allocators

=== `malloc` and Related Functions

The C library interfaces for memory allocation are provided by
`malloc`, `free` and
`realloc`, and the
`calloc` function. In addition to these
generic functions, there are derived functions such as
`strdup` which perform allocation using
`malloc` internally, but do not return
untyped heap memory (which could be used for any object).

The C compiler knows about these functions and can use their
expected behavior for optimizations. For instance, the compiler
assumes that an existing pointer (or a pointer derived from an
existing pointer by arithmetic) will not point into the memory
area returned by `malloc`.

If the allocation fails, `realloc` does not
free the old pointer. Therefore, the idiom `ptr =
realloc(ptr, size);` is wrong because the memory
pointed to by `ptr` leaks in case of an error.

[[sect-Defensive_Coding-C-Use-After-Free]]
==== Use-after-free errors

After `free`, the pointer is invalid.
Further pointer dereferences are not allowed (and are usually
detected by [application]*valgrind*). Less obvious
is that any *use* of the old pointer value is
not allowed, either. In particular, comparisons with any other
pointer (or the null pointer) are undefined according to the C
standard.

The same rules apply to `realloc` if the
memory area cannot be enlarged in-place. For instance, the
compiler may assume that a comparison between the old and new
pointer will always return false, so it is impossible to detect
movement this way.

==== Handling Memory Allocation Errors

Recovering from out-of-memory errors is often difficult or even
impossible. In these cases, `malloc` and
other allocation functions return a null pointer. Dereferencing
this pointer lead to a crash. Such dereferences can even be
exploitable for code execution if the dereference is combined
with an array subscript.

In general, if you cannot check all allocation calls and
handle failure, you should abort the program on allocation
failure, and not rely on the null pointer dereference to
terminate the process. See
<<sect-Defensive_Coding-Tasks-Serialization-Decoders>>
for related memory allocation concerns.

[[sect-Defensive_Coding-C-Allocators-alloca]]
=== `alloca` and Other Forms of Stack-based Allocation

Allocation on the stack is risky because stack overflow checking
is implicit. There is a guard page at the end of the memory
area reserved for the stack. If the program attempts to read
from or write to this guard page, a `SIGSEGV`
signal is generated and the program typically terminates.

This is sufficient for detecting typical stack overflow
situations such as unbounded recursion, but it fails when the
stack grows in increments larger than the size of the guard
page. In this case, it is possible that the stack pointer ends
up pointing into a memory area which has been allocated for a
different purposes. Such misbehavior can be exploitable.

A common source for large stack growth are calls to
`alloca` and related functions such as
`strdupa`. These functions should be avoided
because of the lack of error checking. (They can be used safely
if the allocated size is less than the page size (typically,
4096 bytes), but this case is relatively rare.) Additionally,
relying on `alloca` makes it more difficult
to reorganize the code because it is not allowed to use the
pointer after the function calling `alloca`
has returned, even if this function has been inlined into its
caller.

Similar concerns apply to *variable-length
arrays* (VLAs), a feature of the C99 standard which
started as a GNU extension. For large objects exceeding the
page size, there is no error checking, either.

In both cases, negative or very large sizes can trigger a
stack-pointer wraparound, and the stack pointer and end up
pointing into caller stack frames, which is fatal and can be
exploitable.

If you want to use `alloca` or VLAs for
performance reasons, consider using a small on-stack array (less
than the page size, large enough to fulfill most requests). If
the requested size is small enough, use the on-stack array.
Otherwise, call `malloc`. When exiting the
function, check if `malloc` had been called,
and free the buffer as needed.

[[sect-Defensive_Coding-C-Allocators-Arrays]]
=== Array Allocation

When allocating arrays, it is important to check for overflows.
The `calloc` function performs such checks.

If `malloc` or `realloc`
is used, the size check must be written manually. For instance,
to allocate an array of `n` elements of type
`T`, check that the requested size is not
greater than `((size_t) -1) / sizeof(T)`. See
<<sect-Defensive_Coding-C-Arithmetic>>.

[[sect-Defensive_Coding-C-Allocators-Custom]]
=== Custom Memory Allocators

Custom memory allocates come in two forms: replacements for
`malloc`, and completely different interfaces
for memory management. Both approaches can reduce the
effectiveness of [application]*valgrind* and similar
tools, and the heap corruption detection provided by GNU libc, so
they should be avoided.

Memory allocators are difficult to write and contain many
performance and security pitfalls.

* When computing array sizes or rounding up allocation
requests (to the next allocation granularity, or for
alignment purposes), checks for arithmetic overflow are
required.

* Size computations for array allocations need overflow
checking. See <<sect-Defensive_Coding-C-Allocators-Arrays>>.

* It can be difficult to beat well-tuned general-purpose
allocators. In micro benchmarks, pool allocators can show
huge wins, and size-specific pools can reduce internal
fragmentation. But often, utilization of individual pools
is poor, and external fragmentation increases the overall
memory usage.

=== Conservative Garbage Collection

Garbage collection can be an alternative to explicit memory
management using `malloc` and
`free`. The Boehm-Dehmers-Weiser allocator
can be used from C programs, with minimal type annotations.
Performance is competitive with `malloc` on
64-bit architectures, especially for multi-threaded programs.
The stop-the-world pauses may be problematic for some real-time
applications, though.

However, using a conservative garbage collector may reduce
opportunities for code reduce because once one library in a
program uses garbage collection, the whole process memory needs
to be subject to it, so that no pointers are missed. The
Boehm-Dehmers-Weiser collector also reserves certain signals for
internal use, so it is not fully transparent to the rest of the
program.