Docs/defensive-coding-guide
2
0
Fork 0
Code Issues 3 Pull requests 1 Projects Releases Packages Wiki Activity Actions

Fix xrefs, flatten structure, add ToCs & proper includes everywhere

Browse source
This commit is contained in:
Petr Bokoc 2022-01-13 21:49:39 +01:00
parent 6bb2770bc9
commit e96cdd547e
26 changed files with 1933 additions and 1635 deletions
Download patch file Download diff file Expand all files Collapse all files

1
modules/ROOT/nav.adoc
View file

@ -1,4 +1,3 @@
* xref:index.adoc[Book Information]
* Programming Languages
** xref:programming-languages/C.adoc[The C Programming Language]
** xref:programming-languages/CXX.adoc[The C++ Programming Language]

7
modules/ROOT/pages/Revision_History.adoc
View file

@ -1,8 +1,11 @@
:experimental:
[[appe-Defensive_Coding-Revision_History]]
include::partial$entities.adoc[]
= Revision History
`1.8`:: Thu Jan 13 2021 Petr Bokoc (pbokoc@gmail.com)
Complete book cleanup & publishing on docs.fedoraproject.org
`1.7`:: Sat Sept 18. 2021 Huzaifa Sidhpurwala (huzaifas@redhat.com)
* Add some C-library/syscall specific advice

7
modules/ROOT/pages/features/Features-Authentication.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Authentication]]
= Authentication and Authorization
[[sect-Defensive_Coding-Authentication-Server]]
@ -37,7 +38,7 @@ identity is assured. (Not all operating systems enforce the
trusted ports concept, and the network might not be trusted,
so it is only useful on the local system.)
TLS (<<chap-Defensive_Coding-TLS>>) is the
xref:features/Features-TLS.adoc[Transport Layer Security (TLS)] is the
recommended way for securing connections over untrusted
networks.

8
modules/ROOT/pages/features/Features-TLS.adoc
View file

@ -1,10 +1,10 @@
:experimental:
:toc:
[[chap-Defensive_Coding-TLS]]
= Transport Layer Security (TLS)
include::partial$entities.adoc[]
= Transport Layer Security (TLS)
Transport Layer Security (TLS, formerly Secure Sockets
Layer/SSL) is the recommended way to to protect integrity and
confidentiality while data is transferred over an untrusted
@ -91,7 +91,7 @@ handling input from other clients.
passed between processes. Some TLS implementations add
additional restrictions, and TLS connections generally
cannot be used across `fork` function
calls (see <<sect-Defensive_Coding-Tasks-Processes-Fork-Parallel>>).
calls (see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-Fork-Parallel[`fork` as a Primitive for Parallelism]).
[[sect-Defensive_Coding-TLS-OpenSSL]]
=== OpenSSL Pitfalls

210
modules/ROOT/pages/programming-languages/C-Allocators.adoc
View file

@ -1,210 +0,0 @@
:experimental:
[[sect-Defensive_Coding-C-Allocators]]
== Memory Allocators
=== `malloc` and Related Functions
The C library interfaces for memory allocation are provided by
`malloc`, `free` and
`realloc`, and the
`calloc` function. In addition to these
generic functions, there are derived functions such as
`strdup` which perform allocation using
`malloc` internally, but do not return
untyped heap memory (which could be used for any object).
The C compiler knows about these functions and can use their
expected behavior for optimizations. For instance, the compiler
assumes that an existing pointer (or a pointer derived from an
existing pointer by arithmetic) will not point into the memory
area returned by `malloc`.
If the allocation fails, `realloc` does not
free the old pointer. Therefore, the idiom `ptr =
realloc(ptr, size);` is wrong because the memory
pointed to by `ptr` leaks in case of an error.
[[sect-Defensive_Coding-C-Use-After-Free]]
==== Use-after-free errors
After `free`, the pointer is invalid.
Further pointer dereferences are not allowed (and are usually
detected by [application]*valgrind*). Less obvious
is that any *use* of the old pointer value is
not allowed, either. In particular, comparisons with any other
pointer (or the null pointer) are undefined according to the C
standard.
The same rules apply to `realloc` if the
memory area cannot be enlarged in-place. For instance, the
compiler may assume that a comparison between the old and new
pointer will always return false, so it is impossible to detect
movement this way.
On a related note, `realloc` frees the memory area if the new size is
zero. If the size unintentionally becomes zero, as a result of
unsigned integer wrap-around for instance, the following idiom causes
a double-free.
[source,c]
----
new_size = size + x; /* 'x' is a very large value and the result wraps around to zero */
new_ptr = realloc(ptr, new_size);
if (!new_ptr) {
free(ptr);
}
----
==== Handling Memory Allocation Errors
Recovering from out-of-memory errors is often difficult or even
impossible. In these cases, `malloc` and
other allocation functions return a null pointer. Dereferencing
this pointer lead to a crash. Such dereferences can even be
exploitable for code execution if the dereference is combined
with an array subscript.
In general, if you cannot check all allocation calls and
handle failure, you should abort the program on allocation
failure, and not rely on the null pointer dereference to
terminate the process. See
<<sect-Defensive_Coding-Tasks-Serialization-Decoders>>
for related memory allocation concerns.
[[sect-Defensive_Coding-C-Allocators-alloca]]
=== `alloca` and Other Forms of Stack-based Allocation
Allocation on the stack is risky because stack overflow checking
is implicit. There is a guard page at the end of the memory
area reserved for the stack. If the program attempts to read
from or write to this guard page, a `SIGSEGV`
signal is generated and the program typically terminates.
This is sufficient for detecting typical stack overflow
situations such as unbounded recursion, but it fails when the
stack grows in increments larger than the size of the guard
page. In this case, it is possible that the stack pointer ends
up pointing into a memory area which has been allocated for a
different purposes. Such misbehavior can be exploitable.
A common source for large stack growth are calls to
`alloca` and related functions such as
`strdupa`. These functions should be avoided
because of the lack of error checking. (They can be used safely
if the allocated size is less than the page size (typically,
4096 bytes), but this case is relatively rare.) Additionally,
relying on `alloca` makes it more difficult
to reorganize the code because it is not allowed to use the
pointer after the function calling `alloca`
has returned, even if this function has been inlined into its
caller.
Similar concerns apply to *variable-length
arrays* (VLAs), a feature of the C99 standard which
started as a GNU extension. For large objects exceeding the
page size, there is no error checking, either.
In both cases, negative or very large sizes can trigger a
stack-pointer wraparound, and the stack pointer and end up
pointing into caller stack frames, which is fatal and can be
exploitable.
If you want to use `alloca` or VLAs for
performance reasons, consider using a small on-stack array (less
than the page size, large enough to fulfill most requests). If
the requested size is small enough, use the on-stack array.
Otherwise, call `malloc`. When exiting the
function, check if `malloc` had been called,
and free the buffer as needed.
If portability is not important in your program, an alternative way of
automatic memory management is to leverage the `cleanup` attribute
supported by the recent versions of GCC and Clang. If a local variable
is declared with the attribute, the specified cleanup function will be
called when the variable goes out of scope.
[source,c]
----
static inline void freep(void *p) {
free(*(void**) p);
}
void somefunction(const char *param) {
if (strcmp(param, "do_something_complex") == 0) {
__attribute__((cleanup(freep))) char *ptr = NULL;
/* Allocate a temporary buffer */
ptr = malloc(size);
/* Do something on it, but do not need to manually call free() */
}
}
----
[[sect-Defensive_Coding-C-Allocators-Arrays]]
=== Array Allocation
When allocating arrays, it is important to check for overflows.
The `calloc` function performs such checks.
If `malloc` or `realloc`
is used, the size check must be written manually. For instance,
to allocate an array of `n` elements of type
`T`, check that the requested size is not
greater than `((size_t) -1) / sizeof(T)`. See
<<sect-Defensive_Coding-C-Arithmetic>>.
GNU libc provides a dedicated function `reallocarray` that allocates
an array with those checks performed internally. However, care must
be taken if portability is important: while the interface originated
in OpenBSD and has been adopted in many other platforms, NetBSD
exposes an incompatible behavior with the same interface.
[[sect-Defensive_Coding-C-Allocators-Custom]]
=== Custom Memory Allocators
Custom memory allocates come in two forms: replacements for
`malloc`, and completely different interfaces
for memory management. Both approaches can reduce the
effectiveness of [application]*valgrind* and similar
tools, and the heap corruption detection provided by GNU libc, so
they should be avoided.
Memory allocators are difficult to write and contain many
performance and security pitfalls.
* When computing array sizes or rounding up allocation
requests (to the next allocation granularity, or for
alignment purposes), checks for arithmetic overflow are
required.
* Size computations for array allocations need overflow
checking. See <<sect-Defensive_Coding-C-Allocators-Arrays>>.
* It can be difficult to beat well-tuned general-purpose
allocators. In micro benchmarks, pool allocators can show
huge wins, and size-specific pools can reduce internal
fragmentation. But often, utilization of individual pools
is poor, and external fragmentation increases the overall
memory usage.
=== Conservative Garbage Collection
Garbage collection can be an alternative to explicit memory
management using `malloc` and
`free`. The Boehm-Dehmers-Weiser allocator
can be used from C programs, with minimal type annotations.
Performance is competitive with `malloc` on
64-bit architectures, especially for multi-threaded programs.
The stop-the-world pauses may be problematic for some real-time
applications, though.
However, using a conservative garbage collector may reduce
opportunities for code reduce because once one library in a
program uses garbage collection, the whole process memory needs
to be subject to it, so that no pointers are missed. The
Boehm-Dehmers-Weiser collector also reserves certain signals for
internal use, so it is not fully transparent to the rest of the
program.

216
modules/ROOT/pages/programming-languages/C-Language.adoc
View file

@ -1,216 +0,0 @@
:experimental:
[[sect-Defensive_Coding-C-Language]]
== The Core Language
C provides no memory safety. Most recommendations in this section
deal with this aspect of the language.
[[sect-Defensive_Coding-C-Undefined]]
=== Undefined Behavior
Some C constructs are defined to be undefined by the C standard.
This does not only mean that the standard does not describe
what happens when the construct is executed. It also allows
optimizing compilers such as GCC to assume that this particular
construct is never reached. In some cases, this has caused
GCC to optimize security checks away. (This is not a flaw in GCC
or the C language. But C certainly has some areas which are more
difficult to use than others.)
Common sources of undefined behavior are:
* out-of-bounds array accesses
* null pointer dereferences
* overflow in signed integer arithmetic
[[sect-Defensive_Coding-C-Pointers]]
=== Recommendations for Pointers and Array Handling
Always keep track of the size of the array you are working with.
Often, code is more obviously correct when you keep a pointer
past the last element of the array, and calculate the number of
remaining elements by subtracting the current position from
that pointer. The alternative, updating a separate variable
every time when the position is advanced, is usually less
obviously correct.
<<ex-Defensive_Coding-C-Pointers-remaining>>
shows how to extract Pascal-style strings from a character
buffer. The two pointers kept for length checks are
`inend` and `outend`.
`inp` and `outp` are the
respective positions.
The number of input bytes is checked using the expression
`len > (size_t)(inend - inp)`.
The cast silences a compiler warning;
`inend` is always larger than
`inp`.
[[ex-Defensive_Coding-C-Pointers-remaining]]
.Array processing in C
====
[source,c]
----
include::example$C-Pointers-remaining.adoc[]
----
====
It is important that the length checks always have the form
`len > (size_t)(inend - inp)`, where
`len` is a variable of type
`size_t` which denotes the *total*
number of bytes which are about to be read or written next. In
general, it is not safe to fold multiple such checks into one,
as in `len1 + len2 > (size_t)(inend - inp)`,
because the expression on the left can overflow or wrap around
(see <<sect-Defensive_Coding-C-Arithmetic>>), and it
no longer reflects the number of bytes to be processed.
[[sect-Defensive_Coding-C-Arithmetic]]
=== Recommendations for Integer Arithmetic
Overflow in signed integer arithmetic is undefined. This means
that it is not possible to check for overflow after it happened,
see <<ex-Defensive_Coding-C-Arithmetic-bad>>.
[[ex-Defensive_Coding-C-Arithmetic-bad]]
.Incorrect overflow detection in C
====
[source,c]
----
include::example$C-Arithmetic-add.adoc[]
----
====
The following approaches can be used to check for overflow,
without actually causing it.
* Use a wider type to perform the calculation, check that the
result is within bounds, and convert the result to the
original type. All intermediate results must be checked in
this way.
* Perform the calculation in the corresponding unsigned type
and use bit fiddling to detect the overflow.
<<ex-Defensive_Coding-C-Arithmetic-add_unsigned>>
shows how to perform an overflow check for unsigned integer
addition. For three or more terms, all the intermediate
additions have to be checked in this way.
[[ex-Defensive_Coding-C-Arithmetic-add_unsigned]]
.Overflow checking for unsigned addition
====
[source,c]
----
include::example$C-Arithmetic-add_unsigned.adoc[]
----
====
* Compute bounds for acceptable input values which are known
to avoid overflow, and reject other values. This is the
preferred way for overflow checking on multiplications,
see <<ex-Defensive_Coding-C-Arithmetic-mult>>.
[[ex-Defensive_Coding-C-Arithmetic-mult]]
.Overflow checking for unsigned multiplication
====
[source,c]
----
include::example$C-Arithmetic-mult.adoc[]
----
====
Basic arithmetic operations are commutative, so for bounds checks,
there are two different but mathematically equivalent
expressions. Sometimes, one of the expressions results in
better code because parts of it can be reduced to a constant.
This applies to overflow checks for multiplication `a *
b` involving a constant `a`, where the
expression is reduced to `b > C` for some
constant `C` determined at compile time. The
other expression, `b && a > ((unsigned)-1) /
b`, is more difficult to optimize at compile time.
When a value is converted to a signed integer, GCC always
chooses the result based on 2's complement arithmetic. This GCC
extension (which is also implemented by other compilers) helps a
lot when implementing overflow checks.
Sometimes, it is necessary to compare unsigned and signed
integer variables. This results in a compiler warning,
*comparison between signed and unsigned integer
expressions*, because the comparison often gives
unexpected results for negative values. When adding a cast,
make sure that negative values are covered properly. If the
bound is unsigned and the checked quantity is signed, you should
cast the checked quantity to an unsigned type as least as wide
as either operand type. As a result, negative values will fail
the bounds check. (You can still check for negative values
separately for clarity, and the compiler will optimize away this
redundant check.)
Legacy code should be compiled with the [option]`-fwrapv`
GCC option. As a result, GCC will provide 2's complement
semantics for integer arithmetic, including defined behavior on
integer overflow.
[[sect-Defensive_Coding-C-Globals]]
=== Global Variables
Global variables should be avoided because they usually lead to
thread safety hazards. In any case, they should be declared
`static`, so that access is restricted to a
single translation unit.
Global constants are not a problem, but declaring them can be
tricky. <<ex-Defensive_Coding-C-Globals-String_Array>>
shows how to declare a constant array of constant strings.
The second `const` is needed to make the
array constant, and not just the strings. It must be placed
after the `*`, and not before it.
[[ex-Defensive_Coding-C-Globals-String_Array]]
.Declaring a constant array of constant strings
====
[source,c]
----
include::example$C-Globals-String_Array.adoc[]
----
====
Sometimes, static variables local to functions are used as a
replacement for proper memory management. Unlike non-static
local variables, it is possible to return a pointer to static
local variables to the caller. But such variables are
well-hidden, but effectively global (just as static variables at
file scope). It is difficult to add thread safety afterwards if
such interfaces are used. Merely dropping the
`static` keyword in such cases leads to
undefined behavior.
Another source for static local variables is a desire to reduce
stack space usage on embedded platforms, where the stack may
span only a few hundred bytes. If this is the only reason why
the `static` keyword is used, it can just be
dropped, unless the object is very large (larger than
128 kilobytes on 32-bit platforms). In the latter case, it is
recommended to allocate the object using
`malloc`, to obtain proper array checking, for
the same reasons outlined in <<sect-Defensive_Coding-C-Allocators-alloca>>.

311
modules/ROOT/pages/programming-languages/C-Libc.adoc
View file

@ -1,311 +0,0 @@
:experimental:
include::partial$entities.adoc[]
[[sect-Defensive_Coding-C-Libc]]
== The C Standard Library
Parts of the C standard library (and the UNIX and GNU extensions)
are difficult to use, so you should avoid them.
Please check the applicable documentation before using the
recommended replacements. Many of these functions allocate
buffers using `malloc` which your code must
deallocate explicitly using `free`.
[[sect-Defensive_Coding-C-Absolutely-Banned]]
=== Absolutely Banned Interfaces
The functions listed below must not be used because they are
almost always unsafe. Use the indicated replacements instead.
* `gets`
⟶ `fgets`
* `getwd`
⟶ `getcwd`
or `get_current_dir_name`
* `readdir_r` ⟶ `readdir`
* `realpath` (with a non-NULL second parameter)
⟶ `realpath` with NULL as the second parameter,
or `canonicalize_file_name`
The constants listed below must not be used, either. Instead,
code must allocate memory dynamically and use interfaces with
length checking.
* `NAME_MAX` (limit not actually enforced by
the kernel)
* `PATH_MAX` (limit not actually enforced by
the kernel)
* `_PC_NAME_MAX` (This limit, returned by the
`pathconf` function, is not enforced by
the kernel.)
* `_PC_PATH_MAX` (This limit, returned by the
`pathconf` function, is not enforced by
the kernel.)
The following structure members must not be used.
* `f_namemax` in `struct
statvfs` (limit not actually enforced by the kernel,
see `_PC_NAME_MAX` above)
[[sect-Defensive_Coding-C-Avoid]]
=== Functions to Avoid
The following string manipulation functions can be used securely
in principle, but their use should be avoided because they are
difficult to use correctly. Calls to these functions can be
replaced with `asprintf` or
`vasprintf`. (For non-GNU targets, these
functions are available from Gnulib.) In some cases, the
`snprintf` function might be a suitable
replacement, see <<sect-Defensive_Coding-C-String-Functions-Length>>.
* `sprintf`
* `strcat`
* `strcpy`
* `vsprintf`
Use the indicated replacements for the functions below.
* `alloca` ⟶
`malloc` and `free`
(see <<sect-Defensive_Coding-C-Allocators-alloca>>)
* `putenv` ⟶
explicit `envp` argument in process creation
(see xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-environ[Specifying the Process Environment])
* `setenv` ⟶
explicit `envp` argument in process creation
(see xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-environ[Specifying the Process Environment])
* `strdupa` ⟶
`strdup` and `free`
(see <<sect-Defensive_Coding-C-Allocators-alloca>>)
* `strndupa` ⟶
`strndup` and `free`
(see <<sect-Defensive_Coding-C-Allocators-alloca>>)
* `system` ⟶
`posix_spawn`
or `fork`pass:attributes[{blank}]/pass:attributes[{blank}]`execve`pass:attributes[{blank}]/
(see xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-execve[Bypassing the Shell])
* `unsetenv` ⟶
explicit `envp` argument in process creation
(see xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-environ[Specifying the Process Environment])
[[sect-Defensive_Coding-C-String-Functions-Length]]
=== String Functions with Explicit Length Arguments
The C run-time library provides string manipulation functions
which not just look for NUL characters for string termination,
but also honor explicit lengths provided by the caller.
However, these functions evolved over a long period of time, and
the lengths mean different things depending on the function.
[[sect-Defensive_Coding-C-Libc-snprintf]]
==== `snprintf`
The `snprintf` function provides a way to
construct a string in a statically-sized buffer. (If the buffer
size is allocated on the heap, consider use
`asprintf` instead.)
[source,c]
----
include::example$C-String-Functions-snprintf.adoc[]
----
The second argument to the `snprintf` call
should always be the size of the buffer in the first argument
(which should be a character array). Elaborate pointer and
length arithmetic can introduce errors and nullify the
security benefits of `snprintf`.
In particular, `snprintf` is not well-suited
to constructing a string iteratively, by appending to an
existing buffer. `snprintf` returns one of
two values, `-1` on errors, or the number of
characters which *would have been written to the
buffer if the buffer were large enough*. This means
that adding the result of `snprintf` to the
buffer pointer to skip over the characters just written is
incorrect and risky. However, as long as the length argument
is not zero, the buffer will remain null-terminated. <<ex-Defensive_Coding-C-String-Functions-snprintf-incremental>>
works because `end -current > 0` is a loop
invariant. After the loop, the result string is in the
`buf` variable.
[[ex-Defensive_Coding-C-String-Functions-snprintf-incremental]]
.Repeatedly writing to a buffer using `snprintf`
====
[source,c]
----
include::example$C-String-Functions-snprintf-incremental.adoc[]
----
====
If you want to avoid the call to `strlen`
for performance reasons, you have to check for a negative
return value from `snprintf` and also check
if the return value is equal to the specified buffer length or
larger. Only if neither condition applies, you may advance
the pointer to the start of the write buffer by the number
return by `snprintf`. However, this
optimization is rarely worthwhile.
Note that it is not permitted to use the same buffer both as
the destination and as a source argument.
[[sect-Defensive_Coding-C-Libc-vsnprintf]]
==== `vsnprintf` and Format Strings
If you use `vsnprintf` (or
`vasprintf` or even
`snprintf`) with a format string which is
not a constant, but a function argument, it is important to
annotate the function with a `format`
function attribute, so that GCC can warn about misuse of your
function (see <<ex-Defensive_Coding-C-String-Functions-format-Attribute>>).
[[ex-Defensive_Coding-C-String-Functions-format-Attribute]]
.The `format` function attribute
====
[source,c]
----
include::example$C-String-Functions-format.adoc[]
----
====
[[sect-Defensive_Coding-C-Libc-strncpy]]
==== `strncpy`
The `strncpy` function does not ensure that
the target buffer is null-terminated. A common idiom for
ensuring NUL termination is:
[source,c]
----
include::example$C-String-Functions-strncpy.adoc[]
----
Another approach uses the `strncat`
function for this purpose:
[source,c]
----
include::example$C-String-Functions-strncat-as-strncpy.adoc[]
----
[[sect-Defensive_Coding-C-Libc-strncat]]
==== `strncat`
The length argument of the `strncat`
function specifies the maximum number of characters copied
from the source buffer, excluding the terminating NUL
character. This means that the required number of bytes in
the destination buffer is the length of the original string,
plus the length argument in the `strncat`
call, plus one. Consequently, this function is rarely
appropriate for performing a length-checked string operation,
with the notable exception of the `strcpy`
emulation described in <<sect-Defensive_Coding-C-Libc-strncpy>>.
To implement a length-checked string append, you can use an
approach similar to <<ex-Defensive_Coding-C-String-Functions-snprintf-incremental>>:
[source,c]
----
include::example$C-String-Functions-strncat-emulation.adoc[]
----
In many cases, including this one, the string concatenation
can be avoided by combining everything into a single format
string:
[source,c]
----
include::example$C-String-Functions-strncat-merged.adoc[]
----
But you should must not dynamically construct format strings
to avoid concatenation because this would prevent GCC from
type-checking the argument lists.
It is not possible to use format strings like
`"%s%s"` to implement concatenation, unless
you use separate buffers. `snprintf` does
not support overlapping source and target strings.
==== `strlcpy` and `strlcat`
Some systems support `strlcpy` and
`strlcat` functions which behave this way,
but these functions are not part of GNU libc.
`strlcpy` is often replaced with
`snprintf` with a `"%s"`
format string. See <<sect-Defensive_Coding-C-Libc-strncpy>> for a caveat
related to the `snprintf` return value.
To emulate `strlcat`, use the approach
described in <<sect-Defensive_Coding-C-Libc-strncat>>.
==== ISO C11 Annex K *pass:attributes[{blank}]`_s` functions
ISO C11 adds another set of length-checking functions, but GNU
libc currently does not implement them.
==== Other `strn*` and `stpn*` functions
GNU libc contains additional functions with different variants
of length checking. Consult the documentation before using
them to find out what the length actually means.
=== Using tricky syscalls or library functions
==== `readlink`
This is the hardest system call to use correctly because of everything you have to do
* The buf should be of PATH_MAX length, that includes space for the terminating NUL character.
* The bufsize should be `sizeof(buf) - 1`
* `readlink` return value should be caught as a signed integer (ideally type `ssize_t`).
* It should be checked for < 0 for indication of errors.
* The caller needs to '\0' -terminate the buffer using the returned value as an index.
==== `chroot`
* Target dir should be writable only by root (this implies owned by).
* Must call `chdir` immediately after chroot or you are not really in the changed root.
==== `stat`, `lstat`, `fstatat`
* These functions have an inherent race in that you operate on the path name which could change in the mean time. Using fstat is recommended when stat is used.
* If `S_ISLNK` macro is used, the stat buffer MUST come from lstat or from fstatat with `AT_SYMLINK_NOFOLLOW`
* If you are doing something really important, call fstat after opening and compare the before and after stat buffers before trusting them.
==== `setgid`, `setuid`:
* Call these in the right order: groups and then uid.
* Always check the return code.
* If `setgid` & `setuid` are used, supplemental groups are not reset. This must be done with setgroups or initgroups before the uid change.

135
modules/ROOT/pages/programming-languages/C-Other.adoc
View file

@ -1,135 +0,0 @@
:experimental:
[[sect-Defensive_Coding-C-Other]]
== Other C-related Topics
[[sect-Defensive_Coding-C-Wrapper-Functions]]
=== Wrapper Functions
Some libraries provide wrappers for standard library functions.
Common cases include allocation functions such as
`xmalloc` which abort the process on
allocation failure (instead of returning a
`NULL` pointer), or alternatives to relatively
recent library additions such as `snprintf`
(along with implementations for systems which lack them).
In general, such wrappers are a bad idea, particularly if they
are not implemented as inline functions or preprocessor macros.
The compiler lacks knowledge of such wrappers outside the
translation unit which defines them, which means that some
optimizations and security checks are not performed. Adding
`__attribute__` annotations to function
declarations can remedy this to some extent, but these
annotations have to be maintained carefully for feature parity
with the standard implementation.
At the minimum, you should apply these attributes:
* If you wrap function which accepts are GCC-recognized format
string (for example, a `printf`-style
function used for logging), you should add a suitable
`format` attribute, as in <<ex-Defensive_Coding-C-String-Functions-format-Attribute>>.
* If you wrap a function which carries a
`warn_unused_result` attribute and you
propagate its return value, your wrapper should be declared
with `warn_unused_result` as well.
* Duplicating the buffer length checks based on the
`__builtin_object_size` GCC builtin is
desirable if the wrapper processes arrays. (This
functionality is used by the
`-D_FORTIFY_SOURCE=2` checks to guard
against static buffer overflows.) However, designing
appropriate interfaces and implementing the checks may not
be entirely straightforward.
For other attributes (such as `malloc`),
careful analysis and comparison with the compiler documentation
is required to check if propagating the attribute is
appropriate. Incorrectly applied attributes can result in
undesired behavioral changes in the compiled code.
[[sect-Defensive_Coding-C-Common-Mistakes]]
=== Common mistakes
==== Mistakes in macros
A macro is a name given to a block of C statements as a pre-processor
directive. Being a pre-processor the block of code is transformed by
the compiler before being compiled.
A macro starts with the preprocessor directive, #define. It can
define a single value or any 'substitution', syntactically valid or
not.
A common mistake when working with macros is that programmers treat
arguments to macros like they would functions. This becomes an issue
when the argument may be expanded multiple times in a macro.
For example:
macro-misuse.c
[source,C]
----
#define simple(thing) do { \
if (thing < 1) { \
y = thing; \
} \
else if (thing > 100) { \
y = thing * 2 + thing; \
} \
else { \
y = 200; \
} \
} while (0)
int main(void) {
int x = 200;
int y = 0;
simple(x++);
return 0;
}
----
Each pass through the simple() macro would mean that x could be
expanded in-place each time 'thing' was mentioned.
The 'main' function would be processed and expanded as follows:
macro-misuse-post-processing.c
[source,C]
----
int main(void) {
int x = 200;
int y = 0;
do {
if ( x++ < 1) {
y = x++;
}
else if (thing > 100) {
y = x++ * 2 + x++;
}
else {
x = 200;
}
} while (0)
return 0;
}
----
Each evaluation of the argument to 'simple' (x++) would be executed
each time it was referenced.
While this may be 'expected' behaviour by the original creator, large
projects may have programmers who were unaware of how the macro may
expand and this may introduce unexpected behaviour, especially if the
value is later used as indexing into an array or able to be
overflowed.

865
modules/ROOT/pages/programming-languages/C.adoc
View file

@ -1,13 +1,866 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-C]]
= The C Programming Language
include::C-Language.adoc[]
[[sect-Defensive_Coding-C-Language]]
== The Core Language
include::C-Libc.adoc[]
C provides no memory safety. Most recommendations in this section
deal with this aspect of the language.
include::C-Allocators.adoc[]
[[sect-Defensive_Coding-C-Undefined]]
=== Undefined Behavior
include::C-Other.adoc[]
Some C constructs are defined to be undefined by the C standard.
This does not only mean that the standard does not describe
what happens when the construct is executed. It also allows
optimizing compilers such as GCC to assume that this particular
construct is never reached. In some cases, this has caused
GCC to optimize security checks away. (This is not a flaw in GCC
or the C language. But C certainly has some areas which are more
difficult to use than others.)
Common sources of undefined behavior are:
* out-of-bounds array accesses
* null pointer dereferences
* overflow in signed integer arithmetic
[[sect-Defensive_Coding-C-Pointers]]
=== Recommendations for Pointers and Array Handling
Always keep track of the size of the array you are working with.
Often, code is more obviously correct when you keep a pointer
past the last element of the array, and calculate the number of
remaining elements by subtracting the current position from
that pointer. The alternative, updating a separate variable
every time when the position is advanced, is usually less
obviously correct.
<<ex-Defensive_Coding-C-Pointers-remaining>>
shows how to extract Pascal-style strings from a character
buffer. The two pointers kept for length checks are
`inend` and `outend`.
`inp` and `outp` are the
respective positions.
The number of input bytes is checked using the expression
`len > (size_t)(inend - inp)`.
The cast silences a compiler warning;
`inend` is always larger than
`inp`.
[[ex-Defensive_Coding-C-Pointers-remaining]]
.Array processing in C
====
[source,c]
----
include::example$C-Pointers-remaining.adoc[]
----
====
It is important that the length checks always have the form
`len > (size_t)(inend - inp)`, where
`len` is a variable of type
`size_t` which denotes the *total*
number of bytes which are about to be read or written next. In
general, it is not safe to fold multiple such checks into one,
as in `len1 + len2 > (size_t)(inend - inp)`,
because the expression on the left can overflow or wrap around
(see <<sect-Defensive_Coding-C-Arithmetic>>), and it
no longer reflects the number of bytes to be processed.
[[sect-Defensive_Coding-C-Arithmetic]]
=== Recommendations for Integer Arithmetic
Overflow in signed integer arithmetic is undefined. This means
that it is not possible to check for overflow after it happened,
see <<ex-Defensive_Coding-C-Arithmetic-bad>>.
[[ex-Defensive_Coding-C-Arithmetic-bad]]
.Incorrect overflow detection in C
====
[source,c]
----
include::example$C-Arithmetic-add.adoc[]
----
====
The following approaches can be used to check for overflow,
without actually causing it.
* Use a wider type to perform the calculation, check that the
result is within bounds, and convert the result to the
original type. All intermediate results must be checked in
this way.
* Perform the calculation in the corresponding unsigned type
and use bit fiddling to detect the overflow.
<<ex-Defensive_Coding-C-Arithmetic-add_unsigned>>
shows how to perform an overflow check for unsigned integer
addition. For three or more terms, all the intermediate
additions have to be checked in this way.
[[ex-Defensive_Coding-C-Arithmetic-add_unsigned]]
.Overflow checking for unsigned addition
====
[source,c]
----
include::example$C-Arithmetic-add_unsigned.adoc[]
----
====
* Compute bounds for acceptable input values which are known
to avoid overflow, and reject other values. This is the
preferred way for overflow checking on multiplications,
see <<ex-Defensive_Coding-C-Arithmetic-mult>>.
[[ex-Defensive_Coding-C-Arithmetic-mult]]
.Overflow checking for unsigned multiplication
====
[source,c]
----
include::example$C-Arithmetic-mult.adoc[]
----
====
Basic arithmetic operations are commutative, so for bounds checks,
there are two different but mathematically equivalent
expressions. Sometimes, one of the expressions results in
better code because parts of it can be reduced to a constant.
This applies to overflow checks for multiplication `a *
b` involving a constant `a`, where the
expression is reduced to `b > C` for some
constant `C` determined at compile time. The
other expression, `b && a > ((unsigned)-1) /
b`, is more difficult to optimize at compile time.
When a value is converted to a signed integer, GCC always
chooses the result based on 2's complement arithmetic. This GCC
extension (which is also implemented by other compilers) helps a
lot when implementing overflow checks.
Sometimes, it is necessary to compare unsigned and signed
integer variables. This results in a compiler warning,
*comparison between signed and unsigned integer
expressions*, because the comparison often gives
unexpected results for negative values. When adding a cast,
make sure that negative values are covered properly. If the
bound is unsigned and the checked quantity is signed, you should
cast the checked quantity to an unsigned type as least as wide
as either operand type. As a result, negative values will fail
the bounds check. (You can still check for negative values
separately for clarity, and the compiler will optimize away this
redundant check.)
Legacy code should be compiled with the [option]`-fwrapv`
GCC option. As a result, GCC will provide 2's complement
semantics for integer arithmetic, including defined behavior on
integer overflow.
[[sect-Defensive_Coding-C-Globals]]
=== Global Variables
Global variables should be avoided because they usually lead to
thread safety hazards. In any case, they should be declared
`static`, so that access is restricted to a
single translation unit.
Global constants are not a problem, but declaring them can be
tricky. <<ex-Defensive_Coding-C-Globals-String_Array>>
shows how to declare a constant array of constant strings.
The second `const` is needed to make the
array constant, and not just the strings. It must be placed
after the `*`, and not before it.
[[ex-Defensive_Coding-C-Globals-String_Array]]
.Declaring a constant array of constant strings
====
[source,c]
----
include::example$C-Globals-String_Array.adoc[]
----
====
Sometimes, static variables local to functions are used as a
replacement for proper memory management. Unlike non-static
local variables, it is possible to return a pointer to static
local variables to the caller. But such variables are
well-hidden, but effectively global (just as static variables at
file scope). It is difficult to add thread safety afterwards if
such interfaces are used. Merely dropping the
`static` keyword in such cases leads to
undefined behavior.
Another source for static local variables is a desire to reduce
stack space usage on embedded platforms, where the stack may
span only a few hundred bytes. If this is the only reason why
the `static` keyword is used, it can just be
dropped, unless the object is very large (larger than
128 kilobytes on 32-bit platforms). In the latter case, it is
recommended to allocate the object using
`malloc`, to obtain proper array checking, for
the same reasons outlined in <<sect-Defensive_Coding-C-Allocators-alloca>>.
[[sect-Defensive_Coding-C-Libc]]
== The C Standard Library
Parts of the C standard library (and the UNIX and GNU extensions)
are difficult to use, so you should avoid them.
Please check the applicable documentation before using the
recommended replacements. Many of these functions allocate
buffers using `malloc` which your code must
deallocate explicitly using `free`.
[[sect-Defensive_Coding-C-Absolutely-Banned]]
=== Absolutely Banned Interfaces
The functions listed below must not be used because they are
almost always unsafe. Use the indicated replacements instead.
* `gets`
⟶ `fgets`
* `getwd`
⟶ `getcwd`
or `get_current_dir_name`
* `readdir_r` ⟶ `readdir`
* `realpath` (with a non-NULL second parameter)
⟶ `realpath` with NULL as the second parameter,
or `canonicalize_file_name`
The constants listed below must not be used, either. Instead,
code must allocate memory dynamically and use interfaces with
length checking.
* `NAME_MAX` (limit not actually enforced by
the kernel)
* `PATH_MAX` (limit not actually enforced by
the kernel)
* `_PC_NAME_MAX` (This limit, returned by the
`pathconf` function, is not enforced by
the kernel.)
* `_PC_PATH_MAX` (This limit, returned by the
`pathconf` function, is not enforced by
the kernel.)
The following structure members must not be used.
* `f_namemax` in `struct
statvfs` (limit not actually enforced by the kernel,
see `_PC_NAME_MAX` above)
[[sect-Defensive_Coding-C-Avoid]]
=== Functions to Avoid
The following string manipulation functions can be used securely
in principle, but their use should be avoided because they are
difficult to use correctly. Calls to these functions can be
replaced with `asprintf` or
`vasprintf`. (For non-GNU targets, these
functions are available from Gnulib.) In some cases, the
`snprintf` function might be a suitable
replacement, see <<sect-Defensive_Coding-C-String-Functions-Length>>.
* `sprintf`
* `strcat`
* `strcpy`
* `vsprintf`
Use the indicated replacements for the functions below.
* `alloca` ⟶
`malloc` and `free`
(see <<sect-Defensive_Coding-C-Allocators-alloca>>)
* `putenv` ⟶
explicit `envp` argument in process creation
(see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-environ[Specifying the Process Environment])
* `setenv` ⟶
explicit `envp` argument in process creation
(see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-environ[Specifying the Process Environment])
* `strdupa` ⟶
`strdup` and `free`
(see <<sect-Defensive_Coding-C-Allocators-alloca>>)
* `strndupa` ⟶
`strndup` and `free`
(see <<sect-Defensive_Coding-C-Allocators-alloca>>)
* `system` ⟶
`posix_spawn`
or `fork`pass:attributes[{blank}]/pass:attributes[{blank}]`execve`pass:attributes[{blank}]/
(see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-execve[Bypassing the Shell])
* `unsetenv` ⟶
explicit `envp` argument in process creation
(see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-environ[Specifying the Process Environment])
[[sect-Defensive_Coding-C-String-Functions-Length]]
=== String Functions with Explicit Length Arguments
The C run-time library provides string manipulation functions
which not just look for NUL characters for string termination,
but also honor explicit lengths provided by the caller.
However, these functions evolved over a long period of time, and
the lengths mean different things depending on the function.
[[sect-Defensive_Coding-C-Libc-snprintf]]
==== `snprintf`
The `snprintf` function provides a way to
construct a string in a statically-sized buffer. (If the buffer
size is allocated on the heap, consider use
`asprintf` instead.)
[source,c]
----
include::example$C-String-Functions-snprintf.adoc[]
----
The second argument to the `snprintf` call
should always be the size of the buffer in the first argument
(which should be a character array). Elaborate pointer and
length arithmetic can introduce errors and nullify the
security benefits of `snprintf`.
In particular, `snprintf` is not well-suited
to constructing a string iteratively, by appending to an
existing buffer. `snprintf` returns one of
two values, `-1` on errors, or the number of
characters which *would have been written to the
buffer if the buffer were large enough*. This means
that adding the result of `snprintf` to the
buffer pointer to skip over the characters just written is
incorrect and risky. However, as long as the length argument
is not zero, the buffer will remain null-terminated. <<ex-Defensive_Coding-C-String-Functions-snprintf-incremental>>
works because `end -current > 0` is a loop
invariant. After the loop, the result string is in the
`buf` variable.
[[ex-Defensive_Coding-C-String-Functions-snprintf-incremental]]
.Repeatedly writing to a buffer using `snprintf`
====
[source,c]
----
include::example$C-String-Functions-snprintf-incremental.adoc[]
----
====
If you want to avoid the call to `strlen`
for performance reasons, you have to check for a negative
return value from `snprintf` and also check
if the return value is equal to the specified buffer length or
larger. Only if neither condition applies, you may advance
the pointer to the start of the write buffer by the number
return by `snprintf`. However, this
optimization is rarely worthwhile.
Note that it is not permitted to use the same buffer both as
the destination and as a source argument.
[[sect-Defensive_Coding-C-Libc-vsnprintf]]
==== `vsnprintf` and Format Strings
If you use `vsnprintf` (or
`vasprintf` or even
`snprintf`) with a format string which is
not a constant, but a function argument, it is important to
annotate the function with a `format`
function attribute, so that GCC can warn about misuse of your
function (see <<ex-Defensive_Coding-C-String-Functions-format-Attribute>>).
[[ex-Defensive_Coding-C-String-Functions-format-Attribute]]
.The `format` function attribute
====
[source,c]
----
include::example$C-String-Functions-format.adoc[]
----
====
[[sect-Defensive_Coding-C-Libc-strncpy]]
==== `strncpy`
The `strncpy` function does not ensure that
the target buffer is null-terminated. A common idiom for
ensuring NUL termination is:
[source,c]
----
include::example$C-String-Functions-strncpy.adoc[]
----
Another approach uses the `strncat`
function for this purpose:
[source,c]
----
include::example$C-String-Functions-strncat-as-strncpy.adoc[]
----
[[sect-Defensive_Coding-C-Libc-strncat]]
==== `strncat`
The length argument of the `strncat`
function specifies the maximum number of characters copied
from the source buffer, excluding the terminating NUL
character. This means that the required number of bytes in
the destination buffer is the length of the original string,
plus the length argument in the `strncat`
call, plus one. Consequently, this function is rarely
appropriate for performing a length-checked string operation,
with the notable exception of the `strcpy`
emulation described in <<sect-Defensive_Coding-C-Libc-strncpy>>.
To implement a length-checked string append, you can use an
approach similar to <<ex-Defensive_Coding-C-String-Functions-snprintf-incremental>>:
[source,c]
----
include::example$C-String-Functions-strncat-emulation.adoc[]
----
In many cases, including this one, the string concatenation
can be avoided by combining everything into a single format
string:
[source,c]
----
include::example$C-String-Functions-strncat-merged.adoc[]
----
But you should must not dynamically construct format strings
to avoid concatenation because this would prevent GCC from
type-checking the argument lists.
It is not possible to use format strings like
`"%s%s"` to implement concatenation, unless
you use separate buffers. `snprintf` does
not support overlapping source and target strings.
==== `strlcpy` and `strlcat`
Some systems support `strlcpy` and
`strlcat` functions which behave this way,
but these functions are not part of GNU libc.
`strlcpy` is often replaced with
`snprintf` with a `"%s"`
format string. See <<sect-Defensive_Coding-C-Libc-strncpy>> for a caveat
related to the `snprintf` return value.
To emulate `strlcat`, use the approach
described in <<sect-Defensive_Coding-C-Libc-strncat>>.
==== ISO C11 Annex K *pass:attributes[{blank}]`_s` functions
ISO C11 adds another set of length-checking functions, but GNU
libc currently does not implement them.
==== Other `strn*` and `stpn*` functions
GNU libc contains additional functions with different variants
of length checking. Consult the documentation before using
them to find out what the length actually means.
=== Using tricky syscalls or library functions
==== `readlink`
This is the hardest system call to use correctly because of everything you have to do
* The buf should be of PATH_MAX length, that includes space for the terminating NUL character.
* The bufsize should be `sizeof(buf) - 1`
* `readlink` return value should be caught as a signed integer (ideally type `ssize_t`).
* It should be checked for < 0 for indication of errors.
* The caller needs to '\0' -terminate the buffer using the returned value as an index.
==== `chroot`
* Target dir should be writable only by root (this implies owned by).
* Must call `chdir` immediately after chroot or you are not really in the changed root.
==== `stat`, `lstat`, `fstatat`
* These functions have an inherent race in that you operate on the path name which could change in the mean time. Using fstat is recommended when stat is used.
* If `S_ISLNK` macro is used, the stat buffer MUST come from lstat or from fstatat with `AT_SYMLINK_NOFOLLOW`
* If you are doing something really important, call fstat after opening and compare the before and after stat buffers before trusting them.
==== `setgid`, `setuid`:
* Call these in the right order: groups and then uid.
* Always check the return code.
* If `setgid` & `setuid` are used, supplemental groups are not reset. This must be done with setgroups or initgroups before the uid change.
[[sect-Defensive_Coding-C-Allocators]]
== Memory Allocators
=== `malloc` and Related Functions
The C library interfaces for memory allocation are provided by
`malloc`, `free` and
`realloc`, and the
`calloc` function. In addition to these
generic functions, there are derived functions such as
`strdup` which perform allocation using
`malloc` internally, but do not return
untyped heap memory (which could be used for any object).
The C compiler knows about these functions and can use their
expected behavior for optimizations. For instance, the compiler
assumes that an existing pointer (or a pointer derived from an
existing pointer by arithmetic) will not point into the memory
area returned by `malloc`.
If the allocation fails, `realloc` does not
free the old pointer. Therefore, the idiom `ptr =
realloc(ptr, size);` is wrong because the memory
pointed to by `ptr` leaks in case of an error.
[[sect-Defensive_Coding-C-Use-After-Free]]
==== Use-after-free errors
After `free`, the pointer is invalid.
Further pointer dereferences are not allowed (and are usually
detected by [application]*valgrind*). Less obvious
is that any *use* of the old pointer value is
not allowed, either. In particular, comparisons with any other
pointer (or the null pointer) are undefined according to the C
standard.
The same rules apply to `realloc` if the
memory area cannot be enlarged in-place. For instance, the
compiler may assume that a comparison between the old and new
pointer will always return false, so it is impossible to detect
movement this way.
On a related note, `realloc` frees the memory area if the new size is
zero. If the size unintentionally becomes zero, as a result of
unsigned integer wrap-around for instance, the following idiom causes
a double-free.
[source,c]
----
new_size = size + x; /* 'x' is a very large value and the result wraps around to zero */
new_ptr = realloc(ptr, new_size);
if (!new_ptr) {
free(ptr);
}
----
==== Handling Memory Allocation Errors
Recovering from out-of-memory errors is often difficult or even
impossible. In these cases, `malloc` and
other allocation functions return a null pointer. Dereferencing
this pointer lead to a crash. Such dereferences can even be
exploitable for code execution if the dereference is combined
with an array subscript.
In general, if you cannot check all allocation calls and
handle failure, you should abort the program on allocation
failure, and not rely on the null pointer dereference to
terminate the process. See
xref:tasks/Tasks-Serialization.adoc#sect-Defensive_Coding-Tasks-Serialization-Decoders[Recommendations for Manually-written Decoders]
for related memory allocation concerns.
[[sect-Defensive_Coding-C-Allocators-alloca]]
=== `alloca` and Other Forms of Stack-based Allocation
Allocation on the stack is risky because stack overflow checking
is implicit. There is a guard page at the end of the memory
area reserved for the stack. If the program attempts to read
from or write to this guard page, a `SIGSEGV`
signal is generated and the program typically terminates.
This is sufficient for detecting typical stack overflow
situations such as unbounded recursion, but it fails when the
stack grows in increments larger than the size of the guard
page. In this case, it is possible that the stack pointer ends
up pointing into a memory area which has been allocated for a
different purposes. Such misbehavior can be exploitable.
A common source for large stack growth are calls to
`alloca` and related functions such as
`strdupa`. These functions should be avoided
because of the lack of error checking. (They can be used safely
if the allocated size is less than the page size (typically,
4096 bytes), but this case is relatively rare.) Additionally,
relying on `alloca` makes it more difficult
to reorganize the code because it is not allowed to use the
pointer after the function calling `alloca`
has returned, even if this function has been inlined into its
caller.
Similar concerns apply to *variable-length
arrays* (VLAs), a feature of the C99 standard which
started as a GNU extension. For large objects exceeding the
page size, there is no error checking, either.
In both cases, negative or very large sizes can trigger a
stack-pointer wraparound, and the stack pointer and end up
pointing into caller stack frames, which is fatal and can be
exploitable.
If you want to use `alloca` or VLAs for
performance reasons, consider using a small on-stack array (less
than the page size, large enough to fulfill most requests). If
the requested size is small enough, use the on-stack array.
Otherwise, call `malloc`. When exiting the
function, check if `malloc` had been called,
and free the buffer as needed.
If portability is not important in your program, an alternative way of
automatic memory management is to leverage the `cleanup` attribute
supported by the recent versions of GCC and Clang. If a local variable
is declared with the attribute, the specified cleanup function will be
called when the variable goes out of scope.
[source,c]
----
static inline void freep(void *p) {
free(*(void**) p);
}
void somefunction(const char *param) {
if (strcmp(param, "do_something_complex") == 0) {
__attribute__((cleanup(freep))) char *ptr = NULL;
/* Allocate a temporary buffer */
ptr = malloc(size);
/* Do something on it, but do not need to manually call free() */
}
}
----
[[sect-Defensive_Coding-C-Allocators-Arrays]]
=== Array Allocation
When allocating arrays, it is important to check for overflows.
The `calloc` function performs such checks.
If `malloc` or `realloc`
is used, the size check must be written manually. For instance,
to allocate an array of `n` elements of type
`T`, check that the requested size is not
greater than `((size_t) -1) / sizeof(T)`. See
<<sect-Defensive_Coding-C-Arithmetic>>.
GNU libc provides a dedicated function `reallocarray` that allocates
an array with those checks performed internally. However, care must
be taken if portability is important: while the interface originated
in OpenBSD and has been adopted in many other platforms, NetBSD
exposes an incompatible behavior with the same interface.
[[sect-Defensive_Coding-C-Allocators-Custom]]
=== Custom Memory Allocators
Custom memory allocates come in two forms: replacements for
`malloc`, and completely different interfaces
for memory management. Both approaches can reduce the
effectiveness of [application]*valgrind* and similar
tools, and the heap corruption detection provided by GNU libc, so
they should be avoided.
Memory allocators are difficult to write and contain many
performance and security pitfalls.
* When computing array sizes or rounding up allocation
requests (to the next allocation granularity, or for
alignment purposes), checks for arithmetic overflow are
required.
* Size computations for array allocations need overflow
checking. See <<sect-Defensive_Coding-C-Allocators-Arrays>>.
* It can be difficult to beat well-tuned general-purpose
allocators. In micro benchmarks, pool allocators can show
huge wins, and size-specific pools can reduce internal
fragmentation. But often, utilization of individual pools
is poor, and external fragmentation increases the overall
memory usage.
=== Conservative Garbage Collection
Garbage collection can be an alternative to explicit memory
management using `malloc` and
`free`. The Boehm-Dehmers-Weiser allocator
can be used from C programs, with minimal type annotations.
Performance is competitive with `malloc` on
64-bit architectures, especially for multi-threaded programs.
The stop-the-world pauses may be problematic for some real-time
applications, though.
However, using a conservative garbage collector may reduce
opportunities for code reduce because once one library in a
program uses garbage collection, the whole process memory needs
to be subject to it, so that no pointers are missed. The
Boehm-Dehmers-Weiser collector also reserves certain signals for
internal use, so it is not fully transparent to the rest of the
program.
[[sect-Defensive_Coding-C-Other]]
== Other C-related Topics
[[sect-Defensive_Coding-C-Wrapper-Functions]]
=== Wrapper Functions
Some libraries provide wrappers for standard library functions.
Common cases include allocation functions such as
`xmalloc` which abort the process on
allocation failure (instead of returning a
`NULL` pointer), or alternatives to relatively
recent library additions such as `snprintf`
(along with implementations for systems which lack them).
In general, such wrappers are a bad idea, particularly if they
are not implemented as inline functions or preprocessor macros.
The compiler lacks knowledge of such wrappers outside the
translation unit which defines them, which means that some
optimizations and security checks are not performed. Adding
`__attribute__` annotations to function
declarations can remedy this to some extent, but these
annotations have to be maintained carefully for feature parity
with the standard implementation.
At the minimum, you should apply these attributes:
* If you wrap function which accepts are GCC-recognized format
string (for example, a `printf`-style
function used for logging), you should add a suitable
`format` attribute, as in <<ex-Defensive_Coding-C-String-Functions-format-Attribute>>.
* If you wrap a function which carries a
`warn_unused_result` attribute and you
propagate its return value, your wrapper should be declared
with `warn_unused_result` as well.
* Duplicating the buffer length checks based on the
`__builtin_object_size` GCC builtin is
desirable if the wrapper processes arrays. (This
functionality is used by the
`-D_FORTIFY_SOURCE=2` checks to guard
against static buffer overflows.) However, designing
appropriate interfaces and implementing the checks may not
be entirely straightforward.
For other attributes (such as `malloc`),
careful analysis and comparison with the compiler documentation
is required to check if propagating the attribute is
appropriate. Incorrectly applied attributes can result in
undesired behavioral changes in the compiled code.
[[sect-Defensive_Coding-C-Common-Mistakes]]
=== Common mistakes
==== Mistakes in macros
A macro is a name given to a block of C statements as a pre-processor
directive. Being a pre-processor the block of code is transformed by
the compiler before being compiled.
A macro starts with the preprocessor directive, #define. It can
define a single value or any 'substitution', syntactically valid or
not.
A common mistake when working with macros is that programmers treat
arguments to macros like they would functions. This becomes an issue
when the argument may be expanded multiple times in a macro.
For example:
macro-misuse.c
[source,C]
----
#define simple(thing) do { \
if (thing < 1) { \
y = thing; \
} \
else if (thing > 100) { \
y = thing * 2 + thing; \
} \
else { \
y = 200; \
} \
} while (0)
int main(void) {
int x = 200;
int y = 0;
simple(x++);
return 0;
}
----
Each pass through the simple() macro would mean that x could be
expanded in-place each time 'thing' was mentioned.
The 'main' function would be processed and expanded as follows:
macro-misuse-post-processing.c
[source,C]
----
int main(void) {
int x = 200;
int y = 0;
do {
if ( x++ < 1) {
y = x++;
}
else if (thing > 100) {
y = x++ * 2 + x++;
}
else {
x = 200;
}
} while (0)
return 0;
}
----
Each evaluation of the argument to 'simple' (x++) would be executed
each time it was referenced.
While this may be 'expected' behaviour by the original creator, large
projects may have programmers who were unaware of how the macro may
expand and this may introduce unexpected behaviour, especially if the
value is later used as indexing into an array or able to be
overflowed.

324
modules/ROOT/pages/programming-languages/CXX.adoc
View file

@ -1,8 +1,324 @@
:experimental:
:toc:
[[chap-Defensive_Coding-CXX]]
= The C++ Programming Language
include::partial$entities.adoc[]
include::CXX-Language.adoc[]
= The {cpp} Programming Language
include::CXX-Std.adoc[]
[[sect-Defensive_Coding-CXX-Language]]
== The Core Language
C++ includes a large subset of the C language. As far as the C
subset is used, the recommendations in xref:programming-languages/C.adoc#chap-Defensive_Coding-C[Defensive Coding in C] apply.
=== Array Allocation with `operator new[]`
For very large values of `n`, an expression
like `new T[n]` can return a pointer to a heap
region which is too small. In other words, not all array
elements are actually backed with heap memory reserved to the
array. Current GCC versions generate code that performs a
computation of the form `sizeof(T) * size_t(n) + cookie_size`, where `cookie_size` is
currently at most 8. This computation can overflow, and GCC
versions prior to 4.8 generated code which did not detect this.
(Fedora 18 was the first release which fixed this in GCC.)
The `std::vector` template can be used instead
an explicit array allocation. (The GCC implementation detects
overflow internally.)
If there is no alternative to `operator new[]`
and the sources will be compiled with older GCC versions, code
which allocates arrays with a variable length must check for
overflow manually. For the `new T[n]` example,
the size check could be `n || (n > 0 && n >
(size_t(-1) - 8) / sizeof(T))`. (See xref:programming-languages/C.adoc#sect-Defensive_Coding-C-Arithmetic[Recommendations for Integer Arithmetic]) If there are
additional dimensions (which must be constants according to the
{cpp} standard), these should be included as factors in the
divisor.
These countermeasures prevent out-of-bounds writes and potential
code execution. Very large memory allocations can still lead to
a denial of service. xref:tasks/Tasks-Serialization.adoc#sect-Defensive_Coding-Tasks-Serialization-Decoders[Recommendations for Manually-written Decoders]
contains suggestions for mitigating this problem when processing
untrusted data.
See xref:tasks/programming-languages/C.adoc#sect-Defensive_Coding-C-Allocators-Arrays[Array Allocation]
for array allocation advice for C-style memory allocation.
=== Overloading
Do not overload functions with versions that have different
security characteristics. For instance, do not implement a
function `strcat` which works on
`std::string` arguments. Similarly, do not name
methods after such functions.
=== ABI compatibility and preparing for security updates
A stable binary interface (ABI) is vastly preferred for security
updates. Without a stable ABI, all reverse dependencies need
recompiling, which can be a lot of work and could even be
impossible in some cases. Ideally, a security update only
updates a single dynamic shared object, and is picked up
automatically after restarting affected processes.
Outside of extremely performance-critical code, you should
ensure that a wide range of changes is possible without breaking
ABI. Some very basic guidelines are:
* Avoid inline functions.
* Use the pointer-to-implementation idiom.
* Try to avoid templates. Use them if the increased type
safety provides a benefit to the programmer.
* Move security-critical code out of templated code, so that
it can be patched in a central place if necessary.
The KDE project publishes a document with more extensive
guidelines on ABI-preserving changes to {cpp} code, link:++https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B++[Policies/Binary
Compatibility Issues With {cpp}]
(*d-pointer* refers to the
pointer-to-implementation idiom).
[[sect-Defensive_Coding-CXX-Language-CXX11]]
=== {cpp}0X and {cpp}11 Support
GCC offers different language compatibility modes:
* [option]`-std=c++98` for the original 1998 {cpp}
standard
* [option]`-std=c++03` for the 1998 standard with the
changes from the TR1 technical report
* [option]`-std=c++11` for the 2011 {cpp} standard. This
option should not be used.
* [option]`-std=c++0x` for several different versions
of {cpp}11 support in development, depending on the GCC
version. This option should not be used.
For each of these flags, there are variants which also enable
GNU extensions (mostly language features also found in C99 or
C11):
* [option]`-std=gnu++98`
* [option]`-std=gnu++03`
* [option]`-std=gnu++11`
Again, [option]`-std=gnu++11` should not be used.
If you enable {cpp}11 support, the ABI of the standard {cpp} library
`libstdc++` will change in subtle ways.
Currently, no {cpp} libraries are compiled in {cpp}11 mode, so if
you compile your code in {cpp}11 mode, it will be incompatible
with the rest of the system. Unfortunately, this is also the
case if you do not use any {cpp}11 features. Currently, there is
no safe way to enable {cpp}11 mode (except for freestanding
applications).
The meaning of {cpp}0X mode changed from GCC release to GCC
release. Earlier versions were still ABI-compatible with {cpp}98
mode, but in the most recent versions, switching to {cpp}0X mode
activates {cpp}11 support, with its compatibility problems.
Some {cpp}11 features (or approximations thereof) are available
with TR1 support, that is, with [option]`-std=c++03` or
[option]`-std=gnu++03` and in the
`<tr1/*>` header files. This includes
`std::tr1::shared_ptr` (from
`<tr1/memory>`) and
`std::tr1::function` (from
`<tr1/functional>`). For other {cpp}11
features, the Boost {cpp} library contains replacements.
[[sect-Defensive_Coding-CXX-Std]]
== The C++ Standard Library
The C++ standard library includes most of its C counterpart
by reference, see xref:programming-languages/C.adoc#chap-Defensive_Coding-C[Defensive Coding in C].
[[sect-Defensive_Coding-CXX-Std-Functions]]
=== Functions That Are Difficult to Use
This section collects functions and function templates which are
part of the standard library and are difficult to use.
[[sect-Defensive_Coding-CXX-Std-Functions-Unpaired_Iterators]]
==== Unpaired Iterators
Functions which use output operators or iterators which do not
come in pairs (denoting ranges) cannot perform iterator range
checking.
(See <<sect-Defensive_Coding-CXX-Std-Iterators>>)
Function templates which involve output iterators are
particularly dangerous:
* `std::copy`
* `std::copy_backward`
* `std::copy_if`
* `std::move` (three-argument variant)
* `std::move_backward`
* `std::partition_copy_if`
* `std::remove_copy`
* `std::remove_copy_if`
* `std::replace_copy`
* `std::replace_copy_if`
* `std::swap_ranges`
* `std::transform`
In addition, `std::copy_n`,
`std::fill_n` and
`std::generate_n` do not perform iterator
checking, either, but there is an explicit count which has to be
supplied by the caller, as opposed to an implicit length
indicator in the form of a pair of forward iterators.
These output-iterator-expecting functions should only be used
with unlimited-range output iterators, such as iterators
obtained with the `std::back_inserter`
function.
Other functions use single input or forward iterators, which can
read beyond the end of the input range if the caller is not careful:
* `std::equal`
* `std::is_permutation`
* `std::mismatch`
[[sect-Defensive_Coding-CXX-Std-String]]
=== String Handling with `std::string`
The `std::string` class provides a convenient
way to handle strings. Unlike C strings,
`std::string` objects have an explicit length
(and can contain embedded NUL characters), and storage for its
characters is managed automatically. This section discusses
`std::string`, but these observations also
apply to other instances of the
`std::basic_string` template.
The pointer returned by the `data()` member
function does not necessarily point to a NUL-terminated string.
To obtain a C-compatible string pointer, use
`c_str()` instead, which adds the NUL
terminator.
The pointers returned by the `data()` and
`c_str()` functions and iterators are only
valid until certain events happen. It is required that the
exact `std::string` object still exists (even
if it was initially created as a copy of another string object).
Pointers and iterators are also invalidated when non-const
member functions are called, or functions with a non-const
reference parameter. The behavior of the GCC implementation
deviates from that required by the {cpp} standard if multiple
threads are present. In general, only the first call to a
non-const member function after a structural modification of the
string (such as appending a character) is invalidating, but this
also applies to member function such as the non-const version of
`begin()`, in violation of the {cpp} standard.
Particular care is necessary when invoking the
`c_str()` member function on a temporary
object. This is convenient for calling C functions, but the
pointer will turn invalid as soon as the temporary object is
destroyed, which generally happens when the outermost expression
enclosing the expression on which `c_str()`
is called completes evaluation. Passing the result of
`c_str()` to a function which does not store
or otherwise leak that pointer is safe, though.
Like with `std::vector` and
`std::array`, subscribing with
`operator[]` does not perform bounds checks.
Use the `at(size_type)` member function
instead. See <<sect-Defensive_Coding-CXX-Std-Subscript>>.
Furthermore, accessing the terminating NUL character using
`operator[]` is not possible. (In some
implementations, the `c_str()` member function
writes the NUL character on demand.)
Never write to the pointers returned by
`data()` or `c_str()`
after casting away `const`. If you need a
C-style writable string, use a
`std::vector<char>` object and its
`data()` member function. In this case, you
have to explicitly add the terminating NUL character.
GCC's implementation of `std::string` is
currently based on reference counting. It is expected that a
future version will remove the reference counting, due to
performance and conformance issues. As a result, code that
implicitly assumes sharing by holding to pointers or iterators
for too long will break, resulting in run-time crashes or worse.
On the other hand, non-const iterator-returning functions will
no longer give other threads an opportunity for invalidating
existing iterators and pointers because iterator invalidation
does not depend on sharing of the internal character array
object anymore.
[[sect-Defensive_Coding-CXX-Std-Subscript]]
=== Containers and `operator[]`
Many sequence containers similar to `std::vector`
provide both `operator[](size_type)` and a
member function `at(size_type)`. This applies
to `std::vector` itself,
`std::array`, `std::string`
and other instances of `std::basic_string`.
`operator[](size_type)` is not required by the
standard to perform bounds checking (and the implementation in
GCC does not). In contrast, `at(size_type)`
must perform such a check. Therefore, in code which is not
performance-critical, you should prefer
`at(size_type)` over
`operator[](size_type)`, even though it is
slightly more verbose.
The `front()` and `back()`
member functions are undefined if a vector object is empty. You
can use `vec.at(0)` and
`vec.at(vec.size() - 1)` as checked
replacements. For an empty vector, `data()` is
defined; it returns an arbitrary pointer, but not necessarily
the NULL pointer.
[[sect-Defensive_Coding-CXX-Std-Iterators]]
=== Iterators
Iterators do not perform any bounds checking. Therefore, all
functions that work on iterators should accept them in pairs,
denoting a range, and make sure that iterators are not moved
outside that range. For forward iterators and bidirectional
iterators, you need to check for equality before moving the
first or last iterator in the range. For random-access
iterators, you need to compute the difference before adding or
subtracting an offset. It is not possible to perform the
operation and check for an invalid operator afterwards.
Output iterators cannot be compared for equality. Therefore, it
is impossible to write code that detects that it has been
supplied an output area that is too small, and their use should
be avoided.
These issues make some of the standard library functions
difficult to use correctly, see <<sect-Defensive_Coding-CXX-Std-Functions-Unpaired_Iterators>>.

8
modules/ROOT/pages/programming-languages/Go.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Go]]
= The Go Programming Language
This chapter contains language-specific recommendations for Go.
@ -96,8 +97,7 @@ spontaneously.
Several packages in the `encoding` hierarchy
provide support for serialization and deserialization. The usual
caveats apply (see
<<chap-Defensive_Coding-Tasks-Serialization>>).
caveats apply (see xref:tasks/Tasks-Serialization.adoc[Serialization and Deserialization]).
As an additional precaution, the `Unmarshal`
and `Decode` functions should only be used with

252
modules/ROOT/pages/programming-languages/Java-Language.adoc
View file

@ -1,252 +0,0 @@
:experimental:
[[sect-Defensive_Coding-Java-Language]]
== The Core Language
Implementations of the Java programming language provide strong
memory safety, even in the presence of data races in concurrent
code. This prevents a large range of security vulnerabilities
from occurring, unless certain low-level features are used; see
<<sect-Defensive_Coding-Java-LowLevel>>.
[[sect-Defensive_Coding-Java-Language-ReadArray]]
=== Increasing Robustness when Reading Arrays
External data formats often include arrays, and the data is
stored as an integer indicating the number of array elements,
followed by this number of elements in the file or protocol data
unit. This length specified can be much larger than what is
actually available in the data source.
To avoid allocating extremely large amounts of data, you can
allocate a small array initially and grow it as you read more
data, implementing an exponential growth policy. See the
`readBytes(InputStream, int)` function in
<<ex-Defensive_Coding-Java-Language-ReadArray>>.
[[ex-Defensive_Coding-Java-Language-ReadArray]]
.Incrementally reading a byte array
====
[source,java]
----
include::example$Java-Language-ReadArray.adoc[]
----
====
When reading data into arrays, hash maps or hash sets, use the
default constructor and do not specify a size hint. You can
simply add the elements to the collection as you read them.
[[sect-Defensive_Coding-Java-Language-Resources]]
=== Resource Management
Unlike C++, Java does not offer destructors which can deallocate
resources in a predictable fashion. All resource management has
to be manual, at the usage site. (Finalizers are generally not
usable for resource management, especially in high-performance
code; see <<sect-Defensive_Coding-Java-Language-Finalizers>>.)
The first option is the
`try`-`finally` construct, as
shown in <<ex-Defensive_Coding-Java-Language-Finally>>.
The code in the `finally` block should be as short as
possible and should not throw any exceptions.
[[ex-Defensive_Coding-Java-Language-Finally]]
.Resource management with a `try`-`finally` block
====
[source,java]
----
include::example$Java-Finally.adoc[]
----
====
Note that the resource allocation happens
*outside* the `try` block,
and that there is no `null` check in the
`finally` block. (Both are common artifacts
stemming from IDE code templates.)
If the resource object is created freshly and implements the
`java.lang.AutoCloseable` interface, the code
in <<ex-Defensive_Coding-Java-Language-TryWithResource>> can be
used instead. The Java compiler will automatically insert the
`close()` method call in a synthetic
`finally` block.
[[ex-Defensive_Coding-Java-Language-TryWithResource]]
.Resource management using the `try`-with-resource construct
====
[source,java]
----
include::example$Java-TryWithResource.adoc[]
----
====
To be compatible with the `try`-with-resource
construct, new classes should name the resource deallocation
method `close()`, and implement the
`AutoCloseable` interface (the latter breaking
backwards compatibility with Java 6). However, using the
`try`-with-resource construct with objects that
are not freshly allocated is at best awkward, and an explicit
`finally` block is usually the better approach.
In general, it is best to design the programming interface in
such a way that resource deallocation methods like
`close()` cannot throw any (checked or
unchecked) exceptions, but this should not be a reason to ignore
any actual error conditions.
[[sect-Defensive_Coding-Java-Language-Finalizers]]
=== Finalizers
Finalizers can be used a last-resort approach to free resources
which would otherwise leak. Finalization is unpredictable,
costly, and there can be a considerable delay between the last
reference to an object going away and the execution of the
finalizer. Generally, manual resource management is required;
see <<sect-Defensive_Coding-Java-Language-Resources>>.
Finalizers should be very short and should only deallocate
native or other external resources held directly by the object
being finalized. In general, they must use synchronization:
Finalization necessarily happens on a separate thread because it is
inherently concurrent. There can be multiple finalization
threads, and despite each object being finalized at most once,
the finalizer must not assume that it has exclusive access to
the object being finalized (in the `this`
pointer).
Finalizers should not deallocate resources held by other
objects, especially if those objects have finalizers on their
own. In particular, it is a very bad idea to define a finalizer
just to invoke the resource deallocation method of another object,
or overwrite some pointer fields.
Finalizers are not guaranteed to run at all. For instance, the
virtual machine (or the machine underneath) might crash,
preventing their execution.
Objects with finalizers are garbage-collected much later than
objects without them, so using finalizers to zero out key
material (to reduce its undecrypted lifetime in memory) may have
the opposite effect, keeping objects around for much longer and
prevent them from being overwritten in the normal course of
program execution.
For the same reason, code which allocates objects with
finalizers at a high rate will eventually fail (likely with a
`java.lang.OutOfMemoryError` exception) because
the virtual machine has finite resources for keeping track of
objects pending finalization. To deal with that, it may be
necessary to recycle objects with finalizers.
The remarks in this section apply to finalizers which are
implemented by overriding the `finalize()`
method, and to custom finalization using reference queues.
[[sect-Defensive_Coding-Java-Language-Exceptions]]
=== Recovering from Exceptions and Errors
Java exceptions come in three kinds, all ultimately deriving
from `java.lang.Throwable`:
* *Run-time exceptions* do not have to be
declared explicitly and can be explicitly thrown from any
code, by calling code which throws them, or by triggering an
error condition at run time, like division by zero, or an
attempt at an out-of-bounds array access. These exceptions
derive from from the
`java.lang.RuntimeException` class (perhaps
indirectly).
* *Checked exceptions* have to be declared
explicitly by functions that throw or propagate them. They
are similar to run-time exceptions in other regards, except
that there is no language construct to throw them (except
the `throw` statement itself). Checked
exceptions are only present at the Java language level and
are only enforced at compile time. At run time, the virtual
machine does not know about them and permits throwing
exceptions from any code. Checked exceptions must derive
(perhaps indirectly) from the
`java.lang.Exception` class, but not from
`java.lang.RuntimeException`.
* *Errors* are exceptions which typically
reflect serious error conditions. They can be thrown at any
point in the program, and do not have to be declared (unlike
checked exceptions). In general, it is not possible to
recover from such errors; more on that below, in <<sect-Defensive_Coding-Java-Language-Exceptions-Errors>>.
Error classes derive (perhaps indirectly) from
`java.lang.Error`, or from
`java.lang.Throwable`, but not from
`java.lang.Exception`.
The general expectation is that run-time errors are avoided by
careful programming (e.g., not dividing by zero). Checked
exception are expected to be caught as they happen (e.g., when
an input file is unexpectedly missing). Errors are impossible
to predict and can happen at any point and reflect that
something went wrong beyond all expectations.
[[sect-Defensive_Coding-Java-Language-Exceptions-Errors]]
==== The Difficulty of Catching Errors
Errors (that is, exceptions which do not (indirectly) derive
from `java.lang.Exception`), have the
peculiar property that catching them is problematic. There
are several reasons for this:
* The error reflects a failed consistenty check, for example,
`java.lang.AssertionError`.
* The error can happen at any point, resulting in
inconsistencies due to half-updated objects. Examples are
`java.lang.ThreadDeath`,
`java.lang.OutOfMemoryError` and
`java.lang.StackOverflowError`.
* The error indicates that virtual machine failed to provide
some semantic guarantees by the Java programming language.
`java.lang.ExceptionInInitializerError`
is an example—it can leave behind a half-initialized
class.
In general, if an error is thrown, the virtual machine should
be restarted as soon as possible because it is in an
inconsistent state. Continuing running as before can have
unexpected consequences. However, there are legitimate
reasons for catching errors because not doing so leads to even
greater problems.
Code should be written in a way that avoids triggering errors.
See <<sect-Defensive_Coding-Java-Language-ReadArray>>
for an example.
It is usually necessary to log errors. Otherwise, no trace of
the problem might be left anywhere, making it very difficult
to diagnose related failures. Consequently, if you catch
`java.lang.Exception` to log and suppress all
unexpected exceptions (for example, in a request dispatching
loop), you should consider switching to
`java.lang.Throwable` instead, to also cover
errors.
The other reason mainly applies to such request dispatching
loops: If you do not catch errors, the loop stops looping,
resulting in a denial of service.
However, if possible, catching errors should be coupled with a
way to signal the requirement of a virtual machine restart.

141
modules/ROOT/pages/programming-languages/Java-LowLevel.adoc
View file

@ -1,141 +0,0 @@
:experimental:
[[sect-Defensive_Coding-Java-LowLevel]]
== Low-level Features of the Virtual Machine
[[sect-Defensive_Coding-Java-Reflection]]
=== Reflection and Private Parts
The `setAccessible(boolean)` method of the
`java.lang.reflect.AccessibleObject` class
allows a program to disable language-defined access rules for
specific constructors, methods, or fields. Once the access
checks are disabled, any code can use the
`java.lang.reflect.Constructor`,
`java.lang.reflect.Method`, or
`java.lang.reflect.Field` object to access the
underlying Java entity, without further permission checks. This
breaks encapsulation and can undermine the stability of the
virtual machine. (In contrast, without using the
`setAccessible(boolean)` method, this should
not happen because all the language-defined checks still apply.)
This feature should be avoided if possible.
[[sect-Defensive_Coding-Java-JNI]]
=== Java Native Interface (JNI)
The Java Native Interface allows calling from Java code
functions specifically written for this purpose, usually in C or
C++.
The transition between the Java world and the C world is not
fully type-checked, and the C code can easily break the Java
virtual machine semantics. Therefore, extra care is needed when
using this functionality.
To provide a moderate amount of type safety, it is recommended
to recreate the class-specific header file using
[application]*javah* during the build process,
include it in the implementation, and use the
[option]`-Wmissing-declarations` option.
Ideally, the required data is directly passed to static JNI
methods and returned from them, and the code and the C side does
not have to deal with accessing Java fields (or even methods).
When using `GetPrimitiveArrayCritical` or
`GetStringCritical`, make sure that you only
perform very little processing between the get and release
operations. Do not access the file system or the network, and
not perform locking, because that might introduce blocking.
When processing large strings or arrays, consider splitting the
computation into multiple sub-chunks, so that you do not prevent
the JVM from reaching a safepoint for extended periods of time.
If necessary, you can use the Java `long` type
to store a C pointer in a field of a Java class. On the C side,
when casting between the `jlong` value and the
pointer on the C side,
You should not try to perform pointer arithmetic on the Java
side (that is, you should treat pointer-carrying
`long` values as opaque). When passing a slice
of an array to the native code, follow the Java convention and
pass it as the base array, the integer offset of the start of
the slice, and the integer length of the slice. On the native
side, check the offset/length combination against the actual
array length, and use the offset to compute the pointer to the
beginning of the array.
[[ex-Defensive_Coding-Java-JNI-Pointers]]
.Array length checking in JNI code
====
[source,java]
----
include::example$Java-JNI-Pointers.adoc[]
----
====
In any case, classes referring to native resources must be
declared `final`, and must not be serializeable
or clonable. Initialization and mutation of the state used by
the native side must be controlled carefully. Otherwise, it
might be possible to create an object with inconsistent native
state which results in a crash (or worse) when used (or perhaps
only finalized) later. If you need both Java inheritance and
native resources, you should consider moving the native state to
a separate class, and only keep a reference to objects of that
class. This way, cloning and serialization issues can be
avoided in most cases.
If there are native resources associated with an object, the
class should have an explicit resource deallocation method
(<<sect-Defensive_Coding-Java-Language-Resources>>) and a
finalizer (<<sect-Defensive_Coding-Java-Language-Finalizers>>) as a
last resort. The need for finalization means that a minimum
amount of synchronization is needed. Code on the native side
should check that the object is not in a closed/freed state.
Many JNI functions create local references. By default, these
persist until the JNI-implemented method returns. If you create
many such references (e.g., in a loop), you may have to free
them using `DeleteLocalRef`, or start using
`PushLocalFrame` and
`PopLocalFrame`. Global references must be
deallocated with `DeleteGlobalRef`, otherwise
there will be a memory leak, just as with
`malloc` and `free`.
When throwing exceptions using `Throw` or
`ThrowNew`, be aware that these functions
return regularly. You have to return control manually to the
JVM.
Technically, the `JNIEnv` pointer is not
necessarily constant during the lifetime of your JNI module.
Storing it in a global variable is therefore incorrect.
Particularly if you are dealing with callbacks, you may have to
store the pointer in a thread-local variable (defined with
`__thread`). It is, however, best to avoid the
complexity of calling back into Java code.
Keep in mind that C/C++ and Java are different languages,
despite very similar syntax for expressions. The Java memory
model is much more strict than the C or C++ memory models, and
native code needs more synchronization, usually using JVM
facilities or POSIX threads mutexes. Integer overflow in Java
is defined, but in C/C++ it is not (for the
`jint` and `jlong` types).
[[sect-Defensive_Coding-Java-MiscUnsafe]]
=== `sun.misc.Unsafe`
The `sun.misc.Unsafe` class is unportable and
contains many functions explicitly designed to break Java memory
safety (for performance and debugging). If possible, avoid
using this class.

256
modules/ROOT/pages/programming-languages/Java-SecurityManager.adoc
View file

@ -1,256 +0,0 @@
:experimental:
[[sect-Defensive_Coding-Java-SecurityManager]]
== Interacting with the Security Manager
The Java platform is largely implemented in the Java language
itself. Therefore, within the same JVM, code runs which is part
of the Java installation and which is trusted, but there might
also be code which comes from untrusted sources and is restricted
by the Java sandbox (to varying degrees). The *security
manager* draws a line between fully trusted, partially
trusted and untrusted code.
The type safety and accessibility checks provided by the Java
language and JVM would be sufficient to implement a sandbox.
However, only some Java APIs employ such a capabilities-based
approach. (The Java SE library contains many public classes with
public constructors which can break any security policy, such as
`java.io.FileOutputStream`.) Instead, critical
functionality is protected by *stack
inspection*: At a security check, the stack is walked
from top (most-nested) to bottom. The security check fails if a
stack frame for a method is encountered whose class lacks the
permission which the security check requires.
This simple approach would not allow untrusted code (which lacks
certain permissions) to call into trusted code while the latter
retains trust. Such trust transitions are desirable because they
enable Java as an implementation language for most parts of the
Java platform, including security-relevant code. Therefore, there
is a mechanism to mark certain stack frames as trusted (<<sect-Defensive_Coding-Java-SecurityManager-Privileged>>).
In theory, it is possible to run a Java virtual machine with a
security manager that acts very differently from this approach,
but a lot of code expects behavior very close to the platform
default (including many classes which are part of the OpenJDK
implementation).
[[sect-Defensive_Coding-Java-SecurityManager-Compatible]]
=== Security Manager Compatibility
A lot of code can run without any additional permissions at all,
with little changes. The following guidelines should help to
increase compatibility with a restrictive security manager.
* When retrieving system properties using
`System.getProperty(String)` or similar
methods, catch `SecurityException`
exceptions and treat the property as unset.
* Avoid unnecessary file system or network access.
* Avoid explicit class loading. Access to a suitable class
loader might not be available when executing as untrusted
code.
If the functionality you are implementing absolutely requires
privileged access and this functionality has to be used from
untrusted code (hopefully in a restricted and secure manner),
see <<sect-Defensive_Coding-Java-SecurityManager-Privileged>>.
[[sect-Defensive_Coding-Java-SecurityManager-Activate]]
=== Activating the Security Manager
The usual command to launch a Java application,
[command]`java`, does not activate the security manager.
Therefore, the virtual machine does not enforce any sandboxing
restrictions, even if explicitly requested by the code (for
example, as described in <<sect-Defensive_Coding-Java-SecurityManager-Unprivileged>>).
The [option]`-Djava.security.manager` option activates
the security manager, with the fairly restrictive default
policy. With a very permissive policy, most Java code will run
unchanged. Assuming the policy in <<ex-Defensive_Coding-Java-SecurityManager-GrantAll>>
has been saved in a file `grant-all.policy`,
this policy can be activated using the option
[option]`-Djava.security.policy=grant-all.policy` (in
addition to the [option]`-Djava.security.manager`
option).
[[ex-Defensive_Coding-Java-SecurityManager-GrantAll]]
.Most permissve OpenJDK policy file
====
[source,java]
----
grant {
permission java.security.AllPermission;
};
----
====
With this most permissive policy, the security manager is still
active, and explicit requests to drop privileges will be
honored.
[[sect-Defensive_Coding-Java-SecurityManager-Unprivileged]]
=== Reducing Trust in Code
The <<ex-Defensive_Coding-Java-SecurityManager-Unprivileged>> example
shows how to run a piece code of with reduced privileges.
[[ex-Defensive_Coding-Java-SecurityManager-Unprivileged]]
.Using the security manager to run code with reduced privileges
====
[source,java]
----
include::example$Java-SecurityManager-Unprivileged.adoc[]
----
====
The example above does not add any additional permissions to the
`permissions` object. If such permissions are
necessary, code like the following (which grants read permission
on all files in the current directory) can be used:
[source,java]
----
include::example$Java-SecurityManager-CurrentDirectory.adoc[]
----
[IMPORTANT]
====
Calls to the
`java.security.AccessController.doPrivileged()`
methods do not enforce any additional restriction if no
security manager has been set. Except for a few special
exceptions, the restrictions no longer apply if the
`doPrivileged()` has returned, even to
objects created by the code which ran with reduced privileges.
(This applies to object finalization in particular.)
The example code above does not prevent the called code from
calling the
`java.security.AccessController.doPrivileged()`
methods. This mechanism should be considered an additional
safety net, but it still can be used to prevent unexpected
behavior of trusted code. As long as the executed code is not
dynamic and came with the original application or library, the
sandbox is fairly effective.
The `context` argument in <<ex-Defensive_Coding-Java-SecurityManager-Unprivileged>>
is extremely important—otherwise, this code would increase
privileges instead of reducing them.
====
For activating the security manager, see <<sect-Defensive_Coding-Java-SecurityManager-Activate>>.
Unfortunately, this affects the virtual machine as a whole, so
it is not possible to do this from a library.
[[sect-Defensive_Coding-Java-SecurityManager-Privileged]]
=== Re-gaining Privileges
Ordinarily, when trusted code is called from untrusted code, it
loses its privileges (because of the untrusted stack frames
visible to stack inspection). The
`java.security.AccessController.doPrivileged()`
family of methods provides a controlled backdoor from untrusted
to trusted code.
[IMPORTANT]
====
By design, this feature can undermine the Java security model
and the sandbox. It has to be used very carefully. Most
sandbox vulnerabilities can be traced back to its misuse.
====
In essence, the `doPrivileged()` methods
cause the stack inspection to end at their call site. Untrusted
code further down the call stack becomes invisible to security
checks.
The following operations are common and safe to perform with
elevated privileges.
* Reading custom system properties with fixed names,
especially if the value is not propagated to untrusted code.
(File system paths including installation paths, host names
and user names are sometimes considered private information
and need to be protected.)
* Reading from the file system at fixed paths, either
determined at compile time or by a system property. Again,
leaking the file contents to the caller can be problematic.
* Accessing network resources under a fixed address, name or
URL, derived from a system property or configuration file,
information leaks not withstanding.
The <<ex-Defensive_Coding-Java-SecurityManager-Privileged>> example
shows how to request additional privileges.
[[ex-Defensive_Coding-Java-SecurityManager-Privileged]]
.Using the security manager to run code with increased privileges
====
[source,java]
----
include::example$Java-SecurityManager-Privileged.adoc[]
----
====
Obviously, this only works if the class containing the call to
`doPrivileged()` is marked trusted (usually
because it is loaded from a trusted class loader).
When writing code that runs with elevated privileges, make sure
that you follow the rules below.
* Make the privileged code as small as possible. Perform as
many computations as possible before and after the
privileged code section, even if it means that you have to
define a new class to pass the data around.
* Make sure that you either control the inputs to the
privileged code, or that the inputs are harmless and cannot
affect security properties of the privileged code.
* Data that is returned from or written by the privileged code
must either be restricted (that is, it cannot be accessed by
untrusted code), or must be harmless. Otherwise, privacy
leaks or information disclosures which affect security
properties can be the result.
If the code calls back into untrusted code at a later stage (or
performs other actions under control from the untrusted caller),
you must obtain the original security context and restore it
before performing the callback, as in <<ex-Defensive_Coding-Java-SecurityManager-Callback>>.
(In this example, it would be much better to move the callback
invocation out of the privileged code section, of course.)
[[ex-Defensive_Coding-Java-SecurityManager-Callback]]
.Restoring privileges when invoking callbacks
====
[source,java]
----
include::example$Java-SecurityManager-Callback.adoc[]
----
====

649
modules/ROOT/pages/programming-languages/Java.adoc
View file

@ -1,11 +1,650 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Java]]
= The Java Programming Language
include::Java-Language.adoc[]
[[sect-Defensive_Coding-Java-Language]]
== The Core Language
include::Java-LowLevel.adoc[]
Implementations of the Java programming language provide strong
memory safety, even in the presence of data races in concurrent
code. This prevents a large range of security vulnerabilities
from occurring, unless certain low-level features are used; see
<<sect-Defensive_Coding-Java-LowLevel>>.
include::Java-SecurityManager.adoc[]
[[sect-Defensive_Coding-Java-Language-ReadArray]]
=== Increasing Robustness when Reading Arrays
External data formats often include arrays, and the data is
stored as an integer indicating the number of array elements,
followed by this number of elements in the file or protocol data
unit. This length specified can be much larger than what is
actually available in the data source.
To avoid allocating extremely large amounts of data, you can
allocate a small array initially and grow it as you read more
data, implementing an exponential growth policy. See the
`readBytes(InputStream, int)` function in
<<ex-Defensive_Coding-Java-Language-ReadArray>>.
[[ex-Defensive_Coding-Java-Language-ReadArray]]
.Incrementally reading a byte array
====
[source,java]
----
include::example$Java-Language-ReadArray.adoc[]
----
====
When reading data into arrays, hash maps or hash sets, use the
default constructor and do not specify a size hint. You can
simply add the elements to the collection as you read them.
[[sect-Defensive_Coding-Java-Language-Resources]]
=== Resource Management
Unlike C++, Java does not offer destructors which can deallocate
resources in a predictable fashion. All resource management has
to be manual, at the usage site. (Finalizers are generally not
usable for resource management, especially in high-performance
code; see <<sect-Defensive_Coding-Java-Language-Finalizers>>.)
The first option is the
`try`-`finally` construct, as
shown in <<ex-Defensive_Coding-Java-Language-Finally>>.
The code in the `finally` block should be as short as
possible and should not throw any exceptions.
[[ex-Defensive_Coding-Java-Language-Finally]]
.Resource management with a `try`-`finally` block
====
[source,java]
----
include::example$Java-Finally.adoc[]
----
====
Note that the resource allocation happens
*outside* the `try` block,
and that there is no `null` check in the
`finally` block. (Both are common artifacts
stemming from IDE code templates.)
If the resource object is created freshly and implements the
`java.lang.AutoCloseable` interface, the code
in <<ex-Defensive_Coding-Java-Language-TryWithResource>> can be
used instead. The Java compiler will automatically insert the
`close()` method call in a synthetic
`finally` block.
[[ex-Defensive_Coding-Java-Language-TryWithResource]]
.Resource management using the `try`-with-resource construct
====
[source,java]
----
include::example$Java-TryWithResource.adoc[]
----
====
To be compatible with the `try`-with-resource
construct, new classes should name the resource deallocation
method `close()`, and implement the
`AutoCloseable` interface (the latter breaking
backwards compatibility with Java 6). However, using the
`try`-with-resource construct with objects that
are not freshly allocated is at best awkward, and an explicit
`finally` block is usually the better approach.
In general, it is best to design the programming interface in
such a way that resource deallocation methods like
`close()` cannot throw any (checked or
unchecked) exceptions, but this should not be a reason to ignore
any actual error conditions.
[[sect-Defensive_Coding-Java-Language-Finalizers]]
=== Finalizers
Finalizers can be used a last-resort approach to free resources
which would otherwise leak. Finalization is unpredictable,
costly, and there can be a considerable delay between the last
reference to an object going away and the execution of the
finalizer. Generally, manual resource management is required;
see <<sect-Defensive_Coding-Java-Language-Resources>>.
Finalizers should be very short and should only deallocate
native or other external resources held directly by the object
being finalized. In general, they must use synchronization:
Finalization necessarily happens on a separate thread because it is
inherently concurrent. There can be multiple finalization
threads, and despite each object being finalized at most once,
the finalizer must not assume that it has exclusive access to
the object being finalized (in the `this`
pointer).
Finalizers should not deallocate resources held by other
objects, especially if those objects have finalizers on their
own. In particular, it is a very bad idea to define a finalizer
just to invoke the resource deallocation method of another object,
or overwrite some pointer fields.
Finalizers are not guaranteed to run at all. For instance, the
virtual machine (or the machine underneath) might crash,
preventing their execution.
Objects with finalizers are garbage-collected much later than
objects without them, so using finalizers to zero out key
material (to reduce its undecrypted lifetime in memory) may have
the opposite effect, keeping objects around for much longer and
prevent them from being overwritten in the normal course of
program execution.
For the same reason, code which allocates objects with
finalizers at a high rate will eventually fail (likely with a
`java.lang.OutOfMemoryError` exception) because
the virtual machine has finite resources for keeping track of
objects pending finalization. To deal with that, it may be
necessary to recycle objects with finalizers.
The remarks in this section apply to finalizers which are
implemented by overriding the `finalize()`
method, and to custom finalization using reference queues.
[[sect-Defensive_Coding-Java-Language-Exceptions]]
=== Recovering from Exceptions and Errors
Java exceptions come in three kinds, all ultimately deriving
from `java.lang.Throwable`:
* *Run-time exceptions* do not have to be
declared explicitly and can be explicitly thrown from any
code, by calling code which throws them, or by triggering an
error condition at run time, like division by zero, or an
attempt at an out-of-bounds array access. These exceptions
derive from from the
`java.lang.RuntimeException` class (perhaps
indirectly).
* *Checked exceptions* have to be declared
explicitly by functions that throw or propagate them. They
are similar to run-time exceptions in other regards, except
that there is no language construct to throw them (except
the `throw` statement itself). Checked
exceptions are only present at the Java language level and
are only enforced at compile time. At run time, the virtual
machine does not know about them and permits throwing
exceptions from any code. Checked exceptions must derive
(perhaps indirectly) from the
`java.lang.Exception` class, but not from
`java.lang.RuntimeException`.
* *Errors* are exceptions which typically
reflect serious error conditions. They can be thrown at any
point in the program, and do not have to be declared (unlike
checked exceptions). In general, it is not possible to
recover from such errors; more on that below, in <<sect-Defensive_Coding-Java-Language-Exceptions-Errors>>.
Error classes derive (perhaps indirectly) from
`java.lang.Error`, or from
`java.lang.Throwable`, but not from
`java.lang.Exception`.
The general expectation is that run-time errors are avoided by
careful programming (e.g., not dividing by zero). Checked
exception are expected to be caught as they happen (e.g., when
an input file is unexpectedly missing). Errors are impossible
to predict and can happen at any point and reflect that
something went wrong beyond all expectations.
[[sect-Defensive_Coding-Java-Language-Exceptions-Errors]]
==== The Difficulty of Catching Errors
Errors (that is, exceptions which do not (indirectly) derive
from `java.lang.Exception`), have the
peculiar property that catching them is problematic. There
are several reasons for this:
* The error reflects a failed consistenty check, for example,
`java.lang.AssertionError`.
* The error can happen at any point, resulting in
inconsistencies due to half-updated objects. Examples are
`java.lang.ThreadDeath`,
`java.lang.OutOfMemoryError` and
`java.lang.StackOverflowError`.
* The error indicates that virtual machine failed to provide
some semantic guarantees by the Java programming language.
`java.lang.ExceptionInInitializerError`
is an example—it can leave behind a half-initialized
class.
In general, if an error is thrown, the virtual machine should
be restarted as soon as possible because it is in an
inconsistent state. Continuing running as before can have
unexpected consequences. However, there are legitimate
reasons for catching errors because not doing so leads to even
greater problems.
Code should be written in a way that avoids triggering errors.
See <<sect-Defensive_Coding-Java-Language-ReadArray>>
for an example.
It is usually necessary to log errors. Otherwise, no trace of
the problem might be left anywhere, making it very difficult
to diagnose related failures. Consequently, if you catch
`java.lang.Exception` to log and suppress all
unexpected exceptions (for example, in a request dispatching
loop), you should consider switching to
`java.lang.Throwable` instead, to also cover
errors.
The other reason mainly applies to such request dispatching
loops: If you do not catch errors, the loop stops looping,
resulting in a denial of service.
However, if possible, catching errors should be coupled with a
way to signal the requirement of a virtual machine restart.
[[sect-Defensive_Coding-Java-LowLevel]]
== Low-level Features of the Virtual Machine
[[sect-Defensive_Coding-Java-Reflection]]
=== Reflection and Private Parts
The `setAccessible(boolean)` method of the
`java.lang.reflect.AccessibleObject` class
allows a program to disable language-defined access rules for
specific constructors, methods, or fields. Once the access
checks are disabled, any code can use the
`java.lang.reflect.Constructor`,
`java.lang.reflect.Method`, or
`java.lang.reflect.Field` object to access the
underlying Java entity, without further permission checks. This
breaks encapsulation and can undermine the stability of the
virtual machine. (In contrast, without using the
`setAccessible(boolean)` method, this should
not happen because all the language-defined checks still apply.)
This feature should be avoided if possible.
[[sect-Defensive_Coding-Java-JNI]]
=== Java Native Interface (JNI)
The Java Native Interface allows calling from Java code
functions specifically written for this purpose, usually in C or
C++.
The transition between the Java world and the C world is not
fully type-checked, and the C code can easily break the Java
virtual machine semantics. Therefore, extra care is needed when
using this functionality.
To provide a moderate amount of type safety, it is recommended
to recreate the class-specific header file using
[application]*javah* during the build process,
include it in the implementation, and use the
[option]`-Wmissing-declarations` option.
Ideally, the required data is directly passed to static JNI
methods and returned from them, and the code and the C side does
not have to deal with accessing Java fields (or even methods).
When using `GetPrimitiveArrayCritical` or
`GetStringCritical`, make sure that you only
perform very little processing between the get and release
operations. Do not access the file system or the network, and
not perform locking, because that might introduce blocking.
When processing large strings or arrays, consider splitting the
computation into multiple sub-chunks, so that you do not prevent
the JVM from reaching a safepoint for extended periods of time.
If necessary, you can use the Java `long` type
to store a C pointer in a field of a Java class. On the C side,
when casting between the `jlong` value and the
pointer on the C side,
You should not try to perform pointer arithmetic on the Java
side (that is, you should treat pointer-carrying
`long` values as opaque). When passing a slice
of an array to the native code, follow the Java convention and
pass it as the base array, the integer offset of the start of
the slice, and the integer length of the slice. On the native
side, check the offset/length combination against the actual
array length, and use the offset to compute the pointer to the
beginning of the array.
[[ex-Defensive_Coding-Java-JNI-Pointers]]
.Array length checking in JNI code
====
[source,java]
----
include::example$Java-JNI-Pointers.adoc[]
----
====
In any case, classes referring to native resources must be
declared `final`, and must not be serializeable
or clonable. Initialization and mutation of the state used by
the native side must be controlled carefully. Otherwise, it
might be possible to create an object with inconsistent native
state which results in a crash (or worse) when used (or perhaps
only finalized) later. If you need both Java inheritance and
native resources, you should consider moving the native state to
a separate class, and only keep a reference to objects of that
class. This way, cloning and serialization issues can be
avoided in most cases.
If there are native resources associated with an object, the
class should have an explicit resource deallocation method
(<<sect-Defensive_Coding-Java-Language-Resources>>) and a
finalizer (<<sect-Defensive_Coding-Java-Language-Finalizers>>) as a
last resort. The need for finalization means that a minimum
amount of synchronization is needed. Code on the native side
should check that the object is not in a closed/freed state.
Many JNI functions create local references. By default, these
persist until the JNI-implemented method returns. If you create
many such references (e.g., in a loop), you may have to free
them using `DeleteLocalRef`, or start using
`PushLocalFrame` and
`PopLocalFrame`. Global references must be
deallocated with `DeleteGlobalRef`, otherwise
there will be a memory leak, just as with
`malloc` and `free`.
When throwing exceptions using `Throw` or
`ThrowNew`, be aware that these functions
return regularly. You have to return control manually to the
JVM.
Technically, the `JNIEnv` pointer is not
necessarily constant during the lifetime of your JNI module.
Storing it in a global variable is therefore incorrect.
Particularly if you are dealing with callbacks, you may have to
store the pointer in a thread-local variable (defined with
`__thread`). It is, however, best to avoid the
complexity of calling back into Java code.
Keep in mind that C/C++ and Java are different languages,
despite very similar syntax for expressions. The Java memory
model is much more strict than the C or C++ memory models, and
native code needs more synchronization, usually using JVM
facilities or POSIX threads mutexes. Integer overflow in Java
is defined, but in C/C++ it is not (for the
`jint` and `jlong` types).
[[sect-Defensive_Coding-Java-MiscUnsafe]]
=== `sun.misc.Unsafe`
The `sun.misc.Unsafe` class is unportable and
contains many functions explicitly designed to break Java memory
safety (for performance and debugging). If possible, avoid
using this class.
[[sect-Defensive_Coding-Java-SecurityManager]]
== Interacting with the Security Manager
The Java platform is largely implemented in the Java language
itself. Therefore, within the same JVM, code runs which is part
of the Java installation and which is trusted, but there might
also be code which comes from untrusted sources and is restricted
by the Java sandbox (to varying degrees). The *security
manager* draws a line between fully trusted, partially
trusted and untrusted code.
The type safety and accessibility checks provided by the Java
language and JVM would be sufficient to implement a sandbox.
However, only some Java APIs employ such a capabilities-based
approach. (The Java SE library contains many public classes with
public constructors which can break any security policy, such as
`java.io.FileOutputStream`.) Instead, critical
functionality is protected by *stack
inspection*: At a security check, the stack is walked
from top (most-nested) to bottom. The security check fails if a
stack frame for a method is encountered whose class lacks the
permission which the security check requires.
This simple approach would not allow untrusted code (which lacks
certain permissions) to call into trusted code while the latter
retains trust. Such trust transitions are desirable because they
enable Java as an implementation language for most parts of the
Java platform, including security-relevant code. Therefore, there
is a mechanism to mark certain stack frames as trusted (<<sect-Defensive_Coding-Java-SecurityManager-Privileged>>).
In theory, it is possible to run a Java virtual machine with a
security manager that acts very differently from this approach,
but a lot of code expects behavior very close to the platform
default (including many classes which are part of the OpenJDK
implementation).
[[sect-Defensive_Coding-Java-SecurityManager-Compatible]]
=== Security Manager Compatibility
A lot of code can run without any additional permissions at all,
with little changes. The following guidelines should help to
increase compatibility with a restrictive security manager.
* When retrieving system properties using
`System.getProperty(String)` or similar
methods, catch `SecurityException`
exceptions and treat the property as unset.
* Avoid unnecessary file system or network access.
* Avoid explicit class loading. Access to a suitable class
loader might not be available when executing as untrusted
code.
If the functionality you are implementing absolutely requires
privileged access and this functionality has to be used from
untrusted code (hopefully in a restricted and secure manner),
see <<sect-Defensive_Coding-Java-SecurityManager-Privileged>>.
[[sect-Defensive_Coding-Java-SecurityManager-Activate]]
=== Activating the Security Manager
The usual command to launch a Java application,
[command]`java`, does not activate the security manager.
Therefore, the virtual machine does not enforce any sandboxing
restrictions, even if explicitly requested by the code (for
example, as described in <<sect-Defensive_Coding-Java-SecurityManager-Unprivileged>>).
The [option]`-Djava.security.manager` option activates
the security manager, with the fairly restrictive default
policy. With a very permissive policy, most Java code will run
unchanged. Assuming the policy in <<ex-Defensive_Coding-Java-SecurityManager-GrantAll>>
has been saved in a file `grant-all.policy`,
this policy can be activated using the option
[option]`-Djava.security.policy=grant-all.policy` (in
addition to the [option]`-Djava.security.manager`
option).
[[ex-Defensive_Coding-Java-SecurityManager-GrantAll]]
.Most permissve OpenJDK policy file
====
[source,java]
----
grant {
permission java.security.AllPermission;
};
----
====
With this most permissive policy, the security manager is still
active, and explicit requests to drop privileges will be
honored.
[[sect-Defensive_Coding-Java-SecurityManager-Unprivileged]]
=== Reducing Trust in Code
The <<ex-Defensive_Coding-Java-SecurityManager-Unprivileged>> example
shows how to run a piece code of with reduced privileges.
[[ex-Defensive_Coding-Java-SecurityManager-Unprivileged]]
.Using the security manager to run code with reduced privileges
====
[source,java]
----
include::example$Java-SecurityManager-Unprivileged.adoc[]
----
====
The example above does not add any additional permissions to the
`permissions` object. If such permissions are
necessary, code like the following (which grants read permission
on all files in the current directory) can be used:
[source,java]
----
include::example$Java-SecurityManager-CurrentDirectory.adoc[]
----
[IMPORTANT]
====
Calls to the
`java.security.AccessController.doPrivileged()`
methods do not enforce any additional restriction if no
security manager has been set. Except for a few special
exceptions, the restrictions no longer apply if the
`doPrivileged()` has returned, even to
objects created by the code which ran with reduced privileges.
(This applies to object finalization in particular.)
The example code above does not prevent the called code from
calling the
`java.security.AccessController.doPrivileged()`
methods. This mechanism should be considered an additional
safety net, but it still can be used to prevent unexpected
behavior of trusted code. As long as the executed code is not
dynamic and came with the original application or library, the
sandbox is fairly effective.
The `context` argument in <<ex-Defensive_Coding-Java-SecurityManager-Unprivileged>>
is extremely important—otherwise, this code would increase
privileges instead of reducing them.
====
For activating the security manager, see <<sect-Defensive_Coding-Java-SecurityManager-Activate>>.
Unfortunately, this affects the virtual machine as a whole, so
it is not possible to do this from a library.
[[sect-Defensive_Coding-Java-SecurityManager-Privileged]]
=== Re-gaining Privileges
Ordinarily, when trusted code is called from untrusted code, it
loses its privileges (because of the untrusted stack frames
visible to stack inspection). The
`java.security.AccessController.doPrivileged()`
family of methods provides a controlled backdoor from untrusted
to trusted code.
[IMPORTANT]
====
By design, this feature can undermine the Java security model
and the sandbox. It has to be used very carefully. Most
sandbox vulnerabilities can be traced back to its misuse.
====
In essence, the `doPrivileged()` methods
cause the stack inspection to end at their call site. Untrusted
code further down the call stack becomes invisible to security
checks.
The following operations are common and safe to perform with
elevated privileges.
* Reading custom system properties with fixed names,
especially if the value is not propagated to untrusted code.
(File system paths including installation paths, host names
and user names are sometimes considered private information
and need to be protected.)
* Reading from the file system at fixed paths, either
determined at compile time or by a system property. Again,
leaking the file contents to the caller can be problematic.
* Accessing network resources under a fixed address, name or
URL, derived from a system property or configuration file,
information leaks not withstanding.
The <<ex-Defensive_Coding-Java-SecurityManager-Privileged>> example
shows how to request additional privileges.
[[ex-Defensive_Coding-Java-SecurityManager-Privileged]]
.Using the security manager to run code with increased privileges
====
[source,java]
----
include::example$Java-SecurityManager-Privileged.adoc[]
----
====
Obviously, this only works if the class containing the call to
`doPrivileged()` is marked trusted (usually
because it is loaded from a trusted class loader).
When writing code that runs with elevated privileges, make sure
that you follow the rules below.
* Make the privileged code as small as possible. Perform as
many computations as possible before and after the
privileged code section, even if it means that you have to
define a new class to pass the data around.
* Make sure that you either control the inputs to the
privileged code, or that the inputs are harmless and cannot
affect security properties of the privileged code.
* Data that is returned from or written by the privileged code
must either be restricted (that is, it cannot be accessed by
untrusted code), or must be harmless. Otherwise, privacy
leaks or information disclosures which affect security
properties can be the result.
If the code calls back into untrusted code at a later stage (or
performs other actions under control from the untrusted caller),
you must obtain the original security context and restore it
before performing the callback, as in <<ex-Defensive_Coding-Java-SecurityManager-Callback>>.
(In this example, it would be much better to move the callback
invocation out of the privileged code section, of course.)
[[ex-Defensive_Coding-Java-SecurityManager-Callback]]
.Restoring privileges when invoking callbacks
====
[source,java]
----
include::example$Java-SecurityManager-Callback.adoc[]
----
====

16
modules/ROOT/pages/programming-languages/Python.adoc
View file

@ -1,8 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Python]]
= The Python Programming Language
Python provides memory safety by default, so low-level security
@ -11,21 +11,21 @@ interpreter or standard library itself.
Other sections with Python-specific advice include:
* xref:../tasks/Tasks-Temporary_Files.adoc#chap-Defensive_Coding-Tasks-Temporary_Files[Dealing with temp files]
* xref:tasks/Tasks-Temporary_Files.adoc#chap-Defensive_Coding-Tasks-Temporary_Files[Dealing with temp files]
* xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-Creation[Creating Safe Processes]
* xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-Creation[Creating Safe Processes]
* xref:../tasks/Tasks-Serialization.adoc#chap-Defensive_Coding-Tasks-Serialization[Serialization and Deserialization], in
particular xref:../tasks/Tasks-Serialization.adoc#sect-Defensive_Coding-Tasks-Serialization-Library[Library Support for Deserialization]
* xref:tasks/Tasks-Serialization.adoc#chap-Defensive_Coding-Tasks-Serialization[Serialization and Deserialization], in
particular xref:tasks/Tasks-Serialization.adoc#sect-Defensive_Coding-Tasks-Serialization-Library[Library Support for Deserialization]
* xref:../tasks/Tasks-Cryptography.adoc#sect-Defensive_Coding-Tasks-Cryptography-Randomness[Randomness]
* xref:tasks/Tasks-Cryptography.adoc#sect-Defensive_Coding-Tasks-Cryptography-Randomness[Randomness]
== Dangerous Standard Library Features
Some areas of the standard library, notably the
`ctypes` module, do not provide memory safety
guarantees comparable to the rest of Python. If such
functionality is used, the advice in xref:../programming-languages/C.adoc#chap-Defensive_Coding-C[Defensive Coding in C] should be followed.
functionality is used, the advice in xref:programming-languages/C.adoc#chap-Defensive_Coding-C[Defensive Coding in C] should be followed.
== Run-time Compilation and Code Generation

7
modules/ROOT/pages/programming-languages/Shell.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Shell]]
= Shell Programming and [application]*bash*
include::partial$entities.adoc[]
@ -296,7 +297,7 @@ it is not possible for a process to tell if the process feeding
data (or the process consuming data) exited normally or with
an error.
See <<sect-Defensive_Coding-Tasks-Processes-Creation>>
See xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-Creation[Creating Safe Processes]
for additional details on creating child processes.
[[sect-Defensive_Coding-Shell-Temporary_Files]]

13
modules/ROOT/pages/programming-languages/Vala.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Vala]]
= The Vala Programming Language
Vala is a programming language mainly targeted at GNOME developers.
@ -11,12 +12,12 @@ unlike C# and Java, Vala does not attempt to provide memory safety:
Vala is compiled to C, and the C code is compiled with GCC using
typical compiler flags. Basic operations like integer arithmetic
are directly mapped to C constructs. As a results, the
recommendations in xref:../programming-languages/C.adoc#chap-Defensive_Coding-C[Defensive Coding in C] apply.
recommendations in xref:programming-languages/C.adoc#chap-Defensive_Coding-C[Defensive Coding in C] apply.
In particular, the following Vala language constructs can result in
undefined behavior at run time:
* Integer arithmetic, as described in xref:../programming-languages/C-Language.adoc#sect-Defensive_Coding-C-Arithmetic[Recommendations for Integer Arithmetic].
* Integer arithmetic, as described in xref:programming-languages/C.adoc#sect-Defensive_Coding-C-Arithmetic[Recommendations for Integer Arithmetic].
* Pointer arithmetic, string subscripting and the
`substring` method on strings (the
@ -26,11 +27,11 @@ is the responsibility of the calling code to ensure that the
arguments being passed are valid. This applies even to cases
(like `substring`) where the implementation
would have range information to check the validity of indexes.
See xref:../programming-languages/C-Language.adoc#sect-Defensive_Coding-C-Pointers[Recommendations for Pointers and Array Handling]
See xref:programming-languages/C.adoc#sect-Defensive_Coding-C-Pointers[Recommendations for Pointers and Array Handling]
* Similarly, Vala only performs garbage collection (through
reference counting) for `GObject` values. For
plain C pointers (such as strings), the programmer has to ensure
that storage is deallocated once it is no longer needed (to
avoid memory leaks), and that storage is not being deallocated
while it is still being used (see xref:../programming-languages/C-Allocators.adoc#sect-Defensive_Coding-C-Use-After-Free[Use-after-free errors]).
while it is still being used (see xref:programming-languages/C.adoc#sect-Defensive_Coding-C-Use-After-Free[Use-after-free errors]).

7
modules/ROOT/pages/tasks/Tasks-Cryptography.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Tasks-Cryptography]]
= Cryptography
== Primitives
@ -50,7 +51,7 @@ but strongly discouraged
These primitives are difficult to use in a secure way. Custom
implementation of security protocols should be avoided. For
protecting confidentiality and integrity of network
transmissions, TLS should be used (xref:../features/Features-TLS.adoc#chap-Defensive_Coding-TLS[Transport Layer Security]).
transmissions, TLS should be used (xref:features/Features-TLS.adoc#chap-Defensive_Coding-TLS[Transport Layer Security]).
In particular, when using AES in CBC mode, it is necessary to
add integrity checking by other means, preferably using

5
modules/ROOT/pages/tasks/Tasks-Descriptors.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[sect-Defensive_Coding-Tasks-Descriptors]]
= File Descriptor Management
File descriptors underlie all input/output mechanisms offered by

6
modules/ROOT/pages/tasks/Tasks-File_System.adoc
View file

@ -1,15 +1,15 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Tasks-File_System]]
= File System Manipulation
In this chapter, we discuss general file system manipulation, with
a focus on access files and directories to which an other,
potentially untrusted user has write access.
Temporary files are covered in their own chapter, xref:../tasks/Tasks-Temporary_Files.adoc#chap-Defensive_Coding-Tasks-Temporary_Files[Temporary Files].
Temporary files are covered in their own chapter, xref:tasks/Tasks-Temporary_Files.adoc#chap-Defensive_Coding-Tasks-Temporary_Files[Temporary Files].
[[sect-Defensive_Coding-Tasks-File_System-Unowned]]
== Working with Files and Directories Owned by Other Users

7
modules/ROOT/pages/tasks/Tasks-Library_Design.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Tasks-Library_Design]]
= Library Design
Through this section, the term *client code*
@ -126,7 +127,7 @@ Several attributes are global and affect all code in the
process, not just the library that manipulates them.
* environment variables
(see xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-secure_getenv[Accessing Environment Variables])
(see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-secure_getenv[Accessing Environment Variables])
* umask

5
modules/ROOT/pages/tasks/Tasks-Packaging.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Tasks-Packaging]]
= RPM Packaging
This chapter deals with security-related concerns around RPM

8
modules/ROOT/pages/tasks/Tasks-Processes.adoc
View file

@ -1,16 +1,16 @@
:experimental:
:toc:
[[sect-Defensive_Coding-Tasks-Processes]]
= Processes
include::partial$entities.adoc[]
= Processes
[[sect-Defensive_Coding-Tasks-Processes-Creation]]
== Creating Safe Processes
This section describes how to create new child processes in a
safe manner. In addition to the concerns addressed below, there
is the possibility of file descriptor leaks, see xref:../tasks/Tasks-Descriptors.adoc#sect-Defensive_Coding-Tasks-Descriptors-Child_Processes[Preventing File Descriptor Leaks to Child Processes].
is the possibility of file descriptor leaks, see xref:tasks/Tasks-Descriptors.adoc#sect-Defensive_Coding-Tasks-Descriptors-Child_Processes[Preventing File Descriptor Leaks to Child Processes].
=== Obtaining the Program Path and the Command-line Template

11
modules/ROOT/pages/tasks/Tasks-Serialization.adoc
View file

@ -1,7 +1,8 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Tasks-Serialization]]
= Serialization and Deserialization
Protocol decoders and file format parsers are often the
@ -13,7 +14,7 @@ robustly in languages which are not memory-safe.
[[sect-Defensive_Coding-Tasks-Serialization-Decoders]]
== Recommendations for Manually-written Decoders
For C and C++, the advice in xref:../programming-languages/C-Language.adoc#sect-Defensive_Coding-C-Pointers[Recommendations for Pointers and Array Handling] applies. In
For C and C++, the advice in xref:programming-languages/C.adoc#sect-Defensive_Coding-C-Pointers[Recommendations for Pointers and Array Handling] applies. In
addition, avoid non-character pointers directly into input
buffers. Pointer misalignment causes crashes on some
architectures.
@ -107,7 +108,7 @@ reject fragments which are inconsistent.
fragments against the unfragmented PDU length (if they are
present). Check that the last byte in the fragment does not
lie after the end of the unfragmented PDU. Avoid integer
overflows in these computations (see xref:../programming-languages/C-Language.adoc#sect-Defensive_Coding-C-Arithmetic[Recommendations for Integer Arithmetic]).
overflows in these computations (see xref:programming-languages/C.adoc#sect-Defensive_Coding-C-Arithmetic[Recommendations for Integer Arithmetic]).
[[sect-Defensive_Coding-Tasks-Serialization-Fragmentation-ID]]
=== Fragment IDs
@ -140,7 +141,7 @@ Protocol.)
== Library Support for Deserialization
There are too many subtleties when dealing with Deserialization to be discussed here.
A more detailed and updated guide is available as https://cheatsheetseries.owasp.org/cheatsheets/Deserialization_Cheat_Sheet.html[OWASP Deserialization Cheat Sheet]
A more detailed and updated guide is available as https://cheatsheetseries.owasp.org/cheatsheets/Deserialization_Cheat_Sheet.html[OWASP Deserialization Cheat Sheet].
[[sect-Defensive_Coding-Tasks-Serialization-XML]]
== XML Serialization

11
modules/ROOT/pages/tasks/Tasks-Temporary_Files.adoc
View file

@ -1,20 +1,21 @@
:experimental:
:toc:
include::partial$entities.adoc[]
[[chap-Defensive_Coding-Tasks-Temporary_Files]]
= Temporary Files
In this chapter, we describe how to create temporary files and
directories, how to remove them, and how to work with programs
which do not create files in ways that are safe with a shared
directory for temporary files. General file system manipulation
is treated in a separate chapter, xref:../tasks/Tasks-File_System.adoc#chap-Defensive_Coding-Tasks-File_System[File System Manipulation].
is treated in a separate chapter, xref:tasks/Tasks-File_System.adoc#chap-Defensive_Coding-Tasks-File_System[File System Manipulation].
Secure creation of temporary files has four different aspects.
* The location of the directory for temporary files must be
obtained in a secure manner (that is, untrusted environment
variables must be ignored, see xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-secure_getenv[Accessing Environment Variables).
variables must be ignored, see xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-secure_getenv[Accessing Environment Variables).
* A new file must be created. Reusing an existing file must be
avoided (the `/tmp` race
@ -151,7 +152,7 @@ the [option]`-rf` and [option]`--` options.
There are two ways to make a function or program which excepts a
file name safe for use with temporary files. See
xref:../tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-Creation[Creating Safe Processes]
xref:tasks/Tasks-Processes.adoc#sect-Defensive_Coding-Tasks-Processes-Creation[Creating Safe Processes]
for details on subprocess creation.
* Create a temporary directory and place the file there. If