TAOSSA Chapter 7

Ch 7 - Program building blocks

Useful to study recurring code patterns, focusing on areas where developers might make security-relevant mistakes

Auditing variable use

Different techniques for recognising variable and data structure misuse

Variable relationships

The more variables used to represent state, the higher the chances of error
Search for variables that are related to each other, determine their intended relationships, and then determine whether there’s a way to desynchronize these variables from each other
This usually means finding a block of code that alters one variable in a fashion inconsistent with the other variables
Go through the code quickly (in a function) and identify variable relationships, then make one pass to see whether any vars can be desynchronised
Well-designed application keeps variable relationships to a minimum
Data hiding - concealing complex relationships in separate subsystems so that the internals aren’t exposed to callers
Data hiding can make your job harder by spreading complex relationships across multiple files and functions
Examples of data hiding include private variables in a C++ class and the buffer management subsystem in OpenSSH

Structure and object mismanagement

Applications often use large structures to manage program and session state, and group related data elements
Familiarise yourself with the interfaces to learn the purpose of objects and their constituent members
One goal of auditing object-oriented code is to determine whether it’s possible to desynchronise related structure members or leave them in an unexpected or inconsistent state to cause the application to perform some sort of unanticipated operation
Structure mismanagement bugs tend to be quite subtle - the code to manage structures is spread out into several small functions that are individually quite simple. Therefore, any vulnerabilities tend to be a result of aggregate, emergent behaviour occurring across multiple functions
One major problem area in this structure management code is low-level language issues, such as type conversion, negative values, arithmetic boundaries, and pointer arithmetic. The reason is that management code tends to perform a lot of length calculations and comparisons
Similarly to structures, objects can be left in an inconsistent state
Potential for subtle vulnerabilities caused by incorrect assumptions of implicit member functions, e.g. overloaded operators

Variable initialisation

Reading a value from a variable before it is initialised. Two cases:

• Variable was intended to be initialised at the beginning of the function but the developer forgot to specify an initialiser in the declaration
• A code path exists where the variable is accidentally used without ever being initialised

Most vulnerabilities of this nature occur when a function takes an abnormal code path
Functions that allocate a number of variables commonly have an epilogue that cleans up objects to avoid memory leaks when an error occurs. If these vars have not been allocated, this is potentially exploitable
In C++ code, pay close attention to member variables in objects - unexpected code paths can leave objects in an inconsistent or partially uninitialised state
The best way to begin examining this code is by looking at constructor functions to see whether any constructors neglect to initialise certain elements of the object
Destructors are automatically called during the function epilogue for objects declared in the function, similar to the case of vars freed in an epilogue above

Arithmetic boundaries

Structured process for identifying these vulnerabilities (see Ch 6 for details):

1. Discover operations that, if a boundary condition could be triggered, would have security-related consequences (primarily length-based calculations and comparisons)
2. Determine a set of values for each operand that trigger the relevant arithmetic boundary wrap
3. Determine whether this code path can be reached with values within the set determined in step 2

For step 3 (is this what solvers can be used for?):

• Identify the data type of the variable involved
• Determine at which points the variable is assigned a value
• Determine constraints on the variable from assignment until the vulnerable operation
• Determine supporting code path constraints

Type confusion

Union data types are used when structures or objects are required to represent multiple data types depending on an external condition, e.g. representing different opaque objects read off the network
Occasionally, application developers confuse what the data in a union represents. This can have disastrous consequences on an application, particularly when integer data types are confused with pointer data types, or complex structures of one type are confused with another
Most vulnerabilities of this nature stem from misinterpreting a variable used to define what kind of data the structure contains

Lists and tables

Errors in implementing routines that add and modify these data structures, leading to inconsistencies in these data structures
Points to address with examining the algorithm:

• Does the algorithm deal correctly with manipulating list elements when the list is empty?
• What are the implications of duplicate elements?
• Do previous and next pointers always get updated correctly?
• Are data ranges accounted for correctly?

Empty lists: often list structure members or global variables are used to point to the head of a list and potentially the tail of the list. Look for mistakes in updating these variables. Code that doesn’t deal with head and tail elements correctly isn’t common, but it can occur, particularly when list management is decentralised (that is, there’s no clean interface for list management, so management happens haphazardly at different points in the code)
Duplicate elements: elements containing identical keys (data values used to characterise the structure as unique) could cause the two elements to get confused, resulting in the wrong element being selected from the list
Previous and next pointer updates: Often happens if the program treats the current member as the head or tail of a list
Data ranges: in ordered lists, the elements are sorted into some type of order based on a data member that distinguishes each list element. Often each data element in the list represents a range of values
Nuances with this:

• Can overlapping data ranges be supplied?
• Can replacement data ranges (duplicate elements) be supplied?
• Does old or new data take precedence?
• What happens when 0 length data ranges are supplied?

Hashing algorithms: hash tables often implemented as an array of linked lists. They use the list element as input to a hash function. The resulting hash value is used as an index to an array
Important questions:

• Is the hashing algorithms susceptible to invalid results? E.g. when algorithm uses modulus, force it to return negative results (negative dividend). Or force to have many collisions
• What are the implications of invalidating elements? Some algorithms prune elements based on conditions. Potentially incorrect unlinking

Auditing control flow

Internal control flow; loops and branches

Looping constructs

Data processing loops - interpret user-supplied data and construct output based on this data
Common errors:

• The terminating conditions don’t account for destination buffer sizes or don’t correctly account for destination sizes in some cases
• The loop is post-test when it should be pretest
• A break or continue statement is missing or incorrectly placed
• Some misplaced punctuation causes the loop to not do what it’s supposed to

Terminating Conditions

Some loops have multiple terminating conditions when processing user data
The set of terminating conditions in a loop might not adequately account for all possible error conditions, or the implementation of the checks is incorrect
Main problems when calculating lengths:

• The loops fail to account for a buffer’s size
• A size check is made, but it’s incorrect

When you read complex functions containing nested loops, these types of suspect loop constructs can be difficult to spot
With size checks off-by-one errors are common, in string processing
Occasionally, when loops terminate in an unexpected fashion, variables can be left in an inconsistent state
Another off-by-one error occurs when a variable is incorrectly checked to ensure that it’s in certain boundaries before it’s incremented and used
Loops that can write multiple data elements in a single iteration might also be vulnerable to incorrect size checks, e.g. because of character escaping or expansion that weren’t adequately taken into account by the loop’s size checking
A loop’s size check could be invalid because of a type conversion, an arithmetic boundary condition, operator misuse, or pointer arithmetic error

Post-test vs pretest loops

Pretest loops tend to be used primarily; post-test loops are used in some situations out of necessity or for personal preference
Post-test loops should be used when the body of the loop always needs to be performed at least one time. Look for potential situations where execution of the loop body can lead to an unexpected condition. One thing to look out for is the conditional form of the loop performing a sanity check that should be done before the loop is entered
With pre-test loops - if code following a loop expects that the loop body has run at least once, an attacker might be able to intentionally skip the loop entirely and create an exploitable condition

Punctuation errors

E.g. semicolon at the end of the line with the for loop - empty loop
See chapter 6 as well

Flow transfer statements

Dual use of break in C (loops/switch) can be confusing
Developers might assume that a break statement can break out of any nested block and use it in an incorrect place
Or they might assume the statement breaks out of all surrounding loops instead of just the most immediate loop
Another problem is using a continue statement inside a switch statement to restart the switch comparison

Switch statements

A common pitfall that developers fall into when using switch statements is to forget the break statement at the end of each case clause
When the break statement is left out on purpose, programmers often leave a comment (such as /* FALLTHROUGH */ for lint) indicating that the omission of the break statement is intentional
Check if there are any unaccounted for case

Auditing functions

What program state changes because of that call? What things can possibly go wrong with that function? What role do arguments play in how that function operates?
Focus on arguments and aspects of the function that users can influence in some way
Four main vulnerability types:

• Return values are misinterpreted or ignored.
• Arguments supplied are incorrectly formatted in some way.
• Arguments get updated in an unexpected fashion.
• Some unexpected global program state change occurs because of the function call.

Function audit logs

Create a per-function log - purpose and side effects; return values type and meaning, conditions that cause errors, erroneous return values

Return value testing and interpretation

If a return value is misinterpreted or simply ignored, the program might take incorrect code paths as a result, which can have severe security implications

Ignoring return values

Ignoring a return value could cause an error condition to go undetected
Often programmers forget to test malloc or realloc return value for failure
Realloc failures may be exploitable
Other memory allocation functions, especially if they involve copying data
Note where the return value (for functions where it indicates success or failure) is not tested
Note the error conditions returned by the function
Effects of ignoring return value depend on the structure of the caller

Mistinterpreting return values

A return value could be misinterpreted in two ways: a programmer might simply misunderstand the meaning of the return value, or the return value might be involved in a type conversion that causes its intended meaning to change
First one often happens when a team of programmers is developing an application and using third-party code and libraries
Example: on UNIX snprintf returns typically returns how many bytes it would have written to the destination, had there been enough room
Systematic approach when finding misinterpreted values:

1. Determine the intended meaning of the return value for the function. If code is documented, verifying that the function returns what the documenter says it does is still important.
2. Look at each location in the application where the function is called and see what it does with the return value. Is it consistent with that return value’s intended meaning?

Occasionally, the fault of a misinterpreted return value isn’t with the calling function, but with the called function
Finding these cases:

1. Determine all the points in a function where it might return. Usually there are multiple points where it might return because of errors and one point at which it returns because of successful completion.
2. Examine the value being returned. Is it within the range of expected return values? Is it appropriate for indicating the condition that caused the function to return?

The second type of misinterpretation (type conversion) is an extension of the first. Determine what type conversions occur when a the return value is tested (conversion rules?) or stored (target variable type?)

Function side-effects

A function that does not generate any side-effects is referentially transparent - that is, the function call can be replaced directly with the return value. A function that causes side-effects isreferentially opaque
Interesting are the specific function side effects: manipulating arguments passed by reference (value-result arguments) and manipulating globally scoped variables
One common situation is when realloc() is used to resize a buffer passed as a pointer argument. Then the calling function has a pointer that was not updated after a call to realloc(), or the new allocation size is incorrect because of a length miscalculation
“Outdated pointer” bugs are often spread out b/w several functions Make note of security-relevant functions that manipulate pass-by-reference arguments, as well as the specific manner in which they perform this manipulation.
These kinds of argument manipulations often use opaque pointers with an associated set of manipulation functions.
This type of manipulation is also an inherent part of C++ classes, as they implicitly pass a reference to the this pointer. However, C++ member functions can be harder to review due to the number of implicit functions that may be called and the fact that the code paths do not follow a more direct procedural structure.
Determining risk of pass-by-reference manipulation:

1. Find all locations in a function where pass-by-reference arguments are modified, particularly structure arguments.
2. Differentiate between mandatory modification and optional modification. Mandatory modification occurs every time the function is called; optional modification occurs when an abnormal situation arises. Programmers are more likely to overlook exceptional conditions related to optional modification.
3. Examine how calling functions use the modified arguments after the function has returned.

Also, note when arguments aren’t updated when they should be. Pay close attention to what happens when functions return early because of some error: Are arguments that should be updated not updated for some reason?
Auditing functions that modify global variables is similar but the vulnerabilities introduced might be more subtle. Especially for the code that can run at any point in the program, e.g. exception handler or signal handler
In object-oriented programs, it can be much harder to determine whether global variables are susceptible to misuse because of unexpected modification. The difficulty arises because the order of execution of constituent member functions often isn’t clear.

Argument meaning

When auditing a function for vulnerabilities related to incorrect arguments being supplied, the process is as follows:

1. List the type and intended meaning of each argument to a function.
2. Examine all the calling functions to determine whether type conversions or incorrect arguments could be supplied.

Check for type conversions. They may become an issue if the interoperation of the argument can change based on the sign change
MultiByteToWideChar() - length is misinterpreted: destination buffer in wide chars, not in bytes. Confusing the two sizes (e.g. by specifying sizeof(buf)) leads to an overflow.
The more difficult the function is to figure out, the more likely it is that it will be used incorrectly
You should be able to answer any questions about a functions quirks and log the answers so that the information is easily accessible later.
Be especially mindful of type conversions that happen with arguments, such as truncation when dealing with short integers, because they are susceptible to boundary issues

Auditing memory management

Allocation-check-copy logs

Recording variations in allocation sizes of memory blocks, length checks on the block, how data elements are copied in that block
Beware of custom allocators

• Unanticipated conditions. Length miscalculations can arise when unanticipated conditions occur during data processing
• Data assumptions. In code dealing with binary data (e.g. proprietary file formats and protocols) programmers tend to be more trusting of the content
  E.g. assumptions about a data element’s largest possible size, even when a length is specified before the variable-length data field
• Order of actions. Actions that aren’t performed in the correct order can also result in length miscalculation
• Multiple length calculations on the same input. A common situation is data being processed with an initial pass to determine the length and then a subsequent pass to perform the data copy

Allocation functions

Watch for erroneous handling of requests instead of assuming these custom routines are sound. Audit custom allocators as you would any other complex code - by keeping a log of the semantics of these routines and noting possible error conditions and the implications of those errors.
Typical issues to look for:

• Is it legal to allocate 0 bytes? Requesting an allocation of 0 bytes on most OS allocation routines is legal. A chunk of a certain minimum size (typically 12 or 16 bytes) is returned. This piece of information is important when you’re searching for integer-related vulnerabilities - a custom alloc call can be a sanitising wrapper to malloc
• Does the allocation routine perform rounding on the requested size? An allocation routine potentially exposes itself to an integer overflow vulnerability when it rounds a requested size up to the next relevant boundary without performing any sanity checks on the request size first
• Are other arithmetic operations performed on the request size? Another potential for integer overflows - when an application performs an extra layer of memory management on top of the OS’s management. E.g. the application memory management routines request large memory chunks from the OS and then divide it into smaller chunks for individual requests. Some sort of header is usually prepended to the chunk and hence the size of such a header is added to the requested chunk size.
  Similar situation with reallocation routines when they don’t have sanity checking.
• Are the data types for request sizes consistent? Many typing issues from Ch 6 are relevant for allocators - any type conversion mistake usually leads to memory corruption.
  Allocators that use 16 -bit sizes are even easier to overflow.
  Similar issues with LP64 arch - long and size_t are 64bit, while int is only 32bit.
  Important case - when values passed to memory allocation routines are signed. If an allocation routine doesn’t do anything except pass the integer to the OS, it might not matter whether the size parameter is signed. If the routine is more complex and performs calculations and comparisons based on the size parameter, however, whether the value is signed is definitely important. Usually, the more complicated the allocation routine, the more likely it is that the signed condition of size parameters can become an issue.
• Is there a maximum request size?* Sometimes developers build in a maximum limit for how much memory the code allocates. This often works as a sanitiser.
• Is a different size memory chunk than was requested ever returned? Essentially all integer-wrapping vulnerabilities become exploitable bugs for one reason: A different size memory chunk than was requested is returned. When this happens, there’s the potential for exploitation. Occasionally a memory allocation routine can resize a memory request.

Allocator scorecards and error domains

Create allocator scorecard: Prototype, is 0 bytes legal, rounds to X bytes, additional operations, maximum size, exceptional circumstances, notes, errors. Signedness and 16-bit issues can be inferred from function prototype.
Error domain is a set of values that, when supplied to the function, generate one of the exceptional conditions that could result in memory corruption.

Double frees

1. Free then allocated to other data, overwritten and freed again - with crafted data can lead to code execution
2. Block can be entered in the free block list twice (not possible on Windows and glibc - they check that block passed to free() is in use). Can also lead to code exec

Track each path throughout a variable’s lifespan to see whether it’s accidentally deallocated with the free() function more than once.
Especially pay attention when auditing C++ code. Sometimes keeping track of an object’s internal state is difficult, and unexpected states could lead to double-frees. Be mindful of members that are freed in more than one member function in an object (such as a regular member function and the destructor), and attempt to determine whether the class is ever used in such a way that an object can be destructed when some member variables have already been freed.
Many operating systems’ reallocation routines free a buffer that they’re supposed to reallocate if the new size for the buffer is 0. This is true on most UNIX implementations. Therefore, if an attacker can cause a call to realloc() with a new size of 0, that same buffer might be freed again later; there’s a good chance the buffer that was just freed will be written into.

TAOSSA Chapter 5-6

Ch 5 - Memory corruption

All memory corruption vulnerabilities should be treated as exploitable until proved otherwise

Buffer overflows

Process memory layout
Stack overflows
The runtime stack, activation records (function frames). Stack usually grows downward (Full Descending?). ESP / EBP. Calling conventions
Exploiting stack overflows
SEH attacks. Convenient method for exploiting stack overflows on a Windows system b/c the exception handler registration structures are located on the stack. Stack overflow followed by any exception
Off-by-one errors. Often in dealing with C strings NUL byte is not accounted for correctly. Easy to exploit on x86 by overwriting LSB of saved EBP (also little-endianness combined with FD stack)
Heap overflows. Heap management. malloc(). Exploiting via marking the next block as free and causing a single controlled fixed size value to be written to a controlled location

Popular targets of heap overwrites

• Global Offset Table (GOT), process linkage table (PLT)
• Exit handlers (Unix)
• Lock pointers (Win) in process environment block PEB
• Exception handling routines in PEB (Win)
• Function pointers
• Global and static data overflows. Usually result in application-specific attacks. No runtime structures to control

Shellcode

Writing the code
Finding your code in memory. Shellcode must be position-independent

Protection mechanisms

Stack cookies (canary values). Does not prevent against overwriting adjacent local vars, only saved frame pointer and return address; or against SEH overwrite (so SEH-based exploitation was developed because of stack cookies?)
Heap implementation hardening. Header cookie related to a global cookie and chunk’s address. Or additional checks on unlink operation. Similar deficiencies
Non-executable stack and heap. ROP bypasses this
Address space layout randomisation. Limitations: find something in memory that’s in a static location; or bruteforce where possible
SafeSEH. When exception is triggered, EH target addresses are checked. Bypasses depend on specific implementation. Function pointer obfuscation. Obfuscate any sensitive pointers stored in globally visible data structures. Limitations - can still overwrite app-specific pointers.

Assessing memory corruption impact

Where is the buffer located in memory?
What other data is overwritten? Pay special attention to any variables in the overflow path that mitigate exploit attempts (e.g. pointers that are freed before return)
How many bytes can be overwritten?
What data can be used to corrupt memory? Sometimes attacker does not control what data is used to overwrite memory. Often happens with off-by-ones
Are memory blocks shared? Determining whether memory-block-sharing vulnerabilities are exploitable is usually complicated and application specific
What protections are in place?

Ch 6 - C language issues

Background

Data storage overview

Chars are usually signed in implementations
Integers, floats, bit fields
Integers have precision and width, for signed ints width = precision + 1
Typedef as aliasing Signed representations: sign+magnitude; one’s complement; two’s complement (the most common) Byte order - big endian (RISC various), little endian (x86)
Common lengths: char type - signed by default and take up 1 byte. The short type takes 2 bytes, and int takes 4 bytes. The long type is also 4 bytes, and long long is 8 bytes. Also known as ILP32LL
64 bit architectures tend to be LP64 (long, long long, and pointer are 64 bit)

Arithmetic boundary conditions

Numeric overflow / underflow (wrapping). Used to manipulate length checks
NB: chars and shorts in arithmetic expressions are converted to ints first

Unsigned integer boundaries

In a typical case memory is allocated after multiplying user controlled values, then for loop is used to fill the memory x86 detects integer overflows but C cannot access the mechanism (OF flag)

Signed integer boundaries

In C spec, the result of under/overflow with signed integers is implementation defined and could include a machine trap. Usually though it is well defined, predictable and does not lead to exceptions
2’s complement used very commonly

Type conversions

Explicit vs. implicit type conversions. Conversions of integers are most interesting

Conversion rules

• These rules are two’s complement specific
• Value-preserving vs value-changing conversions
• Widening - zero extension (unsigned source) and sign extension (signed source). If a narrow signed type is converted to a wider unsigned type, sign extension occurs(!).
• Narrowing - only truncation.
• Converting signed to unsigned of the same width does not modify bit pattern

Simple conversions

• Casts (to the specified type)
• Assignments (to the type of the left operand)
• Function prototypes (to the types in the prototype; if not prototype, default argument promotion)
• Return statement (to the type in the function definition)

Integer promotions

Each integer data type is assigned an integer conversion rank (top to btm):

1. long long int, unsigned long long int
2. long int, unsigned long int
3. unsigned int, int
4. unsigned short, short
5. char, unsigned char, signed char
6. _Bool

You can substitute higher rank by lower rank types. Bit fields are narrower than their base type
Variables of integer types higher than int do not get promoted. Smaller (lower rank/narrower type) ones get taken to an int. If a value-preserving transformation to an int can be done, it is; otherwise a value-preserving conversion to an unsigned int is performed
Almost everything is converted to an int. To unsigned int - only unsigned int fields with 32 bits or some implementation specific integer types
In K&R days promotions were unsigned-preserving (e.g. unsigned char -> unsigned int)

Integer promotion applications

• Unary + operator promotes
• Unary - operator promotes first then does negation by 2’s complement, regardless whether the promoted operand is signed
• Unary ~ operator - 1’s complement after promotion
• Bitwise shifts promote both operands; result is the same type as promoted left operand
• Switch statements: controlling expression is promoted, then all switch constants converted to the resulting promoted type
• Function invocations: for functions w/o prototypes - when called, default argument promotions apply. Each argument is promoted, and any floats converted to doubles

Usual arithmetic conversions

Transforming 2 operands in an expression into a common real type

1. Floating point takes precedence (int -> float, less precise -> more precise)
2. Apply integer promotions (when neither is float). => Types narrower than int are promoted to an int
3. If both operands are the same type after int promotions, done
4. If operands are same signedness, different type, converted to the wider type (these will be always wider than int)
5. If the unsigned type is wider than or same width as a signed type - signed is converted to the type of the unsigned operand
6. If the signed type wider than unsigned type & value-preserving conversion is possible - transform everything into the signed integer
7. It the signed type wider than unsigned type & value-preserving conversion is impossible - take the type of the signed integer type, convert it into a corresponding unsigned integer type, then convert both operands to that type. E.g. unsigned int and long int (if long int width is the same as int) converts to unsigned long int

Usual arithmetic conversion applications

• Addition (when both arguments are of arithmetic type)
• Subtraction between 2 arithmetic types
• Multiplicative - always arithmetic types, always converted
• Comparison - always converted, result is an int = 0 or 1
• Binary bitwise operators require integer operands, conversion applies
• Questions mark operator - compiler decides on the type of the operand based on the types of 2nd and 3rd arguments, first arg does not affect

Type conversion summary

In addition to the above, sizeof is of type size_t which is unsigned integer type
Auditing tip: check assembly for conversions, beware of optimisations.

Type conversion vulnerabilities

Implicit type conversions are the source of vulnerabilities

Signed/Unsigned conversions

The most common case is simple conversions b/w signed and unsigned integers, esp. in assignments, function calls, or typecasts
Calling a function that expects an unsigned int with a negative parameter is a common case. Negative value gets interpreted as a huge positive int
Many libc routines use an argument of type size_t which is unsigned int the same width as pointer.
Important not to get a negative parameter into read(), recvfrom(), memcpy(), memset(), bcopy(), snprintf(), strncat(), strncpy(), and malloc()

Sign extension

Can occur:

• because of typecast, assignment, or function call
• if a signed type smaller than an integer is promoted via the integer promotions
• as a result of the usual arithmetic conversions applied after integer promotions because a signed integer type could be promoted to a larger type, such as long long

In some cases sign extension is value changing (e.g. a negative value from char to unsigned int) and has an unexpected result
Programmers often forget that char and short types are signed, especially in network code that deals with signed integer lengths or code processing binary or text data one char at a time
Another place where programmers forget whether small types are signed occurs with use of the ctype libc functions
Programmers rarely intend for their smaller data types to be sign-extended when they are converted, and the presence of sign extension often indicates a bug
Sign extension is somewhat difficult to locate in C, but it shows up well in assembly code as the movsx eax, [XXX] instruction; zero extension is xor eax, eax / mov al, [XXX]

Truncation

When a larger type is converted into a smaller type - can only happen as a result of an assignment, a typecast, or a function call that has a prototype
A good place to look at - in structure definitions, especially in network-oriented code

Comparisons

In comparisons, the compiler first performs integer promotions on the operands and then follows the usual arithmetic conversions on the operands to get compatible types. These promotions and conversions might result in value changes (because of sign change), and the comparison screws up
gcc -Wall does not warn on impossible condition checks (e.g. unsigned < 0), but gcc -W does
Pay particular attention to comparisons that protect allocation, array indexing, and copy operations
Watch for unsigned integer values that cause their peer operands to be promoted to unsigned integers. sizeof and strlen() are classic examples

Operators

Each operator has associated type promotions that are performed on each of its operands implicitly which could produce some unexpected results

Sizeof operator

One of the most common mistakes with sizeof is accidentally using it on a pointer instead of its target (ok to use sizeof(array) but not sizeof(pointer))
Often shows up as a result of editing when a buffer is moved from being within a function to being passed into a function
Look for it in expressions that cause operands to be converted to unsigned values

Unexpected results

2 primary issues with arithmetic operators: boundary conditions related to storage of integer types and issues with conversions that occur in expressions
On 2’s complement machines there are only a few C operators where signedness of operands can affect the result of operations. Their underlying implementation is sign-aware
Comparisons plus right shift >>, division /, modulus %
Right shift - problems when the left operand is signed (and negative). Easy to locate in assembly, look for sar mnemonic Division - when one operand is negative, the result is also negative. Apps often of not account for this possibility
Modulus - same when negative dividend. Often used with fixed-size arrays (hash tables)
Look for div (unsigned) vs idiv (signed) mnemonic in the x86 assembly

Pointer arithmetic

When pointers are subtracted, the result is a signed integer type ptrdiff_t
When C does arithmetic involving a pointer, it does the operation relative to the size of the pointer’s target.
Pointer to an object is treated as an array composed of one element of that object
You can add an integer type to a pointer type or a pointer type to an integer type, but you can’t add a pointer type to a pointer type
E1[E2] is equivalent to (*((E1)+(E2)))
Resulting type of the addition between an integer and a pointer is the type of the pointer

Vulnerabilities

Plenty of vulnerabilities that involve manipulation of character pointers essentially boil down to miscounting buffer sizes
Also, developers mistakenly perform arithmetic on pointers without realising that their integer operands are being scaled by the size of the pointer’s target

Other C nuances

Order of evaluation

C doesn’t guarantee the order of evaluation of operands or the order of assignments from expression “side effects”.
Example is macros or functions with side effects used several times in an expression

Structure padding

Structure members don’t have to be laid out contiguously in memory.
The order of members is guaranteed to follow the order programmers specify, but structure padding can be used between members to facilitate alignment and performance needs.
More visible on 64bit structures.
Comparing memory of 2 “identical” structs with different padding content will lead to incorrect results, or lead to double-free errors

Precedence

Sometimes a precedence mistake is made but occurs in such a way that it doesn’t totally disrupt the program
Precedence of the bitwise & and | operators, especially when you mix them with comparison and equality operators
Same with assignment, but these usually result in a compiler warning

Macros / Preprocessor

Parenthesising of params is important Problems with order of evaluation and side-effects exist

Typos

Programmers can make many simple typographic errors that might not affect program compilation or disrupt a program’s runtime processes, but these typos could lead to security-relevant problems
= in comparisons instead of ==
== instead of = in for loop initial assignment
& instead of && - even if there isn’t an issue caused by the difference between bitwise and logical AND operations in some situations, there’s still the critical problem of short-circuit evaluation (for the logical &&) and guaranteed order of execution
Semi-columns at the end of an if(); statement
Accidental octal conversion. E.g. 040 is 32 decimal
Missing comment sign */ at the end of the line will hide code until the end of the next comment
Missing {} in if statements Commenting out an if’s “then” part together with its “;” will make the next operator “then” part.

TAOSSA Chapter 1-4 summary

I've recently dug up these old notes of mine - they are probably from around 2008. I was jotting down points from The Art of Software Security Assessment as I was going through the book. 
Don't expect this to be a perfect summary, rather a snapshot of things I thought were worth remembering at the time. So, minimal formatting and no fancy sentences :) 

Oh and I asked @mdowd if he would be terribly upset about me posting this, he said no.

Ch 1 - Software vulnerability fundamentals

Augment source audit with black-box testing for best results
Design / implementation / operational vulnerabilities

Common threads

• Input and data flow. Tracing input is one of the main tasks
• Trust relationships (between components on an interface). Often the transitivity of trust plays an important role
• Assumptions and misplaced trust. Example - assumptions on the structure of input by developers
• Interfaces. Misplaced trust e.g. in cases when the interface is more exposed to external access than developers think
• Environmental attacks. OS, hardware, networks, file system etc
• Exceptional conditions

Ch 2 - Design review

Something about software design fundamentals
Issues in algorithms. Errors in business logic or in the key algorithms
Abstraction and decomposition

Trust relationships

Trust boundaries
Simple and complex trust relationships. Chain of trust
Strong coupling: look out for any strong intermodule coupling across trust boundaries. Usually data validation issues and too much trust b/w modules
Strong cohesion: pay special attention to designs that address multiple trust domains within a single module. Modules should be decomposed along trust boundaries.

Design flaws examples

Shatter attacks in Windows - strong coupling
Automountd + rpc.statd - transitive trust

Authentication vulns

“Forgetting” to authenticate
Untrustworthy credentials, e.g verified on a client
Insufficient validation, e.g. in programmatic authN between systems only the client or only the server is authenticated

Authorisation vulns

Web apps often do authZ checks “at the front door” but the actual handler pages omit authZ checks
Often it is possible to authN as a low prig user, but access info from other users, higher privilege
Insecure authorities - inconsistent logic or leaves room for abuse

Other vulns

Accountability - log injection breaks nonrepudiation

Confidentiality

All sorts of crypto problems
CBC (cypher block chaining) is the only good mode for block cyphers
CTR (counter) mode is the best for stream cyphers. IV (init vector) reuse can lead to trouble
Key exchange can be subject to MITM
Storing more sensitive data than needed, or for longer period. Lack of encryption, obsolete algorithms, data obfuscation instead of encryption. Hash issues
Salt values
“Bait-and-Switch” attacks, or hash collision.

Threat modelling

Microsoft stuff. DFDs
Attack trees
DREAD ratings
Prioritising implementation review based on thread modelling

Ch 3 - Operational review

Insecure defaults - of the application and of the base platform or OS
Access control issues
Unnecessary services
Secure channels
Spoofing
Network profile exposure (unnecessary services)

Web-specific issues

• HTTP request methods.
• Directory indexing.
• File handlers (server-side) leading to source disclosure. Uploads.
• Misconfiguration of external auth.
• Default site installed.
• Verbose error messages.
• Admin interface is public facing.

Development protective measures

• Non-executable stack
• Stack protection (canaries)
• Heap protection
• ASLR
• Registration of function pointers (wrapping in a check for unauthorised modification)
• Virtual machines (only prevent low-level vulnerabilities)

Host-based protective measures

• Object (e.g. memory) and file system permissions. Permissions are difficult, can be screwed up
• Restricted accounts. Details of access are important
• Chroot jails (more effective when combined with a restricted account). Does not limit network access
• System virtualisation
• Kernel protections. System call gateway as a natural trust boundary. Additional checks by kernel modules, e.g. SELinux
• Host based firewalls
• Anti-malware applications
• File and object change monitors. Reactive in nature
• Host passed ID/PS

Network-based protective measures

• Network segmentation, on all levels of OSI
• NAT
• VPNs
• Network ID/PS

Ch 4 - Application review process

Code review is a fundamentally creative process; it is also a skill, not strictly a knowledge problem
Some businesses focus on easy-to-detect issues (even of low risk) for the fear of issues found and published by someone else; not subtle complex ones
A good reviewer can do 100 to 1000 LoC an hour, depending on code. 100 kLoC takes less than double the time of 50 kLoC
Information gathering: dev interviews, dev docs, standards, source profiling, system profiling

Application review

Do not do a waterfall-like review. The time you’re best qualified to find more abstract design and logic vulnerabilities is toward the end of the review
Design review is not always the best to begin with - e.g. when there is no docs
Initial phase, then iterate the process 2-3 times a day: plan-work-reflect
Initial preparation: top-down (good only when the design docs are good, which is rare); bottom-up (could end up reviewing a lot of irrelevant code) or hybrid

Hybrid approach questions

• General application purpose
• Assets and entry points
• Components and modules
• Intermodule relationships
• Fundamental security expectations
• Major trust boundaries

Planning: consider goals (depending on the stage and level of understanding); pick the right strategy; create a master ideas list; pick a target/goal; coordinate
Work: keep good notes; don’t fall down rabbit holes (can be waste of time); take breaks
Reflect: status check; re-evaluate; peer review

Code Navigation

Code flow navigation - control-flow sensitive or data-flow sensitive. Surprisingly, not used much.
It is more effective to review functions in isolation and trace the code flow only when absolutely necessary (!)
Forward- and back-tracing. Back-tracing usually start from candidate points
Back-tracing examines fewer code flows - easier to do; but misses logic problems or anything not covered by candidate points

Code-auditing strategies

Code comprehension, candidate point, design generalisation

Code comprehension strategies

• Trace malicious input - difficult in OO code, especially poorly designed. In these cases, do some module or class review to understand the design. “5 or 6 code files before the system manages to do anything with the input”
• Analyse a module - reading code file line by line, not tracing or drilling-down. Very popular among experienced reviewers. Especially good for framework and glue code. Easy to go off-track
• Analyse an algorithm. Less likely to go off-track. Focus on pervasive and security critical algorithms
• Analyse a class or object. Study interface and implementation of an important object. Good for OO code (obviously). Less likely to go off-track than analysing a module
• Trace black box hits. Fuzz then investigate crashes. Check Shellcoder’s Handbook - Fault Injection chapter

Candidate point strategies

• General approach. Trace from potential vulnerabilities to user input.
• Automated source code analysis tool. Similar. Limited to a set of potentially vulnerable idioms
• Simple lexical candidate points
• Simple binary candidate points
• Black-box generated candidate points. Mostly crash analysis. Microsoft’s gflags is useful for heap overflows - “heap paging” functionality in the debugged process. LD_PRELOAD in Linux is useful. Corruption can happen in a buffer or an array (or heap)
• Application-specific candidate points. Similarities to previously found vulnerable patterns

Design generalisation patterns

• Model the system. Detailed modelling for security critical components
• Hypothesis testing. Guess an abstraction and then test the validity of this guess
• Deriving purpose and function. Somewhat similar; pick key programmatic elements and summarise them
• Design conformity check. Look at the “grey areas” and common code paths. Look for discrepancies b/w spec and implementation

Code-auditing tactics

Internal flow analysis. Intra-procedural and intra-module analysis. Especially error-checking branches and pathological code paths

• Error-checking branches - code paths that are followed when validity checks result in an error. Do not dismiss them
• Pathological code paths - functions with many small and nonterminating branches - branches that don’t result in abrupt termination of the current function. Exponential explosion of similar code paths

Subsystem and dependency analysis
Re-reading code. At least 2 passes
Desk-checking (symbolic execution)
Test cases. Be wary of input data from other modules. Don’t assume the same level of danger as external input, but a be a bit suspicious about it. Boundary cases

Code auditor’s toolbox

Code navigators: cscope, ctags, source navigator, code surfer (slicing!), understand (scripting)
Debuggers: gdb, OllyDbg, SoftICE (yeah right), (Immunity Debugger),
Binary navigation: IDA Pro, BinNavi Fuzzing: SPIKE, (Sulley)

Fiddling with Nexus 4 boot image

TLDR; How to modify any system to set ro.debuggable=1 without rebuilding it from source. This setting will make any apk debuggable on the device.

Get the existing boot image off the phone

dd if=/dev/block/mmcblk0p6 of=/mnt/sdcard/boot.img # on the phone
adb pull /mnt/sdcard/boot.img # on your computer

/dev/block/mmcblk0p6 is Nexus 4's boot partition.

Install abootimg from https://github.com/coruus/abootimg. The rest of the process below is stolen from this page.

Extract and unpack initrd

mkdir boot 
cd boot
abootimg -x /tmp/boot.img

mkdir initrd
cd initrd
cat ../initrd.img | gunzip | cpio -vid

Edit default.prop, setting anything you want, including ro.debuggable=1.

Repack initrd and boot image

cd initrd
find . | cpio --create --format='newc' | gzip > ../myinitrd.img
 
cd ..
abootimg --create myboot.img -f bootimg.cfg -k zImage -r myinitrd.img

Flash to phone

adb reboot-bootloader
fastboot flash boot myboot.img

Android Studio for refactoring obscure decompiled code

"It's in Foreign" @thegrugq

A while ago, I've been experimenting with using Android Studio for refactoring decompiled code.

1. Export Java sources, from whatever decompiler works 
2. "Import project" from sources in Android Studio 
3. Use Shift-Fn-F6 to rename classes, methods etc 

What's best is that Studio (hurray for IntelliJ IDEA) is sometimes intelligently estimates types of variables and offers reasonably meaningful names:

Using Proguard to deobfuscate code

TLDR; Optimisation features of Proguard can be useful for removing some "obfuscations" that add dead code and screw up control flow.

Proguard comes with Android Studio or can be installed from homebrew on Mac, that one version is newer:

$brew install proguard

Then use a generic script similar to this one:

-injars      <obfuscated jar>
-outjars     <result>
-libraryjars $HOME/Library/Android/sdk/platforms/android-19/android.jar ; or similar
-optimizationpasses 10 
-dontobfuscate
-dontpreverify
-printusage

-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-verbose

-keepattributes *Annotation*

 
-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends android.app.admin.DeviceAdminReceiver
-keep public class * extends android.view.View {
 public <init>(android.content.Context);
        public <init>(android.content.Context, android.util.AttributeSet);
 public <init>(android.content.Context, android.util.AttributeSet, int);
 public void set*(...);
}
-keepclasseswithmembers class * {
 public <init>(android.content.Context, android.util.AttributeSet);
}
-keepclasseswithmembers class * {
 public <init>(android.content.Context, android.util.AttributeSet, int);
}
-keepclassmembers class * extends android.content.Context {
 public void *(android.view.View);
 public void *(android.view.MenuItem);
}
-keepclassmembers class * implements android.os.Parcelable {
 static ** CREATOR;
}
-keepclassmembers class **.R$* {
 public static <fields>;
}
-keepclassmembers class * {
 @android.webkit.JavascriptInterface <methods>;
}
-keepclasseswithmembernames class * {
    native <methods>;
}
-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}
-dontwarn android.support.**

Run as proguard @deob.conf and you'll end up with a more readable version of your obfuscated code.

Re-signing VirusTotal samples to install them on a non-jailbroken iPhone

Sir Leigh Teabing: In which year did a Harvard scholar outrow an Oxford man at Henley? 
Robert Langdon: [reluctantly] Surely such a travesty has never occurred.

Why would one even want to do this? Let's say, you're doing some sort of zomg malware research and you want to verify something. You can either use a jailbroken phone, code signing is kinda moot there, or a real phone :)

VT samples of iPhone apps are sometimes encrypted, tough luck then. Most of the time they are already stripped of DRM. So you can download one and rename it to *.ipa. Then the fun begins.
I'll assume you already have a working Xcode setup and can sign and run a test app on your device. If not, Google it, there are bazillions of how-tos on that topic.

The best and most fool-proof application for re-signig is iReSign, or you can do the signing manually, using commands similar to ones here (codesign is the main one, duh) or here. IReSign requires you to chose the application to resign, the certificate and the profile. Entitlements can be usually omitted, the app will attempt it best at recovering the ones required. The ID field can be left unchanged if your certificate allows for * to be signed, otherwise change it to something that you can actually sign for.

Tips

• If in the process of re-signing VT ipas you get "Getting certificate ID's failed" from iReSign and the cert list is empty, don't worry, this is temporary after 16 Feb 2016 because Apple's root cert has expired. To save you some googling - http://stackoverflow.com/a/35399656 ("Show expired certificates" in keychain, then delete the old ones, install the new one).

• iReSign needs the correct provisioning profile that applies both to the ipa file being re-signed and your iPhone. It looks like setting them is a little contrived, what finally worked for me was using “iOS Team Provisioning Profile: *” from Xcode. Download it, or a similar one from developer centre. It massively depends on your particular setup, though. The simplest way to test yours is to create a base iOS project with Xcode, fiddle with settings (do not leave them on "Automatic") in "Build Settings"->"Code Signing" and try "Project"->"Run" for your dummy project. If it does install and run, use the same identity and provisioning profile for iReSign.

• You are likely to get (if you install via Xcode, iTunes install will silently fail) "Can't install application. The Info.plist for application at... specifies a CFBundleExecutable of ..., which is not executable" at some stage. This is because VirusTotal samples do not maintain Unix file permissions and the main executable needs to be, well, executable. Unpack and "chmod +x" then repack the ipa, either before or after re-signing: https://github.com/maciekish/iReSign/issues/45.
• Sometimes troubles with signing are due to a mismatch of the certificate in the provisioning profile and the keys you have. See here for how to check the DeveloperCertificates in the provisioning profile (and don't forget to split lines in your .pem to 63 char lengths). Then again, overall signing is described in a million places on the Internet :)

More Meta Than Regehr

Context

This is another meta-meta-meta-meta-model page to show up in Google search. John, you're welcome :D
 

More IDA Pro plugins for OS X - HexRaysCodeXplorer

Another little thing I did. Compiling was rather painless - mostly changing __LINUX__ to __MAC__ and recovering a few files from pre-6.9 version. It looks like the format of "custom view handlers" in Hex-Rays SDK has changed between IDA 6.8 and 6.9.

So the diff is here and the (IDA 6.8) plugin is here (32bit) + here (64bit).

IMMV - I have only tested that it compiles, displays context menu and C-Tree graph.

I'll maintain a branch for IDA Pro 6.8 OS X for a brief while at https://github.com/agelastic/HexRaysCodeXplorer/tree/OSX_IDA68

Compiling non-OSX IDA pro plugins on OS X, or sanity ala Einstein

"Insanity: doing the same thing over and over again and expecting different results." - Albert Einstein

Recently I was looking at more IDA Pro on OS X than normally, so I experimented with themes a bit, e.g. Consonance. Then I found IDASkins. Lo and behold, there are tons of Windows IDA versions for which it has been already precompiled in the github repo, but no OS X ones.

OS X compilation in cmake files in IDASkins is marked "untested" and it is, in fact, a bit buggy. After some fiddling I got the compilation to work, although one of the resulting plugins (one for Ida 64) crashes everything, no idea why :). I'm mostly looking at 32bit ARM anyways, so one plugin is good enough for me.

The diff that made the build work is in this gist. It contains mostly cosmetic changes, plus a weird fact that "if(APPLE)" didn't work properly in the cmake I had (from homebrew?). Plus I haven't used make before :) YMMV

Plugin for 32bit OS X IDA Pro 6.8 is here.

Oh and while I am compiling plugins, here are the two precompiled WWDC plugins for the same version of IDA Pro. No tricks in compiling it from source, just stick it into "plugins" dir of IDA SDK and have a universal or i386 libcapstone.a (the one from home-brew is x64 only, I believe).