Slides for a college course at City College San Francisco. Based on "The Shellcoder's Handbook: Discovering and Exploiting Security Holes ", by Chris Anley, John Heasman, Felix Lindner, Gerardo Richarte; ASIN: B004P5O38Q.
Instructor: Sam Bowne
Class website: https://samsclass.info/127/127_S18.shtml
3. Vulnerability
• A flaw in a system that allows an attacker
to do something the designer did not
intend, such as
– Denial of service (loss of availability)
– Elevating privileges (e.g. user to
Administrator)
– Remote Code Execution (typically a remote
shell)
4. Exploit
• Exploit (v.)
– To take advantage of a vulnerability and
cause a result the designer did not intend
• Exploit (n.)
– The code that is used to take advantage of a
vulnerability
– Also called a Proof of Concept (PoC)
5. 0Day and Fuzzer
• 0Day
– An exploit that has not been publicly
disclosed
– Sometimes used to refer to the vulnerability
itself
• Fuzzer
– A tool that sends a large range of unexpected
input values to a system
– The purpose is to find bugs which could later
be exploited
6. Memory Management
• Specifically for Intel 32-bit architecture
• Most exploits we'll use involve
overwriting or overflowing one portion of
memory into another
• Understanding memory management is
therefore crucial
7. Instructions and Data
• There is no intrinsic difference between
data and executable instructions
– Although there are some defenses like Data
Execution Prevention
• They are both just a series of bytes
• This ambiguity makes system exploitation
possible
8. Program Address Space
• Created when a program is run, including
– Actual program instructions
– Required data
• Three types of segments
– .text contains program instructions (read-
only)
– .data contains static initialized global
variables (writable)
– .bss contains uninitialized global variables
(writable)
9. Stack
• Last In First Out (LIFO)
• Most recently pushed data is the first popped
• Ideal for storing transitory information
– Local variables
– Information relating to function calls
– Other information used to clean up the stack
after a function is called
• Grows down
– As more data is added, it uses lower address
values
10. Heap
• Holds dynamic variables
• Roughly First In First Out (FIFO)
• Grows up in address space
11. Program Layout in RAM
• From link Ch 1a (.bss = Block Started by Symbols)
14. Assembly Language
• Different versions for each type of
processor
• x86 – 32-bit Intel (most common)
• x64 – 64-bit Intel
• SPARC, PowerPC, MIPS, ARM – others
• Windows runs on x86 or x64
• x64 machines can run x86 programs
• Most malware is designed for x86
15. Instructions
• Mnemonic followed by operands
• mov ecx 0x42
– Move into Extended C register the value 42
(hex)
• mov ecx is 0xB9 in hexadecimal
• The value 42 is 0x42000000
• In binary this instruction is
• 0xB942000000
16. Endianness
• Big-Endian
– Most significant byte first
– 0x42 as a 64-bit value would be 0x00000042
• Little-Endian
– Least significant byte first
– 0x42 as a 64-bit value would be 0x42000000
• Network data uses big-endian
• x86 programs use little-endian
17. IP Addresses
• 127.0.0.1, or in hex, 7F 00 00 01
• Sent over the network as 0x7F000001
• Stored in RAM as 0x0100007F
18. Operands
• Immediate
– Fixed values like 0x42
• Register
– eax, ebx, ecx, and so on
• Memory address
– Denoted with brackets, like [eax]
20. Registers
• General registers
– Used by the CPU during execution
• Segment registers
– Used to track sections of memory
• Status flags
– Used to make decisions
• Instruction pointer
– Address of next instruction to execute
21. Size of Registers
• General registers are all 32 bits in size
– Can be referenced as either 32bits (edx) or 16
bits (dx)
• Four registers (eax, ebx, ecx, edx) can
also be referenced as 8-bit values
– AL is lowest 8 bits
– AH is higher 8 bits
23. General Registers
• Typically store data or memory addresses
• Normally interchangeable
• Some instructions reference specific
registers
– Multiplication and division use EAX and EDX
• Conventions
– Compilers use registers in consistent ways
– EAX contains the return value for function
calls
24. Flags
• EFLAGS is a status register
• 32 bits in size
• Each bit is a flag
• SET (1) or Cleared (0)
25. Important Flags
• ZF Zero flag
– Set when the result of an operation is zero
• CF Carry flag
– Set when result is too large or small for
destination
• SF Sign Flag
– Set when result is negative, or when most
significant bit is set after arithmetic
• TF Trap Flag
– Used for debugging—if set, processor executes
only one instruction at a time
26. EIP (Extended Instruction Pointer)
• Contains the memory address of the next
instruction to be executed
• If EIP contains wrong data, the CPU will
fetch non-legitimate instructions and
crash
• Buffer overflows target EIP
29. Simple Instructions
• mov destination, source
– Moves data from one location to another
• Intel format is favored by Windows
developers, with destination first
35. NOP
• Does nothing
• 0x90
• Commonly used as a NOP Sled
• Allows attackers to run code even if they
are imprecise about jumping to it
36. The Stack
• Memory for functions, local variables, and
flow control
• Last in, First out
• ESP (Extended Stack Pointer) – top of stack
• EBP (Extended Base Pointer) – bottom of
stack
• PUSH puts data on the stack
• POP takes data off the stack
38. Function Calls
• Small programs that do one thing and
return, like printf()
• Prologue
– Instructions at the start of a function that
prepare stack and registers for the function
to use
• Epilogue
– Instructions at the end of a function that
restore the stack and registers to their state
before the function was called
41. Conditionals
• test
– Compares two values the way AND does, but
does not alter them
– test eax, eax
• Sets Zero Flag if eax is zero
• cmp eax, ebx
– Sets Zero Flag if the arguments are equal
42. Branching
• jz loc
– Jump to loc if the Zero Flag is set
• jnz loc
– Jump to loc if the Zero Flag is cleared
43. C Main Method
• Every C program has a main() function
• int main(int argc, char** argv)
– argc contains the number of arguments on
the command line
– argv is a pointer to an array of names
containing the arguments
44. Example
• cp foo bar
• argc = 3
• argv[0] = cp
• argv[1] = foo
• argv[2] = bar