In languages like Python or Java, memory is managed for you. If a list has 10 items and you ask for the 11th, you get an Exception. In C or C++, you have direct access to memory. If you copy 100 bytes into a buffer that can only hold 50, the program doesn't stop. It writes into the adjacent memory. This is a Buffer Overflow. It allows attackers to overwrite critical data structures, including the "Return Address" that tells the CPU what to do next.
The Stack Layout (x86)
Memory grows downwards. When a function is called, a "Stack Frame" is created containing:
1. Local Variables: (e.g., `char buffer[64]`)
2. Saved EBP: The Base Pointer of the previous function.
3. Return Address (EIP): The address of the instruction to execute when the function finishes.
If we write past `buffer[64]`, we hit EBP, then we hit EIP.
1. Controlling EIP (Instruction Pointer)
EIP is the most important register. It points to the next instruction.
If we overwrite the Return Address with 0x41414141 ('AAAA'), when the function returns, the CPU tries to jump to memory address 0x41414141.
Since that address doesn't exist, the program crashes (Segmentation Fault).
The Exploit: Instead of 'AAAA', we put the address of our malicious code.
2. Shellcode
Shellcode is raw machine code (Assembly) designed to spawn a shell (`/bin/sh` or `cmd.exe`).
It must be small, position-independent, and often free of "bad characters" (like Null bytes \x00 which terminate C strings).
3. The NOP Sled
We don't know the exact memory address where our shellcode starts.
Solution: Prepend the shellcode with a massive slide of NOP instructions (`\x90`).
NOP means "No Operation". The CPU just slides down them until it hits the shellcode.
We just need to jump somewhere in the NOP sled.
4. Mitigations (Why it's harder now)
ASLR (Address Space Layout Randomization): The OS randomizes the stack location every time. You can't hardcode addresses.
DEP / NX (Data Execution Prevention): Mark the stack as "Non-Executable". The CPU refuses to run code sitting on the stack.
Stack Canaries: A random secret value placed before the Return Address. If you overflow the buffer, you destroy the Canary. The function checks the Canary before returning and aborts if it's dead.
Modern exploits rely on leaking pointers to defeat ASLR and using ROP (Return Oriented Programming) to defeat DEP.