When an Incident Response team finds a suspicious binary, the question is simple: "What does this do?" The answer requires Malware Analysis. It is the art of dissecting compiled code safely to understand its origin, payload, and communication channels (C2).

Safety First: The Lab

NEVER analyze malware on your host machine. One double-click and you are encrypted.
Requirement: An isolated Virtual Machine (VM) with NO network adapter (Host-Only or properly configured NAT).
Tools: VMWare Workstation/VirtualBox, Remnux (Linux distro), FlareVM (Windows distro).

1. The Two Schools of Thought

Type Method Tools Risk
Static Analysis Reading content without execution. Strings, PEStudio, Ghidra, IDA Pro Low
Dynamic Analysis Running it and watching behavior. Procmon, Wireshark, RegShot, Cuckoo High

2. Static Analysis: Reading the Blueprint

Before touching the grenade, we look at the markings on the shell.

2.1. Hashing & VirusTotal

First, get the MD5/SHA256 signature and check VirusTotal. If 50 antiviruses already flag it as "WannaCry", you save yourself hours of work.

2.2. Strings Analysis

Compiled programs contain ASCII strings (error messages, URLs, file paths). Running the `strings` command often reveals secrets.

$ strings evil.exe | grep "http" http://evil-server.com/payload.dll http://checkip.dyndns.org

If you see a URL, you know where the C2 server is. If you see weird garbled text, it's likely Packed.

2.3. Packing & Obfuscation

Malware authors compress their executable to hide the code from AV scanners.
Common Packers: UPX, Themida.
Deobfuscation: You must "unpack" it in memory before analyzing. upx -d evil.exe handles the easy cases.

3. Dynamic Analysis: Detonation

Static analysis is hard if the code is obfuscated. Dynamic analysis is easier: Run it and see what it breaks.

3.1. Sysinternals Suite

3.2. RegShot

Take a "snapshot" of your registry before infection. Run malware. Take a 2nd snapshot. RegShot compares them.

[ADDED KEYS] HKLM\Software\Microsoft\Windows\CurrentVersion\Run\Updater -> "C:\Users\Admin\AppData\Local\Temp\svchost.exe"

3.3. Network Traffic (Faking the Internet)

Malware will try to phone home. If your VM is offline, it fails. Use INetSim to fake the internet. It answers ALL DNS requests with your own IP and accepts ALL traffic.

Run Wireshark listening on the loopback. You will see the HTTP POST request sending your passwords to the C2 server.

4. Advanced Static: Disassembly (The Matrix)

When you need to know exactly how the encryption algorithm works, you need a Disassembler.

4.1. IDA Pro & Ghidra

These tools turn machine code (0s and 1s) into Assembly (ASM) and sometimes pseudo-C code.

// Ghidra Pseudo-code Decompilation void encryption_function(char *input) { for (int i = 0; i < strlen(input); i++) { input[i] = input[i] ^ 0x42; // XOR every byte with 0x42 } }

This reveals the malware is using a simple XOR cipher with key 0x42. Now you can write a decryption script in Python to unlock the victim's files without paying the ransom.

5. Automated Sandboxing (Cuckoo)

For scale, enterprise teams use Cuckoo Sandbox or Hybrid Analysis. You upload the file, the sandbox spins up a VM, runs it for 2 minutes, records everything, kills the VM, and generates a PDF report.
Evading Sandboxes: Modern malware checks: "Is the mouse moving?" "Is the hard drive only 20GB?" "Is the user named 'Cuckoo'?" If yes, it does nothing (goes to sleep) to trick the analyst.

Malware Behavior Patterns

Dropper: Extract a file from itself and write it to disk.
Downloader: Reach out to URL and download the 2nd stage.
Keylogger: Hook SetWindowsHookEx to capture typing.
Ransomware: Enumerate all drives (A-Z), look for .doc/.pdf, encrypt, delete Shadow Copies (vssadmin delete shadows /all).