Attacks as data: why a plain text format beats a script

Security teams already build lightweight replay scripts for specific exploits. Most of them by hand, and none of them alike. Exploitmatic formalizes that idea with a grammar: an attack becomes a plain text file, a chain of steps that captures values and evaluates an assert on each response.

The important move is that the file is data, not code. That one decision drives everything else.

Data, never code

A solution is bytes in, bytes out, and a set of tests over the response. The runtime never executes anything from a solution. It replays the steps and reports what the asserts found.

That means the corpus has no arbitrary-code supply chain. There is no template engine and no injected interpreter. When you run a solution you are not running someone else’s program; you are running your own runtime against a payload that is just bytes and tests. The blast radius of a bad file is a failed assert, not code execution on your machine.

A human can read it, an LLM can write it

The format is plain text on purpose. A step block reads like a checklist:

tls malformed-heartbeat
  send hex "1803030003014000"
  receive 70000 bytes
  expect the response to match "18030[123]40"

A human sees the identity and the bytes, then the test. An LLM sees the same grammar, and because the runtime is deterministic, a file an LLM writes is exactly what runs. The result lines give it a clean signal to fix and retry.

That is the loop: describe the attack in your own words, let an LLM draft the file, review it, then run it. Natural language is an authoring aid, never a runtime input.

The result is derived, never stored

The runtime prints what was sent and what came back, then the test that ran on each step. One line per step, then a result line. There is no artifact file; the console output is the result.

The result comes from the tests, never from the file. A solution cannot claim a success it did not earn, because the report only repeats what the asserts found on this run.

Plain text, no YAML

Line-based, one instruction per line, comments with #, validated at load. There are no aliases, anchors, or YAML CVEs. The strictness lives in the loader, not in a second language to learn.

That is the trade we wanted: a format with no syntax to remember, and a runtime that refuses to run a bad file.

The grammar is the contract

The grammar is open, and so are the parser and the validator. A solution either parses or it does not, and the error says which line and why. That makes the corpus reviewable by machines and humans alike, and it makes authoring something an LLM can be coached to do well.

Exploitation comes first, and the verification is what proves it worked. The format is how both stay honest.