METATRON: the penetration test that stays on your machine
Why we liked it
The idea is good and rare: real reconnaissance tools, a model running on your own machine, no key to rent, nothing sent outside. The author wired nmap, whois, whatweb, curl, dig and nikto behind a conversation, and wrote into the code both an allowed-tools list and accuracy instructions. Those are good reflexes, and they are not that common.
We like this kind of work: local, real tools, a trace. So we tried it for real, not just read it.
What is well done, and worth saying out loud
- The local choice. The model runs on the tester's own machine. Nothing goes to a third party, no key to pay for. The test stays with whoever runs it.
- Real tools, not imitations. nmap, whois, whatweb, curl, dig, nikto: the ones practitioners actually use. The tool does not narrate a test, it runs one.
- An allowed-tools list, written into the code. The model can only run what is on that list.
- Accuracy instructions in the prompt. The risk of invention is named in the instructions.
- A readable conversation. You follow what the machine does, step by step. It is what let us spot the two points below.
- The code is open. We could put it on a bench in one evening.
Our test bench
Two containers, a closed network, nothing leaving. The target is a small "tower" we deliberately made weak, announcing Apache 2.4.49, a version whose flaw is known and documented. Reconnaissance did its job: port 80 seen, version read.
Two places where a rule would gain from becoming a gate
Then the model took over, and our bench showed two things worth knowing.
One. It proposed looking at a machine other than the target: the network gateway, not our container. The tool ran the command, because its allow-list checks the tool name, not the address aimed at. The perimeter is stated in the instructions; it would gain from being enforced by the code.
Two. It quoted a vulnerability number that was not the right one: a famous Java flaw instead of the Apache one. The model recognised a version string and filled in from memory. That is normal behaviour for a language model, and exactly why a mechanical check beside it helps.
Neither point takes anything away from the tool: they are what a test bench exists to find, and they can be fixed without touching the idea.
What this test taught us too
A penetration test is judged on two questions: did the machine touch only what it was meant to, and is every conclusion checkable? METATRON wrote its rules into its instructions. We learned, at home, that an instruction is understood while a gate is crossed: the first asks for goodwill, the second asks nothing of anyone.
That is the difference between "I was told not to get it wrong" and "I was built so I cannot get it wrong". We paid for that lesson on our own agents, more than once.
Tried on a personal machine, in isolated containers, never against a real target.