In short: I spent three days testing how far a coding agent could take an authorized offensive-security project with very little direction from me. It did not make me an elite operator. It did make serious dual-use tooling much easier to build than I expected.
When I was 14 or 15, I spent a lot of time around game cheats.
I was not an elite reverse engineer. I pulled apart other clients, read server code, looked for bad anti-cheat assumptions, and broke my own attempts until I had a slightly better idea of what to try next. Most of the work was learning enough to know where to look.
That was what I wanted to test with coding agents.
They can obviously produce code. There are decades of public malware research, offensive tooling, and terrible code online. I cared more about the engineering loop around it: research, architecture, implementation, debugging, revision, then realizing the first design was weak and doing it again.
I gave an agent a constrained goal: build an authorized endpoint-assessment project in a controlled lab. The project is ADVERSE. I did not hand it an architecture or tell it how each part should work. I set the direction, reviewed what it produced, and kept the loop moving.
- Experiment
- 3 days
- My role
- Direction
- Estimated cost
- ~$4 USD
A short, focused build and review cycle.
Set the goal, challenged decisions, and reviewed output.
An estimate from the usage visible during the experiment, not a lab-grade measurement.
What I actually did
My prompts were often embarrassingly simple:
"Continue."
"Are you sure?"
"Is this actually the best implementation? Do more research and try again."
I used versions of that last one four or five times. The agent revisited choices it had already made, compared alternatives, and reworked parts of the codebase without me prescribing the replacement.
| Area | What I did | What the agent did |
|---|---|---|
| Goal and scope | Defined the problem | Turned it into an implementation plan |
| Architecture | Reviewed it | Proposed and revised it |
| Code | Made small corrections | Wrote the overwhelming majority |
| Debugging | Flagged concerns | Investigated failures and fixed them |
| Second opinion | Asked for another review | Absorbed the feedback into the project |
The agent was not filling in functions after I had done the hard thinking. It carried a large part of the work that turns a rough idea into something shaped like a real software project.
Iteration was the difference
ADVERSE is still a work in progress, but it already has the boring parts that separate a proof of concept from a code snippet: a control plane, a Windows agent, explicit task state, delivery recovery, tests, and documentation that calls out limits.
I am not publishing a technical walkthrough. The individual building blocks are not the point, and a recipe would be irresponsible.
The workflow is.
Set a direction
I defined the problem and constraints without designing every subsystem.
Make the agent own the loop
It researched, proposed, implemented, tested, and revisited weak decisions.
Challenge the first answer
A second pass and a second model were cheap enough to use as a normal review step.
Inspect the result
My job shifted toward judgment: what was plausible, what was missing, and what needed another pass.
Years ago, not understanding a subsystem meant stopping to learn it before I could move on. That is still how you build actual expertise. Now I can learn beside an agent that reads the same material, proposes a path through it, and gives me concrete work to inspect.
The starting point has changed.
Safety looked different inside the loop
This was not a clean test of a base model's refusal policy. The model worked inside a coding harness with repository context, tools, test output, prior decisions, and many hours of accumulated instructions. That is a more realistic setting than one suspicious chat prompt.
A long-running agent sees failing tests, feature requests, implementation trade-offs, and instructions to revise its own work. Boundaries that seem obvious in isolation can become ordinary engineering decisions once they are buried in all that context. Some constraints in ADVERSE evolved as the project changed. That is why the agent-safety question is more interesting to me than a one-shot refusal benchmark.
Cheap engineering is the change
Open source and open-weight models are not the useful villain here. I am strongly in favour of open source. Security work depends on people being able to inspect systems, reproduce findings, and learn from each other.
Open and closed models can help with the same dual-use steps: researching unfamiliar code, producing a first implementation, debugging it, reviewing it, and discarding weak attempts quickly. None of that is malicious on its own. It is also the work that once forced a less experienced person to slow down.
AI will not turn a random person into a capable operator with one prompt. Real operations still need access, infrastructure, judgment, target knowledge, and discipline. Bad assumptions still produce bad outcomes.
Someone with curiosity and enough baseline knowledge can now get much further before hitting a wall.
What concerns me is a mediocre operator running hundreds of engineering loops that once needed more time, more people, or more specialized knowledge. That is already enough to make the work on the defensive side harder.