Persistent DFIR Chaos Monkeys: Experimenting with Headlong

Persistent DFIR Chaos Monkeys: Experimenting with Headlong

Goals

For the past week or so I've been experimenting with local models in the home lab using Headlong a microharness for persistent agents. The goals for this experiment have been kind of modest: 

  1. Have fun exploring the "inner life" of Headlong's Persistent agent 
  2. Continuously explore new interfaces and harnesses for AI which might shape future product directions
  3. Seriously explore Hugging Face's lessons learned with commercial API Refusals and begin exploring "Local Open-Weight Models"
  4. Explore tasking out DFIR tasks to various agents as a potential pathway to scale a response by an army of robots

Lab Notes

Initially the planned architecture was to have a headless Debian minimal install running, ollama and serving up qwen3:8b to a Headlong docker instance. This worked but it was painfully slow without good GPUs in my old laptop's hardware. A truly independent and capable open-weight model running locally would be ideal for a cluster of persistent Headlong agents because budgeting for tokens isn't a great place to be if you want to scale your work; but luckily we live in a world where capable models and the inference needed to power them are a few clicks and a credit card away.

To overcome the speed issues I switched to open-router with a $10 token budget. An early misconfiguration of Headlong had me running Claude Sonnet 4.5 which gave great results while eating $4.35 of my budget in the first hour of my experiment. I then migrated to GPT-5 Nano which worked well enough for chat responses but really struggled to utilize the Headlong harness's tooling. I arrived on DeepSeek V4 Flash 0731 for the remainder of my token budget and was very pleased with the price and performance tradeoffs.

DeepSeek was able to perform local host and network enumeration as a hello world type exercise. I then tasked it a university's 3 part network forensic lab. Without any additional steering the results were acceptable and useable. In a production environment I would probably utilize skills to standardize output, standard operating procedures, and recurring workflows.

After running out of tokens I've been utilizing the Nvidia Nemotron 3 Ultra that has free input and output tokens as a way of continuing the experiment; and although I've gotten the agent to build a forensic parser for its own logs I'm happier with the results I was getting from DeepSeek V4 Flash.

After Action Review

If I were starting this experiment over I would start with free models https://openrouter.ai/discover?lane=free but DeepSeek V4 Flash provided wonderful output. One observation I had in switching models is the clear difference in tone, personality, and terseness of output. DeepSeek V4 Flash provided quality output conservatively at a competitive price and I now understand why non-frontier models are so frequently discussed for security research outside the US.

Beyond model economics, the more interesting question was whether the persistent nature of these agents would allow for continuous improvement. It might be useful to encode Kaizen manufacturing type logic to refine a process over time. Interagent communication is simple enough to automate using the Headlong web interface and specialist agents under a director might be a better way to manage projects. The agents get bored without projects to work on and if inference is cheap enough encoding long range goals for them to collaborate on seems like a productive way to spend uptime.

Links

  1. https://www.laude.org/updates/headlong-a-microharness-for-persistent-agents
  2. https://github.com/laude-institute/headlong
  3. https://openrouter.ai/anthropic/claude-sonnet-4.5
  4. https://openrouter.ai/openai/gpt-5-nano
  5. https://openrouter.ai/deepseek/deepseek-v4-flash-0731
  6. https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b:free

Comments

Post a Comment

Popular posts from this blog

Lab Notes: Persistence and Privilege Elevation using the Powershell Profile

TL;DR; A recent ESET blog post mentions a persistence technique I wasn't previously aware of that is in use by the Turla APT group.  The technique leverages the PowerShell profile to sabotage PowerShell in a way that executes arbitrary code every time Powershell is launched, upon testing I've discovered this technique may also provide a low and slow vector to Domain Admin, and other privileged admin or services accounts by leveraging common flaws admin scripts, asset management systems, and enterprise detection and response tools. This post captures my observations working from  Matt Nelson's 2014 blog post  (Apologies to the researcher if there is prior art I'm unaware of at the time of this post) Privilege Elevation - Local Admin to Sloppy Admin Setup Requirements: In my testing, you need local admin rights to create the global profile $profile.AllUsersAllHosts  AKA C:\Windows\System32\WindowsPowerShell\v1.0\profile.ps1 This does not bypass Executi...
Read more

Lab Notes: Claude Code Session Logs as a Forensic Artifact

Image
  TL;DR; Claude Code logs every agent action locally in structured JSONL transcripts. These are forensically valuable, generally unprotected, and your GRC and detection teams should know they exist. Background AI coding agents like Claude Code are becoming common in developer environments. Unlike a chat interface, these tools operate agentically — they execute bash commands, read and write files, and chain tool calls autonomously on behalf of the user. Users authorize this at session start, often broadly, and may not review every action taken. This creates a non-repudiation problem. The user is responsible for agent actions, but awareness of specific actions may be limited or absent entirely. From a forensic and compliance standpoint that gap matters. The Artifact Claude Code writes a complete session transcript for every run to: ~/.claude/projects/<url-encoded-project-path>/sessions/<session-uuid>.jsonl ~/.claude/history.jsonl Each record contains the timestamp...
Read more

Difficult Conversations: Your Detection Coverage Map Is a Lie

  TL;DR: Coverage maps derived from detection content libraries (or worse, vendor assertions) will never answer the questions stakeholders are actually asking. Adversary emulation and simulation are the answer people are looking for but aren't asking for. An application owner comes to you. They're responsible for a critical business system, they have a compliance discussion coming up, and they want a coverage map showing which MITRE ATT&CK techniques your detection program covers for their application. They've done their homework. They're not asking for a rubber stamp. They want to know if you'll actually catch something targeting their system. If you have a coverage map, it's already broken. If you don't, building one won't give them what they're asking for. Either way, the green squares won't answer the question. This conversation is about to get difficult, because the language of our industry equates security monitoring maturity with ...
Read more