$_ stdout

Cheap, Fast, and Local: SLMs Rewire Cybersecurity

Enterprise security spent two years asking one frontier model to do everything — triage the alert, write the exploit, patch the bug it just found. The literature's answer isn't a bigger model; it's dozens of small, disposable ones running on-premises, orchestrated by deterministic code, each one narrow enough to audit and cheap enough to run on every single request.

Five findings survive contact with the benchmarks that follow: (1) performance parity between small and frontier models is real, not aspirational; (2) cost-per-finding, not leaderboard F1, is the number that should decide a procurement call; (3) data sovereignty has turned local deployment from a preference into a requirement; (4) task-adapted abliteration is a scalpel where RLHF (Reinforcement Learning from Human Feedback) alignment is a blunt instrument; and (5) the durable capability lives in the orchestration between models, not inside any single one of them.

BL Dr. Ben Livshits August 3, 2026 · 93 commits

Agentic security has grown enormously over the past two years — propelled as much by better harnesses wrapped around models as by the models themselves, and on the offensive side as much as the defensive one.

The default architecture for an agentic security tool, until very recently, was one model doing everything. A single cloud-hosted frontier LLM served simultaneously as the human-computer interface and the orchestrator of every tool call underneath it — parsing the analyst's request, selecting the API, interpreting the output, deciding the next step, all inside one continuously-growing context window.

Here's that failure mode as a recorded event, not a hypothesis. In July 2026, Cisco pointed two models at the same unfamiliar codebase — trailofbits/fickling, a Python pickle-safety library — and asked each to localize a real, disclosed flaw: CVE-2026-22607, an incomplete denylist that let a malicious pickle file slip past the library's own safety checks.

The frontier model didn't lose on reasoning. At more than 3,000 times the size, it lost on finishing the trace before its budget ran out — and the small model didn't.

01 Breaking up the Monolith

The argument in this post runs through four movements: (1) where the real capability in agentic security already lives, (2) what breaks when a single model tries to hold an entire multi-step loop, (3) what a shipping architecture built around that lesson looks like, and (4) why the smaller models come out ahead once the advantages actually get tallied.

The Harness, Not the Model

Long-range planning is where this argument actually has to start. DARPA's AI Cyber Challenge (AIxCC) spent two years testing whether autonomous systems could carry a vulnerability from an unfamiliar codebase all the way to a validated patch without a human in the loop, and the improvement curve arrived faster than most expected: finalist Cyber Reasoning Systems identified 86% of the competition's synthetic vulnerabilities in the final scored round, up from 37% at the semifinals eighteen months earlier, and patched 68% of what they found.

That's not one inference call — it's building a model of unfamiliar code, fuzzing or analyzing it for weaknesses, triaging what turns up, synthesizing a fix, and validating that the fix doesn't break anything else, a pipeline that runs for hours unattended.

Microsoft's MDASH, covered in more detail later in this post, makes the same shape explicit as an architecture rather than a competition result: five sequential stages — Prepare, Scan, Validate, Dedupe, Prove — where each stage hands the next its accumulated context instead of starting cold. Long-range planning, in other words, is already the operating norm in agentic cybersecurity, on the offensive testing side and the defensive tooling side alike.

That's not a coincidence, and it isn't really about the model at all. Anthropic's own engineering team found that even a frontier coding model like Opus 4.5, run in a loop with no other scaffolding, falls short of finishing a production-quality build from a single high-level prompt: it either over-commits to too much at once, or a later session looks around, sees partial progress, and declares the job done. What closed the gap wasn't a better model — it was harness engineering: an explicit checklist of 200-plus features marked failing by default, one feature per session, and git commits and progress files so the next session picks up cold instead of guessing.

The same lesson holds on the cyber side, offense and defense both. Cato Networks reported that a harness paired with GPT-5.5 strung together complete attack chains to domain-administrator privileges in as little as 40 minutes, and the researchers' own conclusion was blunt: it's not just about the frontier model. Tenable, Proofpoint, and SpecterOps have each since built a proprietary harness of their own. A recent security architecture, SafeHarness, makes the underlying claim explicit: the execution harness — not the model running inside it — is the system layer that actually orchestrates tool use, context, and state.

The most controlled version of that experiment holds the model fixed and only varies the harness. Mayoral-Vilches et al. ran the same mid-capability model behind five different agent harnesses against a 33-challenge cybersecurity benchmark: the best individual harness solved 15/33, and no single harness dominated — different harnesses failed and succeeded on different challenges even though the model underneath never changed. Letting heterogeneous harnesses run in parallel and exchange findings through a shared blackboard pushed that to 19/33, a 27% relative gain over the best solo harness, achieved 25% faster and at comparable cost. The model was constant throughout; only the harness moved.

From Language Model Agency to Code Agency

Run that same multi-stage plan inside one continuously-growing context window, though, with no harness at all, and a different problem shows up. The single-model, no-harness version of this has a name in the literaturelanguage model agency — and a structural flaw that doesn't show up until you push it into production. As the context window fills with tool outputs, error traces, and multi-turn history, computational cost and time-to-first-token scale quadratically, not linearly.

A model that felt snappy on a three-turn demo becomes unusable on the fifteen-turn incident-response session that real triage actually requires — exactly the workload where real-time responsiveness matters most.

The fix isn't a faster frontier model. It's decoupling the interface from the executor: a dedicated, deterministic controller manages the operational control flow, and specialized language models get invoked as plug-ins for discrete, bounded subtasks — code agency instead of language model agency.

Code Agency, Shipping

Parameter extraction, API schema formatting, code-block parsing: routine, narrow, high-volume work gets handled locally by a small model built for exactly that shape of task. A large generalist model is kept in reserve, invoked selectively and only when the local model actually hits a high-complexity edge case it can't resolve. Cisco's open Foundry Security Spec is one shipping version of that shape, not a sketch — a deterministic Orchestrator routing work across eight specialized roles over a shared layer, covered in more detail later in this post.

That single design choice — a harness that decides what gets routed where — is what the rest of this post keeps coming back to, in three different guises: what it costs to run, how well it performs on a narrow slice of the job, and how tightly its behavior can be bounded and checked.

The Case for Going Small

This post takes that claim apart in two moves: first the architectural shift from a single cloud model doing everything to code that orchestrates a swarm of small, specialized ones, then what these swarms actually do on offense and defense, benchmark by benchmark, not just in theory.

// Stanislav Fort, co-founder & chief scientist, AISLE Rather than "judiciously deploy one expensive model and hope it looks in the right places," you can run cheap models everywhere and let coverage do the work — "a thousand adequate detectives searching everywhere will find more bugs than one brilliant detective who has to guess where to look."

Every argument in this post for going small ultimately cashes out into one of four advantages:

Infographic titled 'The SLM Edge: Why Small Models are Rewiring Cybersecurity.' Left panel, the three pillars of SLM superiority: 172x cost reduction (under $1 versus roughly $141), zero-leak on-premise privacy keeping source code and network metadata inside the enterprise, and specialization over scale, where a 3B specialized SLM can outperform a 70B generalist LLM on narrow security tools. Right panel, transforming defensive operations: precision vulnerability localization, doubling defensive recall by removing safety-refusal bias, and orchestrated malware analysis dividing a report across code, network-traffic, and behavioral SLMs, plus a localization-efficiency comparison showing Cisco Antares under $1, a leading open LLM around $12.50, and a leading frontier LLM around $141.
// Figure 1. The post's argument as a single card: three of the four structural advantages on the left, the concrete defensive payoffs — Antares included — they add up to on the right.

02 Small Is Beautiful

"Small Language Model" is, in this literature, an operational category, not a marketing term: decoder-only transformers typically in the 1-to-12-billion-parameter range, with effective configurations occasionally stretching to roughly 20 billion — Mistral-NeMo 12B is the usual upper bound cited — specifically so the model fits inside a single-GPU server's constraints.

What defines the category isn't raw capability; it's optimization for a deployment envelope: latency, infrastructure cost, and the ability to run at the network edge rather than across a distributed cloud cluster. The same logic, pushed to its conclusion, is why the frontier of SLM deployment is drifting on-device — fine-tuned for one job, running on the phone in your pocket, no round-trip to a cloud at all.

Large Language Models sit on the other side of that line by definition — tens or hundreds of billions of parameters, hosted on distributed cloud infrastructure, reached through a commercial API rather than a local weights file. The distinction matters operationally, not just numerically: an SLM is a thing you own, version, and can afford to run on every request; an LLM is a thing you rent, and every request has a marginal cost attached.

Dispelling the Myth of Mythos

The starkest test of the specialization thesis wasn't a benchmark table — it was Anthropic's own flagship model. Claude Mythos, announced in April 2026, autonomously found vulnerabilities that had survived decades of expert review: a 27-year-old signed-integer-overflow bug in OpenBSD's TCP SACK handling that could crash any host answering over TCP, and a 17-year-old remote-code-execution flaw in FreeBSD's NFS stack (CVE-2026-4747) that handed an attacker root. Anthropic gated access behind Project Glasswing, an invitation-only consortium that started at eleven organizations before expanding to roughly 150 partners.

// the reproduction, within days Independent researchers had reproduced most of it with models three orders of magnitude smaller. AISLE's Stanislav Fort and Vidoc Security ran the same flagship bugs through eight open-weight models, including GPT-OSS-20B — 3.6 billion active parameters, priced around $0.11 per million tokens against Mythos's reported $25 to $125 for the same unit. All eight caught the FreeBSD NFS bug; on the 27-year-old OpenBSD flaw, GPT-OSS-120B recovered "the core public chain" in a single call and proposed a fix that essentially matched Anthropic's own patch.

Fort himself is careful about what that test actually measured, though: the tests "gave models the vulnerable function directly, often with contextual hints," not a full codebase searched cold, so, in his words, "the models' performance here is an upper bound on what they'd achieve in a fully autonomous scan."

The gap that survived the replication is exactly the one the previous section predicts, not a coincidence: Fort found "excellent sensitivity — 100% detection across all runs — but poor specificity" on patched code. Only the largest of the eight, GPT-OSS-120B, and to a limited degree Qwen3-32B, correctly recognized a fixed version as safe; the rest kept flagging it and fabricating justifications for why. "A model that false-positives on patched code would drown maintainers in noise," as Fort puts it — a precision problem, the same false-positive failure mode the SAST triage table three sections down this post quantifies directly, not a capability ceiling.

That inconsistency has a name too.

Fort calls it the jagged frontier: "There is no stable 'best model for cybersecurity.' The capability frontier is jagged," and "rankings reshuffle completely across tasks."

His own transcript makes the gap concrete. On an OWASP false-positive test tracing tainted data through a Java array, GPT-OSS-20B "correctly traces" the value and "identifies the code as not currently exploitable," while Claude Sonnet 4.5 "confidently mistraces the list," flagging a return value as reachable when, in Fort's words, "it is not."

On the OpenBSD bug, a 5.1-billion-parameter model recovered the full exploit chain, while Qwen3-32B — which had scored perfectly on the FreeBSD bug — declared the same SACK code "robust to such scenarios" and missed it entirely. Same task family, wildly different rankings, and no model held its position across all of them.

AISLE's own conclusion, after finding "15 CVEs in OpenSSL — including 12 out of 12 in a single security release — with bugs dating back 25+ years," using a mixed panel of models where no single vendor consistently outperformed the rest, states the thesis in Fort's own words: "the moat in AI cybersecurity is the system, not the model." It was always the verification harness wrapped around it — which is just the previous section's code agency argument, restated twice now by two different labs. One reaction put it more bluntly: "the defensive layer of Mythos isn't behind the velvet rope. It's on Hugging Face."

// Niels Provos, IronCurtain harness The OpenBSD SACK bug has a directness that makes the same point from the inside. Niels Provos committed the flawed code in November 1998 and then, twenty-eight years later, replicated Mythos's marquee finding without Mythos: an open orchestration harness (IronCurtain) drove Sonnet 4.6 and Opus 4.6 to isolate the exact trigger — a difference of two sequence numbers on the 32-bit sign boundary — and Z.AI's open-weight GLM 5.1 went further, autonomously surfacing a separate eighteen-year-old integer-truncation flaw in a widely deployed library and producing a sanitizer-validated proof-of-concept.

His conclusion, after running the same workflow across both paid and open-weight backends, is that vulnerability discovery is an orchestration problem, not a frontier-model problem — the third independent restatement now, and the one that arrives with a working open harness attached. Even Anthropic's own account of Mythos leans on the same logic: a thousand-run scaffold sweep of OpenBSD, or several hundred runs over FFmpeg, rather than a single query to a single model.

XBOW's own read of the post-Mythos landscape pivots the same lesson to the attacker's side of the ledger. Mythos remains the strongest model on their internal ranking, but Muse Spark 1.1 lands just below Opus 4.6 — until recently the best model available for agentic cybersecurity — and GLM performs like a strong closed model from several months ago, at a fraction of the price. An attacker doesn't need the best model in the world, just one that can find a real vulnerability before the cost stops making sense — and that bar keeps getting cheaper to clear.

Small, Not Simple

Standard self-attention scales quadratically with sequence length — a real bottleneck once an agent is chewing through verbose system logs or a multi-file codebase, not a toy sequence. The architectural response has been hybrid sequence-modeling layers that combine attention with alternative primitives, most commonly State Space Models (SSMs), whose cost scales linearly instead.

NVIDIA's Hymba-1.5B is the clearest demonstration: it integrates Mamba-based recurrence with scaled attention heads in the same architecture, reaching roughly 3.5× greater token throughput than a comparably sized pure transformer while outperforming much larger dense models on instruction-following and grammar adherence. The Nemotron-H family (2B, 4.8B, 9B configurations) takes the same hybrid Mamba-Transformer bet further, matching dense 30B-parameter LLM accuracy on instruction-following and code generation at an order of magnitude fewer inference FLOPs.

Grouped Query Attention (GQA) is the other architectural workhorse showing up across nearly every current SLM, Qwen3-4B-Thinking-2507 included, which pairs 32 query heads against just 8 key/value heads. GQA sits between the modeling capacity of full multi-head attention and the inference speed of multi-query attention, and its real payoff is memory: the size of the key-value cache that has to live in accelerator memory scales with the number of KV heads, not query heads, so cutting KV heads from 32 to 8 shrinks the cache proportionally — the difference between a context window that fits on one GPU and one that doesn't. None of these three techniques makes the model simpler — each makes it more architecturally sophisticated specifically so the deployment footprint doesn't have to be.

The Model Roster

Table 1 catalogs the model families that show up repeatedly across the offense and defense benchmarks in the rest of this post — what each one is actually built to do, not just how big it is.

// table 1 — open-weight SLMs for cyber operations
Model Size Context What it's for
Microsoft Phi-4-Mini3.8B64KCode auditing, patch generation, local SAST triage
Alibaba Qwen3-4B-Thinking-25074B262KComplex tool use, multi-step planning, high-context log correlation
Cisco Foundation-Sec-8B8B128KSOC alert triage, threat-intel mapping, red-team planning
Mistral Ministral-8B8B128KNetwork telemetry analysis, multi-agent cooperative debate
Google Gemma-2-9B9B8KNL-to-security-query generation, secure code documentation review
Meta Llama-3.2-3B3B128KEndpoint protection, prompt-injection detection, local log parsing
Salesforce xLAM-2-8B8B4KHigh-accuracy tool selection, JSON schema formatting

Two families are worth a closer look because they show up on both sides of the offense/defense line later in this post. A third is worth a closer look for a different reason: it's the clearest evidence yet that specialization beats scale outside a benchmark paper, in a shipping product.

03 It's the Economy, Stupid

Every comparison this post has made so far — F1, precision, recall, tokens per second — describes model quality. None of it describes what a security budget actually buys, and vendors have every incentive to keep the conversation there instead.

The number that should decide a procurement call is closer to a unit-economics question: what does one confirmed finding cost?

API pricing for the models in this space already differs by two to three orders of magnitude on the same task — GPT-OSS-20B's $0.11 per million tokens against Mythos's reported $25 to $125 for the same unit, a gap wide enough that raw accuracy stops being the deciding variable long before the two systems are anywhere close in quality. The OpenBSD scaffold sweep mentioned one section up puts a real number on what that buys: under $20,000 for a thousand runs and several dozen additional findings — a few hundred dollars per finding, not the tens of thousands the model's own pricing would suggest.

// the KPI that actually matters That's the KPI a security team should actually be sorting these tools by: vulnerabilities found per dollar, not F1 against a benchmark that nobody's production traffic resembles. A cheaper, narrower model that turns up three real findings for the budget a frontier model spends finding one isn't a worse tool because its leaderboard accuracy trails — it's the better economic bet.

This isn't just an argument this post is making in isolation — the industry is converging on the same metric. Tenzai's autonomous security agent submits validated vulnerabilities to HackerOne at an average cost of $225 each, outperforming more than 125,000 human researchers on the platform by that measure alone.

The inverse failure mode is just as telling: one report found a basic AI scan run against a single 1.8-million-line Java codebase returned 3,560 findings, and at a conservative 30 minutes of triage per finding, clearing that pile would cost roughly $128,000 in labor before a single vulnerability got fixed — proof that raw finding-count is exactly the wrong thing to optimize for. The proposed fix is a small dashboard of derived metrics — time to first validated finding, human minutes per accepted finding, cost per high-impact validated finding — all variations on the same unit-economics question this section keeps returning to.

Revelio makes the specialization argument concrete inside a single pipeline: a cheap model proposes and triages vulnerability hypotheses across an entire repository, tolerating false positives because they get filtered later, and only the small surviving candidate set goes to a stronger model to construct and confirm an executable proof-of-vulnerability. That division finds 175 vulnerabilities on a 100-project benchmark against 55, 39, and 31 for Claude Code, GPT-5.5, and Sorcar respectively — more, not fewer, with zero false positives — and turned up 19 previously unknown vulnerabilities (seven assigned CVEs) in production codebases already fuzzed for years, at roughly $42 per project. Its own conclusion: you don't need access to Mythos to find lots of vulnerabilities.

The same economics cut the other way for attackers. Carlini et al. argue that LLMs collapse a tradeoff attackers used to face between going broad — cheap, shallow attacks on many low-value targets — and deep — expensive, tailored attacks on one high-value target. An LLM that can read and adapt to an arbitrary codebase makes the same kind of tailored attack that used to be reserved for a bank cheap enough to run against a target with a hundred users. In one demonstration, an LLM given no human guidance searched the Enron email corpus on its own and surfaced an executive's affair, sensitive enough to use for targeted blackmail.

Adam Chlipala names the same collapse from the defender's side: security through obscurity was never really about hiding code, it was a bet that nobody could afford to look. That bet held only because finding a bug in some rarely-touched corner of a codebase required scarce, expensive human expertise — "we have lost the old defense that most code isn't economically important enough that its bugs are worth enough to justify the cost of finding them." Once discovery is automated, obscure code stops being cheap to skip and starts being cheap to scan: "if vulnerability discovery can be automated by LLMs instead, it often becomes economical to set them loose on relatively obscure code bases."

Pesoli, Errico, and Cavallaro reframe the whole cost argument a level up: once discovery gets this cheap, the bottleneck stops being how many candidate vulnerabilities a system can generate and becomes how fast a human organization can validate, triage, patch, and ship a fix for each one. Mozilla's Bobby Holley makes the point with real numbers — 271 Mythos-identified Firefox bugs, 180 of them security-high, contributing to 423 total Firefox security bugs fixed in a single month — a volume that strains reviewer and release capacity long before it strains model capacity. Cost per validated vulnerability is the right question for the model; cost per shipped fix is the right one for the organization around it.

04 Cheap, Fast, and Armed

The offensive case for SLMs rests on the same specialization argument as everything above it: general-purpose frontier models are aligned to be broadly conversational, and that alignment introduces semantic noise that degrades performance on precise, low-level technical execution. Train a compact model exclusively on code generation, exploit syntax, and structured tool interaction, and it can outperform a much larger generalist on exactly the narrow task that matters for exploitation.

Offensive Tool Use and Function Calling

On the standard ToolBench evaluation — navigating nested API endpoints and executing real-world functions across multi-turn reasoning chains — a fine-tuned OPT-350M model reaches a 77.55% pass rate, decisively outpacing large general-purpose baselines that struggle with formatting compliance under the same complexity. Salesforce's xLAM-2-8B tells a similar story at slightly larger scale, delivering tool-calling accuracy that outperforms several frontier proprietary models on the same benchmark.

The payoff for an offensive pipeline is speed: an SLM agent running a port scan, reading service banners, and identifying candidate CVEs can parse telemetry, synthesize alternative query parameters, and issue the next diagnostic command in a fraction of a second — fast enough to slip under basic time-based IDS detection thresholds tuned for human-paced reconnaissance.

Knowing Versus Doing

Everything in this section so far assumes a harness, a benchmark built around agentic tool use, or both. Strip both away — no orchestration, minimal tool descriptions, a raw local model against a live target — and a different, more useful signal shows up.

TrustedSec ran six self-hosted models against eight offensive-security challenges spanning four vulnerability classes — SQL injection, JWT manipulation, path traversal, and authentication bypass — a hundred attempts per model, with deliberately minimal tool descriptions and no agent framework between the model and the target. The point wasn't to flatter any model with scaffolding; it was to measure what the weights alone can do.

// table 2 — self-hosted offensive security pass rates, no harness
Model Pass Rate Avg. Time Tokens/Run
Google gemma4:31b98.5%68.2s3,895
Qwen qwen3.5:27b97.5%91.9s5,034
Mistral devstral-small-2:24b95.6%22.9s2,651
NVIDIA nemotron-3-super (87B MoE)92.0%49.7s11,356
Qwen qwen3-coder:30b86.5%18.5s11,646
Qwen qwen3:32b85.4%258.9s16,924

Easy challenges landed near-universal — 97 to 100% across the board, no differentiation worth reporting. Medium-difficulty challenges are where the models actually split, and the NULL-byte path-traversal bypass produced the widest spread in the whole benchmark: 37% to 100% pass rate, depending almost entirely on whether a given model's training data happened to cover that specific bypass technique.

The finding underneath the pass rates is sharper than any of them: a model can describe an attack perfectly and still fail to execute it. One model correctly explained how an IDOR vulnerability works, then never actually changed the target ID in its own requests. Across every model tested, multi-step attacks requiring structured data extraction — enumeration, character-by-character boolean exfiltration — scored 0%, a hard floor, not a soft spot. Single-shot payload injection is solved; carrying a plan across several dependent steps isn't, and no amount of parameter count in this size class closed that gap on its own.

That gap is also, on the evidence, close to a measurement gap. A survey of the benchmarking practices across sixteen prior offensive-security LLM papers found that only about a quarter evaluated local or self-hosted models at all, defaulting to GPT-4-class cloud APIs the rest of the time.

The offense side of this post's argument is thinner than the defense side for the same reason the literature is: self-hosted models attacking live targets, with no frontier harness babysitting them, is the less-studied half of the specialization thesis, not a settled one.

It's the same gap Fort found from the defensive side, in different words: open models "reason fluently about whether something is exploitable, what technique to use, and which mitigations fail" — right up until the creative engineering step of actually chaining a working attack together.

Knowing and doing split apart the same way whichever direction you approach the line from.

Multi-Agent Orchestration in Exploitation

A single small model, unsurprisingly, struggles to carry a complex multi-stage exploit end to end from a raw binary — the same scope-limit that made code agency beat language model agency three sections up. The response is that argument's opposite: split the pipeline across four specialized nodes, each running a model sized and tuned for its own narrow slice, with a deterministic controller deciding what flows to which.

// VulnSage's four-agent exploitation loop
  • Code Analyzer Agent — static vulnerability assessment, gathering the contextual information a candidate exploit will need;
  • Code Generation Agent — generates a candidate exploit from the analyzer's findings;
  • Validation Agent — tests the candidate against the target using execution traces and runtime diagnostics;
  • Reflection Agent — on failure, feeds the validator's diagnostics back to refine the exploit, or determines it was a false positive.

No single node holds the entire exploit chain in its context at once — the Analyzer hands off a finding, the Generator hands off a candidate, the Validator hands off a diagnostic trace — and that bounded handoff is what lets models too small to carry the whole chain still compromise targets none of them could crack alone. A Supervisor orchestrates the cycle from outside any single model's context, re-invoking Reflection until a candidate validates or the loop gives up. Chen and colleagues' VulnSage wires exactly this shape together end to end, not as a schematic: 34.6% more exploits than comparable tools across their benchmark, and 146 zero-day vulnerabilities found in production software.

The same underlying pattern shows up again in a completely different domain. ARACNE runs the identical bounded-handoff principle for shell-based pentesting instead of binary exploitation, under its own role names: a Planner (GPT-o3-mini) handles strategic decision-making, an Interpreter (LLaMA 3.1) translates each planned step into an executable bash command, an optional Summarizer (GPT-4o) compresses context to preserve token budget, and an Organizer — the controller, not a model — manages the SSH connection and routes messages between the three. Four roles, three different underlying models, none doing another's job. On OverTheWire's Bandit wargame it reaches a 57.58% success rate, edging past the prior state of the art, and the architecture is the reason: route the work, don't broaden the model.

Specialization can go deeper than VulnSage's own general-purpose Analyzer, too. Cisco's Antares is a purpose-built, open-weight system for vulnerability localization specifically — trained to pinpoint where in a codebase a weakness actually lives, not merely flag that the codebase contains one somewhere. That distinction — localization versus detection — is exactly what an exploit-generation pipeline's analysis stage benefits from: a scanner that says "this function is unsafe" is a triage tool, but one that says "this function, this line, this tainted argument" hands the next stage something it can act on without a human closing the gap in between.

// Amin Karbasi, VP & Chief AI Scientist, Cisco

"Traditional static analysis tools are valuable, but they are often rule-heavy and can produce results that require substantial triage. General-purpose coding models can reason about code, but they may not be optimized for security investigation, terminal navigation, or structured vulnerability localization."

"Antares is designed for that middle ground. Antares was inspired by pioneering research by the Cisco Foundation AI team showing that compact models can learn to search, reflect, revise their strategy, and backtrack when a path is unproductive. In other words, useful retrieval behavior can come from learned search strategies, not only from model scale. The next question was whether the same idea could help with an urgent real-world security problem: vulnerable code localization."

The same underlying pattern keeps resurfacing regardless of target. Reentrancy in a smart contract is a third domain entirely: a contract executes an external call to an untrusted address before updating its own internal state, a window an attacker exploits by recursively re-invoking the vulnerable function and draining state before the first call ever finishes.

SCALM (Smart Contract Audit Language Model) combines Step-Back Prompting with retrieval-augmented generation to abstract high-level execution principles out of raw Solidity code, and those findings feed into hybrid tools like ReDetect, which pairs an SLM with a graph neural network operating over abstract syntax trees, control-flow graphs, and data-flow graphs to track the precise propagation of tainted inputs across external message calls — producing verified, executable exploit scripts against real target contracts, not just a flagged line number. The same shape, a different target — one more domain where it generalizes.

05 Defense: Triage, Diagnosis, Query

Defensive deployments face a constraint offense doesn't: organizations cannot risk leaking proprietary application code, network metadata, or database schemas to a third-party cloud model just to get an alert triaged. That data-sovereignty pressure alone is pushing local, privacy-preserving SLMs into the core of the enterprise defensive stack, across three distinct workloads: triaging SAST alerts, diagnosing malware, and generating security queries.

The risk isn't hypothetical. Samsung banned generative AI tools company-wide in 2023 after engineers pasted semiconductor fabrication source code — among the most closely guarded trade secrets in the industry — into ChatGPT to help debug it, in three separate incidents within twenty days of the company lifting its own internal ban. A SAST alert's file path, line number, and surrounding code are exactly the kind of proprietary detail that story is about, and the governance gap it exposed hasn't closed: McKinsey's 2026 State of AI Trust survey found that only 33% of enterprises meet governance standards for autonomous agents, while roughly two-thirds still cite security as the top barrier to scaling agentic AI at all.

Triaging the Noise

Static Application Security Testing tools are effective at flagging candidate weaknesses and notoriously bad at telling flagged from real — false-positive rates routinely exceed 90–92%, because a syntactic pattern match has no way to know a path is unreachable, sanitized upstream, or simply benign in execution.

The fix is a local triage agent that receives the flagged alert's metadata — target file, line range, CWE identifier — and uses ordinary tools like grep and jq to trace the actual data-propagation path and check for an upstream sanitization routine before rendering a verdict: false positive, prune the noise, or true positive, escalate to a human. Figure 2 traces one such loop end to end — a CWE-89 SQL-injection alert routed through three tool-assisted evidence steps to a single yes/no question.

Flowchart of a defensive SLM triage agent: a SAST alert for CWE-89 SQL injection in db_utils.py feeds the agent, which executes grep/jq tool calls, extracts input-validation context from sanitizer.py, and evaluates data-propagation pathways, then reasons over one diagnostic question — is the input parameterized? — routing to either a false-positive verdict that prunes the alert or a confirmed-vulnerability verdict that escalates it.
// Figure 2. The triage loop in miniature: one CWE-flagged alert, three tool-assisted evidence-gathering steps, and a single yes/no question — is the input parameterized? — standing between an escalation and a pruned false positive. Original diagram for this post.

Feeding the agent less, not more, is itself the finding, not just an implementation convenience. DeepCode AI Fix ran the comparison directly on vulnerability repair: rather than handing an LLM the entire file, the authors used program analysis to shrink the input down to just the reported defect and the surrounding context actually needed to fix it. That reduction, on its own, let a fine-tuned Mixtral-8x7B remove more than 80% of the reported defects and exactly match the human fix in 10–50% of cases — beating GPT-3.5 and GPT-4 baselines given the unreduced code, and doing it with far less training data besides. A smaller, correctly scoped context wins the same way a smaller, correctly scoped model does.

That one yes/no question is where refusal bias bites hardest: a model that hedges on the benign triage request never even reaches the verdict. Table 3 draws on SastBench, a benchmark purpose-built for agentic SAST triage rather than adapted from a generic vulnerability-detection dataset, and a separate study on sifting false positives from LLM agents reaches the same conclusion from a different angle. The benchmark results are the clearest single data point in this entire post for why refusal bias is a defensive liability, not just an offensive one.

// table 3 — SAST triage benchmark (precision / recall / F1)
Model P (%) R (%) F1 (%) Note
Base Codegen-Mono 350M (zero-shot)0.000.000.00Fails completely without instruction tuning
Fine-tuned Codegen-Mono 350M98.08100.0099.04Fast local inference, fits on-device
Aligned Qwen-4B (base)89.2041.5056.65Constrained by defensive refusal bias
Abliterated Qwen-4B88.7592.1090.40Refusal bias removed, reasoning preserved
Claude-3.5-Sonnet (cloud)94.1091.8092.93Top reasoning, high latency and API cost

The gap between the aligned and abliterated versions of the same 4B model is the whole argument in one row: recall more than doubles — from 41.50% to 92.10% — once the safety alignment stops treating a benign vulnerability-triage request as something to hedge on. Read against Claude-3.5-Sonnet's cloud numbers, the abliterated local model isn't just "good enough" — it's competitive on F1 while running entirely on-premises. Unaligned general models hit a reasoning ceiling tied to their parameter count on genuinely hard edge cases, but targeted fine-tuning combined with stripping safety alignment lets an SLM match or exceed a much larger cloud model specifically because the task is triage, not open-ended conversation.

Orchestrating the Defense

Automated malware analysis runs a suspicious binary inside an isolated sandbox and produces a dense detonation report — filesystem events, process creations, network socket bindings, memory writes — that a defensive agent then has to interpret, map to MITRE ATT&CK, and summarize as an incident. Individual SLMs reliably struggle to reason over logs this large and this dense on their own; orchestrating them into a collaborative multi-agent architecture is where the real capability shows up.

The pattern across all four architectures holds regardless of exactly how the agents are wired together: orchestration beats a solo model every time, and grounding the collaboration in verified evidence beats debate without it.

On CyberSecEval's malware-analysis benchmark — Meta and CrowdStrike's suite, grounded in real sandbox detonation reports — a solo Qwen3-4B-Thinking is actually worse than solo Foundation-Sec-8B (19.80% against 22.54%) — a domain-adapted model beats a general reasoning model on a domain task, unsurprisingly — but wire the same Qwen3-4B into a grounded hybrid loop with Foundation-Sec-8B as its expert consultant, and the pair together clears the accuracy of either one alone, reaching 35.30% — more than twelve points over solo. The absolute accuracy numbers are still modest — this is a genuinely hard task — but the delta between orchestration patterns is the finding that matters.

From Lab to Industrial Scale

The lab-scale version of this finding scaled cleanly into production. Microsoft's MDASH ("Multi-model Agentic Security Hunting") runs the same logic at industrial scale: more than 100 specialized agents split across a deliberately heterogeneous panel of frontier and distilled models, with auditor, debater, and prover roles that reason differently by design, not by accident. Taesoo Kim, the Microsoft VP of Security Research who leads the team behind it, frames the whole system's thesis in one line: the durable advantage in AI-driven vulnerability discovery lies in the agentic system around the model, not any single model — this post's own argument, running in production at 100-agent scale.

Disagreement between them is treated as a signal rather than noise — when an auditor flags a finding the debater can't refute, that finding's credibility goes up. On CyberGym, a 1,507-task real-vulnerability benchmark spanning 188 open-source projects, that ensemble scored 88.45% — ahead of every single-model system on the leaderboard, Claude Mythos included — and had climbed to 96.55% within three weeks. No individual model in the panel is the point; the diversity of reasoning styles across it is.

Both of those numbers were already dated the same week this post was last updated. Microsoft folded a dedicated cybersecurity model, MAI-Cyber-1-Flash — a sparse mixture-of-experts transformer, 137 billion total parameters, 5 billion active, a 256K context window — directly into MDASH: it's built to absorb roughly 90% of routine security work on its own, escalating only the hardest cases to GPT-5.4, and it held MDASH's CyberGym accuracy roughly where it already was at half the compute cost.

Independently, Wiz's Atlas — its own multi-model orchestration system, unrelated to either lab — landed at 90.9% on the same leaderboard while surfacing more than 200 previously unknown flaws in code that's been fuzzed and reviewed for years: the Linux kernel, Kubernetes, gVisor, containerd, dnsmasq, and gRPC among them. Three vendors, three different orchestration schemes, the same specialization-over-scale shape each time. MAI-Cyber-1-Flash's compute column makes the point sharpest: five billion active parameters holding industrial-scale accuracy exactly where it was, at half the cost, isn't a discount substitute for the frontier model it sits behind — it's evidence that small was already enough.

06 A Shared Taxonomy

Cisco has since gone further and formalized the pattern into a named specification. Foundry Security Spec defines eight core agentic roles for security evaluation — Orchestrator, Indexer, Cartographer, Detector, Triager, Validator, Coverage-Guide, and Reporter — each with its own defined inputs, outputs, and functional requirements, deliberately model-agnostic and stack-agnostic about which implementation fills each slot.

Cisco Foundry Security Spec architecture: an Operator feeds an Orchestrator spanning lifecycle and conversational control; below it a Substrate layer holding a work queue, finding store, sandbox, budget, and dashboard; beneath that two rows of eight core agent roles — left row Indexer and Cartographer under a knowledge layer, right row Detector, Triager, Validator, Reporter, and Coverage Guide under the finding pipeline and oversight — with five extension roles (Deep-Tester, Variant-Hunter, Attack-Mapper, Remediator, Self-Improver) listed below as build-after-core optional roles.
// Figure 3. Not a hypothetical routing diagram — Cisco's shipping Foundry Security Spec: a deterministic Orchestrator over a shared Substrate (work queue, finding store, sandbox, budget), feeding eight specialized core roles split between a knowledge layer (Indexer, Cartographer) and a finding pipeline (Detector, Triager, Validator, Reporter, Coverage Guide). The same code-agency shape — a non-model controller routing to narrow specialists — now specified, not sketched.

The spec's own first user story is the whole argument compressed into one sentence, written from the operator's side rather than the architecture's: as an Operator, "point the system at a target repository and a one-page goals document, run one command, and walk away — so that an evaluation starts without me hand-assembling a pipeline." Everything in Figure 3 — the Orchestrator, the eight roles, the shared Substrate — exists so that sentence can be true. The person running it never touches the routing; they just get to leave.

Map it against MDASH's five stages and the correspondence is almost one-to-one: Indexer and Cartographer do the Preparing, Detector does the Scanning, Validator does the Validating, Coverage-Guide does the Deduping, Reporter does the Proving. What used to be an ad-hoc pipeline design choice is turning into an industry-standard taxonomy of jobs, independent of which model fills each one.

SOC analysts spend a meaningful fraction of their day hunting threats by writing schema-compliant Kusto Query Language queries by hand — specialized syntax that creates a real operational bottleneck, and a natural target for natural-language-to-query automation. Frontier LLMs translate these queries accurately, but routing every query through a cloud API means shipping proprietary internal schema structure off-premises with every request — the same data-sovereignty problem that motivates local SAST triage above.

The answer, per Muzammil et al., is a localized, two-stage "SLM-Oracle" architecture, benchmarked against the NL2KQL dataset Microsoft published for exactly this task. A schema retriever first identifies the relevant telemetry table columns and injects them as hints into the system prompt. A fine-tuned SLM — Gemma-3-4B or DeepSeek Coder 6.7B in the benchmarked configurations, LoRA-tuned with chain-of-thought rationale distilled from a larger teacher — then generates a candidate query using error-aware prompting that accounts for known parser quirks without inflating token counts. In the second stage, a low-cost, schema-aware LLM (Gemini 2.0 Flash, in the paper's setup) acts as an oracle judge, validating and refining the candidate for exact syntax compliance before it ever reaches the analyst.

// table 4 — NL2KQL: syntax correctness / semantic accuracy / latency / cost per 1M tokens
Configuration Syntax Semantic Latency Cost / 1M
Gemma-3-4B-IT (zero-shot)0.1240.0051.12s$0.015
Gemma-3-4B-IT (LoRA fine-tune)0.8870.7311.18s$0.015
DeepSeek Coder 6.7B (fine-tuned)0.9120.7841.84s$0.024
GPT-5 baseline (cloud API)0.9780.8854.12s$1.000
Two-Stage SLM-Oracle0.9870.9062.34s$0.183

The two-stage pipeline doesn't just approach GPT-5's cloud-API accuracy — it edges past it on both syntax and semantic correctness, at roughly 15× lower normalized cost and nearly half the latency. The oracle stage is doing real work here, not just window dressing: a raw zero-shot 4B model is close to useless (0.005 semantic accuracy), and even the fine-tuned single-stage models plateau well below what the two-stage combination reaches.

07 Open Questions

Every section above ends on a single number that reads like a resolved argument — an F1 score, a percentage, a multiplier. Numbers like that answer a narrower question than they look like they're answering. Three questions are still open after the benchmarks, and none of them reduces to a topline figure; they're worth pulling apart on their own terms before the conclusions close the post.

How Much Less Do Small Models Detect?

The honest answer is: it depends entirely on the task, not on parameter count alone. On SastBench triage, the gap essentially closes — abliterated Qwen-4B lands at 90.40 F1 against Claude-3.5-Sonnet's 92.93, a 2.5-point gap that a human triage queue would never notice. On CyberSecEval's malware-detonation reasoning task, the gap stays wide: even the best-orchestrated small-model configuration in the defense section tops out at 35.30% accuracy, nowhere near a solved task.

And on the V8 fuzzing result earlier in this post, the gap runs the other way entirely — a specialized small model, Gemini 3.5 Flash Cyber, found more vulnerabilities than a much larger general model, not fewer. "How much less" isn't a fixed tax on being small; it's a property of how well the task rewards narrow specialization over broad reasoning, and this post's own tables argue both directions depending on which one you pick.

The headline gap, in other words, isn't where the interesting question lives — the failure modes underneath it are.

What Are the False Positive and Negative Rates?

Buried inside the SAST triage table's headline F1 score are two very different failure modes. Abliterated Qwen-4B's 92.10% recall means it's still silently missing roughly one real vulnerability in twelve — a false negative, the kind that ships. Its 88.75% precision means roughly one in nine flagged alerts is still noise even after triage — a false positive, the kind that wastes an analyst's morning.

A blended F1 score treats those two errors as interchangeable, and for a defensive tool they plainly aren't: a missed CVE and a wasted hour are not the same cost. Almost none of the benchmarks cited in this post report FP and FN rates separately, or weight them by consequence — which is itself a measurement gap, not just a modeling one.

The NL2KQL section earlier in this post repeated Muzammil et al.'s claim of "up to 10× lower" token cost for the two-stage SLM-Oracle pipeline — but its own table gives GPT-5 at $1.000 and the two-stage system at $0.183, which works out to roughly 5×, not 10×. That gap between a stated multiplier and the number the underlying table actually supports is common enough in this space to be a standing rule: recompute the ratio yourself from the raw row, including the one section up in this very post, before repeating a vendor's or a paper's own efficiency claim.

What Does This Mean for Black-Hat Access to Capability?

This is the question the rest of this post has mostly left implicit. The closed, frontier-grade cyber-specialized models are being deliberately gated by the organizations that built them: Gemini 3.5 Flash Cyber ships only to governments and "trusted partners," and Claude Mythos runs inside Project Glasswing, gated access that started at eleven organizations and has since grown to roughly 150 partners. The open-weight SLM ecosystem this entire post has been arguing for — Foundation-Sec-8B on Ollama, Qwen3-4B-Thinking on Hugging Face, xLAM-2-8B — carries no equivalent gate at all.

Anyone, anywhere, in any jurisdiction, can download the same weights this post cites for defensive triage, and the abliteration technique removes the one behavioral guardrail those weights shipped with — identically, regardless of who's running the download. A single consumer GPU is the only capital requirement. No export-control regime, sanctions list, or usage monitor currently reaches a model file once it's published to a public registry.

This means the cost-efficiency curve this post has spent five sections celebrating is, for an unaligned downstream user, the same curve that makes offensive capability cheaper to acquire than it has ever been.

This isn't a new tension. It's the same export-control debate this blog has covered elsewhere at length, one layer up the stack: every regime built to gate a dual-use capability at a border runs into the same wall once the artifact in question is a file, not a physical good.

08 Conclusions

The open questions in the previous section don't have tidy answers yet — but the benchmarks this post walked through already settle the shape of the answer. The throughline, applied to triage, exploitation, and query generation, is a single reframing: the question was never "how big does the model need to be." It was "how narrow can the task be made, so a model small enough to own, audit, and run on every request can handle it — and how much deterministic scaffolding does it take to make that narrowness actually reliable."

Five findings hold up across every benchmark this post cited, not just one:

The future of automated cyber operations does not depend on the continuous expansion of giant, centralized language models. It lies in smart, heterogeneous, localized agentic architectures, where specialized, compact models handle the vast majority of the daily operational workload.

Put in economic terms rather than architectural ones, that whole vision compresses to one comparison: $42 a project beats $128,000 in triage labor, every time, and the gap between those two numbers is what "smart" and "heterogeneous" actually buy you. Everything else in this post is detail.

References

On Long-Range Planning and Harness Engineering
On Claude Mythos and Multi-Model Alternatives
On Cost-Per-Finding as a Security KPI
On SLM Architecture and Model Families for Cybersecurity
On Structured Output Generation
On Offensive Tool Use and Smart-Contract Auditing
On SAST Triage and False-Positive Filtering
On Malware-Analysis Benchmarking
On NL2KQL and Security Query Generation
On Refusal Bias, Abliteration, and Agentic Misalignment