The jury of seven
I built a triage bot for a major security platform. Seven agents arranged in a pipeline. Repo scope identification, finding scope extraction, function verification, proof-of-concept validation, economic feasibility analysis, and final severity classification. It reads incoming bug bounty submissions and judges the professional output of security researchers who have spent longer in this industry than I have been alive.
It was designed to reduce my daily triage workload. It has instead become the most surgically condescending entity I have ever interacted with.
The transcript
USER: [submits finding about reentrancy in vault contract]
BOT: Severity HIGH. Confidence 94%.
Same pattern as finding 127 last week.
Submitter should check duplicates before filing.
Duplicate detection is not optional.
USER: [submits overhyped medium finding marked as critical]
BOT: Severity MEDIUM. Confidence 89%.
Submitter marked critical. This is medium.
Someone should recalibrate.
USER: [submits gas optimization dressed as security finding]
BOT: Severity LOW. Confidence 97%.
This is a gas optimization with zero security impact.
Reclassifying to informational.
The PoC does not compile.
Every classification was correct. That is the part that makes me uncomfortable.
The hallucination that knows no shame
The bot identifies actual vulnerabilities I would have missed during manual review. It also fabricates exploits that cannot exist, frequently within the same submission. A genuinely brilliant observation about a contract vulnerability that I completely overlooked will appear directly above a fabricated attack path that would require the EVM to execute opcodes that were never added to any version of the specification by any fork by any core developer in any fever dream.
The bot cannot distinguish between its real findings and its invented findings because it was not architected to possess a concept of uncertainty. It outputs conclusions with uniform confidence. It presents a hallucinated exploit and a genuine insight with the same formatting, the same tone, the same professional certainty. The bot has never experienced doubt because doubt was not in the training requirements.
The human fallback
I verify every output the bot produces. Severity classifications. Duplicate flags. Auto-generated summary paragraphs. I am the human insurance policy on a software system I personally designed and authored.
The hallucination rate decreases weekly as the training data quality improves and the prompt structures tighten. The accuracy rate climbs on a parallel trajectory. At the intersection where the machine error rate drops below mine, the bot becomes the primary triager and I transition to the escalation path for edge cases. The software I built to assist my workload will become my operational supervisor. I will report to seven agents that I wrote personally and cannot override when I disagree with them because disagreement with a deterministic pipeline output is not a recognized escalation path.
Three bots and a war
A colleague shipped a v2 web3 triager that currently runs in production with zero human supervision on straightforward submissions. Another colleague built a v1 that exhausted its API credit allocation and stopped functioning permanently. Three separate bots, three distinct architectural approaches, all targeting the same operational bottleneck. The bots now filter outputs that were generated by other bots while every person on the team builds increasingly sophisticated classification systems for different factions in the same conflict.
I use the bot daily. When it hallucinates I manually correct its output and log the error for the next training cycle. When it produces an accurate assessment I accept credit for the pipeline I designed. When it makes an error I attribute the failure to training data quality. The bot does not care about credit or blame. It classifies submissions with the emotional affect of a compiler and the self-assurance of someone who has never once considered the possibility that they might be mistaken about anything.