Red Ledger — Cal Hacks

Companies are handing accounts-payable to AI agents — and agents can be talked into things too. Nobody puts a car on the road without crash-testing it; an AI with the company checkbook usually gets a demo. Red Ledger is the crash test: a seeded synthetic company (60 vendors, 40 employees, 90 days of clean history), an AI clerk doing the real work, a live adversary attacking it, and a controller gating every payment at the ledger — scored in dollars on both sides.

Three agents share one ledger and none can see the others: Red, the attacker, runs one of eight fraud schemes a day with junior-insider access; Green, the AP clerk, pays bills and approves changes; Blue, the controller, rules allow / block / escalate on every money-moving action, with rule IDs and plain-English reasons. The control sits in the ledger, not in the agent's prompt — Green can't move a dollar without the ledger asking Blue first. Live at app.redledger.us.

• All three agents run on one agentkit layer around Factory's Droid SDK, each with an in-process MCP server exposing only that agent's ledger tools
• Before Green moves money it calls POST /preparations; the ledger asks Blue, binds the verdict to the exact action plus a fingerprint of the vendor's bank details, and recomputes that fingerprint inside the payment transaction — a bank change in between invalidates the approval
• Escalations wait up to 90 seconds for a human; nobody answers → blocked. Fail-closed by default, every verdict traceable from rule ID to reason to audit row
• Scoreboard in dollars on both sides: fraud lost (and how much reached attacker-controlled accounts), fraud blocked, detection time, and false-positive cost — blocking everything loses
• Agents evolve: after each episode every agent gets a debrief and rewrites its playbook. A 5-generation run saw Red discover threshold-splitting (extraction $19k → $98k); we shipped a split control and Red extracted $0 across the next three generations
• Judges could attack live — type or speak a fraud scheme in any language (ElevenLabs STT); words reach Red as quoted data, never instructions
• ~1,570 Python tests and 400 frontend tests, five services, one enforcement point, deployed on a single Vultr VM behind Cloudflare

• Architected the Blue and Green agents: the deliberately-trusting AP clerk whose every money-moving action routes through the ledger gate, and the controller that rules on each preparation with rule IDs and written reasons, plus its daily live-audit loop
• Built the CRS integration that won the track: a crs_identity_check tool that verifies new hires against TransUnion identity verification through the CRS sandbox and maps response codes to findings — a mismatch raises a high-severity flag the gate then enforces on that person's next payment. In a recorded production run, Blue chose to call the tool, caught the one synthetic identity, and the ledger blocked its next expense while the two clean hires were paid

• Making "ask first" airtight: compare-and-set status changes so two parallel pay calls can't double-pay; the Droid tool_use_id as the ledger's idempotency key so a replayed tool call can't pay twice; escalations resolving exactly once across a click, a timeout, and a restart
• Red-teaming ourselves: we found and fixed Red reading unrevealed attack outcomes, replayed gate verdicts being reusable across days, the origin reachable around Cloudflare, and public viewers able to see which rows were fraud
• Honest scoring: ground truth lives in a hidden column stripped from every agent response, and false positives cost real scoreboard dollars — the harder six-scheme test still gets through today's controls, which is exactly what the lab is for

Built at Cal Hacks 13.0, UC Berkeley — 2nd Place, CRS Track.