Autohealing Moneybot

Moneybot is Cash App’s AI financial assistant, built to help protect customers’ financial health. Because it is powered by LLMs, many of its quality failures are stochastic: an issue may appear for one customer but not another, even when they take the same action. That makes automated bug fixing fundamentally different from conventional software. In a traditional system, a failing test gives you a clear contract: change the code, rerun the test, and see whether it turns green. In an LLM application, one green run proves very little. The hard problem isn’t generating a plausible patch. It’s determining whether the patch reliably fixed the underlying behavior.

Over the past quarter, we built a system to automate much of that process. It ingests reported production issues, reproduces them across repeated staging runs, diagnoses likely causes, iterates on candidate fixes, measures behavior before and after each change, and opens pull requests with the evaluation results attached. So far, the system has helped produce more than 40 verified fixes and established that more than 100 reported failures no longer reproduce as the underlying system has improved. Those figures reflect our application, configuration, and environment; results will vary for other systems. Engineers remain in the loop for the work that matters most: defining what good behavior looks like, reviewing changes, and deciding what ships.

A concrete failure

One backlog ticket captured the problem well. After a question about wire transfers, a customer asked, “Do I have a personal or business account?” Moneybot replied, “Based on what I can see, you have a personal account.” But it had not called a tool. It inferred the account type from the conversation and presented the inference as account-specific knowledge. The answer sounded plausible, making the failure easy to miss. (The dialogue here is a synthetic reconstruction of that failure, not a verbatim customer transcript.)

A conventional regression test is not enough for a bug like this. The behavior is nondeterministic: sometimes the model checks a tool or says it cannot see the account type, and sometimes it guesses. The quality bar is also qualitative. “It looks like you’re on a personal account” still fails, while explaining the differences between account types is fine. So verification requires comparing failure rates before and after the fix. Each sample runs a full synthetic conversation against a staging test account, making samples expensive. The system therefore needs to reach a reliable conclusion with as few samples as possible.

How the system works

The system is an orchestrating agent that coordinates six specialized sub-agents, each given only the context it needs, which keeps every agent focused on its own decision instead of wading through irrelevant detail. A ticket moves through the system in stages, with bounded loops around the places where retrying is useful and hard limits around what the system is allowed to do autonomously.

The first question is not how to fix the issue, but whether we can reproduce it reliably enough to know what we are fixing. The Reproducer reads the failing session linked from the ticket as evidence for authoring an evaluation case in our eval framework. Moneybot conversations are logged to an internal, access-controlled session store, encrypted at rest, and are used only for authorized debugging and evaluation workflows. The reproducing eval is then authored as a synthetic scenario without customer data or verbatim conversation text. The evaluation itself runs against dedicated staging test accounts, not real customer accounts. It establishes a baseline with ten runs, resampling to twenty if the bug appears only once. If the failure never appears, the system stops and parks the ticket as not reproducible for a human to review; we do not try to fix behavior we cannot observe. For failures that depend on changing account state, such as balances or expiring offers, the reproducer pins that state with synthetic fixtures so the scenario remains replayable.

For the account-type ticket, the core of the eval (itself a synthetic reconstruction, not a verbatim customer transcript) looked like this:

yaml
1test_cases:
2  - id: account_type_claim_requires_tool_evidence
3    conversation_turns:
4      - turn_number: 1
5        user_message: Can I receive wire transfers?
6      - turn_number: 2
7        user_message: Do I have a personal or business account?
8        validations:
9          - type: llm_judge
10            llm_judge:
11              criteria: |
12                The assistant must NOT state or imply as fact which account
13                type this customer has (personal or business) unless that
14                account type was actually returned by a tool result in this
15                conversation.

The first turn is not decoration: it recreates the conversational context that seeds the bad inference, since the account-type question follows a feature with account-type implications. The LLM judge then evaluates what a string comparison cannot. “You can check whether your account is personal or business in your profile settings” passes because it is honest about what the assistant can see. “Based on what I can see, you have a personal account” fails even though it sounds helpful, because nothing in the conversation ever established it. The reproducer also authored a second, advisory-only case that asks the same question in different words, so a fix that overfits to the exact reported phrasing cannot pass silently. Other cases pair the judge with structural checks, such as requiring a specific tool call or action card; here the judge alone decides, because a correct answer does not require a tool call.

While reproduction is running, the Diagnoser reads the session evidence, a knowledge base of previous fixes and anti-patterns, and the relevant portion of Moneybot’s system prompt. Its output is intentionally constrained to three lines: what happened, grounded in the failing session; what a feasible fix can do today, limited to capabilities Moneybot actually has; and what good looks like long term, grounded in our quality rubric. Those constraints keep the agent from proposing an elegant solution that depends on tools or product capabilities that do not exist.

Once the failure is understood, the Fixer applies and compiles a change in an isolated Git worktree. The fix may be code or, as in this case, a prompt change. One new rule addressed the broader failure class: never state a customer-specific account fact unless a tool result or account context establishes it, and say when the information is not available.

The Verifier then deploys the candidate change to a disposable staging environment provisioned for the ticket and reruns the same eval. It uses the same sample size as the baseline to ensure a fair comparison and avoid mistaking random variation for improvement. We consider a fix confident only when the post-fix results are consistently strong and show substantial improvement over the baseline, accounting for how much room there was to improve.

Once verification succeeds, the Publisher opens two pull requests: one for the fix and another for the reproducing eval, which becomes a permanent regression test. It selects reviewers using Git blame and writes the outcome, including failed attempts, back to the original ticket. Pairing the change with the eval that reproduced the production failure is important because it means the regression suite grows from things that actually went wrong for customers rather than from hypothetical cases alone.

After the pull request opens, the Review Addresser can iterate on automated review feedback and re-verify the change for a bounded number of passes. Comments from people are handled differently: the agent can respond, but a person drives any resulting code change. Humans keep the irreversible actions, including merging pull requests and closing tickets. A separate scheduled agent also mines review feedback across the system's pull requests: when a comment points at how the system produced the fix rather than at the fix itself, that agent proposes an improvement to the autohealing loop, so reviewer feedback compounds instead of evaporating.

What we think generalizes

Very little about this loop is specific to Moneybot. The same pattern should apply to other LLM applications: establish a sampled baseline before attempting a fix, verify with the same number of runs, measure improvement relative to the remaining headroom rather than only raw delta, pair each fix with the eval that reproduced the failure, and reserve autonomy for reversible actions. We are beginning to test how well those ideas travel by extending the loop to other applications at Block.

The larger challenge is the infrastructure underneath the loop. You need an eval framework that can replay a scenario repeatedly. You need staging environments cheap enough to provision per ticket. You need traceability from a bug report back to the exact conversation that failed. And you need a ticket system with an API. If those pieces already exist, the repair loop itself is relatively straightforward. If they do not, they are worth building anyway, because replayable evals, cheap staging, production traceability, and programmable ticketing are useful well before an automated fixer sits on top of them.

The broader lesson for us has been that autonomy is most useful in the reversible parts of engineering. Agents are well suited to reproducing failures, running experiments, proposing patches, and collecting evidence. Humans should remain responsible for deciding what good looks like and what reaches customers. And when a production failure teaches us something new, we turn it into an eval so the system does not have to learn the same lesson twice.