The System Design Round

“We’re a subscription company. Support gets buried in refund requests and plan-change questions. We want an agent that can just handle it. Where would you start?”

Before anything else — when you say “handle it,” do you mean answer questions about refunds and plans, or actually issue refunds and change plans?

“Both, eventually. Right now a human does both, and it’s slow.”

Those are very different risk levels, so I’d want to split them from the start. Answering questions accurately is one problem. Actually issuing a refund or changing billing is a different one, because a mistake there costs real money and touches a customer’s trust directly. What does the refund policy actually look like — is it a fixed rule, or does it involve judgment calls?

“Mostly rules — refund within 14 days, no questions asked. Outside that window it needs a human’s judgment call.”

That’s a clean line to design around. I’d let the agent handle the in-policy case end to end, and hand anything outside the window straight to a human, no attempt to be clever about it. What tools or systems would the agent actually need access to — a billing API, a CRM, something else?

“A billing API that can issue refunds and change plans, and a CRM with the customer’s history. Both have real write access — this isn’t just a read-only lookup tool.”

Given that a tool call here can move real money, I want a hard guardrail layer that sits between the agent’s reasoning and those write calls, not just a prompt asking it to be careful. Concretely: a rules engine checks eligibility — is this within 14 days, has this customer already gotten a refund this month — before the refund tool is even callable, regardless of what the agent’s reasoning concluded.

“What if the agent gets talked into it? Customers will absolutely try to convince it they qualify when they don’t.”

That’s exactly why the eligibility check can’t live in the prompt. It has to be a deterministic check in code, outside the model’s control entirely, so no amount of clever phrasing from a customer changes the answer. The agent can be as persuadable as it wants in conversation; the tool it calls simply won’t execute if the rule fails.

“Okay. And plan changes?”

Same shape, different rule set — some plan changes are free, some involve proration or a contract term, and I’d expect the proration math to be genuinely fiddly. First phase, I’d actually keep plan changes as agent-drafts-the-change, human-confirms, since the failure mode of a wrong proration is messier to unwind than a straightforward refund.

“What ships first, given we want something live in six weeks?”

The in-policy refund flow only, in shadow mode for the first week or two — the agent proposes the action, a human clicks confirm, and we log how often the agent’s proposal matches what a human would have done. Once that agreement rate is high and we’ve seen a real week of production traffic, we flip refunds to fully automated within the 14-day rule. Plan changes stay human-confirmed a while longer, since the rules are messier and the blast radius of a mistake is larger.

“Our engineers waste hours searching Confluence, Slack, and old Jira tickets for answers that already exist somewhere. We want an internal assistant that just knows the answer. How would you build it?”

Before the retrieval design, one question that changes everything: do all of those sources have the same access permissions, or does some of that content vary by team or seniority?

“It varies a lot, actually. Some Confluence spaces are restricted, some Slack channels are private, and there are Jira tickets with customer PII in them that only certain people should see.”

Then permission-aware retrieval isn’t a nice-to-have here, it’s the central design constraint — a wrong answer is annoying, but leaking a restricted document through the assistant is a much bigger problem, and one that’s easy to introduce accidentally with a naive vector index. How is access actually controlled today across these three systems — is there a common identity or permission model?

“All three use the same SSO identity, but the permission checks live separately in each system’s own access control.”

That’s workable, but it means the retrieval layer can’t just embed everything into one flat index and hope for the best — at query time, I’d need to filter results down to only what this specific user is actually allowed to see, checked against each source’s real permission model, not a cached snapshot that could go stale. What’s the actual query pattern going to look like — a chat interface, something in Slack, an IDE plugin?”

“Probably a Slack bot to start, since that’s where people already ask each other questions.”

Good, that also gives a natural place to collect feedback — a thumbs up or down on each answer is a cheap way to start building an eval set instead of guessing whether the assistant is any good. Given the permission complexity, I’d actually start with the least sensitive source first rather than trying to do all three sources at once.

“Which one’s that?”

Confluence is the most structured and probably the easiest to reason about permission-wise. I’d ship retrieval over public and semi-restricted Confluence spaces first, with the permission filter proven out at that smaller scale, before adding Slack history and Jira, which both have messier structure and more sensitive content mixed in.

“What happens if it gives someone a wrong answer instead of no answer?”

That’s the other reason I want an eval set early — I’d rather the assistant say “I couldn’t find a confident answer” than hallucinate something plausible-sounding from a stale doc. I’d set an explicit confidence threshold based on retrieval score, and below that threshold it says so and links what it did find, rather than guessing.

“Given a one-quarter timeline, what actually ships?”

A Slack bot answering questions over Confluence only, permission-filtered per user, with a thumbs up/down feedback loop building an eval set from day one. Slack and Jira ingestion, plus any Jira PII handling, is phase two, once the permission model and the eval process are both proven at smaller scale.

“We want a coding assistant that can plan a feature, write the code, and review it, using separate agents for each step. But we have a hard per-request cost cap from finance. Design it.”

Before I split anything into multiple agents — what’s driving the multi-agent idea specifically? A single agent with a good loop and the right tools can go pretty far, and every extra agent is extra token spend, which matters a lot given the cap you just mentioned.

“Fair pushback. We assumed separate planner, coder, and reviewer agents would each do their one job better than one agent doing all three.”

That can be true, but it’s worth testing rather than assuming, especially under a token budget. I’d actually start with a single agent doing plan-then-code-then-self-review as one loop, and use that as the baseline both for quality and for cost. Multi-agent only earns its complexity if it measurably beats that baseline on the same eval set.

“Say the baseline isn’t good enough and we do need to split it. How do you keep it inside the budget?”

First, I’d put a shared token budget tracker across the whole request, not a separate budget per agent — otherwise three agents each assuming they have the full budget blows through the cap immediately. Every agent call decrements from one shared counter, and any agent can check how much is left before deciding how much context to pull in.

“What happens when the budget actually runs low mid-task?”

That’s the part I’d design for explicitly rather than let happen by accident. Graceful degradation — if the reviewer agent is running low on budget, it falls back to a cheaper, faster model instead of skipping review entirely, or it reviews only the diff instead of the full file. The system should degrade in a controlled way, not just fail or blow the budget.

“What about a task that just needs more tokens than the cap allows, no matter what you do?”

Then I’d rather the system say so early and clearly — “this task is estimated to exceed budget, here’s a partial plan” — than silently produce a truncated, half-finished result and present it as complete. I’d build a rough cost estimate at the planning stage, before the coder agent starts, so we catch that case before spending most of the budget getting there.

“How do the agents actually hand off work to each other without wasting tokens re-explaining context?”

I’d pass a structured summary between agents rather than the full conversation history — the planner hands the coder a concrete spec, not a transcript of how it arrived there. That’s both cheaper and, honestly, usually produces a cleaner handoff than dumping raw context and hoping the next agent extracts the right part.

“Given all that, what ships first?”

The single-agent plan-code-review baseline, measured against a small eval set of real tickets, with the shared token tracker built in from day one even before any multi-agent split. If the baseline’s quality genuinely falls short on some class of task, I’d split just that piece into a second agent, rather than committing to a three-agent architecture up front on an assumption.