An AI assistant you still double-check has saved you nothing.
I run three mailboxes across three identities — a CTO role, an advisory practice, a personal life. About a hundred emails a day. Every AI inbox tool I looked at fails in one of two ways: the bill scales with the noise rather than the signal, or it drops something that mattered once, and after that you check the inbox anyway. The second failure is the expensive one — it returns you to where you started while still charging you. This is the architecture I built to avoid both, the real numbers from ten days of it running, and the parts that are genuinely hard.
The two failure modes, stated plainly
1. Cost scales with volume, and volume is mostly noise
The naive build hands every incoming email to a model. But an inbox is not a stream of decisions — it's a stream of receipts, newsletters, calendar noise and CC chatter, with a few real asks buried inside. Paying a model to read all of it means paying most of your bill to be told "nothing here." Worse, the cost curve is set by how much noise you receive, which is the one variable you don't control.
2. One false negative and the whole thing is worthless
This is the failure people underestimate. A triage system that misses a real item doesn't degrade gracefully — it collapses. You go back to reading the inbox yourself, and now you're doing that and paying for the assistant. The bar isn't "usually right." The bar is "I no longer open the inbox," and nothing short of that produces any saving at all.
Those two pull against each other. Cheap means looking at less. Trustworthy means missing nothing. The whole engineering problem is buying both, and it is not solved by a better prompt.
The shape of it
Three stages, and the interesting one is the first — a deterministic gate that costs no model tokens at all and throws away most of the work before any model is involved:
The gate runs on a schedule, four times an hour per mailbox, around the clock. It compares unread mail against a small state file of message ids it has already handed over. No new mail, it exits silently and nothing else runs. It also carries two blocklists — by sender domain and by subject substring — for senders that recur forever and never need a judgment call; those get marked read and labelled right there in plain code.
What that actually saved, measured
Ten full days, three mailboxes, every number below pulled from the run log rather than estimated:
| Stage | Count (10 days) | What it cost |
|---|---|---|
| Emails arriving across three mailboxes | 1,034 | — |
| Scheduled gate checks | 2,880 | ~1s of CPU each, zero tokens |
| Checks that reached a model | 567 20% | Model run |
| Runs that surfaced anything | 202 of 567 | — |
| Notifications actually sent to me | 255 25% of inbound | — |
Four out of five scheduled checks never touch a model. That is the entire cost story, and it comes from a hundred lines of ordinary code, not from model choice or prompt tuning. Then a second filter applies: of the runs that do reach a model, roughly two thirds conclude there is nothing worth telling me and say nothing. Twenty-five emails a day out of a hundred produce a notification.
The pattern generalises, and it's the first thing I look for in anyone else's AI feature: put a deterministic filter in front of the expensive judgment, and make it cheap to be wrong in the safe direction. A gate that occasionally wakes the model unnecessarily costs one run. A gate that drops a real email costs the system's credibility. So the gate is built to over-admit, and the model is where the actual discrimination happens.
The false-negative problem, and the fix that isn't obvious
Getting cheap was straightforward. Getting trustworthy took a change I didn't anticipate.
The first version asked one question per email: does this need a reply? If yes, draft one and create a task. If no, one short summary line. That worked for weeks and then missed something in a way that made the flaw obvious: a new travel-booking policy landed — a rule I was now expected to follow, requiring no reply from me at all. The system correctly decided there was nothing to respond to, summarised it in a notification, and moved on. That notification scrolled away in a busy chat within the hour, and the rule was gone with it.
The realisation: in that whole stack, only two things survive — a task and a pin. Everything else is a message in a feed, which means it's temporary. So every email now gets judged twice, on two genuinely different questions:
- Does this need me to DO something? → a dated task, with a priority, plus a drafted reply where one is warranted.
- Does this need me to KNOW something? → pinned, and a read task logged. No reply required, but it is not allowed to just scroll past: a policy change, a contract or money fact, anything touching my role or compensation, a security notice, travel affecting my plans, a decision other people took that binds me.
One rule decides it: does this change something I must do or must know, in a way that would alter a decision of mine? If not, it's disposable — a status update on tracked work, a close-out "thanks", progress on somebody else's task. The categories exist to catch what a human would regret missing, not to sweep in everything that brushes a keyword. That distinction is the difference between a triage system and a second inbox.
Three design decisions that came from using it, not from designing it
One email, one message — because I work this from a phone
The first build sent a single digest per run. It read well and was useless. I action these between meetings, on a phone: a thumbs-up to mark one done, a reply of "send" to approve one specific draft. A digest gives you nothing to react to individually, and replying to it quotes the entire blob. Now each email is its own self-contained message — sender, one-line summary, the priority and why, the reference id, and what was created. The delivery format is dictated by how the work actually gets done, which is not something you can specify up front.
Priority rules live in a file, not in code
Every task gets P1–P4, because a flat list of forty items is not a list you work. The rules that assign it sit in a plain markdown file per context — work priorities and personal priorities are genuinely different, and personal P1 means money leaving the account or travel inside 48 hours, not "someone senior emailed."
That file is injected into the prompt verbatim on every run. The tempting alternative — tell the model to go read the file — fails in a way you cannot see: a skipped read produces unprioritised tasks and no error anywhere. Interpolation makes it impossible to skip. The payoff is that editing a markdown file changes triage behaviour on the next run, with no deploy and no code change.
A degraded run is loud, and it still runs
If the priority file is missing, the job doesn't fail and it doesn't quietly carry on either. It sends a warning saying tasks this run are prioritised by a crude fallback, then triages anyway with rules deliberately blunt enough that the degradation is visible in the output. Mail still has to be processed. Automation that stops on a missing config file is automation you'll be debugging on a Sunday.
What one notification looks like
A real one, names and references changed:
Follow-up on the stale user-data thread raised this morning. The product lead asked for a meeting with the platform owners to agree a concrete action list; the delivery lead then confirmed to the group that this is already agreed as high priority, folded into the September list with a target of 15 Sept, and other work will move to make room. An owner is named and the meeting is being organised.
You're CC'd only, no ask of you. This now has sign-off, an owner and a committed date. I downgraded the existing task from P1 to P3 and pushed it to 28 Aug so you can check it's tracking after the roadmap meeting, rather than it sitting as urgent with nothing left for you to do today.
That closing paragraph is what took the longest to get right. A summary tells you what happened. This tells you what it did on your behalf and what judgment it applied — which is the only way you ever get comfortable enough to stop checking behind it.
Costs and limits
- It is not a filter you can walk away from. The blocklists, the no-task senders, the priority rules — I've tuned all three since it went live, and I expect to keep doing that. What it removes is the reading, not the ownership.
- Drafted replies are never sent automatically. Every outbound message waits for an explicit approval. That is a hard rule, not a setting, and it exists because one confidently-worded wrong email costs more than the whole system saves.
- It runs against my own mailboxes, as me. A company version — many mailboxes, shared inboxes, someone else's data-retention obligations, an audit trail of what an automated system read — is a different build with a different risk profile. Not a copy of this one.
- The measured numbers above are one person's mail over ten days. They're real, and they are not a benchmark for anyone else's volume or noise ratio.
Why I'd point a CTO at this rather than at the tool
The interesting part of this build is not that a model can read email. Everyone knows that. It is that the two things that decide whether an AI feature survives contact with production — what it costs at your real volume, and what happens the first time it's wrong — are both settled by architecture around the model, not by the model. A deterministic gate in front. A second, differently-framed judgment to catch the class of error the first one structurally can't see. Rules in config so behaviour changes without a deploy. Degradation that announces itself. An output that explains what it did, not just what it read.
That's the same checklist I run through in an Engineering Velocity Audit, and the same one that decides whether an AI feature in a technical due diligence is a real capability or a demo with a cost problem it hasn't hit yet. If your team has an AI pilot that works in the demo and nobody can tell you what it costs at ten times the volume, that's the conversation.
Have an AI feature that works in the demo? Let's talk about what it does at real volume.
I read every message and reply personally.