I built an AI that answers support questions. Founders still hated it. Here's what I learned.
Early version of Velor was a great question-answerer.
Customer asks "how do I add a team member?" — perfect response, cited the right doc, even knew about the edge case with SSO.
Founder still got a support ticket the next day. "Can you just add me manually? I can't figure it out."
That's when I realized I'd been solving the wrong problem.
The questions that kill founder productivity aren't the ones that need answers.
"What's the status of my refund?" → needs an answer
"Can you just issue it?" → needs an action
"Where's my order?" → needs an answer
"Can you reroute it to my new address?" → needs an action
"Why did my payroll run fail?" → needs an answer
"Can you reverse the charge?" → needs an action
The answerable questions are annoying but fast. You paste a doc link and move on.
The actionable ones are the ones that ruin your afternoon. You have to log into Stripe, find the transaction, check the amount, decide if it's legitimate, then actually do the thing — all while you're supposed to be building.
So I rebuilt it.
Instead of just grounding answers in docs, it now connects to Stripe, Shopify, Zendesk, and a few others. When a customer asks for a refund, it doesn't say "here's how to request one." It checks the order, confirms with the customer, and processes it — with a signed audit log of every action it took and why.
It still answers questions. But it also does the thing.
What changed:
Deflection rate went up (fewer tickets escalated to founder) — expected.
The surprise: the tickets that did escalate were completely different. Before, founders were getting "can you just do X for me" all day. After, those were gone. The escalations left were genuinely nuanced judgment calls — the kind a founder should be involved in.
If you're running support for an early-stage product, curious what your most draining ticket category is. For most founders I've talked to it's some variation of "I know what needs to happen, I just have to be the one to make it happen."
That's the problem I'm trying to kill.
Demo at velorplatforms.com if you want to see it — no signup, just talk to it.
Moving from answers to actions changes the product from a knowledge layer into an operations layer. The critical feature is the policy system around each action: eligibility rules, value thresholds, approval requirements, reversibility, and a complete receipt. I would track successful autonomous resolutions separately from actions that needed approval or reversal, because that tells buyers where trust is actually being earned.
This sounds like one of those cases where the technical problem was easier than the trust problem. What did founders actually dislike: wrong answers, lack of control, or just not wanting AI talking directly to customers?
The answers vs actions split shows up in app store reviews too. I mine them for a living and people almost never praise an app for explaining something well, they praise it when the thing just happened without them. Same asymmetry in the complaints, 'support told me how to do it' reads as a negative in context, what they wanted was for it to be done. Your escalation quality change is the interesting bit, filtering by required judgment instead of by topic. Most triage sorts by category and gets this wrong. From reviews, the draining category is refunds by a mile, money, trust and permission in one ticket. Curious whether the signed audit log actually matters to buyers in practice, people say they want the receipts, I wonder how many ever open them.
The asymmetry in reviews is a sharper version of the same thing -- "support told me how to do it" reads negative because doing it was the expectation, not a bonus. On the audit log: most customers don't open it. Founders do -- especially early, when they're still deciding whether to trust the system. It's less a customer-facing feature than a founder-trust feature. Worth naming it that way explicitly.
The answer vs action split is a useful boundary. We have been thinking about the same line in Speechara.Ai: the assistant can draft a reply when the user explicitly asks, but it never decides what they should say. Keeping the user in control makes the suggestion easier to trust. The next metric I would watch is not only deflection, but how often a suggested action is reviewed, edited, and approved without founder intervention. How are you handling actions that need confirmation before execution?
The suggestion vs decision boundary is worth naming explicitly -- the edit rate on suggested actions is a better health signal than approval rate alone. High approval + low edits = policy is calibrated. High approval + high edits = right direction, wrong specifics. That's a more useful diagnostic than watching deflection in isolation.
On confirmation before execution: for anything with cost or state implications the customer didn't explicitly agree to, Velor holds in a proposed state and sends the customer the full plan before running it. The confirmation step also surfaces mismatches -- what the AI inferred the customer wanted vs what they actually wanted. That signal comes earlier and is more actionable than a CSAT score after the fact.
The answerable vs actionable split is the right frame, and I think it maps to something even broader in early-stage products: the support tickets that drain founders are the ones where the customer already did the work of diagnosing, they just need you to have the right permissions.
That's a different problem than confusion. Confusion-based tickets are a UX problem. Permission-based tickets are an automation problem. They look similar in a helpdesk queue but need completely different fixes.
The escalation quality shift you described is the real signal here. When the actionable tickets go away, what's left are genuine judgment calls. That's actually what founders should be making. The work changes character, not just volume.
One thing I'd push on: how are you handling cases where the action needs confirmation from the customer before executing? For refunds that's straightforward, but for something like rerouting an order with cost implications, the customer expectation management layer becomes tricky. Curious how Velor handles that moment in the loop.
The permission vs confusion split is a cleaner model than we were using internally -- it changes where you look for the fix. On the confirmation question: for anything with cost or state implications that can't be undone quickly, Velor holds the action in a proposed state. The customer gets a message saying what the AI wants to do and confirms before it runs. The audit log captures both the proposal and the confirmation, so the founder can reconstruct exactly what happened and who said yes. The rerouting case you're describing is exactly the category this pattern was built for -- the AI does the work of figuring out what to do, the human just says whether to do it.
The answer/action split points to a simple approval rule: automate actions you can undo, but ask first when they're costly or hard to reverse. For example, you could auto-approve refunds below a set limit. Rerouting an order after it ships should probably require confirmation. I'd also track how often users approve an action without editing it. That shows whether the policy is building trust, not just cutting ticket volume.
Reversibility is a cleaner gate than confidence -- confidence is internal to the model, reversibility is something the founder can actually reason about. We landed at the same split from a different direction. The approval-rate metric you mention is one we track too: if founders routinely approve without editing, the policy is earning trust. If they're editing heavily, the proposed actions are landing wrong and the KB needs work. That ratio ends up being a better health signal than deflection rate.
The signed audit log is the detail that will sell this, not the deflection rate. At SocialPost.ai the blocker to letting AI touch billing was never capability, it was whether I could reconstruct exactly what the agent did when a customer disputed it. To answer your question: the draining category is the "can you just" refund-adjacent requests, and the one escalation I still want is any customer who is both angry and high LTV, because that call is about the relationship, not the ticket.
The "can I reconstruct what happened" question is the right frame for billing-adjacent trust. Deflection rate proves the system is answering questions -- it doesn't prove the right action was taken. We found the same thing: the audit log does more selling work than almost any other feature because it answers the anxiety that holds founders back from enabling actions in the first place. The angry + high LTV escalation is the right carve-out too -- that's not a support ticket, it's a relationship call.
The answer-versus-action split is a useful product boundary. I’d add a second one: action completion versus action confirmation. A refund can be processed correctly but still leave support unresolved if the webhook is delayed, the customer never sees confirmation, or the operation partially fails. Give every action an idempotency key, provider object ID, and terminal status that stays in the support thread. Then measure not only deflection, but “resolved without founder involvement” and “reopened after automation.” The most valuable escalation bundle is probably current state, last safe transition, and exactly what remains. How are you handling the ugly case where Velor acted successfully but never received confirmation?
The completion vs confirmation split is the right second layer -- we ran into exactly the partial-failure case with Stripe webhooks. Action confirmed, webhook delayed, customer emails again thinking nothing happened. The fix was exactly what you're describing: a terminal status on the action that stays visible in the thread so the founder can see 'refund queued, not yet confirmed by Stripe' vs 'refund settled.' The idempotency key also matters for when the action gets retried -- without it you end up double-processing. Good framing for what the infrastructure under these actions actually needs.
The answer vs action split is the whole product. Replies grounded in docs make demos look great, then the founder still spends the afternoon in Stripe because the ticket was never a question. Wiring the action with a signed audit log of what ran and why is the right bar, because "I handled it" without evidence is just a different kind of support debt. Curious whether founders trust auto refunds more when the confirmation sits on the customer side, or when a human still signs the irreversible ones.
The audit log framing is exactly right -- "I handled it" without evidence trades one trust problem for another. What we've found is that the threshold splits by reversibility: for refunds or cancellations that can be walked back, founders are comfortable with auto-execution as long as they can see the full action record in the thread. For anything irreversible -- a permanent delete, a one-way data export -- they still want to be the ones who click confirm. The audit log is what makes the first bucket feel safe rather than anxious. Without it, even reversible actions feel like they happened in the dark.
This is a sharp lesson in measurement system misalignment. You were optimizing for the wrong metric.
Perfect answer accuracy looked like success (founders literally said "good response"), but it didn't solve the constraint that was actually killing productivity. The real bottleneck wasn't "I don't know the answer" - it was "I know what needs to happen, and I have to be the one to make it happen."
That reframe from "answer questions well" to "eliminate founder actions" is a measurement system shift. When your north star was "answer accuracy," the product was doing great. When your north star became "founder time freed," the whole system was backwards.
The part that stands out most: after rebuilding, the escalations flipped from "can you just do it?" to actual judgment calls. That's when you know you've solved the real problem - the downstream signal changed in a way that makes founder decision-making clearer, not noisier. That's constraint-driven clarity that most products never find.
The "downstream signal" observation is the sharpest thing in this thread. When escalations go from "can you just handle it" to genuine judgment calls, that's the signal you've actually changed the system -- not just the accuracy number. Answer accuracy was a proxy, and the proxy felt great while the real problem was compounding. Measuring "founder time freed" or "actions auto-resolved" is harder to instrument early but it's what actually tells you whether the product is working. Escalation quality is probably the leading indicator most people skip because it's qualitative.
Disclosure: I build Fablino - a personalised bedtime story written and emailed to a kid every evening - so I ship model output straight to a five-year-old with no human in the loop. Different domain, same shape as your refund problem.
The bit I'd add to the confidence-boundary thread above: decide now what the system does when it isn't confident enough to act. We ended up with a hard rule. A separate model grades every story on safety, narrative logic and continuity, the bar is 5 out of 5, and it gets up to three targeted repair passes. If it still can't clear the bar, that story is marked failed and a known-good one goes out instead. Nobody gets a slightly-off story, and nobody gets an empty inbox at bedtime.
Escalation is the obvious equivalent for you, but "I'll fetch a human" at 2am can be worse than a prepared safe action. Feels like there's a third branch between act and hand off.
One practical thing: the model that writes isn't the model that grades. Run one model as both and it marks its own homework.
The third branch is what Velor ended up building -- act, but fall back to a known-safe response if the action can't be verified. For support that's the equivalent of your known-good story: if the refund can't be confirmed or the action pathway is uncertain, send a human-sounding holding response rather than either executing blind or waking someone up at 2am.
The separate grader point is the right architecture. One model generating and grading its own output is exactly the failure mode where confident-wrong beats uncertain-right every time. We run evaluation as a separate step after action selection -- it's the only way to catch cases where the generation path was too optimistic about what it actually did.
The distinction between "needs an answer" and "needs an action" is super sharp. Most AI support bots are just glorified doc search engines, but if a user is asking "can you just do it for me," throwing a doc link at them just pisses them off. Moving from answering questions to actually executing tasks in Stripe/Shopify is where the real magic happens. Great pivot.
Most bots are answer machines because answers are safe -- you can always blame the customer for not reading the response. Actions have failure modes that land on the product. That's why the category hasn't moved despite the obvious demand. The audit log exists partly because of this -- it's the mechanism that lets founders trust an action was right even when they weren't watching.
The answer/action split matches where we ended up drawing the line, though we got there from a different direction. Almost everything on our side is run by AI sessions, and the boundary that survived contact with reality wasn't how confident the model was. It was whether the thing is reversible, and whether a person's name is on it.
Reads, drafts, classifications, anything you can undo: fully automated, nobody reviews them. Anything that sends, publishes, charges or refunds: the AI does all the work and a human only says yes. Same rough 90/10 you're describing, but the split isn't by difficulty, it's by whether a mistake can be walked back. Founders hand over far more than they say they will once the risky 10% still visibly stops at them.
Does Velor draft the action and hold it for one click, or does it ask permission before doing the work?
It’s brutal to watch founders reject a tool you poured your heart into, especially when it does answer questions correctly. The real pain isn’t accuracy—it’s the loss of control and the fear of sounding robotic to their customers. That’s a massive gap: they need AI that feels like their brand, not a generic bot. There’s a clear opportunity to build a “voice-copy” engine that learns from past replies. What’s the one thing founders asked for that you couldn’t build?
The brand voice gap is real and it was the thing that took longer than expected to fix. We built a voice profile into Velor that learns from the founder's own past replies, but casual founder-speak is much harder to capture than formal documentation -- the training signal is usually sparse and inconsistent.
To answer the question: the hardest thing to build well was recognizing returning customers across conversations and having that context already in the room when responding. "This customer has emailed 4 times this month, the last refund was manual, and they're visibly frustrated" -- that state should load automatically, not require the founder to read back through a thread. The relationship-level memory is where we still have the most work left.
This is a great insight. I think the relationship memory layer is probably where AI assistants will become truly valuable.
Generating answers is becoming easier, but understanding the history behind a conversation is much harder. A customer's previous frustrations, preferences, and past interactions create context that a normal chatbot simply doesn't have.
It reminds me that the future of AI assistants may not just be about better models, but about building better memory systems around people and relationships.
Really interesting work. Looking forward to seeing how Velor evolves.
The answer-to-action jump is useful, but I would measure recovery as a separate product capability. In these workflows, “the API call returned” is not the same as “the customer problem is resolved”: a provider response can be lost, a webhook delayed, or a multi-step operation only partly complete. I would give every requested action an immutable action ID, idempotency key, provider object ID, and terminal status, then keep those in the support thread so a human can resume without rerunning it. The best escalation bundle is not just transcript plus reason; it is current state, last safe transition, and exactly what remains. Have you tested the ugly path where Velor acted successfully but never received confirmation?
Yes, that path is the one that breaks trust fastest -- action taken, no confirmation, customer emails again thinking nothing happened. The terminal status in the thread is exactly the fix: the founder sees "refund queued, awaiting Stripe confirmation" rather than a gap. The idempotency key is also critical for when the action gets retried after a missed confirmation -- without it you double-process.
The escalation bundle framing is right. "Current state + last safe transition + what remains" is much more actionable than a transcript. Transcript tells you what was said; that bundle tells you where to pick up. We're instrumenting exactly that -- action ID, provider object ID, terminal status -- and keeping it pinned to the thread.
The reframe from answer to action is the real finding here. Most support AI stops at the first one because it is safer to build. The part worth pushing on is the confidence boundary on the money moving actions specifically. A refund is reversible and cheap to get slightly wrong. A payroll reversal is not. Does Velor use the same threshold for deciding when to act on its own across Stripe, Shopify, and Zendesk, or does it get more conservative specifically where the dollar amount or reversibility changes, even when the question sounds equally clear cut on the surface?
The threshold isn't flat across connectors -- reversibility is the primary axis. For refunds (reversible, bounded amount) Velor will auto-execute when the evidence is clear. For payroll adjustments or data deletions it holds for confirmation even when the surface question sounds equally unambiguous. Dollar amount is a secondary factor -- above a threshold it moves to proposed state regardless of how clear-cut the language looks.
The check isn't just "am I confident in my interpretation" but "if I'm wrong, can the founder walk it back without losing money or customer trust." Those two questions have different answers for Stripe vs. a payroll API, so the gate adjusts.
The distinction between answering a support question and actually resolving it is pretty significant. It sounds like the bigger shift was in what remained for the founder to handle after the easy tickets disappeared.
That's exactly it -- the easy tickets disappearing surfaces what was always there underneath: the hard calls that actually need judgment. Ticket count goes down but founder-attention-per-ticket goes up because everything left is something the AI correctly flagged as needing a human. The workload shape changes more than the total workload, at least at first. The goal is for the hard calls to get easier too over time as Velor learns which judgment calls are actually repeatable.
That workload-shape change is the interesting part. I’d be curious to see whether Velor eventually reduces the judgment-heavy queue too, rather than just making it more visible. If you’re open to continuing the conversation, what’s the best email to reach you at?
That's the next phase -- once Velor has seen enough of the same judgment call, it starts surfacing patterns. "You've approved this refund type 10 times, want to turn that into a rule?" The queue doesn't just stay visible, it becomes learnable over time.
Happy to keep the conversation going -- [email protected]