Building an AI product taught me that the LLM isn’t the hardest part

Spot on. The real bottleneck is almost always the data pipeline and infrastructure, not the core model itself. I usually work with XGBoost models for trading, but recently pivoted to e-commerce price tracking. Handling massive daily time-series data without blowing up server costs is the true challenge. I've been relying heavily on Supabase to manage the database load efficiently while keeping query times fast for historical price lookups. How are you handling the database architecture to support your AI pipeline?

Agreed, and the specific part that surprised me is that the hard problem is usually deciding what to do when the model is confidently wrong in a way you can't detect at runtime. Everything else — prompts, latency, cost — has a knob. That one doesn't, so it becomes a product design question about how much the output is allowed to matter.

Where did it land hardest for you — the eval side, or the UI having to hedge?

The harder-than-expected thing for me wasn't the evaluation logic, it was catching how often the AI I direct made a reasonable-sounding judgment call I never approved. Building Alisio, an invoicing tool, early versions would round a number, or decide two related figures were close enough to merge, all technically defensible, none of it a decision I'd signed off on. The fix wasn't a better prompt, prompts decay over long sessions. It was moving anything touching money out of the model's discretion entirely: hard rules the app checks before rendering a number, so the AI can prepare data but can't be the one deciding what's true. Andalibhq's point about cited sources over verdicts is the presentation layer of the same problem. Mine is upstream of that, whether the number was ever computed the same way twice. Consistency across runs was the real fight, not accuracy on any single run.

The model being the easy part is painfully accurate. With DictaFlow, transcription quality mattered, but users remembered the bugs around it: hotkey timing, text landing in the wrong place, and cleanup changing their wording. A better product test is tracking how often users correct the output or undo an action, not just whether the model returns something plausible. Those recovery actions reveal trust problems faster than benchmarks do.

Same lesson, different product. I'm building a startup idea validation service — the AI part was easy to set up. The hard part was making the output trustworthy enough that a founder would actually make a decision based on it. That's why I switched to manual research with AI just for structure. Users don't trust AI verdicts. They trust cited sources and human judgment.

The false-positive problem seems especially brutal with AI products. A model can produce an impressive result, but if users have to second-guess whether each finding is actually meaningful, the impressive part doesn't matter much. Curious which of those problems took the longest to get under control.