Is there demand for an LLM change-history API?
I crawl the docs of 15 LLM vendors and record every change: deprecations,
shutdown dates, price moves. Vendors overwrite these pages, so nobody keeps the
history ? the page just says something different than it did last month.
The open question is whether that history is worth selling as an API:
before-and-after diffs, when a date moved, what a price used to be.
It is also the kind of dataset that only gets more useful with age. Right now
it is months of history, not years, which is either the reason to wait or the
reason to start.
Before I build the billing side, two things I would rather hear from people who
have paid for a data API than guess at:
- Has a deprecation or a silent price change ever actually cost you something?
Or did you find out late and shrug? - When a data API charges per row returned instead of per request, does that
read as fair, or as a meter you cannot predict?
The strongest signal seems less whether people find the history useful and more whether a past change has already caused enough damage to justify paying for prevention. Have you found cases where a deprecation or pricing change created a concrete cost or incident?
Thanks for pushing on this — it's the right question and the one I was least ready for.
Honest answer: I don't have customer incident data, so I can't tell you someone lost money. What I do have is a case where the deadline itself moved.
On July 28 Google's deprecation page listed October 16, 2026 as the shutdown date for gemini-2.5-pro, -flash and -flash-lite. On August 3 the date was gone. As of today the same page reads "No shutdown date announced" for all three — 33 days later, with no replacement date.
So a team that read that page in late July scheduled a migration against an October deadline. A team that read it a week later saw no deadline at all. Same first-party page, no notice in either direction.
That part I can prove. Whether it has cost anyone real money I genuinely don't know, which is why I asked rather than assumed. I'm going to go looking for that number before I build anything else — thanks for making me say it out loud.
That deadline change is a useful concrete signal. The next thing I’d be curious about is whether any teams actually changed a migration plan because of the conflicting dates — that would get you closer to consequential demand.
For the API, I'd want separate fields for when your crawler first observed a change and when the vendor says it takes effect, plus the captured source. Otherwise a polling delay can look like a vendor announcement date. That distinction would matter if I were reconstructing why a migration was scheduled.
Everyone in this thread is debating whether "history" is valuable in general, but I think the sellable wedge is narrower and doesn't need that question answered first: a deprecation-date-change alert, not a full history API. The Gemini example you gave is the product, basically — you're not selling "here's the diff," you're selling "we'll tell you the moment a vendor quietly moves a date you're already planning around." That's a much smaller, cheaper thing to build and sell than a general-purpose API, it's usable immediately by anyone doing migration planning regardless of whether they'd ever query historical data, and it sidesteps the whole "is history worth paying for" debate by monetizing the one moment everyone in this thread agrees is a real pain: finding out late.
Your real measurement problem isn't "do people care about history" - it's "how many customers actually measure the cost of silent breaking changes?" Most teams don't measure what they don't see. If a vendor changes a price or deprecates something quietly and your team never measures the impact, you don't have a cost to justify a prevention tool. You probably have lost time or overspending without even knowing it, but if you can't measure it, it doesn't count in a purchasing decision. The per-row pricing question is measurement granularity: "I know what I'm paying for" feels fair, but "I can't predict my bill" feels like you're hiding costs. The sale swings on whether your measurement system lets them feel in control of the number.
This is the most useful reply I've had on the idea, so thank you.
You named the thing I couldn't. "How many teams measure the cost of a change they didn't see" is the real question, and I suspect the answer is near zero — which makes this a vitamin until someone has a number. I'll be carrying that framing into every conversation from here.
On per-row: that's the sharpest version of that objection I've heard, and it lands. "I can't predict my bill" is exactly what I'd be answering on every call, and it sounds like predictability is worth more here than precision. I'll be weighing that seriously before I settle on a meter.