Free tiers are a design constraint, not a discount

Three production platforms, no infrastructure bill. That sounds like a procurement story and it isn't. Treating a free tier as a permanent requirement rather than a temporary discount changes the architecture, and it changes it in ways that are mostly good — the same pressure that keeps the bill at zero also keeps the system small.

The systems below are real and running. Where a number appears, it was measured; where a vendor limit appears, check it yourself before relying on it, because pricing pages move faster than blog posts.

The integrations, and why each one was chosen#

The two exceptions are worth naming rather than hiding. Kite Connect is a paid subscription — there is no free tier for placing real orders, and pretending otherwise would be the sort of claim that gets someone's budget wrong. Google Routes is billed per call, and the interesting engineering was bounding it rather than avoiding it.

Running the LLM layer on free models#

This is the part people assume cannot work in production, so it is the part worth the most detail.

Shubh Bhagya's readings, horoscopes and question answering all run on Groq's free tier against open-weight gpt-oss models. Not a trial, not a fallback for when the paid key runs out — that is the production path.

The provider chain is an ordered, comma-separated list, currently two entries pointing at different-sized gpt-oss models. The order is shuffled per request so load spreads across them, and the chain falls through on failure, so a single provider having a bad minute degrades latency instead of returning an error.

Quotas are keyed per model, which is the whole trick#

The single most useful thing I learned here:

And the trap that comes with it: listing two chain entries that resolve to the same model does nothing. They share one bucket while looking like extra capacity — the config appears to have doubled your headroom and has not. That failure is silent, and it is only visible if you know quotas are per-model rather than per-key.

One model had to be thrown out, and the measurement is why#

A third model — a 27B Qwen variant — was removed from the chain deliberately. It rejects the reasoning_effort parameter with a 400, which makes it the one model in the set that cannot be bounded. Unbounded, it reasons until it hits the completion limit, gets truncated mid-thought, and the envelope stripper cleans the partial reasoning down to zero visible characters. A forty-second call returning nothing.

The cost was measurable at the job level: the full two-language horoscope run took 399 seconds with that model in the chain, and 213 seconds without it.

That is the argument for keeping a chain configurable rather than clever. The fix was one entry removed from a list.

Two rules that came out of production#

Bound the reasoning, and output gets better, not worse. Setting reasoning_effort="low" cut completion tokens by roughly 85% on the larger model — 3,072 down to about 302 — and increased the visible output, because less of the budget was spent on reasoning that gets stripped before the reader sees it. Capability is probed at runtime rather than hardcoded: a model that 400s on the parameter is remembered and retried without it. Hardcoded model lists go stale, and two entries in this chain pointed at decommissioned models for days before anyone noticed.

An empty completion is an error, not an answer. The call raises rather than returning an empty string. Before that, a response consisting only of stripped reasoning was indistinguishable from success — and a blank reading got cached and served for a full day.

Prefix caching is not the saving you think#

About 97.6% of a horoscope prompt is byte-identical across the twelve signs — same instructions, same structure, different chart data at the end. That looks like an obvious prefix-caching win.

Measured, Groq's prefix caching hit 1 time in 6. It is opportunistic, not guaranteed, and a cost model that assumes it will hit is a cost model that is wrong five times out of six. Design as though every call is cold; treat a cache hit as a bonus.

The model will ignore your prompt occasionally#

The readings render raw with no markdown filter, so any asterisk reaches the page. The prompt says not to emit markdown. Told not to, the model still emitted bold in 1 of 4 generations, so the rule is enforced by a stripper in code as well.

That ratio is worth internalising for anything user-facing: a prompt instruction is a strong prior, not a guarantee, and anything that must be true gets enforced after generation.

The constraint that shapes everything else#

Shubh Bhagya's storage sits in the US because the Cloud Storage free tier is US-only. Its compute runs in Mumbai because that is where the readers are. Those two facts cannot both be optimised, so every bucket round-trip pays cross-continent latency, permanently.

That is a cost decision creating a latency tax, and it produced the single worst performance bug in the project: the homepage took roughly 13 seconds because it made about 30 sequential bucket round-trips per render.

Scale-to-zero is the other half of the same trade. minScale: 0 means paying nothing while idle and accepting cold starts as the literal price — a deliberate choice rather than an oversight, and the sort of thing that should be written down so nobody "fixes" it later.

Bounding the API that does cost money#

VeoCabs needs road distances between arbitrary Indian cities, and Google Routes bills per call. An agent-facing or bot-crawled quote endpoint is an unbounded bill waiting to happen.

Distance resolution therefore goes through three tiers before anything is billed: curated routes answer from a table, anything else checks a cache in Postgres, and only a genuine miss reaches Google — with the result written back immediately. The cache key is the two endpoints sorted, so Delhi→Jaipur and Jaipur→Delhi are one entry rather than two.

The effect is that cost is bounded by the number of distinct city pairs anyone has ever asked for, rather than by traffic. A route that goes viral is billed once. That property comes entirely from the shape of the cache key.

Email and alerts, chosen on quota#

Transactional email runs on Brevo, and the reason recorded in the code is refreshingly unsentimental: the free tier covers 300 emails a day with no card required. SendGrid and Mailgun would work equally well — Brevo simply won on quota at the time the choice was made. (That number is from the integration's own notes; re-check it before you plan around it.)

Operator alerts go to Telegram rather than email, because the operator already lives in it and replies in seconds. The design detail that matters more than the channel: if Telegram is unconfigured or unreachable, the enquiry is logged and the submission still returns success. A lead is worth more than a notification — failing the form to protect a side effect loses the customer.

What it actually costs you#

Free is not free, and pretending otherwise is how people get burned.

• Cold starts, as the price of scale-to-zero.
• Cross-continent latency, permanently, because the free region and the users are not in the same place.
• Model quality below the frontier. Open-weight gpt-oss models are not the best available. For prose written about data that was already computed deterministically, they are comfortably good enough — but that only holds because the hard reasoning was moved into code first. Free models are viable in production largely to the extent that you have narrowed what the model has to do.
• Vendor lock to whoever's free tier you built around, which is real, and mitigated here only because the provider chain is a config string.

The compensation is that every one of those pressures pushes toward a smaller system: fewer network calls, more caching, less work handed to the model, and constraints written down where the next person will find them.

Cost discipline turns out to be architectural discipline wearing a different hat.