Skip to content

AI & Agentic Systems

An $80K a month inference bill cut to $22K at the same quality

Model spend was growing faster than revenue. An audit of the inference path found the waste; routing, caching and budgets removed it.

Client
Consumer AI product
Industry
Consumer AI
Disclosure
Anonymised under NDA
Lower monthly inference bill
73%Lower monthly inference bill
Per month, down from $80K
$22KPer month, down from $80K
Answer quality after the change
SameAnswer quality after the change

The challenge

Model spend grew with every user: a 30% duplicate-call rate nobody had measured, one model size for every prompt and no per-tenant limits.

Stack

  • Model routing
  • Prompt caching
  • Evals
  • Cost telemetry

What we did

  1. 01Traced the inference path and found a 30% duplicate-call rate
  2. 02Routed high-volume prompts to a smaller model
  3. 03Added prompt caching
  4. 04Set per-tenant budgets with alerts
Generic reference architecture
regressedFEATURE STOREfeast · tectonTRAININGray · vertexREGISTRYmlflowEVAL GATElabelled setCANARYshadow trafficSERVINGvllm · tritonCOST · DRIFTper-token budget

A generic representation of the pattern, not the client’s topology.

How you would buy this

Fixed-scope Delivery Sprint

One outcome, one price: a cloud-cost cut, a platform migration, a ClickHouse or IIoT ingestion path, or a production AI-agent pilot.

Compare all engagements →

Fixed fee · 2–6 weeks

Fixed-scope Delivery Sprint

From

$25,000

fixed fee

One outcome, one price: a cloud-cost cut, a platform migration, a ClickHouse or IIoT ingestion path, or a production AI-agent pilot.

  • Written acceptance criteria per milestone
  • Demo at every milestone
  • Source, runbooks and handover included
  • Milestone billing 30 / 40 / 30
Ask about this engagement

More ai & agentic systems work

AI & Agentic SystemsNDA · Connected hardware

An AI agent that resolves 62% of support tickets

Hardware and cloud device vendor

Non-engineers needed to triage field faults across a device fleet. We built the telemetry path and a retrieval-grounded agent over internal runbooks and ticket history.

Support tickets auto-resolved
62%Support tickets auto-resolved
End-to-end answer at p95
<4 sEnd-to-end answer at p95
Languages, English and Hindi
2Languages, English and Hindi

Start with a scoping call.

Thirty minutes with the founder. Bring your architecture diagram, your cloud bill or your plant drawings, and leave with what is realistic — including when the answer is that you do not need us yet.

Mutual NDA before any detail is shared. First response within 24 hours.