AI Product Engineering · Startups & scale-ups
LLM products that survive a real user.
Retrieval, agents and tool use, built with the parts that are missing from every impressive demo: an evaluation suite wired to the merge gate, cost and latency budgets set in the design, guardrails that are tested, and a trace for every answer.
- RAG & agents
- Eval harnesses
- Cost & latency budgets
Five steps. One of them is the model. The other four are where an LLM feature becomes a product.
- ROUTE40ms
Cheap model decides if the expensive one is needed at all
- RETRIEVE120ms
Your corpus, ranked — every passage carries its citation
- GENERATE740ms
Tool calls, structured output, bounded loop
- VERIFY60ms
Schema, grounding and policy checks before anything is shown
- RESPOND40ms
Answer, sources, and a trace you can open six weeks later
04 fails → escalate
Ungrounded, off-policy or schema-invalid output is withheld and handed to a person. The system is allowed to say it does not know.
Cost and latency are decided here, in the design — not discovered on the first invoice.
The demo was always going to work. LLM features are unusually easy to prototype and unusually hard to make dependable. That gap — not the model, not the prompt — is the entire discipline, and it is the only part worth paying an engineering firm for.
- 1 dayto the demo
A retrieval prototype that answers questions about your documents is an afternoon of work, and it will be genuinely impressive. This is not a differentiator and has not been one for some time.
- The restis the product
Grounding that holds when the question is adversarial. Behaviour when retrieval returns nothing useful. Cost at ten thousand users rather than ten. The trace that explains an answer six weeks after it was given.
- 2human gates, unchanged
AI features are built through the same loop as everything else here — the volume generated, the judgment reviewed. A model in the product does not lower the bar for the code around it.
What we build, specifically. Six decisions that separate an LLM feature you can charge for from one you can only demo. Each of them is a place teams routinely lose a quarter, so each of them is settled at design time rather than after launch.
Retrieval that cites
Chunking and ranking tuned against your actual corpus, not a tutorial default — and every passage carried through to the answer with its source attached. An answer a user cannot check is an answer your support team will be checking instead.
Tools that are safe to call twice
Model-invoked tools are designed like any other integration: narrow, typed, idempotent, and permissioned. A retry after a timeout must not create a second order, and a tool loop needs a hard bound so a confused agent cannot run up a bill.
Structured output, validated
Where the output feeds another system it is schema-constrained and validated before use, with a repair path for the cases that still come back malformed. Parsing prose with a regular expression is not an integration strategy.
State that survives the request
Multi-step work is durable — resumable after a failure, inspectable while running, and cancellable. Agent runs that live only in memory are indistinguishable from lost work the moment anything restarts.
Guardrails on both sides
Input and output policy checks, refusal behaviour that is tested rather than hoped for, prompt-injection handling wherever retrieved or user content reaches the model, and PII handling agreed before the first token is sent anywhere.
Observability built for models
Traces that capture the prompt, the retrieved context, the tool calls and the cost of each request — because when a user reports a bad answer, reproducing it is otherwise impossible, and a system you cannot reproduce is a system you cannot fix.
You cannot ship what you cannot measure. An LLM feature has no compiler and no red test to tell you it broke. Without an evaluation set, every prompt change is a guess, and the way you find out it was a bad one is a customer.
So we build the set from the cases that actually matter: the questions the product exists to answer, the ones it must refuse, the adversarial ones, and every failure discovered during the build — which is the highest-quality source of eval cases there is, and the one most teams throw away.
Thresholds are wired to the merge gate, not to a dashboard. A change that improves one answer and regresses grounding across the suite does not ship, and nobody has to notice it in order for that to happen.
The method, in full — evals, not vibes.
One prompt change. It improved the answer that prompted it — and broke three cases nobody was looking at.
- Grounding0.940.88
- Answer quality0.870.91
- Refusal accuracy0.960.96
Merge blocked · grounding below threshold
Not a ticket, not a dashboard somebody checks on Fridays. The regression stops the change.
Token cost and response time are production numbers. They are budgeted and monitored like any other, because the most common way an AI product dies is not that it stopped working. It is that the unit economics never worked and nobody modelled them until the invoice arrived.
Model routing
Most requests do not need the largest model. Routing cheap-first, with escalation on the cases that warrant it, is usually the single largest cost lever available and it is a design decision, not a later optimisation.
Context discipline
Every token in the prompt is paid for on every call, forever. Retrieval that returns eight good passages beats retrieval that returns forty mediocre ones on cost, on latency and — measurably — on answer quality.
Caching and streaming
Prompt caching for the stable prefix, result caching for the repeated question, streaming so that perceived latency is the first token rather than the last one. The user experience and the invoice improve together.
The model proposes. A person disposes. For anything consequential — money moving, a clinical or legal claim, an irreversible action — the design puts a named human at the point of commitment. Automation removes the typing, never the accountability.
That principle runs in two places at once. It is in the products we build, and it is in how we build them: the same pipeline, the same 2 human gates, on AI work as on everything else. The volume is generated. The judgment is not.
The loop, stage by stage — how we build.
Where we have done this. Two engagements, described at the level the clients permit. One is delivered and named; one is in build and says so.
Botco.ai
RAG-grounded conversational agents for healthcare, behavioural health and government, under HIPAA and SOC 2. The domain where a plausible-sounding wrong answer is not a bug but an incident.
Read the case study In buildAgentic platform for cross-border marketplace selling
An agentic pipeline taking Indian brands onto foreign Amazon marketplaces — scouting through compliance to ad campaigns, with a human checkpoint on every step that commits money or creates a legal obligation.
Read the case studyBe honest with yourself first. This engagement is expensive to run properly and pointless to run badly. If it is the wrong shape for where you are, a different one will serve you better and we will say so on the first call.
This is for you if
- You are building an LLM-backed product, not bolting AI onto a brochure
- It has to be reliable enough to charge for
- You care what it costs and how fast it answers at scale
This isn't for you if
- You want AI added to systems you already run — that is AI Integration & Automation
- You are still deciding whether AI applies at all — start with a free spec
Usually begins as a scoped build — often a nine-week MVP Sprint — and continues as the product finds its shape. The eval sets and cost budgets are handed over with the code, so your team can keep both honest after we have gone.
The ones a technical buyer actually asks. Not covered here? Ask an engineer directly — the address below is read by a person.
info@trivialworks.comWhich models do you build on?
Whichever fits the constraint, and the architecture assumes it will change. We build against a provider-agnostic boundary so a model can be swapped without a rewrite, and the eval suite is what tells you whether the swap was an improvement. Being locked to one vendor at this stage of the market is a risk, not a strategy.
Can you work with our data without it leaving our environment?
Yes. Deployment inside your cloud account, self-hosted or open-weight models where the data cannot leave, and retention and training settings agreed in writing before anything is sent to a third party. For regulated work this is usually the first design constraint rather than a late question.
How do you know the feature actually works?
An evaluation set built from real cases — including the failures found during the build — with thresholds wired to the merge gate, so a change that regresses grounding or refusal accuracy blocks rather than ships. The set is handed over with the code so your team can keep it honest after we are gone.
What does it cost to run, not to build?
That is budgeted during design: expected cost per request and a p95 latency target, then routing, caching and context discipline built to hold them. A design that is comfortable at ten users and ruinous at ten thousand is the most common way an AI product fails commercially rather than technically.
Do you also handle the rest of the product?
Yes — the AI is rarely the whole system. Auth, billing, the admin surfaces, the mobile app and the infrastructure are built by the same team through the same pipeline, which is the point of engaging one team rather than an AI specialist plus an app shop who each blame the other.
We already have production systems. Is this the right service?
Possibly not. Adding AI to systems that already exist and already matter is AI Integration & Automation — same discipline, different constraints. This page is for building an AI-native product. We will tell you which one you are in on the first call.
Bring us the hard half.
Send what you have — a prototype that works on the good cases, a feature that is too expensive to run, or nothing but the problem. You get back a functional specification with the architecture, the eval approach and an estimate. There is no charge for it.