AI-First Architecture for High-Velocity Operations

A pattern shows up in nearly every "AI-powered" internal system we are asked to review. A request arrives, the handler assembles a prompt, a model is called, its response is parsed, and something is written to a database. It demos well. It behaves badly.

The problem is not the model. It is that a non-deterministic, high-latency, occasionally unavailable dependency has been placed on the critical path of an operation that is deterministic, latency sensitive, and required to be correct. Posting a stock movement is not a creative task.

The default that fails

Put a model inline and you adopt four properties, whether or not you wanted them:

• Latency floor. A frontier model round trip is rarely under 600 ms and often several seconds. Every user action now costs that.
• Partial availability. Provider incidents become your incidents. A warehouse cannot stop receiving goods because an API is rate limiting.
• Non-determinism. The same input can produce a different write. Reconciliation and audit become guesswork.
• Unbounded cost. Spend scales with traffic rather than with value delivered.

None of this argues against using models. It argues against using them as a general-purpose execution engine.

Three planes

The split we use is deliberately boring: separate the system into three planes and let each have the properties it needs.

The rule that follows is short enough to keep in your head: a model may propose a write, but never perform one. Proposals land in a queue with a confidence score and a provenance record. The system of record applies them under the same constraints it applies to a human operator.

If removing the model turns your product into a broken system rather than a slower one, the model is in the wrong plane.

A reference shape

For an operations platform — inventory, field service, order management — this is the shape we keep landing on:

ingest        →  extraction worker  →  proposals table  →  review  →  apply
(email, PDF,     (model call, async,    (structured,        (human or   (plain SQL,
 photo, form)     retried, cached)       scored, sourced)     rules)      in a txn)

The extraction worker is the only component that talks to a model, and it is asynchronous. Nothing user-facing waits on it. If the provider is down, proposals queue up and the operator keys entries manually — degraded, not broken.

The proposals table is the contract between the two worlds. It is worth designing carefully:

create table extraction_proposals (
  id            uuid primary key default gen_random_uuid(),
  tenant_id     uuid not null references tenants(id),
  source_kind   text not null,           -- 'email' | 'pdf' | 'photo'
  source_ref    text not null,           -- where it came from, for audit
  target_table  text not null,           -- what it wants to write
  payload       jsonb not null,          -- the proposed row
  confidence    numeric(4,3) not null,
  model         text not null,           -- which model and version
  status        text not null default 'pending'
                check (status in ('pending','applied','rejected','superseded')),
  reviewed_by   uuid references users(id),
  created_at    timestamptz not null default now()
);

create index on extraction_proposals (tenant_id, status, created_at desc);

Two details matter more than they look. source_ref means you can always answer "why does the system believe this?" — six months later, in front of an auditor. And model means that when a provider silently changes a model's behaviour, you can find every row it touched.

Where AI earns its place

Having pushed models off the critical path, it is worth being precise about where they genuinely pay for themselves.

Unstructured input, structured output

Supplier invoices arrive as PDFs, photographs, and forwarded email bodies with no consistent layout. Extraction is the highest-value use we see: it removes hours of keying per week, tolerates being wrong because a human confirms, and gets cheaper every year.

Build-time acceleration

The largest gain is not in the running system at all. It is in construction — scaffolding CRUD, generating migrations from a schema sketch, writing the first pass of tests, translating a spec into a typed client. This is where "AI-first" quietly earns most of its speed, and it carries zero runtime risk because the output is reviewed before it ships.

Retrieval over your own corpus

"Which SOP covers a damaged pallet from this supplier?" is a search problem with a natural-language front door. It fails soft: a bad answer wastes a minute, it does not corrupt stock levels.

Failure modes we have hit

• The invisible retry. An extraction worker retried on timeout, but the first call had already completed downstream. Duplicate proposals, both applied. Fix: idempotency key on (source_ref, model), unique index, and let the second insert lose.
• Confidence theatre. A model's self-reported confidence is not calibrated and should never gate an automatic apply on its own. We now calibrate against a labelled sample per document type before any threshold is trusted.
• Prompt as schema. Output shape enforced only by prompt wording drifts the moment the model updates. Enforce with a real schema — structured output or a validator that rejects and retries — and treat a validation failure as a normal path.
• Silent tenant leakage. A retrieval index built without a tenant filter returned another customer's documents. This is the failure that ends contracts. Tenant scoping belongs in the query, not the prompt.

Checklist before you ship

1. Turn the model off. Does the product degrade, or break? It should degrade.
2. Can every model-influenced row answer "what produced this, from what source, using which model version"?
3. Is there a unique constraint that makes a duplicated call harmless?
4. Is the output validated against a schema you control, with a defined path when validation fails?
5. Is tenant scope enforced in the data layer rather than the prompt?
6. Is there a per-tenant spend ceiling, and what happens when it is reached?

None of this is exotic. It is the same discipline you would apply to any unreliable third-party dependency. The only novelty is that this particular dependency is persuasive enough that teams forget to apply it.