Productionizing AI applications

A promising POC that cannot survive real users, real data, or real load is not almost done. It is a different job. Productionizing AI applications is the work that turns that demo into software: evaluation, data foundations, identity, logging, and a path you can roll back. The model is usually the part you already have.

We do this as custom AI development: first slice in your staging environment, your repos, your IP, fixed scope after discovery. This post is the punch list. No invented customer uptime. No package price.

If you cannot name the workflow or the number, you are not productionizing. You are decorating a demo. See POC versus production.

Admit what the POC actually proved

Most POCs prove that a model can answer a planted question on a clean file. They do not prove permissions, freshness, latency under load, or what happens when the source document changes.

Write down, honestly:

• Which question it answered, and who picked the examples.
• Which data it saw, and whether that data was a production export with the ACLs stripped.
• Whether a human still retypes the output into a system of record.
• Whether anyone but the original builder can run it.

If the answers are "demo data," "laptop," and "only me," you have a spike. Treat it as input to a build plan, not as version 0.9.

Taking over a stalled POC is ordinary work. The usual gaps are eval, data foundations, security, and load — not a larger model.

Put it in an environment you can operate

Production means your cloud, your identity, your logs. A vendor notebook or a shared demo host is not an environment. When the original engineer leaves, the system should still start.

Minimum to call it yours:

• Code in your repositories, including prompts, model pins, and retrieval config.
• Secrets in a store you control, not in a chat history.
• The same identity model as the rest of the app. No god-mode service account "just for the pilot."
• Logs you can query: who called, what was retrieved, whether a write happened, what it cost.
• A rollback you have run in staging.

If you cannot hand the repo to another team with a runbook, you do not have a production app. You have a dependency on a person.

Eval and permissions are the first two tickets

Eval is the test suite for behavior you cannot assert with a unit test. Permissions are how you avoid turning the assistant into a bypass.

Eval:

• Cases from real, redacted tickets, including ones that already failed.
• A pass bar that blocks promote.
• A re-run when the prompt, the model, or the corpus changes.

Permissions:

• Retrieval cannot return a file the user could not open in the source.
• Prompt logs inherit the data class of their contents.
• High-risk writes wait for a human. See agents if the POC was secretly a write path.

Skip either ticket and you will learn it from a customer or an auditor. RAG fails this way more often than any other shape.

Load, cost, and the ugly path are part of the slice

Demos are one user, one question, a warm model. Production is a queue, a retry, a timeout, and a bill.

Before you invite a second team:

• Set rate and cost limits so a loop is a page.
• Test the miss path: no retrieval, conflicting sources, PII in the input.
• Test the write path in dry-run. Double-click must not create two refunds.
• Measure latency on a cold start and on a large document, not only on the demo file.

What "good" looks like on our delivery page is a staging slice in about 30 days after a one-week discovery, with you owning the code. That is a target for a thin slice, not a promise that every POC lands in a month. A messy source of record will eat the calendar. That is a data problem.

Handover is the definition of done

If only the original builder can explain the prompt, you are not done. Done looks like:

• An owner on the inventory row, not a department.
• Docs a second engineer can use.
• Eval score and last promote date on the same row.
• A choice to run it yourselves or keep a team under an SLA. Either way, no lock-in.

Month-to-month or a capped statement of work. No license kickbacks. If the shop will not put the code in your repos, you are buying another demo host.

Count labor hours on the workflow you are productionizing. If the hours do not cover the build on conservative inputs, say so before you polish the UI.

If the POC is already embarrassing you in meetings, the next step is a scoped plan to make it survivable — or a recommendation to stop. See custom AI development.