Provider integration
Claude, OpenAI or an open-weights model behind a stable internal interface, so switching providers later is a configuration change rather than a refactor.
AI & Automation
LLM & API integration
This is the narrower job of adding a model to software that already exists and already works. The constraint is that the existing thing must keep working. The AI feature cannot be allowed to slow down a page, break a deploy, or take the application down when a provider has a bad hour.
Practically that means treating the model provider as what it is: a third-party network dependency with variable latency and occasional outages, and building accordingly.
How I work on it
Process
Adjusted to the job, but this is the shape it usually takes.
-
Isolate the dependency
All model calls go through one module. That boundary is what makes provider changes, caching and instrumentation possible later.
-
Make it asynchronous
Anything slower than a second moves to a queue or a stream. Model calls do not belong in a blocking request cycle.
-
Instrument from day one
Latency, token count, error rate and cost per call, recorded before launch rather than added after the first surprising bill.
-
Load test the failure path
Verify the application behaves correctly when the provider returns 429s and 500s, because eventually it will.
Questions
Things people ask
By keeping the provider behind your own interface and your prompts and evaluation set in your repository. The models differ enough that switching is never free, but with that boundary in place it is a day of work rather than a rebuild.
No. That exposes your API key, removes any ability to rate limit, and makes cost uncontrollable. Calls go through your backend, which authenticates the user and enforces quotas.
It depends entirely on how repetitive the inputs are. On features with a large shared context. A system prompt plus a fixed document set, provider-side prompt caching commonly removes a substantial share of input cost. On fully unique inputs it saves nothing, and I will tell you that before you build for it.