From the first API call to streaming, function calling, RAG, and cost management in production. A complete technical guide for 2026, with examples in Python and JavaScript.
The first decision affects cost, quality, and latency. For most cases: start with each provider's mini/Haiku/Flash tier and move up to frontier only if quality is insufficient.
| Criterion | Choice |
|---|---|
| Maximum quality | Each provider's frontier model — see /ranking |
| Cost-efficiency | Each provider's mini/Haiku/Flash tier |
| Speed (real-time chatbot) | Groq (Llama), Claude Haiku, Gemini Flash |
| Privacy / on-premises | Llama, Mistral, Qwen via Ollama |
| Long context (200K+) | Claude Sonnet, Gemini Pro |
| Code & programming | Claude, ChatGPT, DeepSeek Coder |
Production tip: use LiteLLM as an abstraction layer. Write the code once and swap models by changing a single environment variable. Saves rewrites when pricing or quality shifts.
Every integration starts with a simple call. The pattern is the same across all major providers: messages with roles (system, user, assistant). The model string in the examples below is illustrative — check each provider's current model list before shipping.
from openai import OpenAI
client = OpenAI(api_key="your-api-key-here")
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Explain what RAG is in 2 paragraphs."
}
],
max_tokens=500,
temperature=0.7,
)
print(response.choices[0].message.content)
print(f"Tokens used: {response.usage.total_tokens}")import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
const response = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [
{
role: "system",
content: "You are a helpful assistant.",
},
{
role: "user",
content: "Explain what RAG is in 2 paragraphs.",
},
],
max_tokens: 500,
temperature: 0.7,
});
console.log(response.choices[0].message.content);
console.log(`Tokens: ${response.usage.total_tokens}`);With streaming, tokens appear as they are generated — just like ChatGPT. Perceived speed improves dramatically without reducing total processing time.
with client.chat.completions.stream(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Write a poem about AI"}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True) # prints token by token// app/api/chat/route.ts
import { OpenAIStream, StreamingTextResponse } from "ai"; // Vercel AI SDK
import OpenAI from "openai";
export const runtime = "edge";
const openai = new OpenAI();
export async function POST(req: Request) {
const { messages } = await req.json();
const response = await openai.chat.completions.create({
model: "gpt-4o-mini",
stream: true,
messages,
});
const stream = OpenAIStream(response);
return new StreamingTextResponse(stream);
}Function calling lets the LLM decide when and how to call functions you define. It is the foundation of agents — the model can query external data, call APIs, and execute actions.
import json
tools = [
{
"type": "function",
"function": {
"name": "get_model_price",
"description": "Retrieves the price of an LLM model by name",
"parameters": {
"type": "object",
"properties": {
"model_name": {
"type": "string",
"description": "Model name, e.g. gpt-4o-mini"
}
},
"required": ["model_name"]
}
}
}
]
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "How much does this model cost?"}],
tools=tools,
tool_choice="auto",
)
if response.choices[0].finish_reason == "tool_calls":
tool_call = response.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)
# Execute the real function here
result = get_model_price(args["model_name"])
print(f"Model called: {tool_call.function.name}({args})")RAG (Retrieval-Augmented Generation) lets the LLM answer based on your data without fine-tuning. You retrieve relevant chunks from a vector store and inject them into the request context.
import numpy as np
# 1. Generate embedding for the user's question
def get_embedding(text: str) -> list[float]:
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
# 2. Retrieve relevant chunks (simplified — use pgvector/Pinecone in prod)
def search_context(question: str, documents: list[dict]) -> str:
q_emb = get_embedding(question)
# Compute cosine similarity with each document
scores = []
for doc in documents:
score = np.dot(q_emb, doc["embedding"]) / (
np.linalg.norm(q_emb) * np.linalg.norm(doc["embedding"])
)
scores.append((score, doc["text"]))
# Return top-3 most relevant
top3 = sorted(scores, reverse=True)[:3]
return "\n---\n".join([t for _, t in top3])
# 3. Inject context into the prompt
question = "How much does this model cost?"
context = search_context(question, documents)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": f"""Answer ONLY based on the context below.
If the answer is not in the context, say you don't know.
CONTEXT:
{context}"""
},
{"role": "user", "content": question}
]
)
print(response.choices[0].message.content)In production, caching is the most effective tool for cutting costs. Identical or very similar requests do not need to reach the LLM.
import hashlib
import redis
import json
r = redis.Redis(host="localhost", port=6379, db=0)
def llm_with_cache(messages: list, model: str = "gpt-4o-mini", ttl: int = 3600) -> str:
# Generate deterministic cache key
cache_key = hashlib.sha256(
json.dumps({"model": model, "messages": messages}, sort_keys=True).encode()
).hexdigest()
# Check cache
cached = r.get(cache_key)
if cached:
return json.loads(cached)["content"]
# Call LLM
response = client.chat.completions.create(model=model, messages=messages)
content = response.choices[0].message.content
# Save to cache
r.setex(cache_key, ttl, json.dumps({"content": content}))
return contentmax_tokens to the minimum neededRelated content: