Blog
Articles about LLM observability, cost tracking, and building AI infrastructure.
Building Cascades and Guardrails into Your LLM Agent: From Demo to Production
From a naive triage agent to cascaded, guarded, and logged at every step. Twice, once in pydantic-ai and once in majordomo-llm, so you can see what each idiom gives you.
Read- llm
Beyond the Agent Framework: Two Layers Every Production App Needs
Cascades for reliability, guardrails for governance. The two layers that turn an LLM demo into a production agent, illustrated with a customer support triage agent.
- llm
Running a Live Model A/B Test with Majordomo Experiments
Replay tells you how a cheaper model performs on historical traffic. Experiments tell you how it performs on the traffic you're getting right now. Splitting by weight, reading per-arm metrics, and deciding when to promote.
- llm
How to Test a Model Switch Without Breaking Production
Replay a month of your own production traffic against the candidate model, then read the cost delta, the latency delta, and the requests where the two models disagreed.
- llm
How to Track LLM Costs by Team and Feature
Provider dashboards break spend down by model, which is the one dimension you cannot act on. Tagging requests with feature, team, and environment, and the SQL to query them.
- llm
Your AI Stack Needs a Control Plane
Your database has query logs, your services have traces, and your LLM calls have a monthly invoice. What the missing layer needs to do, and what it looks like in practice.