Problem
How much decision freedom should an LLM have in a customer-facing workflow, and where should deterministic business rules take over?
My role
I designed 542 IVR as AI Architect, then delivered it as AI Solutions Manager.
What changed
542 IVR combines a coordinator agent and a tariff agent, with deterministic business rules defining their decision boundaries.
The architecture supports self-hosted and cloud-hosted models. Retrieval-augmented generation (RAG) provides access to relevant information; safety controls, model-assisted evaluation and observability support production operation.
Results
The coordinator handles 100K+ calls per month, with 218K+ calls in the first ~2 months. For the tariff agent, ~23% of recommendations become approved sales leads, with 1,000+ leads. These are approved leads, not completed purchases.
20% model. 80% engineering.
A working principle, not a measured breakdown.
Self-hosted and cloud-hosted models
Orchestration & retrieval
Coordinate agent workflows and retrieve relevant enterprise information through retrieval-augmented generation (RAG).
Guardrails
Apply safety controls around model inputs, outputs and agent behavior.
Business rules
Use deterministic business rules to define which decisions and actions an agent can take.
Tool & backend actions
Connect agents to business services through defined tool and backend interfaces.
Evaluation
Assess response quality and task outcomes with model-assisted evaluation.
Observability
Trace agent workflows and model calls, and monitor production behavior.
“A production LLM agent is 20% model and 80% engineering.”
Related posts
Our production tariff agent understands customer needs, recommends a plan and handles the flow from approval to activation.
LinkedIn post ↗Posted ·The leadership behind the work
Explore the career experiences behind these responsibilities in my desk notebook (in Turkish).