Skip to main content

​
Deployment

The repo ships Docker Compose for local and single-host use, and Kubernetes manifests for production — namespace, deployment, configmap, ingress, a separate admin ingress, and an HPA.
Admin routes have their own ingress. Keep it internal — it exposes tenant cache controls, diagnostics, and session data, gated only by ADMIN_API_KEY.

​
Health probes

Kafka is probed and reported in readiness but only fails it when HEALTH_READY_REQUIRE_KAFKA=true — so by default a broker outage cannot pull call-serving pods out of rotation.

​
Metrics

Prometheus, with pre-configured Grafana dashboards, plus Alertmanager in the Compose stack.
The dedicated metrics port is unauthenticated by design and must be network-restricted. Do not expose it through a public ingress.
Metrics worth alerting on:
  • escalation_bridge_failed_total — a background escalation failed and the caller got a recovery prompt instead of a human.
  • Provider outcome metrics for ElevenLabs — these reveal circuit-breaker trips before callers report bad audio.
Start every “the agent feels slow” investigation at /call-logs/latency for p50/p95 rather than at a single call. One slow call is usually a provider hiccup; a moved p95 is a configuration change, and the two have completely different fixes.

​
Whose latency is it

“The agent is slow” is two different problems, and the metrics separate them. tts_time_to_first_frame_ms — the gap between deciding to speak and the first audio frame reaching the caller — is labelled by provider and by response mode, so the same query answers whether the speech vendor got slower or whether your own pipeline did.
Split before you optimise. A tts_time_to_first_frame_ms that rose while latency_llm_first_token_ms held flat is a speech-vendor problem, and no amount of prompt tuning will move it — switching the audio provider will. The reverse is equally true, and the two are routinely confused because the caller experiences both as the same silence.

​
Auth transport rollout

GET /admin/alerts/auth-transport evaluates rollout state as ok / warn / error with actionable alert codes, and GET /admin/auth/transport shows the legacy fallback toggle and stream-secret readiness. Use these while migrating off legacy query auth — they tell you whether any tenant still depends on it before you turn it off.

​
Tenant cache

Idle entries also expire automatically per TENANT_CACHE_IDLE_TTL_SECS (default 3600s).

​
Twilio webhook diagnostics

When inbound calls fail to resolve to the right tenant:

​
Resilience

  • Retries with exponential backoff on external API calls.
  • Circuit breaker on ElevenLabs, so transport and 5xx/429 failures don’t cascade.
  • Graceful shutdown with resource cleanup.
  • Rate limiting per IP and per tenant, with automatic bucket cleanup.

​
Load testing

A load-test/ harness ships with the repo for validating capacity before a launch.
Last modified on August 29, 2026