01
- Proxy setup
- Config.yaml, Postgres, Redis and routing worked out so the gateway is a thing you deploy with the rest of the platform rather than a one-off.
02
- Virtual keys and budgets
- Team-level keys with real spend limits, rate limits and model allowlists, plus alerting that fires before the cap not after.
03
- Model routing
- Model groups, context-size fallbacks, latency routing and cooldown tuning set from real traffic rather than defaults.
04
- Bedrock and Vertex integration
- AWS and GCP identity wired through the proxy correctly, with IAM scoped to the specific models in use and audit trails you can read.
05
- Observability
- Langfuse, LangSmith, Datadog or plain OTLP exporters set up, so spend and latency live in the tool your team already uses.
06
- Managed Tech
- Day-to-day care of the proxy: upgrades without surprises, config changes reviewed before they land, database kept healthy, and someone to call when a provider decides to rotate an endpoint overnight.