Roll out an LLM gateway

This is the admin's runbook for putting an existing gateway product in front of Claude Code across a team. It assumes the gateway is already installed per its vendor's documentation and meets the requirements below. Developers configuring their own machine want Connect Claude Code to an LLM gateway; the full list of what Claude Code sends lives in the compatibility guide.

Before you start

You need three things:

1. A gateway on HTTPS at its final address. Serve the exact URL you will hand out, not one that redirects. Redirects can drop the request body or strip the credential header, and model discovery treats any redirect as a failure so the credential cannot leak to the target.
2. A provider credential for the gateway. For the Anthropic API that is a Console API key; for a cloud provider, cloud credentials with model access (see Amazon Bedrock, Google Vertex AI or Microsoft Foundry).
3. A way to push files to laptops, such as MDM or configuration management. Set up Claude Code for your organisation compares options if you have none.

What the gateway must do

Serving GET /v1/models is optional but lets Claude Code fill the model picker from the gateway.

Three credentials, three placeholders

Most failed rollouts I have debugged came down to confusing these. The checkpoints below name them explicitly.

Step 1: prove the gateway routes your models

Send a minimal request with whatever key currently works (an admin key, a test key, or your own developer key if the product has no admin credential):

curl "https://claude-gw.northwind.internal/v1/messages" \
  -H "Authorization: Bearer <admin-key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-sonnet-4-6","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}'

Checkpoint. 200 with a content field: the gateway reached the provider. 404: that model name is not routed. 401 from the provider: the gateway's provider credential is wrong. Repeat for every model name in your routing table, because anyone selecting an untested name gets a 404.

Step 2: issue a key per developer

Create one credential per person using your product's key management. Shared keys defeat both per-person attribution and clean offboarding. Re-run the step 1 request with a freshly issued <dev-key>.

Checkpoint. 200 means the developer key works end to end. A 401 here, when step 1 passed, means the key is wrong or not yet active.

Note which header the gateway reads, because it decides the client variable: Authorization: Bearer means ANTHROPIC_AUTH_TOKEN, x-api-key means ANTHROPIC_API_KEY.

Step 3: run Claude Code through it yourself

Type these straight into a terminal (not a .env or settings file) so closing the terminal undoes them:

export ANTHROPIC_BASE_URL=https://claude-gw.northwind.internal
export ANTHROPIC_AUTH_TOKEN="<dev-key>"
claude -p "Answer with the single word: routed"

Checkpoint. You get an answer, and the gateway log shows a POST to /v1/messages with 200. Claude Code appends a query string such as ?beta=true, so match on the path.

If it fails:

• Not logged in. Check the gateway log. Empty log: no credential reached the session, so re-export in this shell. A 401 whose body mentions x-api-key: the gateway wants that header, so use ANTHROPIC_API_KEY instead.
• Failed to authenticate. API Error: 401. A credential was sent and refused. If the log shows the 401 coming from api.anthropic.com or your provider, the developer key was fine and the gateway's provider credential is the problem.
• Apparent hang. A wrong or unreachable base URL causes retries with backoff that can run for minutes. If no request appears in the gateway log, ANTHROPIC_BASE_URL is not pointing at the gateway.

Step 4: distribute the configuration

What to send

Costs explains which background functions use the Haiku-class model.

Through managed settings (recommended)

Put the values in the env block of a managed settings file and push it:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://claude-gw.northwind.internal",
    "CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY": "1"
  },
  "apiKeyHelper": "/opt/northwind/bin/claude-gw-key"
}

A managed ANTHROPIC_BASE_URL beats a developer's shell export and every lower-precedence settings file. The apiKeyHelper approach is the tidy one: a single command that authenticates to your secrets store as the logged-in user, so each laptop fetches its own key and nothing per-person goes in the file. The alternative is delivering keys through your existing secrets process and having developers set ANTHROPIC_AUTH_TOKEN.

Things to avoid and edge cases:

• Never combine a gateway credential with forceLoginMethod, forceLoginOrgUUID or forceLoginGatewayUrl. The first two, with any value, block ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN and apiKeyHelper at startup. Developers see This machine's managed settings require a first-party login, or Administrator policy requires a Cloud gateway sign-in when forceLoginMethod is "gateway" or forceLoginGatewayUrl is set.
• Server-managed settings will not reach these sessions. That delivery route needs a direct connection to api.anthropic.com. Use file-based managed settings, which enforce the same keys. See Server-managed settings.
• The desktop app reads its own third-party inference configuration, not managed settings; push that file via MDM as well.
• CI runners need the base URL and credential in the runner environment.
• WSL on managed Windows reads Windows managed settings only when wslInheritsWindowsSettings is true.

To go further and forbid any other destination, see the allowedProviders example in Other LLM gateways.

HIPAA and gateways

Gateway-routed sessions are not eligible for the HIPAA configuration; HIPAA setup lists which sign-in and connection methods are. You can still restrict features with the managed keys covered in Monitoring usage, but they neither make a session eligible nor cover everything the configuration changes. No managed key turns cloud sessions off, and no setting alone strips Anthropic credentials from child processes. Do not reach for CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC as a substitute: it also kills the auto-updater and leaves WebFetch on.

Without settings distribution

Send each developer the gateway URL, their personal key, which variable to put it in (ANTHROPIC_AUTH_TOKEN or ANTHROPIC_API_KEY) and any conditional variables with values. Point them at the connect page. Naming the variable saves them guessing.

Checkpoint. On a developer machine, claude opens straight into a session with no login screen. In /status, the Anthropic base URL line shows the gateway and, for managed delivery, Setting sources includes managed settings.

Step 5: verify from a developer machine

Test from a laptop, not the gateway host, so the real network path is exercised. A streaming request checks the endpoint, streaming pass-through and routing in one go:

curl -N "https://claude-gw.northwind.internal/v1/messages" \
  -H "Authorization: Bearer <dev-key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-sonnet-4-6","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"list three colours"}]}'

data: lines should trickle in. Everything arriving at once after a pause means the gateway is buffering, which will stall Claude Code. On Windows, pipe the body into curl.exe with --data-binary '@-' to avoid PowerShell quoting problems.

Then open claude and send a message:

• Login prompt. In /status, if Setting sources lacks managed settings, the file never arrived. If it is there, the credential is what is missing.
• Failed to authenticate. The gateway log says which credential failed: its own rejection names the developer key; a 401 from the provider means the provider credential.
• One-time key approval prompt. Expected on first use with ANTHROPIC_API_KEY. ANTHROPIC_AUTH_TOKEN takes over silently.

If you use fast mode, run /fast too. Its availability check bypasses the gateway, so it can report unavailable or disabled while inference is fine; the skip variables in the table fix that.

Finally, find your message in the gateway logs. The credential identifies the developer and the x-claude-code-session-id header groups requests into sessions.

Keeping it healthy

When sizing per-key rate limits, remember the client retries transient failures, 429 included, up to 10 times with backoff and honours Retry-After.

Controlling upgrades

Some behaviour is baked into each Claude Code version, so an upgrade can change things with no gateway change at all. Pin a tested version with requiredMaximumVersion in settings, or DISABLE_UPDATES if you ship Claude Code yourself (see Setup). Before raising the pin, read the release notes and repeat step 3.

Model configuration covers each of these variables.

Related

• Other LLM gateways
• Connect Claude Code to an LLM gateway
• Gateway compatibility guide
• Managed settings
• Set up Claude Code for your organisation