A LiteLLM proxy setup puts the API keys you already hold from model providers behind one OpenAI-compatible endpoint that only your own apps can reach. You run it with Docker Compose on a pinned release, hand each app a virtual key with its own budget, and keep the gateway off the public internet.

Key Takeaways:

Use only your own official provider API keys; do not resell or share gateway access.

Pin the image to a release tag and keep secrets in a protected .env file.

Virtual keys, budgets and per-key limits need the Postgres database.

Publish the port on 127.0.0.1 and reach it over SSH or a TLS reverse proxy.

Back up .env with the database: losing the salt key makes stored credentials unreadable.

What can a LiteLLM gateway do, and what can't it?

LiteLLM's gateway exposes an OpenAI-compatible API, so any OpenAI SDK can point at it. Virtual keys are what you hand to applications instead of raw provider keys, and each can carry its own budget, rate limits and model access.［1］ This procedure is based on the official sources listed below; a complete end-to-end test has not been performed. It targets LiteLLM v1.105.1, whose release notes reference the ghcr.io/berriai/litellm:v1.105.1 image.［2］

A gateway cannot create provider access you do not have, lift a provider's rate limits, or change its terms. It is also not a way to reach AI tools from a restricted network: if Cursor or Copilot fails to connect at all, find the cause of the connection failure before adding another layer.

Keep the scope narrow. Use only API keys issued to you through each provider's official developer platform, for your own apps or your own organization. Do not resell access, share virtual keys with outsiders, or convert a consumer chat web session into an API; requests through the gateway remain your usage under each provider's policies, such as OpenAI's usage policies.［3］

Where should a LiteLLM proxy setup run?

A server you maintain is still a server; the maintenance trade-offs of self-hosting apply, and a rented VPS is not the same thing as a VPN service, as the VPS and VPN comparison explains.

Location	Good for	Main risk
Your own workstation	One developer, local tools	Gateway stops when the machine sleeps
Home server on the LAN	Several of your own devices	Port forwarding mistakes expose it
VPS	Always-on apps you run	Public IP: never publish port 4000 openly

Step 1: Prepare the host and protected secrets

Install Docker Engine with the Compose plugin, then create a working directory. The official quickstart generates the master and salt keys with openssl rand -hex 32; this variant also generates a database password:［1］

mkdir -p ~/litellm && cd ~/litellm
umask 077
printf 'LITELLM_MASTER_KEY=sk-%s\nLITELLM_SALT_KEY=sk-%s\nPOSTGRES_PASSWORD=%s\n' "$(openssl rand -hex 32)" "$(openssl rand -hex 32)" "$(openssl rand -hex 24)" > .env
ls -l .env

Open .env in an editor and add OPENAI_API_KEY= and ANTHROPIC_API_KEY= lines with your own keys; typing them in an editor keeps them out of shell history. The master key is the root credential that authorizes management calls and the Admin UI, so treat it like a root password.［1］ The production guide says it must start with sk-.［4］

Step 2: Write a pinned LiteLLM Docker Compose file

The quickstart's compose file uses a moving tag and publishes port 4000 on all interfaces unless told otherwise, and it recommends pinning a release tag beyond local evaluation.［1］ Save this as compose.yaml:

services:
  litellm:
    image: ghcr.io/berriai/litellm:v1.105.1
    command: ["--config", "/app/config.yaml"]
    ports:
      - "127.0.0.1:4000:4000"
    volumes:
      - ./config.yaml:/app/config.yaml:ro
    environment:
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:?set in .env}
      LITELLM_SALT_KEY: ${LITELLM_SALT_KEY:?set in .env}
      DATABASE_URL: postgresql://litellm:${POSTGRES_PASSWORD:?set in .env}@db:5432/litellm
      OPENAI_API_KEY: ${OPENAI_API_KEY:?set in .env}
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:?set in .env}
      LITELLM_MODE: PRODUCTION
    depends_on:
      db:
        condition: service_healthy
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: litellm
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set in .env}
      POSTGRES_DB: litellm
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U litellm"]
      interval: 5s
      timeout: 5s
      retries: 10
    volumes:
      - postgres_data:/var/lib/postgresql/data
volumes:
  postgres_data:

Docker warns that published ports are reachable from outside the host by default; including 127.0.0.1 limits the port to the Docker host.［5］ LITELLM_MODE set to PRODUCTION stops LiteLLM from loading a local .env by itself.［4］ Remove the provider line you do not use.

Step 3: List models in config.yaml

In config.yaml, model_name is the name your apps send, and litellm_params.model is the provider model the gateway calls.［6］ Replace the placeholders with model IDs from your providers' current documentation:

model_list:
  - model_name: openai-main
    litellm_params:
      model: openai/<openai-model-id>
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude-main
    litellm_params:
      model: anthropic/<anthropic-model-id>
      api_key: os.environ/ANTHROPIC_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  trusted_proxy_ranges: []

The os.environ/ form reads the key from the container environment, and Anthropic models use the anthropic/ prefix.［6］［7］ With store_model_in_db off, the file stays fully authoritative.［6］ An empty trusted_proxy_ranges fits clients that connect directly; set your reverse proxy's range later if you add one.［4］

docker compose up -d
docker compose ps
docker compose logs --tail 50 litellm

Step 4: Issue LiteLLM virtual keys with limits

Virtual keys require the database, and you create them by calling /key/generate with the master key.［8］ Read the master key into a variable without echoing it:

read -rs LITELLM_ADMIN && export LITELLM_ADMIN
curl -s http://127.0.0.1:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_ADMIN" \
  -H 'Content-Type: application/json' \
  -d '{"models": ["openai-main"], "max_budget": 5, "budget_duration": "30d", "rpm_limit": 20}'

After a key crosses its max_budget, requests fail, and budget_duration resets spend at the end of each period; rpm limits can also be set per key.［9］ Budgets are enforced only with a database, so also set spending limits in each provider's own console.［1］ Copy the returned key into the app's secret store; give each app its own key.

Step 5: Verify normal calls and rejections

read -rs APP_KEY && export APP_KEY
curl -s http://127.0.0.1:4000/v1/chat/completions \
  -H "Authorization: Bearer $APP_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model": "openai-main", "messages": [{"role": "user", "content": "Say hello in five words."}]}'

This acceptance table describes what to look for; it is not a record of test results.

Check method	Expected observation	If it fails
Call `openai-main` with the virtual key	A chat completion returns	Check the provider key and model ID in logs
Same call with a made-up key	An authentication error, no completion	Confirm `master_key` is set and the key is not reused
Call `claude-main` with this key	Rejected: model not on the key's list	Recreate the key with an explicit `models` list
Exceed the key's rpm limit	Error responses until the window passes	Check that the database is connected
`curl` port 4000 from another device	Connection refused	Fix the `ports` line; keep `127.0.0.1`

To disable a leaked key without deleting it, post the key to /key/block with the master key; /key/unblock reverses it.［8］

Step 6: Reach the gateway from your other devices

Keep the localhost binding. From your laptop, an SSH local forward carries requests to the server's loopback port: ssh -L 4000:127.0.0.1:4000 user@server. If you are unsure how SSH forwarding differs from a VPN tunnel, compare SSH and VPN tunnels.

For several devices on a LAN, put a reverse proxy with a valid TLS certificate in front of the gateway and restrict it with a firewall; never switch the publish line to all interfaces. Point SDKs at the forwarded or proxied URL with each app's virtual key.

Step 7: Upgrade, back up and stop

Read the next release's notes, change the image tag, then run docker compose pull and docker compose up -d. Release notes publish cosign commands to verify image signatures.［2］

Back up .env and the database together, because changing LITELLM_SALT_KEY later makes stored credentials unreadable:［1］

docker compose exec -T db pg_dump -U litellm litellm > litellm-backup.sql
chmod 600 litellm-backup.sql

docker compose down stops the stack and keeps the database volume; docker compose down -v also deletes it and cannot be undone. When you retire the gateway, revoke the provider keys in each provider's console as well.

Summary

Keep the gateway for your own official API keys and your own apps.
Pin the image, protect .env, and bind port 4000 to localhost.
Map friendly model names to provider models in config.yaml.
Give each app a virtual key with models, budget and rate limits.
Back up secrets with the database before upgrades or shutdown.

FAQ

Is LiteLLM free to self-host?

The open-source gateway is MIT-licensed and free to run; you still pay each provider for the API usage your keys generate.

Do I need Postgres for a simple setup?

Only without virtual keys or budgets. Without a database the master key is the sole credential, and budgets are not enforced.

Can I share my gateway with friends?

You should not. Sharing access to your provider keys can breach provider terms and makes you responsible for other people's usage.

Why must the master key start with sk-?

LiteLLM's virtual key and production guides require the sk- prefix for the master key, so generating it that way avoids startup problems.

Can a gateway bypass regional restrictions on AI services?

No. The gateway forwards your own API requests; it does not change provider availability, account eligibility or the terms that apply to you.

What happens if I lose LITELLM_SALT_KEY?

Credentials stored in the database become unreadable, and you must re-enter them; keep the original .env with your backups.

Should I expose port 4000 on my VPS?

No. Keep it on 127.0.0.1 and use an SSH forward or a TLS reverse proxy restricted to your own devices.

Disclaimer: This guide is general technical information compiled from official documentation, not legal advice or a record of completed testing. Follow each model provider's terms for the API keys you use.

Sources:

LiteLLM Docker Quickstart — https://docs.litellm.ai/docs/proxy/docker_quick_start
LiteLLM v1.105.1 release — https://github.com/BerriAI/litellm/releases/tag/v1.105.1
OpenAI Usage Policies — https://openai.com/policies/usage-policies/
LiteLLM Production Best Practices — https://docs.litellm.ai/docs/proxy/prod
Docker Port Publishing and Mapping — https://docs.docker.com/engine/network/port-publishing/
LiteLLM Proxy Config.yaml — https://docs.litellm.ai/docs/proxy/configs
LiteLLM Anthropic Provider — https://docs.litellm.ai/docs/providers/anthropic
LiteLLM Virtual Keys — https://docs.litellm.ai/docs/proxy/virtual_keys
LiteLLM Budgets and Rate Limits — https://docs.litellm.ai/docs/proxy/users

Sources checked 14 October 2026.

Related Articles:

LiteLLM Proxy Setup: Self-Host a Gateway With Your API Keys