Welcome to SaaS LiteLLM¶

A production-ready platform built on top of LiteLLM with job-based cost tracking for multi-tenant SaaS applications. Build your LLM-powered SaaS without exposing infrastructure complexity to your customers.

What is SaaS LiteLLM?¶

SaaS LiteLLM is a complete SaaS wrapper built on top of LiteLLM that provides:

Core Features:

• Job-Based Cost Tracking - Group multiple LLM calls into business operations and track true costs
• Multi-Tenant Architecture - Isolated teams with independent budgets and access controls
• Hidden Complexity - Teams interact with a clean API, never seeing models, pricing, or infrastructure
• Flexible Pricing - Set your own pricing strategy while tracking actual provider costs
• Production-Ready - Deploy to Railway, includes PostgreSQL, Redis, rate limiting, and monitoring

Key Features¶

🎯 SaaS-Ready Architecture¶

• Job-Based Tracking - Group multiple LLM calls into business operations
• Hidden Complexity - Teams never see models, pricing, or LiteLLM
• Cost Aggregation - Track true costs per job, not per API call
• Usage Analytics - Detailed insights per team and job type

💰 Business Features¶

• Cost Transparency - See actual LiteLLM costs vs. customer pricing
• Flexible Pricing - Flat rate, tiered, or markup-based pricing
• Budget Controls - Per-team credit allocation with suspend/pause capabilities
• Profit Tracking - Calculate margins per job/team

🔧 Technical Features¶

• 🚀 Deploy to Railway with Docker
• 🐘 PostgreSQL database with job tracking schema
• 👥 Team and organization management
• 🔑 Virtual API key generation (hidden from teams)
• 🔄 Multiple LLM providers (OpenAI, Anthropic, etc.)
• ⚡ Redis caching for performance and cost savings
• 📊 Rate limiting per team (TPM/RPM)
• 🎛️ Admin dashboard for team management
• 🌊 Server-Sent Events (SSE) streaming support

Quick Links¶

Quick Example¶

Here's a simple example of using the job-based API:

import requests

API = "http://localhost:8003/api"

# 1. Create job for tracking
job = requests.post(f"{API}/jobs/create", json={
    "team_id": "acme-corp",
    "job_type": "document_analysis",
    "metadata": {"document_id": "doc_123"}
}).json()

job_id = job["job_id"]

# 2. Make LLM call
response = requests.post(f"{API}/jobs/{job_id}/llm-call", json={
    "messages": [
        {"role": "user", "content": "Analyze this document..."}
    ]
}).json()

# 3. Complete job
result = requests.post(f"{API}/jobs/{job_id}/complete", json={
    "status": "completed"
}).json()

print(f"Total cost: ${result['costs']['total_cost_usd']}")

const API = "http://localhost:8003/api";

// 1. Create job for tracking
const jobResponse = await fetch(`${API}/jobs/create`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    team_id: "acme-corp",
    job_type: "document_analysis",
    metadata: { document_id: "doc_123" }
  })
});
const job = await jobResponse.json();
const jobId = job.job_id;

// 2. Make LLM call
const llmResponse = await fetch(`${API}/jobs/${jobId}/llm-call`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    messages: [
      { role: "user", content: "Analyze this document..." }
    ]
  })
});

// 3. Complete job
const result = await fetch(`${API}/jobs/${jobId}/complete`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ status: "completed" })
}).json();

console.log(`Total cost: $${result.costs.total_cost_usd}`);

# 1. Create job
JOB_ID=$(curl -X POST http://localhost:8003/api/jobs/create \
  -H "Content-Type: application/json" \
  -d '{"team_id":"acme-corp","job_type":"document_analysis"}' \
  | jq -r '.job_id')

# 2. Make LLM call
curl -X POST http://localhost:8003/api/jobs/$JOB_ID/llm-call \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Analyze this document..."}]}'

# 3. Complete job
curl -X POST http://localhost:8003/api/jobs/$JOB_ID/complete \
  -H "Content-Type: application/json" \
  -d '{"status":"completed"}'

See more examples

Architecture Overview¶

graph TD
    A[Your SaaS Application] --> B[SaaS API :8003]
    B --> C[LiteLLM Proxy :8002]
    C --> D[PostgreSQL Database]
    C --> E[Redis Cache]
    C --> F[OpenAI]
    C --> G[Anthropic]
    C --> H[Other Providers]

    B -.Job Tracking.-> D

    style B fill:#4CAF50
    style C fill:#2196F3
    style D fill:#FF9800
    style E fill:#F44336

Why This Architecture?

1. Hidden Complexity - Teams interact with your SaaS API, not LiteLLM
2. Job-Based Costs - Track costs for business operations, not individual API calls
3. Flexible Pricing - Charge what you want, track actual costs internally
4. Multi-Tenant - Isolated teams with their own budgets and limits

Learn more about the architecture

Documentation Sections¶

For Admin Users¶

• Admin Dashboard Guide - Manage organizations, teams, and model access
• Organizations - Create and manage organizations
• Teams - Team management and virtual keys
• Model Access Groups - Control model access
• Credits - Allocate and monitor credits

For Developers¶

• Integration Overview - How to integrate the API
• Job Workflow - Understanding job-based tracking
• Streaming - Server-Sent Events (SSE) streaming
• Typed Client - Type-safe Python client
• Error Handling - Handle errors gracefully

For System Administrators¶

• Local Development - Docker Compose setup
• Railway Deployment - Deploy to Railway
• Environment Variables - Configuration guide
• Testing - Run integration tests

Resources¶

• Interactive API Docs: ReDoc | Swagger UI
• Admin Dashboard: http://localhost:3002
• LiteLLM Documentation: docs.litellm.ai

Getting Help¶

Next Steps¶

Ready to get started?

1. Follow the Quickstart Guide
2. Learn about the Architecture
3. Try the Examples
4. Integrate into Your App