# Serverless GPU & LLM Inference Benchmark (2026)

> Independent metrics comparing cold start latency, GPU cost per second, and throughput across RunPod, Modal, Replicate, Together AI, and Lambda Labs.

## Core Benchmark Summary
- Fastest Cold Start (Llama-3 8B): Modal (~1.8s) and Together AI Serverless (instant / warm pooled).
- Lowest Cost per A100/H100 Second: RunPod Serverless ($0.00072/sec for A100 80GB) and Lambda Cloud.
- Best Developer Experience for Custom Containers: Modal and RunPod.
- Best for Pure API Drop-in (OpenAI compatible): Together AI and Groq.

## Provider Performance Table
| Provider | Target GPU | Cold Start (Median) | Hourly Equiv. Rate | Scale to Zero | Container Support |
|---|---|---|---|---|---|
| Modal | A100 40GB | 1.8s | $2.85/hr | Yes | Native Python |
| RunPod | A100 80GB | 4.2s | $2.59/hr | Yes | Custom Docker |
| Replicate | A100 80GB | 6.5s | $4.14/hr | Yes | Cog Containers |
| Together AI | H100 (Shared) | 0.0s (Pooled) | Per Token | N/A | Dedicated Endpoints |
| Lambda Labs | A100 80GB | On-Demand only | $1.89/hr | No | VMs |