Skip to content
A new frontier for open-weight intelligence: Kimi K3.1 Pro is now available on Modal.
Learn more

Model APIs

Create your own frontier

Access the latest in open weight intelligence, optimized for performant inference on any workload.

Deploy the latest open weight models the day they launch.

Get started
  • Deploy frontier models

    Pay only for the tokens you use, with intelligent routing to maximize KV cache hits and minimize cost.

  • Production-grade quality

    Evals verified by leading model labs for production serving performance.

  • Multi-cloud reliability

    Served from Modal's multi-cloud, autoscaling compute pool for consistent reliability.

Production-grade performance

  • Kernels tuned for inference

    Our FlashAttention-4 contributions add KV parallelism and support irregular memory access for decode-heavy workloads.

  • KV cache-aware routing

    Intelligent, stateful routing keeps cache hit rate high for long-running workloads, reducing time to first token and serving costs.

  • Balance work across replicas

    Load-aware placement splits concurrent sessions across replicas and sends new sessions to new capacity during scale-ups.

How we serve trillions of tokens
Throughput and interactivity comparisonAnimated recreation of the supplied chart. Vertical axis: throughput in tokens per minute per GPU. Horizontal axis: interactivity in tokens per second per user. Both curves show points labeled 6, 4, 2, and 1. The supplied comparisons read 5.6x more efficient and 2.8x faster per user. Point coordinates are approximate visual transcriptions, not underlying measurement data.

Designed for security and data privacy

  • Zero Data Retention (ZDR) by default

    Zero data retention means we never store prompts or responses from Model APIs.

  • No data use

    Your data is never used to train models. Inference runs entirely on Modal, so nothing is shared with model labs.

  • Encrypted in transit

    All traffic to and from Model APIs is encrypted in transit with TLS

Read about our policies

Built on the core of inference research

Diagram of a query attending over a sequence of tokens split across four streaming multiprocessors in parallel.

Optimizing the FlashAttention-4 kernel

Key/value parallelism and irregular memory access make the kernel faster for decode-heavy inference.

Read more
Total throughput comparison: baseline 203 tokens per second, MTP 558 tokens per second (2.75 times baseline), and DFlash 875 tokens per second (4.31 times baseline).

Speculation is all you need

How we train custom speculators for state-of-the-art inference performance.

Read more

Ship your first app in minutes

$30 / month free compute