PROVIDER - Serverless GPU compute and model deployment

Modal

Modal turns Python functions into GPU workloads with a decorator and 0 orchestration config, targeting 1 second cold starts with per-second billing that scales to zero, plus queues, schedules and distributed maps for batch pipelines and weight caching; abstraction limits and long-run costs are the gaps.

AI Platform United States 2026.11.19
4.4pts

Detailed Notes

Pros 5 items

Declares GPU-backed functions with a Python decorator, needing 0 lines of container orchestration config
Optimised for 1 second class cold starts, fitting bursty and low-frequency inference work
Bills per second and scales to zero within 1 minute, so idle GPUs cost nothing
Bundles queues, scheduled jobs and distributed maps to orchestrate batch pipelines with 1 primitive set
Caches model weights and image snapshots, cutting repeat deployments under 30 seconds

Cons 4 items

Strong platform abstraction limits low-level customisation
Long-running always-on workloads cost more than reserved GPU instances
Few region choices, so data residency needs availability zone review
Lock-in risk is higher than a generic Kubernetes stack

Service Overview

Modal is a serverless GPU platform where developers declare compute needs in Python and run immediately.

In the AI platform category it is GPU cloud and inference infrastructure, strongest at turning scripts into elastic services.

Key Strengths

  • Python first: decorator deploys
  • Fast cold starts: 1 second class
  • Per-second billing: zero when idle
  • Full primitives: queues and schedules
  • Efficient caching: weights and images

Product Ecosystem

Platform capabilities

Function-level GPU scheduling, serverless autoscaling, persistent storage and caching, queues and cron jobs, distributed maps plus logs and metrics.

Integrations

A Python SDK and CLI pair with mainstream model libraries, vector stores and object storage.

Typical scenarios

On-demand inference APIs, batch data processing, fine-tuning jobs and scheduled generation tasks.

Limitations

  • Abstraction level: limited low-level control
  • Long-run cost: above reserved instances
  • Region count: fewer zones
  • Lock-in: less portable than K8s

Use Cases

  • Bursty inference services (★★★★★): scale to zero saves most
  • Batch data processing (★★★★★): distributed maps
  • Always-on training clusters (★★★☆☆): reserved is cheaper
  • Deep environment control (★★★☆☆): weigh abstraction limits

Pricing

Item Price Notes
Free allowance $0 Monthly compute credits
Usage based Per second Depends on GPU type and memory
Enterprise Contact sales Dedicated capacity and compliance

FAQ

Which GPU types are available?

A range from shared GPUs to current flagship cards; the console lists details.

Are cold starts really that fast?

The vendor targets 1 second starts; image size and dependencies affect reality.

Can it run always-on services?

Yes, though reserved instances are usually cheaper at sustained load.

Are custom images supported?

Yes, through image snapshots and dependency declarations.

Code Examples

import modal

app = modal.App("demo")
image = modal.Image.debian_slim().pip_install("torch", "transformers")

@app.function(gpu="A100", image=image)
def predict(text: str) -> str:
    return f"got {text}"

@app.local_entrypoint()
def main():
    print(predict.remote("hello"))
# Deploy and invoke
pip install modal
modal deploy demo.py

Community & Resources

Visit Modal Website → https://modal.com/
Visit Website ↗