Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMCut Ministral 3 cold start times by 10x with snapshotsMaximize tokens per second in batch processing with vLLMServe an ultra-low-latency chatbot with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsFine-tune an LLM to replace your CEODeploy a stateless MCP with FastMCPImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Run a background coding agent with OpenCodeBuild a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Deploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous
View on GitHub

Text-to-video generation with Mochi

This example demonstrates how to run the Mochi 1 video generation model by Genmo on Modal.

Here’s one that we generated, inspired by our logo:

Note that the Mochi model, at time of writing, requires several minutes on one H100 to produce a high-quality clip of even a few seconds. So a single video generation therefore costs about $0.33 at our ~$5/hr rate for H100s.

Keep your eyes peeled for improved efficiency as the open source community works on this new model. We welcome PRs to improve the performance of this example!

Setting up the environment for Mochi 

At the time of writing, Mochi is supported natively in the diffusers library, but only in a pre-release version. So we’ll need to install diffusers and transformers from GitHub.

Saving outputs 

On Modal, we save large or expensive-to-compute data to distributed Volumes

We’ll use this for saving our Mochi weights, as well as our video outputs.

Downloading the model 

We download the model weights into Volume cache to speed up cold starts. For more on storing model weights on Modal, see this guide.

This download takes five minutes or more, depending on traffic and network speed.

If you want to launch the download first, before running the rest of the code, use the following command from the folder containing this file:

The --detach flag ensures the download will continue even if you close your terminal or shut down your computer while it’s running.

Setting up our Mochi class 

We’ll use the @cls decorator to define a Modal Class which we use to control the lifecycle of our cloud container.

We configure it to use our image, the distributed volume, and a single H100 GPU.

Running Mochi inference 

We can trigger Mochi inference from our local machine by running the code in the local entrypoint below.

It ensures the model is downloaded to a remote volume, spins up a new replica to generate a video, also saved remotely, and then downloads the video to the local machine.

You can trigger it with:

Optional command line flags can be viewed with:

Using these flags, you can tweak your generation from the command line:

Addenda 

The remainder of the code in this file is utility code.