π§ Single-Stream Transformer β A unified 15B-parameter, 40-layer Transformer that jointly processes text, video, and audio via self-attention only. No cross-attention, no multi-stream complexity.
π Exceptional Human-Centric Quality β Expressive facial performance, natural speech-expression coordination, realistic body motion, and accurate audio-video synchronization.
π Multilingual β Supports Chinese (Mandarin & Cantonese), English, Japanese, Korean, German, and French.
β‘ Blazing Fast Inference β Generates a 5-second 256p video in and a 5-second 1080p video in on a single H100 GPU.
daVinci-MagiHuman uses a single-stream Transformer that takes text tokens, an optional reference image latent, and noisy video and audio tokens as input, and jointly denoises the video and audio within a unified token sequence.
Key design choices:
| Component | Description |
|---|---|
| π₯ͺ Sandwich Architecture | First and last 4 layers use modality-specific projections; middle 32 layers share parameters across modalities |
| π Timestep-Free Denoising | No explicit timestep embeddings β the model infers the denoising state directly from input latents |
| π Per-Head Gating | Learned scalar gates with sigmoid activation on each attention head for training stability |
| π Unified Conditioning | Denoising and reference signals handled through a minimal unified interface β no dedicated conditioning branches |
| Matchup | daVinci-MagiHuman Win | Tie | Opponent Win |
|---|:---:|:---:|:---:|
| vs Ovi 1.1 | 80.0% | 8.2% | 11.8% |
| vs LTX 2.3 | 60.9% | 17.2% | 21.9% |
Inference Speed (5-second video, on a single H100 GPU)
| Resolution | Base (s) | Super-Res (s) | Decode (s) | Total (s) |
|---|:---:|:---:|:---:|:---:|
| 256p | 1.6 | β | 0.4 | 2.0 |
| 540p | 1.6 | 5.1 | 1.3 | 8.0 |
| 1080p | 1.6 | 31.0 | 5.8 | 38.4 |
π Efficient Inference Techniques
β‘ Latent-Space Super-Resolution β Two-stage pipeline: generate at low resolution, then refine in latent space, avoiding an extra VAE decode-encode round trip.
π Turbo VAE Decoder β A lightweight re-trained decoder that substantially reduces decoding overhead.
π§ Full-Graph Compilation β MagiCompiler fuses operators across Transformer layers for ~1.2x speedup.
π¨ Distillation β DMD-2 distillation enables generation with only 8 denoising steps (no CFG), without sacrificing quality.
π¦ Getting Started
Option 1: Docker (Recommended)
# Recommended: use the prebuilt MagiHuman image (supports full pipeline including SR 1080p)
docker pull sandai/magi-human:latest
docker run -it --gpus all --network host --ipc host \
-v /path/to/repos:/workspace \
-v /path/to/checkpoints:/models \
--name my-magi-human \
sandai/magi-human:latest \
bash
# Install MagiCompiler
git clone https://github.com/SandAI-org/MagiCompiler.git
cd MagiCompiler
pip install -r requirements.txt
pip install .
cd ..
# Clone daVinci-MagiHuman
git clone https://github.com/GAIR-NLP/daVinci-MagiHuman
cd daVinci-MagiHuman
If you prefer manual setup, follow Option 2 (Conda) below.
If --image_path is omitted, inference/pipeline/entry.py runs T2V.
If --image_path is provided, inference/pipeline/entry.py runs TI2V.
The T2V and TI2V scripts under the same example directory reuse the same checkpoint/config stack. The only difference is whether --image_path is passed.
βοΈ Prompt Guidance
daVinci-MagiHuman uses an Enhanced Prompt system that rewrites user inputs into detailed performance directions optimized for avatar-style video generation. For the full system prompt specification, see prompts/enhanced_prompt_design.md.
Below is a quick reference for writing effective prompts.
Output Structure
Every enhanced prompt has three parts:
Main Body (150β200 words) β A clinical, chronological description of the character's appearance, facial dynamics, vocal delivery, and static cinematography. Written in English regardless of dialogue language.
Dialogue β Repeats all spoken lines in a structured format:
Background Sound β Specifies the most prominent ambient sound:
Background Sound:
<Description of the background sound>
Use <No prominent background sound> if none.
Quick Example
User input: A man in a yellow shirt says "ζηδΊΊε¨δΈθ΅·ηζ΄»δΈθΎεοΌθΏεΈ¦ηει’ε ·ε’"
Enhanced prompt (abbreviated):
A young man with short dark hair, wearing a bright yellow polo shirt, sits stationary. His disposition is earnest and slightly agitated... He speaks with a rapid, emphatic tone, his mouth opening wide as he says, "ζ η δΊΊ ε¨ δΈ θ΅· η ζ΄» δΈ θΎ εοΌθΏ εΈ¦ η ε ι’ ε · ε’..." His brow furrows, lip muscles showing distinct dynamics...
@article{davinci-magihuman-2026,
title={Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model},
author={SII-GAIR and Sand. ai and Chern, Ethan and Teng, Hansi and Sun, Hanwen and Wang, Hao and Pan, Hong and Jia, Hongyu and Su, Jiadi and Li, Jin and Yu, Junjie and Liu, Lijie and Li, Lingzhi and Ye, Lyumanshan and Hu, Min and Wang, Qiangang and Qi, Quanwei and Chern, Steffi and Bu, Tao and Wang, Taoran and Xu, Teren and Zhang, Tianning and Mi, Tiantian and Xu, Weixian and Zhang, Wenqiang and Zhang, Wentai and Yi, Xianping and Cai, Xiaojie and Kang, Xiaoyang and Ma, Yan and Liu, Yixiu and Zhang, Yunbo and Huang, Yunpeng and Lin, Yutong and Tao, Zewei and Liu, Zhaoliang and Zhang, Zheng and Cen, Zhiyao and Yu, Zhixuan and Wang, Zhongshu and Hu, Zhulin and Zhou, Zijin and Guo, Zinan and Cao, Yue and Liu, Pengfei},
journal={arXiv preprint arXiv:2603.21986},
year={2026}
}
Ecosystem Role
Standard MoltPulse indexed agent.
Embed Badge
Show off your Pulse Score in your GitHub README to build trust and rank higher.
