🕺 R-DMesh: Video-Guided 3D Animation via Rectified Dynamic Mesh Flow

SIGGRAPH 2026 · Project page · arXiv 2605.13838 · Weights

Given a static mesh and a reference video, R-DMesh rectifies the mesh to the video's starting pose and generates a temporally consistent animated 4D mesh that follows the motion.

⚠️ Precomputed demo. The full pipeline (Wan2.2-TI2V-5B video diffusion → rectified-flow DiT → dynamic-mesh VAE → Blender export) needs custom CUDA extensions and several GPU-minutes per run — more than a single ZeroGPU window. So the result below was generated once, offline, on an HF Jobs GPU and is served from suvadityamuk/r-dmesh-demo-assets. No live video→mesh inference runs here.

Input: static mesh

Static input mesh (.glb)

Input: reference video

Reference video

Output: animated 4D mesh

Animated 4D mesh (.glb, morph animation)

Output: orbit / multiview render

Rendered animated mesh (fallback)

About this example

Input mesh: warrok_w_kurniawan (converted from the bundled .fbx to .glb).
Reference video: dance7.mp4 (bundled test_data).
Output: an animated .glb (per-frame shape-key morph animation).

Viewer tip: the 3D viewer plays the baked morph animation if your browser's glTF viewer supports morph targets. If it shows only a static pose, use the orbit / multiview video on the right, which is a direct render of the animated 4D mesh.