UniMate

One Unified Model to Animate Diverse Skeletons

Linzhan Mou · Jiahui Lei · Zhiyang Dou · Chenyue Cai · Chaoyue Song · Adam Finkelstein · Szymon Rusinkiewicz

Princeton · UC Berkeley · MIT · NTU

🔥 News

• [2026-10-05] Bring your own rig: rig_preprocess turns any rigged GLB / glTF / FBX, animated or not, into an asset UniMate can animate, with a review step for its joint labels and facing; the sampler takes dataset skeletons and your own assets by reference. UniML3D adds training-ready features, canonical rest-pose meshes and generic / detailed captions. 🛠️
• [2026-09-27] Preview checkpoints are released at HuggingFace; new checkpoints will be synced there. 📦
• [2026-09-06] The training and inference code is released. 🚀
• [2026-08-30] The UniML3D dataset and its data-processing pipeline are released. 🚀
• [2026-08-01] Our Interactive Demo is live — browse our animation results in 3D. 🎮
• [2026-07-18] UniMate is accepted to SIGGRAPH Asia 2026! 🎉

🛠️ Environment Setup

All components share a single conda environment, specified in requirements.txt:

conda create -n unimate python=3.10 -y
conda activate unimate
pip install "setuptools<81"
pip install -r requirements.txt --no-build-isolation

📊 Dataset & Data Processing

We introduce UniML3D, a large-scale dataset of 13,769 text-paired motion clips on 7,430 skeletons covering diverse skeletal topologies — bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects — all brought into a unified canonicalization.

The raw source assets are available on the Hugging Face Hub (collected under UniMate): Mixamo-Animations-Characters, Objaverse-XL-Rigged-Animated and Truebones-ZOO-Annotations (prompts, metadata and renders only). The Truebones ZOO animal motions themselves are a commercial asset pack whose license does not permit redistribution — please purchase the pack directly from Truebones and rebuild the per-clip layout with the build scripts in Truebones-ZOO-Annotations.

The processed dataset is on the Hub as UniML3D: the exported clips with all captions and annotations, the training-ready features, and canonical rest-pose meshes (everything except the Truebones motions and meshes). See data_process/README.md for the full data processing pipeline that turns the raw assets into UniML3D (download → export → rendering → captioning → joint annotation → feature extraction → animation).

🏋️ Training

Training reads the canonicalized clips under dataset/features/<dataset>/. Download them, or build them with stage 4 of the data-processing pipeline (the Truebones clips must be built from the purchased pack):

hf download Linzhan/UniML3D --repo-type dataset --local-dir dataset \
    --include "features/*" --exclude "features/*/videos/*" --exclude "features/*/tpose/*"

Runs are configured by the JSON files in configs/.

Launch with 🤗 Accelerate. Single GPU:

accelerate launch -m unimate.training.train --config configs/uniml3d_60frames_graph_adaln.json

Multi-GPU on one node (e.g. 8 GPUs):

accelerate launch --num_processes 8 -m unimate.training.train --config configs/uniml3d_60frames_graph_adaln.json

python -m unimate.tools.precompute_text_emb --config configs/uniml3d_60frames_graph_adaln.json

📌 Note

The processed UniML3D dataset is released. Its captions were re-processed for this release, so they do not necessarily match the prompts shown on the project page or in the paper. See the released caption style in Truebones · Mixamo · Objaverse. New prompts start with "An object" to generalize across objects.

UniMate is an early step toward text-to-animation for any skeleton, and many motions and skeletons still fail. We believe that scaling up training data — distilled from agents or generated from videos — is a promising direction to close this gap. If you run into failure cases, please open an issue or contact us; they help us improve.

🎬 Inference

Given a rigged 3D asset and a text prompt, UniMate generates articulated motion for arbitrary skeletons in real time — with no per-skeleton retraining and no test-time optimization.

1. Get a model. Download a released checkpoint (configuration, normalization statistics, latest checkpoint), or use the output directory of your own training run:

hf download Linzhan/UniMate --repo-type model --local-dir outputs \
    --include "unimate_uniml3d_f60_v3/*.json" --include "unimate_uniml3d_f60_v3/*.npy" \
    --include "unimate_uniml3d_f60_v3/checkpoints/checkpoint_step_150000.pt"

2a. Animate a dataset skeleton. Download UniML3D into dataset/ (≈ 88 GB; its dataset card shows smaller partial downloads):

hf download Linzhan/UniML3D --repo-type dataset --local-dir dataset

Then name skeletons as <dataset>:<object_type> (truebones:Horse, mixamo, objaverse:<uid>). For example, Garfield, a Gundam, Baymax and a flower from Objaverse-XL, the Mixamo humanoid and a Truebones reindeer (assets/examples/examples.json):

[{"asset": "objaverse:01fcb4e4c36548ca86624b63dfc6b255", "id": "garfield-dance", "prompt": "An object dances in place."},
 {"asset": "objaverse:01fcb4e4c36548ca86624b63dfc6b255", "id": "garfield-fall", "prompt": "An object falls forward."},
 {"asset": "objaverse:42eb6e70ce024c4c9b6ab24118c737b3", "id": "gundam-kick", "prompt": "An object kicks with the right leg."},
 {"asset": "objaverse:42eb6e70ce024c4c9b6ab24118c737b3", "id": "gundam-jump", "prompt": "An object jumps in place."},
 {"asset": "objaverse:e87c7d42f0b3475c9a698da8295244d1", "id": "baymax-walk", "prompt": "An object walks forward."},
 {"asset": "objaverse:e87c7d42f0b3475c9a698da8295244d1", "id": "baymax-punch", "prompt": "An object punches forward with its right arm."},
 {"asset": "objaverse:669a74dd57bbceb71e6e8b1d_fbx", "id": "flower-close", "prompt": "An object closes its petals and bends its stem."},
 {"asset": "mixamo", "id": "mixamo-run", "prompt": "An object runs forward."},
 {"asset": "mixamo", "id": "mixamo-cheer", "prompt": "An object cheers with its arms raised."},
 {"asset": "truebones:Raindeer", "id": "raindeer-walk", "prompt": "An object walks in place."},
 {"asset": "truebones:Raindeer", "id": "raindeer-attack", "prompt": "An object attacks forward."}]

python -m unimate.inference.sample --exp_dir outputs/unimate_uniml3d_f60_v3 \
    --test_cases_json assets/examples/examples.json --num_repetitions 3 --output_dir outputs/samples/examples
# drive each skeleton's canonical mesh: one animated GLB + FBX per motion
# (Truebones meshes are not distributed; SKIP_INVALID=1 skips motions without a mesh)
SKIP_INVALID=1 bash scripts/run_animate_motion.sh outputs/samples/examples

--asset <refs> --prompt <prompts> samples every prompt on every asset without a file. Motions are written to --output_dir (default <exp_dir>/samples); runs into the same directory add to it, and run_animate_motion.sh <dir> drives them all. Training captions, good prompts to start from, are in dataset/features/<dataset>/captions*.json. Mesh driving checks first that motion, conditioning and mesh share one joint order.

2b. Animate your own rigged asset. rig_preprocess turns a rigged GLB / glTF / FBX, animated or not, into an asset directory that --asset takes. It labels the joints with an LLM and stops so you can review the labels and the facing pair before building, since the model is conditioned on both; its guide walks through the review and every option. Three assets processed this way are in assets/examples: a Unitree Go2 quadruped, an eagle and a shark:

python -m unimate.inference.sample --exp_dir outputs/unimate_uniml3d_f60_v3 --asset assets/examples/unitree_go2 \
    --prompt "An object trots forward." "An object rears up on its hind legs." --output_dir outputs/samples/custom
python -m unimate.inference.sample --exp_dir outputs/unimate_uniml3d_f60_v3 --asset assets/examples/eagle \
    --prompt "An object flaps its wings." "An object strikes forward." --output_dir outputs/samples/custom
python -m unimate.inference.sample --exp_dir outputs/unimate_uniml3d_f60_v3 --asset assets/examples/shark \
    --prompt "An object swims and turns around." "An object bites forward." --output_dir outputs/samples/custom
bash scripts/run_animate_motion.sh outputs/samples/custom   # each asset's mesh, animated

🎨 Applications

The same trained model does three more tasks with no extra training. Each is replacement-style sampling: part of the motion is pinned to a known signal and the flow ODE denoises only the rest at every step, so the constraint holds exactly rather than being encouraged by a loss.

Motion in-betweening

Hold chosen keyframes at their ground truth and generate the transitions between them.

{ "mixamo-Air_Squat-000": "An object squats and then stands up." }

KEEP_FRAMES="0,-1" bash scripts/run_sample_motion_inbetween.sh \
    outputs/unimate_uniml3d_f60_v3 cases.json

Text-guided motion editing

Hold chosen joints at their ground-truth motion for every frame and regenerate the rest under a new prompt — keep what should stay, re-animate the rest.

{ "mixamo-Cocky_Head_Turn-000": "An object walks forward." }

KEEP_JOINTS="Neck,Head" bash scripts/run_sample_motion_edit.sh \
    outputs/unimate_uniml3d_f60_v3 cases.json

Motion expansion

Chain several prompts into one long motion. The first segment is generated freely; every later one pins its first few frames to the previous segment's tail, and the segments are stitched at the seam.

{ "mixamo-sequence": ["An object stands up.", "An object walks forward.", "An object turns around in place."] }

EXPAND_OVERLAP=10 bash scripts/run_sample_motion_expand.sh \
    outputs/unimate_uniml3d_f60_v3 cases.json

@article{mou2026unimate,
  title   = {UniMate: One Unified Model to Animate Diverse Skeletons},
  author  = {Mou, Linzhan and Lei, Jiahui and Dou, Zhiyang and Cai, Chenyue and Song, Chaoyue and Finkelstein, Adam and Rusinkiewicz, Szymon},
  journal = {arXiv preprint arXiv:2609.05415},
  year    = {2026}
}