ZeyuLing commited on
Commit
bbb4be9
·
verified ·
1 Parent(s): 46cc25e

Update hftrainer metadata

Browse files
Files changed (1) hide show
  1. README.md +2 -103
README.md CHANGED
@@ -1,104 +1,3 @@
1
- ---
2
- license: mit
3
- tags:
4
- - text-to-motion
5
- - human-motion-generation
6
- - motion-generation
7
- - momask
8
- - humanml3d
9
- library_name: hftrainer
10
- pipeline_tag: text-to-3d
11
- ---
12
-
13
- # MoMask — HumanML3D (hftrainer artifact)
14
-
15
- Self-contained hftrainer re-export of the official **MoMask** text-to-motion
16
- model (Guo et al., CVPR 2024) on **HumanML3D-263**. The RVQ-VAE tokenizer,
17
- masked generative transformer, residual transformer and length estimator are
18
- vendored and **numerically parity-equal to the released checkpoints**
19
- (`convert_momask_checkpoint.py --verify` reports max-abs-diff `0.0` on
20
- generation). The CLIP ViT-B/32 text encoder is **not** stored here; it is
21
- reloaded by name at load time.
22
-
23
- - **Task**: Text-to-Motion (T2M)
24
- - **Representation**: HumanML3D-263 (263-dim, 20 fps, 22-joint SMPL skeleton)
25
- - **Tokenizer**: RVQ-VAE, 6 residual quantizers, codebook 512×512
26
- - **Generator**: MaskTransformer (masked iterative decoding) + ResidualTransformer
27
- - **Text encoder**: CLIP ViT-B/32 (frozen, reloaded by name)
28
- - **Paper**: [MoMask: Generative Masked Modeling of 3D Human Motions](https://arxiv.org/abs/2312.00063)
29
- - **Original code**: https://github.com/EricGuo5513/momask-codes
30
-
31
- ## Files
32
-
33
- | File | Contents |
34
- |---|---|
35
- | `momask_config.json` | architecture config for all sub-modules |
36
- | `vq.safetensors` | RVQ-VAE weights |
37
- | `t2m_trans.safetensors` | MaskTransformer (no CLIP) |
38
- | `res_trans.safetensors` | ResidualTransformer (no CLIP) |
39
- | `length_est.safetensors` | LengthEstimator |
40
- | `Mean.npy` / `Std.npy` | 263-dim de-normalization stats |
41
-
42
- ## Usage
43
-
44
- ```python
45
- from huggingface_hub import snapshot_download
46
- from hftrainer.models.motion.momask import MoMaskBundle
47
- from hftrainer.pipelines.momask import MoMaskPipeline
48
-
49
- path = snapshot_download("ZeyuLing/momask-humanml3d")
50
- bundle = MoMaskBundle.from_pretrained(path)
51
- pipe = MoMaskPipeline(bundle, device="cuda")
52
-
53
- # length picked by the length estimator:
54
- motions = pipe.infer_t2m(["a person walks forward then sits down"]) # list of (T, 263)
55
- # or a fixed length (frames @ 20 fps):
56
- motions = pipe.infer_t2m(["a person walks forward then sits down"], [120])
57
- ```
58
-
59
- ## Evaluation (HumanML3D-263, Guo evaluator)
60
-
61
- 20 repeats, n = 3970. Generation under the official HumanML3D protocol
62
- (standard test split, native 263-dim @ 20 fps).
63
-
64
- | Metric | this artifact | MoMask paper |
65
- |---|---|---|
66
- | FID ↓ | **0.097** | 0.045 |
67
- | R-Precision Top-1 / 2 / 3 ↑ | **0.516 / 0.709 / 0.804** | 0.521 / 0.713 / 0.807 |
68
- | MM-Dist ↓ | **2.990** | 2.958 |
69
- | Diversity → | **9.460** | 9.620 |
70
-
71
- R-Precision, MM-Dist and Diversity match the paper essentially exactly. The
72
- small residual FID gap is a data-processing / evaluation-population difference,
73
- not a generation-quality gap (the decode path is verified parity-equal to the
74
- released MoMask inference).
75
-
76
- ## Citation
77
-
78
- ```bibtex
79
- @inproceedings{guo2024momask,
80
- title = {MoMask: Generative Masked Modeling of 3D Human Motions},
81
- author = {Guo, Chuan and Mu, Yuxuan and Javed, Muhammad Gohar and Wang, Sen and Cheng, Li},
82
- booktitle = {CVPR},
83
- year = {2024}
84
- }
85
- ```
86
-
87
- ## Evaluation (MotionStreamer-272, SMPL retarget path)
88
-
89
- For cross-model comparison with the MotionStreamer / HYMotion-M2M evaluator,
90
- native HumanML3D-263 predictions are retargeted through the validated MDM-style
91
- chain: HML263 -> SMPL `motion_135` (IK refine-80, 20 -> 30 fps) ->
92
- MotionStreamer-272 -> `MotionStreamer272Evaluator`.
93
-
94
- | Metric | MoMask (this artifact) | MS-272 GT/Real |
95
- |---|---:|---:|
96
- | FID ↓ | **114.869** | 0.000 |
97
- | R-Precision Top-1 ↑ | **0.485** | 0.706 |
98
- | R-Precision Top-2 ↑ | **0.650** | 0.857 |
99
- | R-Precision Top-3 ↑ | **0.731** | 0.911 |
100
- | MM-Dist ↓ | **19.411** | 15.007 |
101
- | Diversity → | **25.427** | 27.281 |
102
-
103
- Run details: `n_repeats = 20`, `n_samples_used = 7392`, `skipped_no_pred = 0`.
104
 
 
 
1
+ # MoMask hftrainer Artifact
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
+ Load with `MoMaskBundle.from_pretrained(...)`. CLIP ViT-B/32 is an external runtime dependency and is not stored here.
Free AI Image Generator No sign-up. Instant results. Open Now