Are there independent benchmarks for avatar generation models, or only vendor-/author-reported numbers?

The question

Most performance claims in this space come from the model authors or the vendor selling the model:

• wan-animate reports beating stableanimator, AnimateAnyone, and Champ on automated metrics, and beating runway-act-two and DreamActor-M1 on human eval.
• heygen-avatar-v reports 68.9–85.7% pairwise preference over competitors.
• Closed commercial roundups (Kling 3.0 motion fidelity score 8.4, Aurora "best in class") are vendor-marketing.

Is there an independent benchmarking effort — VBench-class — that compares avatar models on a level playing field?

Why it matters

Without independent benchmarks, picking a model means trusting the vendor or paper authors. For a thread specifically about evaluating model capabilities, this is the load-bearing question.

What we currently believe

Still open. There is no retrieved independent board that ranks the avatar models this thread actually compares (pose-driven SOTA, commercial talking-head / performance-capture, hands, identity) on a shared protocol.

What the 2026-08-21 academic pass did show, from Consensus abstracts only:

• vbench / VBench++ / VBench-2.0 are independent general T2V/I2V suites. They include a subject-identity-inconsistency dimension and a Human Fidelity slice. Retrieved abstracts do not define talking-head, pose-driven, or hand tasks. Author-reported VBench scores (dispose-pose-conditioning) remain uses of a general metric.
• Talking-head academic benches exist: Chen et al. 2020; talking-head-evaluation-benchmarks (THEval 17 models / promised public leaderboards; Zhang 2026 20×7 standardized protocols; Pei 2024 third-party deepfake/talking-face comparison). Abstracts do not name heygen-avatar-v, runway-act-two, or wan-animate.
• Pose-driven fairness is named (humanvid: private training data hampers fair benchmarking) and one author-tied complex-motion bench exists (hypermotionx-bench). No retrieved third-party pose-driven leaderboard.
• artificial-analysis-video-arena remains a general T2V/I2V Elo board, not an avatar board (2026-08-18). This academic pass did not redo those ranks or prices.

Evidence we have

• 2026-05-07-ai-avatar-motion-mimicking-models-survey flags this gap in its "Contradictions and open questions" section.
• DisPose reports VBench improvements over MimicMotion, suggesting VBench is at least applicable as a general metric (dispose-pose-conditioning).
• From 2026-08-18-autoresearch-best-ai-video-generation-tools: artificial-analysis-video-arena is a live independent Elo board for general T2V/I2V. It is not an avatar, identity, or hand leaderboard — it does not close this question.
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: VBench (CVPR 2024) is a 16-dimension open video bench naming subject identity inconsistency; VBench++ adds I2V; VBench-2.0 adds Human Fidelity. None of those abstracts names an avatar task suite.
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: Chen et al. (2020) standardize talking-head eval (identity, lip-sync, quality, natural motion) because author MTurk is unreproducible.
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: THEval (2025) reports 17 models / 85k videos and says leaderboards will be publicly released and updated; the 17 names are not in the abstract.
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: Zhang et al. (2026) report 20 talking-head methods × 7 datasets under sequence-aligned metrics; methods not named.
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: Pei et al. (ACM CSUR 2024) benchmark representative talking-face methods on shared datasets (deepfake framing).
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: HumanVid states inaccessible training data hampers fair pose-driven benchmarking; HyperMotionX Bench is public but author-tied.
• From 2026-08-21-academic-research-independent-avatar-generation-benchmarks: hand-fidelity queries in that pass did not return an animation-generation hand board. See hand-fidelity-comparison-across-avatar-models.

Evidence we need

• The actual THEval / Zhang model lists (full papers or a live leaderboard URL) — abstracts only this pass.
• A third-party pose-driven comparison of AnimateAnyone / MimicMotion / StableAnimator / Wan-Animate on a shared protocol.
• Any board that includes heygen-avatar-v or runway-act-two.
• A hand-specific avatar metric used across models.

How to resolve

• Full-text read of THEval and Zhang 2026 (abstracts cannot name the 17 / 20 methods). Optional /autoresearch or Web Clipper for those paper URLs — do not invent DOIs.
• Do not treat vbench Human Fidelity or artificial-analysis-video-arena Elo as an avatar answer.
• A later pass can look for a live THEval URL; this pass did not fetch landing pages.

Related

• wan-animate
• heygen-avatar-v
• stableanimator
• hand-fidelity-comparison-across-avatar-models
• artificial-analysis-video-arena
• vbench
• talking-head-evaluation-benchmarks
• humanvid
• hypermotionx-bench
• what-are-the-best-tools-the-best-performing-most-cost