Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed
We introduce GroundAnything, a 4B foundation model that unifies diverse grounding tasks with precise localization and parallel decoding, trained through grounding pretraining, AR-to-diffusion conversion with joint objectives, supervised fine-tuning, and GRPO-based reinforcement learning.
Across 30 benchmarks, GroundAnything surpasses the prior overall state of the art among similarly sized AR models and outperforms the larger Qwen3.7-Max. GroundAnything-VLM establishes a new overall state of the art at this scale and remains competitive with GPT-6 Astra.
We analyze decoding strategies in detail, compare with MTP-based generation, and evaluate progressive infrastructure optimizations for practical deployment.
GroundAnything · diffusion / GroundAnything-VLM · autoregressive
Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising.
Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a 4.51× speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems.
Select a model or capability to explore the values.
Loading results…
| # | Model | Score | Zero-shot |
|---|
Unreported and protocol-mismatched results are excluded.
Zero-shot: YES / NO / UNK (unspecified). External references are listed separately.
Details ↗FindingAcross 30 benchmarks, GroundAnything extends broad and precise grounding to pure diffusion, surpassing the prior overall AR state of the art at comparable scale. GroundAnything-VLM sets a new overall best at this scale, remaining competitive with GPT-6 Astra.
QuestionWhich decoding choices govern the speed–accuracy trade-off?
FindingIn direct DLM decoding, the entropy threshold controls the speed–precision trade-off, providing a decoding-effort knob analogous to thinking effort. We favor τ=0.8 in the B=32 COCO sweep. Optional self-speculation improves both quality and throughput over this direct-decoding point, reaching 4.51× AR throughput with a 0.74 pp F1mIoU gap.
COCO decoding sweep; tokens per forward (TPF) and tokens per second (TPS). Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| Linear Self-Speculation | Block size 4 | TPF | 1.41 |
| Linear Self-Speculation | Block size 8 | TPF | 2.01 |
| Linear Self-Speculation | Block size 16 | TPF | 2.21 |
| Linear Self-Speculation | Block size 32 | TPF | 2.18 |
| Linear Self-Speculation | Block size 4 | TPS | 157.7 tokens/s |
| Linear Self-Speculation | Block size 8 | TPS | 215.9 tokens/s |
| Linear Self-Speculation | Block size 16 | TPS | 226.6 tokens/s |
| Linear Self-Speculation | Block size 32 | TPS | 223.8 tokens/s |
| Quadratic Self-Speculation | Block size 4 | TPF | 2.33 |
| Quadratic Self-Speculation | Block size 8 | TPF | 2.99 |
| Quadratic Self-Speculation | Block size 16 | TPF | 3.17 |
| Quadratic Self-Speculation | Block size 32 | TPF | 3.17 |
| Quadratic Self-Speculation | Block size 4 | TPS | 69.2 tokens/s |
| Quadratic Self-Speculation | Block size 8 | TPS | 83.5 tokens/s |
| Quadratic Self-Speculation | Block size 16 | TPS | 88.0 tokens/s |
| Quadratic Self-Speculation | Block size 32 | TPS | 49.9 tokens/s |
| GroundAnything-VLM | Reference AR reference | TPS | 49.6 tokens/s |
COCO; block size B = 32, sub-block size 4. Quality metric: F1mIoU. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| Entropy-Guided Decoding | Entropy threshold τ 0.2 | TPF | 1.02 |
| Entropy-Guided Decoding | Entropy threshold τ 0.4 | TPF | 1.03 |
| Entropy-Guided Decoding | Entropy threshold τ 0.6 | TPF | 1.11 |
| Entropy-Guided Decoding | Entropy threshold τ 0.8 | TPF | 1.24 |
| Entropy-Guided Decoding | Entropy threshold τ 1.0 | TPF | 1.37 |
| Entropy-Guided Decoding | Entropy threshold τ 1.2 | TPF | 1.60 |
| Entropy-Guided Decoding | Entropy threshold τ 0.2 | TPS | 107.3 tokens/s |
| Entropy-Guided Decoding | Entropy threshold τ 0.4 | TPS | 113.0 tokens/s |
| Entropy-Guided Decoding | Entropy threshold τ 0.6 | TPS | 118.2 tokens/s |
| Entropy-Guided Decoding | Entropy threshold τ 0.8 | TPS | 127.0 tokens/s |
| Entropy-Guided Decoding | Entropy threshold τ 1.0 | TPS | 134.3 tokens/s |
| Entropy-Guided Decoding | Entropy threshold τ 1.2 | TPS | 148.7 tokens/s |
| Entropy-Guided Decoding | Entropy threshold τ 0.2 | COCO F1mIoU | 59.97 % |
| Entropy-Guided Decoding | Entropy threshold τ 0.4 | COCO F1mIoU | 59.92 % |
| Entropy-Guided Decoding | Entropy threshold τ 0.6 | COCO F1mIoU | 59.82 % |
| Entropy-Guided Decoding | Entropy threshold τ 0.8 | COCO F1mIoU | 60.72 % |
| Entropy-Guided Decoding | Entropy threshold τ 1.0 | COCO F1mIoU | 54.06 % |
| Entropy-Guided Decoding | Entropy threshold τ 1.2 | COCO F1mIoU | 56.14 % |
| GroundAnything-VLM | Reference AR reference | TPS | 49.6 tokens/s |
| GroundAnything-VLM | Reference AR reference | COCO F1mIoU | 63.70 % |
COCO; entropy guidance uses τ = 0.8 and sub-block size 4. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| Self-Speculative Decoding | Block size 4 | TPF | 1.41 |
| Self-Speculative Decoding | Block size 8 | TPF | 2.01 |
| Self-Speculative Decoding | Block size 16 | TPF | 2.21 |
| Self-Speculative Decoding | Block size 32 | TPF | 2.18 |
| Self-Speculative Decoding | Block size 4 | TPS | 157.7 tokens/s |
| Self-Speculative Decoding | Block size 8 | TPS | 215.9 tokens/s |
| Self-Speculative Decoding | Block size 16 | TPS | 226.6 tokens/s |
| Self-Speculative Decoding | Block size 32 | TPS | 223.8 tokens/s |
| Self-Speculative Decoding | Block size 4 | COCO F1mIoU | 57.75 % |
| Self-Speculative Decoding | Block size 8 | COCO F1mIoU | 60.39 % |
| Self-Speculative Decoding | Block size 16 | COCO F1mIoU | 60.71 % |
| Self-Speculative Decoding | Block size 32 | COCO F1mIoU | 62.96 % |
| Entropy-Guided Decoding | Block size 4 | TPF | 1.18 |
| Entropy-Guided Decoding | Block size 8 | TPF | 1.21 |
| Entropy-Guided Decoding | Block size 16 | TPF | 1.23 |
| Entropy-Guided Decoding | Block size 32 | TPF | 1.24 |
| Entropy-Guided Decoding | Block size 4 | TPS | 134.8 tokens/s |
| Entropy-Guided Decoding | Block size 8 | TPS | 131.7 tokens/s |
| Entropy-Guided Decoding | Block size 16 | TPS | 136.6 tokens/s |
| Entropy-Guided Decoding | Block size 32 | TPS | 127.0 tokens/s |
| Entropy-Guided Decoding | Block size 4 | COCO F1mIoU | 53.91 % |
| Entropy-Guided Decoding | Block size 8 | COCO F1mIoU | 60.93 % |
| Entropy-Guided Decoding | Block size 16 | COCO F1mIoU | 61.58 % |
| Entropy-Guided Decoding | Block size 32 | COCO F1mIoU | 60.72 % |
| GroundAnything-VLM | Reference AR reference | TPS | 49.6 tokens/s |
| GroundAnything-VLM | Reference AR reference | COCO F1mIoU | 63.70 % |
QuestionWhy use diffusion rather than MTP when both can draft for the same AR verifier?
QuestionAt the same candidate length, which drafter achieves higher acceptance?
QuestionHow much wall-clock time does each accepted draft token cost?
FindingBidirectional diffusion with a limited number of refinement steps improves draft-generation efficiency over the tested causal MTP baseline. At K=16, two passes reduce cost per accepted draft token to 4.70 ms; further refinement raises the cost.
Shared verifier-generated COCO prefixes. Accepted draft tokens exclude verifier corrections and bonuses. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| Causal MTP | Candidate length K 4 | Draft acceptance | 47.4 % |
| Causal MTP | Candidate length K 8 | Draft acceptance | 30.6 % |
| Causal MTP | Candidate length K 16 | Draft acceptance | 16.8 % |
| Causal MTP | Candidate length K 32 | Draft acceptance | 8.5 % |
| Causal MTP | Candidate length K 4 | Accepted draft tokens per round | 1.89 tokens |
| Causal MTP | Candidate length K 8 | Accepted draft tokens per round | 2.45 tokens |
| Causal MTP | Candidate length K 16 | Accepted draft tokens per round | 2.69 tokens |
| Causal MTP | Candidate length K 32 | Accepted draft tokens per round | 2.72 tokens |
| DLM (1 step) | Candidate length K 4 | Draft acceptance | 45.2 % |
| DLM (1 step) | Candidate length K 8 | Draft acceptance | 37.4 % |
| DLM (1 step) | Candidate length K 16 | Draft acceptance | 21.8 % |
| DLM (1 step) | Candidate length K 32 | Draft acceptance | 10.3 % |
| DLM (1 step) | Candidate length K 4 | Accepted draft tokens per round | 1.81 tokens |
| DLM (1 step) | Candidate length K 8 | Accepted draft tokens per round | 2.99 tokens |
| DLM (1 step) | Candidate length K 16 | Accepted draft tokens per round | 3.48 tokens |
| DLM (1 step) | Candidate length K 32 | Accepted draft tokens per round | 3.30 tokens |
Decode-only cost includes drafting, verification, rejected work, and commitment; shared visual encoding and prefix prefill are excluded. The stage breakdown uses K = 16. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| Causal MTP | Candidate length K 4 | Time per accepted draft token | 7.15 ms |
| Causal MTP | Candidate length K 8 | Time per accepted draft token | 6.58 ms |
| Causal MTP | Candidate length K 16 | Time per accepted draft token | 7.71 ms |
| Causal MTP | Candidate length K 32 | Time per accepted draft token | 11.14 ms |
| DLM (1 step) | Candidate length K 4 | Time per accepted draft token | 9.31 ms |
| DLM (1 step) | Candidate length K 8 | Time per accepted draft token | 5.89 ms |
| DLM (1 step) | Candidate length K 16 | Time per accepted draft token | 5.38 ms |
| DLM (1 step) | Candidate length K 32 | Time per accepted draft token | 6.88 ms |
| Full round (K = 16) | Drafter Causal MTP | Time per accepted draft token | 7.71 ms |
| Full round (K = 16) | Drafter DLM · 1 step | Time per accepted draft token | 5.38 ms |
| Full round (K = 16) | Drafter DLM · 2 steps | Time per accepted draft token | 4.70 ms |
| Full round (K = 16) | Drafter DLM · 4 steps | Time per accepted draft token | 6.84 ms |
QuestionCan additional DLM refinement repay its extra forward-pass cost?
Refinement uses K = 16. Fixed-deadline results use long-output prefixes with validation-selected policies; they are not full-dataset request throughput. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| DLM | Denoising steps 1 | Draft acceptance | 21.8 % |
| DLM | Denoising steps 2 | Draft acceptance | 37.2 % |
| DLM | Denoising steps 4 | Draft acceptance | 41.9 % |
| DLM | Denoising steps 8 | Draft acceptance | 43.1 % |
| DLM | Denoising steps 1 | Time per accepted draft token | 5.38 ms |
| DLM | Denoising steps 2 | Time per accepted draft token | 4.70 ms |
| DLM | Denoising steps 4 | Time per accepted draft token | 6.84 ms |
| DLM | Denoising steps 8 | Time per accepted draft token | 11.83 ms |
| DLM | Decode budget (ms) 25 | Committed tokens | 4.2 tokens |
| DLM | Decode budget (ms) 50 | Committed tokens | 8.6 tokens |
| DLM | Decode budget (ms) 100 | Committed tokens | 22.4 tokens |
| DLM | Decode budget (ms) 200 | Committed tokens | 47.9 tokens |
| DLM | Decode budget (ms) 400 | Committed tokens | 97.1 tokens |
| Causal MTP | Decode budget (ms) 25 | Committed tokens | 3.7 tokens |
| Causal MTP | Decode budget (ms) 50 | Committed tokens | 10.6 tokens |
| Causal MTP | Decode budget (ms) 100 | Committed tokens | 21.0 tokens |
| Causal MTP | Decode budget (ms) 200 | Committed tokens | 42.0 tokens |
| Causal MTP | Decode budget (ms) 400 | Committed tokens | 84.6 tokens |
QuestionWhich grounding failures benefit from bidirectional attention?
Matched diagnostic windows at K = 8; both DLMs use two refinement steps. Valid-box misses count unmatched valid proposals, not missed ground-truth instances. GUI is ScreenSpot-Pro. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| Causal MTP | Task COCO | Draft format error | 5.01 % |
| Causal MTP | Task Dense200 | Draft format error | 4.13 % |
| Causal MTP | Task DocLayNet | Draft format error | 3.82 % |
| Causal MTP | Task GUI | Draft format error | 0.55 % |
| Causal MTP | Task COCO | Valid-box miss rate | 12.8 % |
| Causal MTP | Task Dense200 | Valid-box miss rate | 24.9 % |
| Causal MTP | Task DocLayNet | Valid-box miss rate | 19.0 % |
| Causal MTP | Task GUI | Valid-box miss rate | 8.0 % |
| Causal MTP | Task COCO | Draft acceptance | 35.6 % |
| Causal MTP | Task Dense200 | Draft acceptance | 28.1 % |
| Causal MTP | Task DocLayNet | Draft acceptance | 31.7 % |
| Causal MTP | Task GUI | Draft acceptance | 49.6 % |
| DLM (causal) | Task Dense200 | Valid-box miss rate | 21.74 % |
| DLM (bidirectional) | Task COCO | Draft format error | 1.80 % |
| DLM (bidirectional) | Task Dense200 | Draft format error | 1.64 % |
| DLM (bidirectional) | Task DocLayNet | Draft format error | 1.48 % |
| DLM (bidirectional) | Task GUI | Draft format error | 0.49 % |
| DLM (bidirectional) | Task COCO | Valid-box miss rate | 9.5 % |
| DLM (bidirectional) | Task Dense200 | Valid-box miss rate | 14.8 % |
| DLM (bidirectional) | Task DocLayNet | Valid-box miss rate | 11.5 % |
| DLM (bidirectional) | Task GUI | Valid-box miss rate | 7.5 % |
| DLM (bidirectional) | Task COCO | Draft acceptance | 55.8 % |
| DLM (bidirectional) | Task Dense200 | Draft acceptance | 58.5 % |
| DLM (bidirectional) | Task DocLayNet | Draft acceptance | 59.6 % |
| DLM (bidirectional) | Task GUI | Draft acceptance | 54.1 % |
QuestionHow can parallel decoding translate into practical inference speedups?
NVIDIA H800 latency comparison; speedups are relative to each mode’s own native PyTorch implementation. Protocol ↗
| Series | Setting | Metric | Value |
|---|---|---|---|
| + BF16 to FP8 | Mode Dynamic Decoding | Time per output token | 6.59 ms/token · 3.48× |
| + BF16 to FP8 | Mode Static Decoding (4) | Time per output token | 6.76 ms/token · 3.45× |
| + BF16 to FP8 | Mode Entropy-Guided Decoding | Time per output token | 7.15 ms/token · 3.51× |
| + BF16 to FP8 | Mode Self-Speculative Decoding | Time per output token | 3.73 ms/token · 4.06× |
TakeawayGrounding as visual evidence extraction naturally suits bidirectional diffusion: related spatial fields share the same image evidence and can be filled jointly, then selectively refined before commitment. This connects the task's structure to parallel decoding.
TakeawayDiffusion is a generation strategy within a Transformer backbone. Its distinctive value lies in suitable workloads, such as parallel filling and structured outputs, where joint prediction and limited refinement can repay their computational cost.
@article{yu2026groundanything,
title={GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed},
author={Yu, Qize and Fan, Lianrui and Ping, Bowen and Ding, Xini and Song, Zetian and Niu, Junbo and Wang, Kaixuan and Chen, Tianxing and Chen, Yue and He, Minghua and others},
journal={arXiv preprint arXiv:2609.39600},
year={2026}
}