Experiments

We evaluate AffordanceVLA on the LIBERO and CALVIN ABC→D simulation benchmarks as well as real-world tasks. We report two variants: AffordanceVLA (w/o stage II), which skips the affordance-augmented robotic co-training, and AffordanceVLA (full), trained with the complete three-stage strategy.

LIBERO Benchmark

Our full model attains a strong average of 95.8% — the highest among the methods compared here and competitive with the best recent VLAs. Strikingly, even without Stage II, AffordanceVLA (w/o stage II) already reaches 86.2%, showing that the decoupled MoT design isolates task-relevant semantics from raw control signals and curbs representation collapse (Q2). The margin narrows only on LIBERO-Long, where extremely long-horizon tasks would further benefit from explicit memory.

CALVIN ABC→D Benchmark

On this zero-shot OOD protocol (train on A/B/C, test on the visually novel Env D), AffordanceVLA (full) reaches a strong average length of 4.33, completing all 5 consecutive tasks in 75.9% of rollouts — competitive among recent VLAs (Q1). By forcing the model to focus on task-critical entities, interaction regions, and spatial layouts, structured affordance prediction makes the perception–action mapping resilient to novel visual disturbances. The substantial jump from the w/o-stage-II variant (3.81) underscores the necessity of Stage II co-training for OOD generalization (Q3).

Real-World Experiments

Across Basic Tasks, AffordanceVLA reaches an average success rate of 88.3%, consistently outperforming the π₀ baseline over diverse objects, colors, and shapes. On Complex Tasks with severe visual aliasing (identical observations, distinct instructions), it unambiguously grounds concise intents into localized affordance heatmaps — e.g., 86.7% / 100.0% on Drawer (pick) / (close) versus π₀'s 46.7% / 40.0% — and sustains long-horizon execution on Pick all the rubbish.

Analysis

Where does the gain come from? A natural concern is whether the improvement stems from high-quality data, added supervision density, or the structured representation itself. We design three controls that each hold one factor fixed:

• Data-Only Control (No-Afd). A plain π₀ architecture on the same Stage II data, without the affordance objective, improves only marginally (LIBERO 92.4%, CALVIN 3.93 vs. π₀'s 3.84). The gain is not data volume per se.
• Frozen-Representation Control (Frozen-Afd). Freezing the affordance expert after Stage I — treating affordance as a fixed external prior — triggers a severe collapse (67.1% / 2.83). Affordance must be co-optimized with the control policy.
• Same-Density Control (Block-wise Tokens). Keeping the losses and data — hence the total supervision density — identical, and only forbidding the three heads from cross-attending, drops results to 90.3% / 3.89. This rules out the “multi-task density” explanation: the structured, jointly-refined representation is what drives the gain.

Together these directly answer Q2: it is the decoupled, jointly-optimized MoT design — not merely more data nor an off-the-shelf affordance module — that prevents collapse and unlocks the gains.

Structured representation (Q1). Removing any single head (Which/Where/How2Act) yields only a graceful degradation rather than a catastrophic collapse — evidence that the three sub-modules are not a brittle Which→Where→How pipeline, but are jointly refined under a shared instruction-aware representation and consumed together by the Action expert. Notably, How2Act's benefit is modest on simple tabletop two-finger settings yet becomes pronounced on complex real-world 6-DoF interactions, exactly where 3D shape and layout priors matter most.

Data Efficiency

Scaling downstream fine-tuning data from 10% to 100%, the vanilla π₀ starts strong (pre-trained weights) but quickly hits a rigid ceiling. AffordanceVLA instead surges: with only 40% of the data it already attains ~92% on LIBERO and an average length above 4.0 on CALVIN, shattering the ceiling of the fully fine-tuned π₀. Because the affordance representation decomposes the perception–action mapping into interpretable sub-problems, each sample supervises not only the action but also object grounding, spatial localization, and 3D reasoning — effectively multiplying the learning signal per sample.

Why Does Affordance Help?

A closer look at failure modes is revealing. In the Toaster task, π₀ performs poorly (toast 26.7% vs. our 86.7%) and its bad cases concentrate at the button-pressing step: instead of extending to press the button, it often closes the gripper as if still doing a pick-and-place, largely disregarding the “press the button” instruction. Even after real-trajectory fine-tuning, π₀ still suffers from weak instruction following — its behavior is driven by the dominant action prior rather than the language command. We offer an intuition for why affordance grounding alleviates this: in an action-only VLA, the low-level action loss is back-propagated directly into the VLM backbone and may gradually erode the instruction-following ability that pre-training endowed. Recent strong policies (π_0.5, π_0.7) implicitly mitigate this with train-only structured supervision; affordance plays a similar — arguably more natural — role, keeping its training signal close to the VLM's semantic space.