Scroll to explore

ICRA Submission · 2026

PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Models

Privileged latent dynamics as training-time supervision only — a future head aligned to short-horizon, contact-centric scene change, discarded entirely before deployment.

Davood Soleymanzadeh¹ Kaidi Zhang² Zhiyuan Zhang² Bihao Zhang¹ Xiao Liang¹ Yu She² Minghui Zheng¹

Paper (Coming Soon) arXiv (Coming Soon) Code (Coming Soon) BibTeX (Coming Soon)

Abstract

Vision-language-action models (VLAs) have shown strong promise for general-purpose robotic manipulation by mapping language instructions and vision observations directly to actions. However, most VLAs primarily condition action prediction on current observations and lack an explicit mechanism for reasoning over future task dynamics, which is particularly important for fine-grained, contact-rich manipulation. We present PHR-VLA, a framework that enables planning-horizon reasoning in VLAs through privileged latent representations of future dynamics. PHR-VLA introduces a lightweight auxiliary future head that, during training, aligns the VLA’s internal representations with latent dynamics extracted from future observations. Evaluation results demonstrate that the most effective planning horizon signals are short-horizon, local, and contact-centric. Patch-level latent dynamics supervision from the wrist camera improves success rate on LIBERO from 84.1% to 88.4% and on real-world disassembly tasks from 63.3% to 82.5%. Patch-level supervision from a third-person camera also improves performance on Meta-World from 56.70% to 57.8%. These results demonstrate that privileged latent dynamics alignment provides an effective training signal for improving anticipatory reasoning in VLA policies.

Overview

Reason over the future without paying for it at inference time.

PHR-VLA aligns action-token representations with privileged latent dynamics computed from planning-horizon demonstration frames during training, then discards the future head and auxiliary encoder entirely at deployment.

84.1% → 88.4%

LIBERO average success rate, with wrist-camera patch-level latent dynamics supervision.

63.3% → 82.5%

real-world contact-rich disassembly success rate on a Franka Emika Panda arm.

56.7% → 57.8%

Meta-World average success rate with third-person patch-level supervision.

Zero

inference-time latency or memory overhead — the future head is only used during training.

PHR-VLA overall performance

LIBERO Meta-World Real-World

PHR-VLA SigLIP 88.4% 57.8% 82.5%

PHR-VLA JEPA 86.2% 59.8% 77.5%

SmolVLA 84.1% 56.7% 63.3%

Diffusion Policy 72.4% 10.5% 51.7%

ACT 24.2% 49.0% 35.8%

PHR-VLA over the strongest baseline (SmolVLA), panel by panel — success rate 1.05×, 1.05×, 1.30×. Bars show average success rate on each domain.

View as table

Method	LIBERO	Meta-World	Real-World
PHR-VLA (SigLIP)	88.4%	57.8%	82.5%
PHR-VLA (JEPA)	86.2%	59.8%	77.5%
SmolVLA	84.1%	56.7%	63.3%
Diffusion Policy	72.4%	10.5%	51.7%
ACT	24.2%	49.0%	35.8%

Method

Supervise how the scene changes, not what it looks like.

Rather than predicting a single absolute future observation, PHR-VLA supervises the change in latent scene representation over the planning horizon — the latent dynamics between consecutive frozen-encoder embeddings of future frames.

y_t:t+H = s_t+1:t+H − s_t:t+H−1

Encode Future Frames

Planning-horizon camera frames I_t+1:t+H are passed through a frozen vision encoder to compute latent targets.

Compute Latent Dynamics

Consecutive latents are subtracted to form a target that emphasizes scene change rather than static content.

Align Action-Token Latents

An auxiliary future head predicts the latent dynamics from the VLA's action-token representations, supervised by L_Align.

PHR-VLA architecture

PHR-VLA framework: privileged finetuning with a future head aligned to latent dynamics during training (left), and efficient inference using only the action head (right)

Fig. 2 — PHR-VLA Framework. Privileged Finetuning (left): planning-horizon frames are encoded by a frozen vision encoder to compute latent-dynamics targets that supervise an auxiliary future head. Efficient Inference (right): the action head runs exactly as in the base VLA.

Added Inference Cost

0ms

Future head & frozen encoder are training-only.

LIBERO Success Rate

88.4%

+4.3 points over SmolVLA.

jump to

future target source

future head

Wrist-camera training (ours)

The wrist camera supplies the planning-horizon target during training — the largest gain (88.4% on LIBERO) because it captures the short-range, contact-centric interactions most relevant to the task.

Training Setup

We adopt SmolVLA (0.45B) as our VLA baseline, which couples a compact SmolVLM-2 backbone with a flow-matching action expert. We fine-tune on a single NVIDIA A100 GPU, which enables controlled ablations of the planning-horizon dynamics-aware alignment objective L_PHR-VLA = L_VLA + λL_Align. The same training setup is used for SmolVLA and all PHR-VLA ablation variants to ensure a fair comparison.

Part I

What PHR-VLA Does

Consistent gains over SmolVLA and every baseline across two simulation benchmarks and a real-world disassembly suite — at zero added inference cost.

01Results

We evaluate PHR-VLA against ACT, Diffusion Policy, Octo, DiT Policy, OpenVLA, TinyVLA, and SmolVLA across two multi-task simulation benchmarks — LIBERO and Meta-World — and a real-world contact-rich disassembly task suite. PHR-VLA uses patch-level, wrist-camera, latent-dynamics supervision at λ = 0.02 as the default configuration.

LIBERO

Method	Spatial ↑	Object ↑	Goal ↑	Long ↑	Average ↑
ACT	30.7%	48.4%	2.0%	15.7%	24.2%
Diffusion Policy	78.3%	92.5%	68.3%	50.5%	72.4%
Octo	78.9%	85.7%	84.6%	51.1%	75.1%
DiT Policy	84.2%	96.3%	85.4%	63.8%	82.4%
OpenVLA	84.7%	88.4%	79.2%	53.7%	76.5%
SmolVLA	86.0%	97.7%	86.7%	66.0%	84.1%
PHR-VLA (ours)	88.7%	98.0%	92.4%	74.4%	88.4%

Success rates across four LIBERO task suites (300 trials per suite). The gains are most pronounced on LIBERO-Long, where PHR-VLA improves performance by +8 points.

Meta-World

Method	Easy ↑	Medium ↑	Hard ↑	Very Hard ↑	Average ↑
ACT	68.2%	49.7%	28.9%	49.4%	49.0%
Diffusion Policy	23.1%	10.7%	1.9%	6.1%	10.5%
TinyVLA	77.6%	21.5%	11.4%	15.8%	31.6%
SmolVLA	82.0%	54.2%	43.9%	46.7%	56.7%
PHR-VLA (ours)	85.2%	55.1%	45.5%	45.3%	57.8%

Success rates across all Meta-World difficulty levels. Because Meta-World provides only a third-person view, wrist-camera fine-grained supervision is unavailable in this domain.

02Real-World Deployment

We deploy PHR-VLA on a 7-DoF Franka Emika Panda arm with parallel grippers, using a fixed third-view camera and a wrist-mounted camera. We design four contact-rich disassembly tasks that each require fine-grained adjustments for successful completion, and collect 100 human teleoperation demonstrations per task at 10 Hz.

Task I — Press & Release Clip. Press the clip to release the object, remove it, then place it in the purple container.

Success

Failure

Task II — Loose Shaft Removal. Remove the object from the loose shaft, then place it in the purple container.

Success

Failure

Task III — Slide & Pull. Slide the object inward, pull it out, then place it in the purple container.

Success

Failure

Task IV — Twist & Extract. Twist the object 90 degrees and pull it out from the shaft, then place it in the purple container.

Success

Failure

A sample of PHR-VLA rollouts across all four tasks, including both successes and failures. Click any clip to expand it.

Real-World Success Rate

Real-world disassembly success rate. PHR-VLA (SigLIP) raises total success from 63.3% (SmolVLA) to 82.5%, with the largest gains on Task IV — the most contact-rich, twist-and-extract task.

Part II

How It Compares

Five controlled ablations isolate which design choices — camera view, target granularity, target formulation, loss weight, and encoder — drive the gains.

03Ablations

We compare three views for the auxiliary target: wrist-mounted, fixed third-person, and a multi-camera variant that concatenates latents from both, using patch-level, latent-dynamics supervision at λ = 0.02. All three views improve over the SmolVLA baseline (84.1%), but the wrist camera yields the largest gain (88.4%) because it captures the short-range, contact-centric interactions that are most task-relevant over the planning horizon.

We compare patch-level against mean-pooled latent targets under latent-dynamics supervision at λ = 0.02, across all three camera views. Patch-level supervision outperforms mean pooling in every view: mean-pooling discards where the scene changes, whereas patch-level targets retain per-region dynamics, giving the future head a denser and more spatially grounded signal.

We compare our latent-dynamics target against directly supervising absolute future latents, holding view, granularity, and λ fixed. Supervising change rather than absolute content relieves the future head of reconstructing static, task-irrelevant scene content already present in the current observation, concentrating the training signal on action-relevant scene evolution.

We sweep the auxiliary loss weight λ using patch-level, wrist-camera, latent-dynamics supervision. Increasing λ from 0.005 to 0.02 improves the average LIBERO success rate from 85.8% to 88.4%; we adopt λ = 0.02 as our default auxiliary loss weight.

PHR-VLA is agnostic to the frozen encoder used to compute planning-horizon targets. We compare SigLIP, a vision-language-aligned image encoder, against V-JEPA 2, a self-supervised video encoder trained explicitly for dynamics prediction. Both encoders consistently exceed their respective no-future-supervision baselines across all three benchmarks.

Benchmark	SmolVLA	PHR-VLA (SigLIP)	PHR-VLA (JEPA)
LIBERO (wrist, patch, dynamics)	84.1%	88.4%	86.2%
Meta-World (patch, dynamics)	56.7%	57.8%	59.8%
Real-world disassembly	63.3%	82.5%	77.5%

On LIBERO and real-world disassembly, SigLIP is the stronger target encoder; on Meta-World, JEPA (with mean-pooled targets) is slightly stronger, reaching 59.8%.

Conclusion and Limitations

PHR-VLA is a training-time planning-horizon supervision framework for VLA policies. It aligns action-token representations with privileged latent dynamics computed from planning-horizon demonstration frames, then discards the future head and auxiliary encoder at inference. PHR-VLA improves success rate consistently across standard manipulation benchmarks and real-world disassembly tasks. Our ablations further show that target structure, not merely the presence of a future-prediction objective, drives the gain: patch-level targets outperform mean-pooled targets, wrist-camera supervision outperforms third-person and multi-camera supervision, and supervising latent dynamics outperforms supervising absolute future latents, consistently across the configurations we tested.

Training-time only, no test-time correction

PHR-VLA is an auxiliary training objective, not an inference-time planner or world model: it only shapes the policy’s representations during fine-tuning and performs no test-time correction.

Vision-only future targets

The future head is currently supervised purely from RGB latent dynamics. Extending it with tactile or force-aware future targets, and object-centric patch supervision, are promising directions for future work.

BibTeX

@article{soleymanzadeh2026phrvla,
  title   = {PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Models},
  author  = {Soleymanzadeh, Davood and Zhang, Kaidi and Zhang, Zhiyuan and Zhang, Bihao and Liang, Xiao and She, Yu and Zheng, Minghui},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}