Peg Insertion
Goal: align a peg with its socket. Success: fully seat the peg.
Tac-Bench
Benchmarking Diverse Tactile Manipulation and Multimodal World Models
RSS Workshop (Tactile Sensing for Robotic Foundation Models)Tac-Bench project video · 2:57
Overview
Tac-Bench is a simulation benchmark for controlled and reproducible evaluation of tactile manipulation and multimodal world models. It controls task, sensor, data, and architecture choices so that gains from touch can be measured rather than confounded.
14
contact-rich tasks
3
tactile sensor types
3
policy fusion architectures
2
physics-aware world-model probes
Task Suite
Fourteen contact-rich tasks span pick-and-place, insertion, assembly, classification, visual occlusion, in-hand orientation, and door manipulation.
USB Insertion
Goal: orient a USB plug to the port. Success: insert it to the seated pose.
Plug Insertion
Goal: align a power plug with its receptacle. Success: seat the plug completely.
Gear Assembly
Goal: place the gear on its shaft. Success: engage the target assembly pose.
Nut-Bolt Screwing
Goal: thread the nut onto the bolt. Success: reach the required screw depth.
Bulb Assembly
Goal: place the bulb into its fixture. Success: achieve the final assembled pose.
Object Search
Goal: find the target through occlusion. Success: make verified target contact.
Peg Reorientation
Goal: reorient a peg with limited visual access. Success: reach the target orientation.
Ball Classification
Goal: sort balls by tactile properties. Success: place each ball in its assigned region.
Test Tube Arrangement
Goal: arrange the test tubes in their holders. Success: place all tubes at target locations.
Stack Cube
Goal: lift and stack the cube. Success: stabilize the cube at its target pose.
Hinge Door
Goal: open the hinged cabinet door. Success: exceed the target hinge angle.
Drawer
Goal: pull the cabinet drawer. Success: reach the required extension.
Sliding Door
Goal: slide the cabinet door open. Success: reach the target travel distance.
Policy Architectures
Tac-Bench controls task data and unimodal encoders while comparing Early Fusion, Compositional Policy, and Cross-Modal Attention.
Early Fusion
EF-DiT and EF-UNet concatenate visual and tactile embeddings before a shared policy network.
Compositional Policy (CP)
CP combines modality-specific diffusion experts through a learned router that assigns consensus weights.
Cross-Modal Attention (CMA)
CMA keeps dedicated modality streams and exchanges information through periodic cross-modal attention gates.
Imitation-learning success rate
TaskEF-DiTEF-UNetCPCMA
Peg Insertion0.120.060.140.10
USB Insertion0.220.060.080.08
Plug Insertion0.180.100.140.10
Ball Classification0.660.740.820.68
Gear Assembly0.100.020.060.04
Bulb Assembly0.040.040.020.06
Nut & Bolt0.220.160.280.06
World-Model Evaluation
Video-fidelity metrics are paired with two functional probes: Closed-Loop Rollout Success (CLRS) and Reachability Gain.
Evaluation Protocols
Imagined visuo-tactile futures are evaluated by whether they can drive inverse dynamics and model-predictive optimization.
Beyond visual fidelity
CLRS decodes actions from imagined goals and executes them in simulation, testing whether generated futures are physically reachable.
Reachability Gain measures whether model-predictive optimization improves an unassisted policy.
Closed-loop rollout examples
USB
World-model target and executed rollout.
Peg
Generated visual goal and simulator execution.
Action-conditioned reachability
Before Optimization · Front view
After Optimization · Front view
Paper
The complete manuscript and source are included with this repository.
@article{anonymous2026tacbench,
title={Tac-Bench: Benchmarking Diverse Tactile Manipulation and Multimodal World Models},
author={Anonymous},
year={2026}
}