Peilin Wu 吴沛琳

Hello! I am an incoming Master's student in Robotics, advised by Prof. Max Simchowitz. I earned my bachelor's degree in Computer Science and Technology from the IEEE Honor Class at Shanghai Jiao Tong University (SJTU).

My research lies broadly in robot learning, with a particular focus on dexterous manipulation, multimodal sensing, and world models. I am interested in how robots can leverage their embodiment and diverse sensory modalities to learn generalizable skills, as well as how scalable task and data generation can help them adapt beyond narrowly defined behaviors. Ultimately, I hope to build robotic systems that continually learn through interaction, reason about the physical world, and develop increasingly general forms of embodied intelligence.

Previously, I was a research intern at Harvard's Embodied Minds Lab, supervised by Prof. Yilun Du, where I studied world models for robotics. At SJTU, I worked with Prof. Weinan Zhang in the Apex Data & Knowledge Management Lab. I also interned at Shanghai AI Lab, working on foundation models and humanoid robots.

CV  /  Email  /  GitHub  /  Google Scholar  

Publications

I'm interested in robotics, generative AIs and reinforcement learning.

Tac-Bench: Benchmarking Diverse Tactile Manipulation and Multimodal World Models

Tac-Bench is a simulation benchmark for evaluating tactile manipulation and multimodal world models across 14 contact-rich tasks and three tactile sensor types. It compares three policy fusion architectures under controlled task, sensor, and data settings to measure the benefits of touch. Beyond visual fidelity, it evaluates whether imagined visuo-tactile futures support successful closed-loop execution and improve policy performance through model-predictive optimization.

Structured 4D Latent World Model for Robot Planning

Learned world models in robotics often lack true 3D understanding because they rely on 2D video. This work introduces a 4D Latent World Model that predicts the evolution of 3D scenes in a sparse voxel latent space, conditioned on observations and text. The model can decode to rich 3D formats (e.g., Gaussian Splatting), enabling more complete and consistent scene understanding. Used as a planner with a goal-conditioned inverse dynamics model, it generates higher-quality, physically consistent, multi-view coherent futures and improves manipulation performance, generalization, and real-world deployment.

LoopSR: Looping Sim-and-Real for Lifelong Policy Adaptation of Legged Robots

This project looked into the lifelong policy adaptation situation. The idea behind was that attention should be paid to leveraging real-world data to fine-tune the policy continuously. The work proposed a pipeline to loop simulated training and real-world data collection, with only a limited amount of data to yield eminent performance in both sim-to-sim and sim-to-real experiments.

Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective

This paper focused on modeling the privileged knowledge distillation problem from a theory-based perspective, where we provided mathematical analysis and a simple but effective framework HIB for the problem. Empirical experiments on both simulated and real-world tasks demonstrate that HIB yields improved generalizability compared to previous methods, which achieved about a 10% performance boost compared with baselines in various tasks.

Services

Hobbies

Outside of research, I am an electric guitarist with two and a half years of band experience during my undergraduate studies. I mainly play ACG music, J-pop, math rock, and Midwest emo. I am also a fan of Japanese anime and comics.

Design and source code from Jon Barron's website