EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning

Kallol Saha*1, Vishal Mandadi*1, Jayaram Reddy*1, Ajit Srikanth1,
Aditya Agarwal2, Bipasha Sen2, Arun Singh3, Madhava Krishna1,
1Robotic Research Center IIIT Hyderabad, 2Massachusetts Institute of Technology , 3University of Tartu
*Equal Contribution. Authors ordered randomly
Looking for MS/PhD positions for FY 2024!

Accepted at ICRA 2024

paper Code

EDMP combines the strengths of classical planning and deep-learning for motion-planning in joint space generalizing to arbitrary scenes directly at inference!

EDMP generalizes to arbitrary scenes and objects directly at inference!

EDMP at the time of training!

Abstract

Classical motion planning for robotic manipulation includes a set of general algorithms that aim to minimize a scene-specific cost of executing a given plan to generate a potentially valid collision-free trajectory. This approach offers remarkable adaptability, as they can be directly used for any new scene without needing specific training datasets. However, without a prior understanding of what diverse valid trajectories are and without specially designed cost functions for a given scene, the overall solutions tend to have low success rates. While deep-learning-based algorithms tremendously improve success rates, they are much harder to adopt without specialized training datasets. We propose EDMP, an Ensemble-of-costs- guided Diffusion for Motion Planning that aims to combine the strengths of classical and deep-learning-based motion planning. Our diffusion-based network is trained on a set of diverse general valid trajectories enabling the model to implicitly learn the properties of a valid trajectory. Like classical planning, for any new scene at the time of inference, we compute collision costs and incorporate this cost at each timestep of the diffusion network to generate valid collision-free trajectories. Instead of a single collision cost that may be insufficient in capturing diverse cues across scenes, we use an ensemble of collision costs to guide the diffusion process, significantly improving the success rate compared to classical planners. As we show in our experiments, EDMP outperforms SOTA deep-learning- based methods in most cases while retaining the generalization capabilities primarily associated with classical planners

Overview

Architecture. EDMP leverages a diffusion model alongside an ensemble of l cost functions. The diffusion model denoises a batch of trajectories τ from t = T to 0, while each cost in the ensemble guides a specific sub-batch. We calculate the gradient ∇J of each collision cost (intersection or swept volume) in the ensemble from robot and environment bounding boxes, using differentiable forward kinematics. After denoising is over, the trajectory with minimum swept volume is chosen as the solution.

Results

Visualizing EDMP's diffusion steps at the time of inference!

EDMP generalizes to any scene without (left) and with (right) holding objects!

Based on diffusion, EDMP generates multimodal trajectories! .. As we show in the paper, the ensemble-of-costs further improves the multimodality!

Acknowledgements

We would like to thank Adam Fishman for assisting with MπNets and providing valuable insights into collision checking and benchmarking.

BibTeX


@misc{saha2023edmp,
      title={EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning}, 
      author={Kallol Saha and Vishal Mandadi and Jayaram Reddy and Ajit Srikanth and Aditya Agarwal and Bipasha Sen and Arun Singh and Madhava Krishna},
      year={2023},
      eprint={2309.11414},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}