MIND-Grasp
Multiview
Imagination-guided
Next-best-action
Discovery
for Occlusion-Aware Grasping
Video
Method
Occluded Scene Reconstruction
Amodal Imagination and Observable-Corridor Discovery
MIND-Grasp imagines amodal 3D target geometry under partial and full occlusion, then builds a target-centric occlusion cloud to guide closed-loop multiview discovery through observable corridors.
Retraction-Aware Grasp Selection
Object-Level Retreat and Collision Checking
MIND-Grasp plans an object-level retreat trajectory and selects a grasp whose gripper-target composite body can safely follow it; otherwise, the dominant blocking object is removed and the retreat plan is updated.
Results
Visualization of MIND-Grasp
Next-Best-View
Retreat Trajectory
Grasp Execution
Speed up x3
Speed up x3
NBV Discovery and Retreat
MIND-Grasp performs observable-corridor-guided NBV discovery, plans a safe gripper-target retreat trajectory, and executes the final grasp in cluttered occluded scenes.
Full-Occlusion Discovery
From Hidden Target to Executable Grasp
For fully occluded targets, MIND-Grasp removes the occluder, updates the amodal 3D model through discovery, and finally grasps the exposed target.
Real-World Demo
Real-Robot Occluded Grasping Demos
Real-world robot demos show MIND-Grasp performing occlusion-aware discovery, retraction-aware grasping, and target retrieval in cluttered tabletop scenes.
Analysis
Multi-View Reconstruction Refinement
Single-View Reconstruction
Multi-View Refinement
Severe-Occlusion Reconstruction
Single-view reconstruction can produce unreliable geometry and pose under heavy occlusion. MIND-Grasp uses occlusion-cloud-guided NBV planning and multiview updates to recover more complete, pose-consistent target models.
NBV Trajectories
NGF
VISO-Grasp
MIND-Grasp
Target-Centric View Planning
NGF often explores globally with longer redundant paths, while VISO-Grasp may resample similar viewpoints. MIND-Grasp focuses on target-centric occlusion clouds, yielding shorter paths with fewer target-critical views.
Retraction Success Analysis
Retraction Collision Check
The bar height reports collision-free retraction rate, with retained feasible grasps shown above each bar. MIND-Grasp improves retraction reliability through RCC-based composite-body retreat checking.
Discovery-to-Grasp Process
Full-Occlusion Discovery
Starting from an invisible target, MIND-Grasp builds an imaginary amodal hypothesis, selects NBV discovery views from the target-centric occlusion cloud, updates target geometry, and performs retraction-aware grasping.
Future Work
MIND-Grasp Failure Cases
Limitations and Extensions
MIND-Grasp may be misled when multiple hidden regions are plausible, and its retreat analysis depends on the quality of amodal 3D hypotheses. Future work will maintain uncertainty-aware multi-hypothesis beliefs, improve closed-loop RGB-D correction, and extend parallel-gripper grasp-remove to richer primitives such as pushing, dragging, opening, and dexterous-hand manipulation.