Grasping with planner

grasping-with-planner

Top-down grasping with a fast axis-locked linear descend and a collision-aware cuRobo fallback. The primary path rises to a hover height, translates over the target while rotating to the grasp yaw, and descends straight down (Z-only, orientation locked) onto the object — which, unlike a goalset planner, happily grips a flat object by simply lowering onto it. If the straight-line solve is infeasible, the same node hands off to the cuRobo planner, which builds a per-observation collision world and searches the whole candidate fan for a reachable, collision-free wrist. This is the grocery_packing grasp motion, distilled to the general single-object case.

Install

This skill depends on the curobo tool bundle (curobo.plan_directed_linear, curobo.plan_to_grasp_poses). cuRobo JIT-compiles CUDA extensions at install time — build isolation must be off and CUDA_HOME must point at a toolkit matching your torch build:

export CUDA_HOME=/usr/local/cuda
uv sync --extra curobo   # (pip: pip install -e "open-robot-skills[curobo]" --no-build-isolation)

See tools/curobo/SKILL.md for the full recipe and gotchas.

When to use

• Default grasping skill whenever curobo is deployed — clean or cluttered.
• Flat / low-profile objects (a butter box, cream cheese) where a goalset planner struggles but a straight lower-on-top succeeds.

When NOT to use

• curobo not deployed. Use grasping-direct-ik instead.
• The graspable region is NOT the OBB centroid — bowl rim, mug / moka-pot / frying-pan handle, or any off-center grasp. The OBB top-down candidates from geometry.top_down_grasp_candidates are centered on the OBB XY, so they slip on hollow centers and miss handles. Use grasping-short-axis for elongated handles, where the centroid is graspable but orientation is what matters.

Recommended subgraph state flow

6 states, in order:

open → compute_grasp → goto_grasp → observe → close → grasped

(grasped is the success-marker noop from sg.add_exit("grasped"), with an edge to END.)

State details:

1. open — type: tool, tool: "robot.open_gripper", inputs: { settle_steps: 40 }.

2. compute_grasp — type: tool, tool: "geometry.top_down_grasp_candidates", inputs: { obb: Ref("in.target_obb") }. Returns candidates: {poses: list[Se3Pose]} — a yaw fan of top-down grasps.

3. goto_grasp — type: script, file scripts/<sg>/grasp_descend_linear.py from this bundle's canonical_scripts. Inputs: grasp_pose = Ref("compute_grasp.candidates.poses.0"), candidate_poses = Ref("compute_grasp.candidates.poses"), target_obb = Ref("in.target_obb"), hover_z = 0.2. Rises to hover_z, translates over the target at the grasp yaw, then Z-locked linear-descends onto it (shallow grip near the perceived top, floored a hair above the object base so the fingers never ram the table). On an infeasible cartesian solve it falls back to curobo.plan_to_grasp_poses over candidate_poses.

4. observe — type: tool, tool: "robot.get_observation", inputs: {}. Captures the arm state AT the grasp (post-descend, pre-close) so ee_pose_at_grasp reflects the real TCP pose the object was gripped at.

5. close — type: tool, tool: "robot.close_gripper", inputs: { settle_steps: 60 }. Edge directly from close to the grasped success marker; the subgraph's on_error: "failed" catches any raise from goto_grasp (both paths failed). Whether the gripper actually closed on the object is checked by the target_held postcondition checkpoint (see ## Checkpoints), NOT by a re-check-and-raise node (none such exists).

   The subgraph publishes two cross-subgraph outputs:

   • ee_pose_at_grasp — the live TCP pose captured at the observe step (which sits between the descend and close). Downstream transporting-objects uses it to compute a drop height that accounts for the panda hand-to-tcp offset and the held object's geometry.
   • grasp_pose — the computed grasp pose emitted by compute_grasp, i.e. what the descend is targeting. Distinct from ee_pose_at_grasp (the actual EE pose at grasp time). Exposing grasp_pose lets the checkpoint author write an output-anchored verifier like predicate=lambda w, o: o["grasp_pose"]["position"]["z"] > 0.01 to catch sub-table grasp poses before they cascade into a target_held=False failure.

   Hard rule on the output binding: the robot.get_observation response is an Observation { cameras: list[CameraFrame]; arms: list[ArmState] }. There is no flat ee_pose field — the EE pose lives at arms[0].ee_pose. The binding must therefore be exactly:

   sg.set_outputs(
       ee_pose_at_grasp=Ref("observe.arms.0.ee_pose"),
       grasp_pose=Ref("compute_grasp.candidates.poses.0"),
   )
   

   Do not write Ref("observe.ee_pose") — that path does not exist and the cross-subgraph binding will silently resolve to None, sending the downstream compute_drop_pose.py into its inferior no-ee_pose_at_grasp fallback (drop height too high, placement misses).

   Cross-subgraph data flow is by name, so any downstream subgraph that declares ee_pose_at_grasp as an input automatically receives it.

   "edges": [ ..., ["close", "grasped"], ["grasped", "END"] ],
   "conditional_edges": {},
   "exit": { "router_field": null, "success_values": ["grasped"] },
   "on_error": "failed"
   

   The lift onto a safe carry height is handled by the next transporting-objects subgraph (its waypoint_move script lifts before lateral motion); do NOT add a lift step here.

Optional candidate-reordering state

For a thin/elongated target (frypan handle, screwdriver, spoon, rod) insert ONE type: script state select_short_axis (scripts/<sg>/select_short_axis.py) between compute_grasp and goto_grasp. Inputs: target_obb = Ref("in.target_obb"), candidate_poses = Ref("compute_grasp.candidates.poses"). It reorders the candidate fan so poses whose finger-opening axis aligns with the OBB's short horizontal axis come first (count and pose values preserved — only the ordering changes), then wire goto_grasp's grasp_pose/candidate_poses against Ref("select_short_axis.poses...") instead. For a deterministic, geometry-locked single short-axis pose use the dedicated grasping-short-axis skill instead.

Required end states

End state	Meaning
grasped	Gripper has closed on the object after the descend. Route to next subgraph (typically transporting-objects).
failed	Grasp-attempt failure: the cartesian descend AND the cuRobo planner fallback both failed (a raise to on_error). Coordinator routes to abort. Lives only in on_error — never declare a failed node.

See also

• references/design_grasp_curobo.md — the Z-locked (fingertip-frame, orientation-LOCK) linear descend and why it beats a blended rotate+descend.
• references/gripper_settle_constants.md — settle-step tunings.
• scripts/{grasp_descend_linear,select_short_axis}.py — canonical scripts.