Tutorial 5: End-Effector Control — IK & PoseAction
| EE Pick & Drop |
Mobile Manipulator |
Source files:
examples/arm/pick_drop_arm_ee_demo.py— low-level callback approachexamples/arm/pick_drop_arm_ee_action_demo.py— action-queue approach
This tutorial shows how to control a robot arm by specifying end-effector (EE) positions instead of joint angles. PyBulletFleet’s built-in IK solver converts Cartesian targets to joint angles automatically.
What you’ll learn:
Using
move_end_effector()for direct EE position controlPoseAction— an action that moves the EE via IKUsing
ee_target_positioninPickAction/DropActionTuning IK behaviour with
IKParamsOrientation control with
target_orientationMobile manipulator IK — auto-detection, wheel locking,
ik_joint_namesdrop_relative_pose— dropping objects relative to the EE position
Prerequisites: Tutorial 4 — Arm Joint Control (joint targets, use_fixed_base, link attachment).
1. Why EE Position Control?
Tutorial 4 uses JointAction to specify exact joint angles for pick/drop positions.
This works, but requires the user to pre-compute the joint angles for every target position.
EE position control inverts the problem: you specify where the end-effector should go (in world coordinates), and the IK solver figures out the joint angles.
Approach |
You specify |
Library computes |
|---|---|---|
|
Joint angles |
— |
|
EE position |
Joint angles via IK |
2. Direct EE Control: move_end_effector()
The simplest way to command the EE:
# Move end-effector to position [x, y, z] in world frame
reachable = arm_agent.move_end_effector([0.0, -0.3, 0.3])
if reachable:
print("IK solution found — joints moving to target")
else:
print("Target unreachable — a best-effort joint command may be used")
move_end_effector() internally:
Calls
_solve_ik()— multi-seed iterative IK solverChecks reachability (
_check_ik_reachability())Sets joint targets via
set_all_joints_targets()Returns
Trueif the IK solution places the EE within tolerance
After calling move_end_effector(), joints move toward the solution over subsequent
update() steps — just like set_all_joints_targets().
With Orientation
To control the EE orientation as well as position, pass a quaternion [x, y, z, w]:
import pybullet as p
# Downward-pointing orientation (Z- axis)
orn = list(p.getQuaternionFromEuler([0, 3.14, 0]))
reachable = arm_agent.move_end_effector(
[0.0, -0.3, 0.3],
target_orientation=orn,
)
Checking Convergence
Use are_joints_at_targets() (no arguments) to check if all joints have reached
the last commanded targets from move_end_effector():
if arm_agent.are_joints_at_targets(tolerance=0.05):
print("EE has reached the target position")
3. PoseAction — EE Control in the Action Queue
PoseAction wraps move_end_effector() as an Action for use in action sequences:
from pybullet_fleet.action import PoseAction
actions = [
PoseAction(target_position=[0.0, -0.3, 0.3], tolerance=0.02),
PoseAction(target_position=[0.0, 0.3, 0.3], tolerance=0.02),
PoseAction(target_position=[0.0, 0.0, 0.75]), # home
]
arm_agent.add_action_sequence(actions)
Parameters:
Parameter |
Type |
Default |
Description |
|---|---|---|---|
|
|
required |
EE target |
|
|
|
Quaternion |
|
|
|
EE link index/name; |
|
|
|
EE Cartesian distance tolerance (metres) |
|
|
|
Motor force for physics mode |
Completion: Joints within default joint tolerance of the IK solution and EE within tolerance of the target position.
Unreachable or non-convergent targets: PoseAction does not fail as
soon as the initial IK reachability check returns False. When IK returns a
candidate solution, it commands those best-effort joint targets and then:
waits for the joints to settle at their commanded targets;
checks the final EE Cartesian pose; and
completes with
ActionStatus.FAILEDunless both the initial reachability check and the final Cartesian check pass.
At the initial failed reachability check, it logs the warning
"IK target is not reachable (best-effort movement will proceed)". On final
failure, action.error_message is "IK target was not reachable" and the
action logger emits it as a failure.
This means the action queue does not stall — the FAILED status propagates
to the next action in the sequence. To handle failures explicitly, check the
action’s status after completion:
action = PoseAction(target_position=[100, 100, 100]) # unreachable
# ... after execution ...
if action.status == ActionStatus.FAILED:
print(f"Failed: {action.error_message}")
move_end_effector()behaviour: For an URDF robot, an unreachable target normally still yields a best-effort IK candidate and joint targets are set. If the solver cannot return any candidate (or the agent is not a URDF robot), it returnsFalsewithout issuing a new joint command. The same IK behaviour is used byPickAction(ee_target_position=...)andDropAction(ee_target_position=...).
4. EE-Based Pick & Drop
PickAction and DropAction accept ee_target_position as an alternative to
joint-level positioning. When set, the action uses IK to move the EE to the
target position before picking/dropping
(see pick_drop_arm_ee_action_demo.py):
from pybullet_fleet.action import PickAction, DropAction, PoseAction, WaitAction
EE_LINK = arm_agent.get_num_joints() - 1
OFFSET = Pose.from_xyz(0, 0, 0.14) # box offset from EE
actions = [
# Pick: move EE to box position via IK, then attach
PickAction(
target_object_id=box.body_id,
use_approach=False,
ee_target_position=[0.0, -0.3, 0.3],
attach_link=EE_LINK,
attach_relative_pose=OFFSET,
),
# Drop: move EE to drop position via IK, then detach
DropAction(
drop_pose=Pose.from_xyz(0.0, 0.3, 0.1),
use_approach=False,
ee_target_position=[0.0, 0.3, 0.3],
),
# Return home
PoseAction(target_position=[0.0, 0.0, 0.75]),
WaitAction(duration=0.5),
]
arm_agent.add_action_sequence(actions)
Key difference from Tutorial 4:
Tutorial 4 (joint targets) |
Tutorial 5 (EE position) |
|---|---|
|
|
|
|
User computes joint angles |
IK solver computes angles |
PickAction and DropAction support both positioning styles. Use
joint_targets when you want to supply the arm configuration explicitly, or
ee_target_position to have IK compute it. They are mutually exclusive for a
single action. A standalone JointAction is still useful when the joint move
is not part of a pick or drop operation.
5. Tuning IK with IKParams
The IK solver uses a multi-seed iterative strategy. You can tune its behaviour
by passing an IKParams dataclass to Agent.from_urdf():
from pybullet_fleet.agent import Agent, IKParams
ik_cfg = IKParams(
max_outer_iterations=5, # refinement iterations per seed (default: 5)
convergence_threshold=0.01, # position error threshold in metres (default: 0.01)
max_inner_iterations=200, # PyBullet IK iterations per attempt (default: 200)
residual_threshold=1e-4, # IK residual threshold (default: 1e-4)
reachability_tolerance=0.02,# tolerance for reachability check in metres (default: 0.02)
seed_quartiles=(0.25, 0.5, 0.75),# joint range quartiles for seed generation (default)
)
arm_agent = Agent.from_urdf(
urdf_path="robots/arm_robot.urdf",
pose=Pose.from_xyz(0, 0, 0),
use_fixed_base=True,
sim_core=sim_core,
ik_params=ik_cfg,
)
The solver tries the following seed strategies (each refined up to
max_outer_iterations times):
Current joint positions — works well when EE is near the target
Quartile seeds — joint-range fractions in
seed_quartiles(default 25 %, 50 %, 75 %) for diversified exploration across the joint space
For most use cases, the defaults work well. Increase max_outer_iterations if
the solver fails to converge for targets requiring large joint displacements.
6. Mobile Manipulator IK
On composite robots (mobile base + arm), IK must solve the arm while leaving the base and drive joints alone. PyBulletFleet’s default joint filtering helps with common URDFs:
FIXED joints are skipped (PyBullet excludes them from IK)
Continuous joints (lower limit ≥ upper limit, e.g. drive wheels) are locked at their current positions
This can make move_end_effector() work without extra configuration, but a
mobile-manipulator application should explicitly name its arm chain when that
joint set is known. The bundled demo does so, making the intended IK degrees of
freedom unambiguous even if the URDF changes.
Explicit control with ik_joint_names
The bundled mobile_manipulator_demo.py
uses ik_joint_names to explicitly list the joints that IK may move, and an
explicit differential controller for base navigation:
from pybullet_fleet.agent import Agent, AgentSpawnParams, IKParams
ik_cfg = IKParams(
ik_joint_names=(
"mount_to_shoulder",
"shoulder_to_elbow",
"elbow_to_wrist",
"wrist_to_end",
),
)
spawn_params = AgentSpawnParams(
urdf_path="mobile_manipulator",
initial_pose=Pose.from_euler(0, 0, 0.3, yaw=0),
ik_params=ik_cfg,
mass=0.0, # kinematic mode
use_fixed_base=False,
controller={
"type": "differential",
"max_linear_vel": 2.0,
"max_linear_accel": 3.0,
"max_angular_vel": 1.5,
"max_angular_accel": 3.0,
},
)
agent = Agent.from_params(spawn_params, sim_core=sim_core)
All movable joints not in ik_joint_names are locked at their current positions.
When ik_joint_names is None (default), auto-detection is used.
The demo first runs a joint-target pick/drop sequence, then an IK sequence:
MoveAction positions the base near the object, PickAction / DropAction
use ee_target_position with the resolved EE link, and JointAction moves to
the carry and home poses between them. Both drops use
drop_relative_pose=Pose.from_xyz(0, 0, 0) so the object is released at the
current EE position. See the source for the complete runnable sequence.
drop_relative_pose — Relative Drop Positioning
When an object is attached to the EE of a mobile manipulator, you often don’t know
the exact world position where it should be dropped (since the base can be anywhere).
Use drop_relative_pose to specify an offset from the object’s current position:
from pybullet_fleet.action import DropAction
# Drop the object exactly where the EE currently is (no offset)
drop = DropAction(
drop_pose=Pose.from_xyz(0, 0, 0), # base navigation target (still used for approach/move-to-drop)
drop_relative_pose=Pose.from_xyz(0, 0, 0), # (0,0,0) = release in place
use_approach=False,
)
# Drop with a small forward offset from EE
drop = DropAction(
drop_pose=Pose.from_xyz(0, 0, 0), # base navigation target
drop_relative_pose=Pose.from_xyz(0.1, 0, 0), # 10cm forward
use_approach=False,
)
When drop_relative_pose is set, the object’s final position is computed as:
current_object_pose × drop_relative_pose — instead of teleporting to drop_pose.
7. Low-Level Callback Approach
For full control, use move_end_effector() in a state-machine callback. The
following is the minimal pattern used by
pick_drop_arm_ee_demo.py:
step_state = 0
_ee_set = False
def pick_drop_ee_callback(sim_core, dt):
global step_state, _ee_set
def _move_ee_once(pos):
global _ee_set
if not _ee_set:
arm_agent.move_end_effector(pos, target_orientation=EE_ORN)
_ee_set = True
def _at_target():
return arm_agent.are_all_joints_at_targets(tolerance=0.05)
def _advance(next_state):
global step_state, _ee_set
step_state = next_state
_ee_set = False
if step_state == 0: # Move to pick position
_move_ee_once([0.0, -0.3, 0.3])
if _at_target():
_advance(1)
elif step_state == 1: # Pick
arm_agent.attach_object(box, parent_link_index=EE_LINK,
relative_pose=Pose.from_xyz(0, 0, 0.14))
_advance(2)
elif step_state == 2: # Move to drop position
_move_ee_once([0.0, 0.3, 0.3])
if _at_target():
_advance(3)
elif step_state == 3: # Drop
arm_agent.detach_object(box)
_advance(0)
sim_core.register_callback(pick_drop_ee_callback, frequency=10)
The runnable demo adds per-robot pick/drop/home presets, passes
target_orientation=EE_ORN for robots that benefit from orientation-constrained
IK, and performs a home move plus a reverse pick/drop cycle before repeating.
The callback issues each EE command once per state and waits for the commanded
joints to converge; it is deliberately lower-level than PoseAction and does
not perform PoseAction’s final Cartesian completion check.
8. Running the Demos
# Action-queue approach (recommended)
python examples/arm/pick_drop_arm_ee_action_demo.py
# Low-level callback approach
python examples/arm/pick_drop_arm_ee_demo.py
# Rail arm — prismatic (linear) + revolute joints with EE control
python examples/arm/rail_arm_demo.py
# Mobile manipulator (base + arm) — kinematic IK pick/drop
python examples/arm/mobile_manipulator_demo.py
Both single-arm EE demos use kinematic mode for fast execution.
Switching Robot Models
All EE demos accept a --robot argument to swap the arm model.
Pass an arm model name or a direct URDF path:
python examples/arm/pick_drop_arm_ee_action_demo.py --robot kuka_iiwa
python examples/arm/pick_drop_arm_ee_demo.py --robot arm_robot
|
Alternatives |
|---|---|
|
|
See Tutorial 6 — Robot Models for the full model resolution system.
9. Prismatic Joints & Rail Arms
The IK solver works with prismatic (linear) joints out of the box. A prismatic joint in the kinematic chain lets the IK solver adjust both the linear position (e.g., rail height) and revolute angles to reach the target.
examples/arm/rail_arm_demo.py
demonstrates a rail arm (1 prismatic Z-axis + 4 revolute = 5 DOF) picking a box
from a high shelf and placing it at a low position:
agent = Agent.from_urdf(
urdf_path="robots/rail_arm_robot.urdf",
pose=Pose.from_xyz(0, 0, 0),
use_fixed_base=True,
sim_core=sim_core,
)
# EE control — IK decides optimal rail height + arm configuration
actions = [
PickAction(
target_object_id=box.body_id,
use_approach=False,
ee_target_position=[0.1, -0.3, 1.0], # high shelf
attach_link=EE_LINK,
attach_relative_pose=Pose.from_xyz(0, 0, 0.14),
),
DropAction(
drop_pose=Pose.from_xyz(0.1, 0.3, 0.3), # low position
use_approach=False,
ee_target_position=[0.1, 0.3, 0.3],
),
]
The demo also shows per-joint tolerance via dict — tightening prismatic accuracy (±5 mm) while keeping revolute joints looser (±0.05 rad):
JointAction(
target_joint_positions={"rail_joint": 0.0, "base_to_shoulder": 0.0, ...},
tolerance={"rail_joint": 0.005, "base_to_shoulder": 0.05, ...},
)
See Tutorial 4 — JointAction Tolerance for the full tolerance reference (scalar, dict, agent-level fallback).
API Summary
API |
Description |
|---|---|
|
Direct EE position command via IK |
|
Check convergence to last targets |
|
Low-level IK solver (internal) |
|
EE control as an Action |
|
IK-based pick positioning |
|
IK-based drop positioning |
|
IK solver tuning parameters |
|
Restrict IK to specific joints |
|
Drop relative to current EE position |
See Also
Tutorial 4 — Arm Joint Control:
JointAction, joint-level pick/drop, tolerance referenceTutorial 2 — Action System: mobile robot pick/drop with
MoveActionRail Arm Demo: prismatic + revolute EE control
Mobile Manipulator Demo: kinematic mobile manipulator with IK pick/drop
Architecture Overview: IK internals, joint control modes