Run the synthetic fleet corridor evaluation

This is an evaluation example, not a fleet-management algorithm supplied by PBF. It shows how an external application can run its own algorithms under equivalent simulation conditions and use PBF’s commands, state and collision observations to compare their outcomes. The two policies below are deliberately simple examples; designing or judging a production algorithm is outside this repository’s scope.

This example runs two independent 20-robot simulations through a narrow corridor. It compares unrestricted bidirectional commands with an external direction gate. It is a synthetic movement workload, not a warehouse delivery benchmark or a reproduction of a field incident.

python -m pybullet_fleet.examples.evaluation.fleet_corridor_evaluation

With no output argument, each run creates a unique pbf-corridor-* directory under the system temporary directory and prints its path. Pass a positional directory to retain reports at a chosen location; an explicitly named directory must not already exist. --dt 0.05 changes the timestep; --cutoff 300 sets the fixed simulated-time window. The cutoff must be an integer number of steps. The command writes uncontrolled.json, direction_gate.json, and comparison.json. Each policy starts in a fresh simulation with the same initial poses, 40 two-leg movement tasks, geometry, motion limits and cutoff. The comparison does not use recorded-command replay. The example lets PBF’s normal run_simulation() loop advance the world; external policy decisions run at PRE_STEP, and the evaluator observes at POST_STEP after collision checks.

Watch a policy in the GUI

Select one policy at a time. The GUI defaults to 1× simulated time and keeps the final view open until you close the window or press Ctrl+C. The JSON report is written when the window closes. Blue robots start in area A; orange robots start in area B.

python -m pybullet_fleet.examples.evaluation.fleet_corridor_evaluation \
  --gui --monitor --policy uncontrolled --cutoff 30
python -m pybullet_fleet.examples.evaluation.fleet_corridor_evaluation \
  --gui --monitor --policy direction_gate --rtf 3

The initial camera is centered on the shortened corridor (x=[-0.75,0.75]). Use the arrow keys or right-drag to pan and = / - or the mouse wheel to zoom. --monitor opens the DataMonitor beside the PyBullet GUI; its Collisions field is the current active pair count, and Tot Collis. is the cumulative count of new PBF collision-margin pair entries. Both include the entire scene, including starts and destinations outside the corridor, and can include robot-wall pairs. They are not counts of geometric overlap only or counts of corridor traffic conflicts. The monitor updates during the run; its window refreshes about every 0.5 wall-clock seconds. The 30-second run is for a quick look and censors unfinished tasks. For the defined 300-second comparison, omit --cutoff; direction gate reaches its last endpoint at about 245 simulated seconds in the tested configuration. --rtf changes live GUI viewing speed, not the simulation timestep. This is a live simulation run, not recorded-result playback. Manual pause or single-step interaction may change command timing, so use the default headless two-policy command for repeatable measurement. GUI mode writes only the selected policy’s report, not comparison.json.

On macOS, native PyBullet GUI can be used from Terminal. If the native window does not work well, the launcher added by PR #48 can show the Linux GUI in a browser (requires Colima and Docker CLI):

scripts/run_linux_gui_macos.sh fleet_corridor_evaluation.py \
  --gui --policy uncontrolled --cutoff 30

That launcher runs in a temporary container; use the native Python command above when you need to retain the JSON report on the Mac.

The JSON includes versioned conditions, measurement definitions, task release, command acknowledgement, first movement and arrival times, external policy decisions, observed proximity/overlap episodes, and aggregate metrics. A task is complete only when the robot arrives at its endpoint and stops. An accepted command is not a completed task. Unissued second legs remain in the workload denominator and are marked unfinished at cutoff.

external_admission_delay_seconds measures time from task release to the external policy’s navigate call. PBF cannot observe a request it has not received, so this is not simulator waiting time. command_to_arrival_seconds starts when the command is issued. completed_per_sim_second, completion fraction, the 60-second completion count, and all-tasks-completed time describe fleet task outcomes in simulated time; wall seconds and RTF describe execution cost. Fixed-window rate can be identical even when task latency differs.

The external evaluator marks corridor crowding when more than three robot reference points occupy x=[-0.75,0.75], y=[-0.17,0.17]. This is a transparent scenario definition, not a PBF congestion verdict or a physical queue. The report retains each observed crowding interval with start/end step and peak occupancy, as well as the aggregate count of over-capacity steps. Kinematic robots pass through one another. The direction gate grants up to two active movement tasks in one direction and alternates direction after a batch finishes. A gate slot is held until endpoint arrival, including travel outside the corridor. It can create substantial admission delay while reducing observed overlap in the corridor; overlap elsewhere may remain.

Collision episodes use PBF’s latest collision-check observation after every completed step. The evaluator still owns episode and corridor classification; it reads only pairs qualified by PBF’s configured collision pipeline. margin_only means signed closest-point distance is positive and at most the configured 0.02 m margin. geometric_overlap means distance is zero or negative; it is not a physics impact. The example reads signed distances from the core’s latest qualifying-pair observation. If an active pair was not rechecked on a step, its episode remains active but that step is not counted as a new distance sample. The separate core_margin_entry_count is the core’s detected threshold-entry count and is not an overlap count or a complete near-miss count. Each episode records pair IDs, step/time, start positions, active task IDs, minimum sampled distance and observed duration. Events that begin and end between samples can still be missed, so smaller --dt improves resolution without guaranteeing continuous detection.

geometric_overlap_episodes counts sampled robot/robot and robot/wall overlap episodes throughout the scene, including the initial areas and destinations. corridor_geometric_overlap_episodes counts the subset with at least one observed step whose pair midpoint lies within the corridor’s x limits. It is therefore possible for both policies to have overlap outside the passage even when the direction gate has no observed corridor overlap.

The report is a scenario-owned artifact. Its IDs and step/time fields permit later trace correlation, but it is not a replay artifact or a general PBF metrics API. Playback, checkpoint/restore, broader input capture and general trace propagation remain separate future capabilities.