Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Humble] nav2-mppi-controller 1.1.14-1 on arm64 installed from binaries crashes on launch #4333

Closed
jimdinunzio opened this issue May 13, 2024 · 9 comments
Closed

[Humble] nav2-mppi-controller 1.1.14-1 on arm64 installed from binaries crashes on launch #4333

jimdinunzio opened this issue May 13, 2024 · 9 comments

Comments

@jimdinunzio
Copy link

Bug report

Required Info:

  • Operating System:
    Ubuntu 22.04 Jammy arm64

  • ROS2 Version:
    Humble, installed from binaries

  • Version or commit hash:
    ros-humble-nav2-mppi-controller/jammy,now 1.1.14-1jammy.20240407.044830 arm64 [installed]

  • DDS implementation:
    default

Steps to reproduce issue

navigation.txt

ros2 launch nav2_bringup navigation_launch.py params_file:=navigation.txt

Expected behavior

nav2-mppi-controller works as expected

Actual behavior

nav2-mppi-controller crashes on launch

nav2-mppi-controller.log

Additional information

When built from source commit 8f097af nav2-mppi-controller does not crash on start up
@SteveMacenski
Copy link
Member

SteveMacenski commented May 14, 2024 •
edited

@pepisg do you see this? I believe you're also on Humble ARM if I recall correctly.

@jimdinunzio please follow the thread in this topic for some possibly relevant information and context about steps forward here: #3767. We ran into a similar situation where build farm binaries were causing a problem that locally compiled did not due to build flags available on the cloud provider jobs that weren't on the local systems.

  1. Did you try with our default launch files and still see the issue with the TB3 simulation (need basis to make sure its not something else going on). Anything odd / different in your MPPI configs (major change to parameters, new critics, etc)
  2. Please compile with debug flags according to the docs here (https://navigation.ros.org/tutorials/docs/get_backtrace.html) and give me the traceback to know more about the crash
  3. What happens if you sudo apt purge ros-humble-nav2-mppi-controller and reinstall it?
  4. Run cat /proc/cpuinfo and give me the output. Compare with https://build.ros2.org/job/Hbin_uJ64__nav2_mppi_controller__ubuntu_jammy_amd64__binary/26/consoleFull#console-section-2 and let me know if there are any notable changes

@pepisg
Copy link
Contributor

pepisg commented May 14, 2024

I'm running iron on ARM with nav2 built from source.

I don't use the binaries so in my case the controller was not crashing, but I found the behavior to be quite erratic: the generated trajectories did not seem to go in accordance with how the critics should score them (sometimes they wouldn't stick to the path even when only PathFollow was in use), and also the optimal trajectory did not seem to be sampled from the generated trajectories.

@jimdinunzio does MPPI work normally for you when building from source from the commit you referenced?

@SteveMacenski
Copy link
Member

@pepisg if you did run the binaries real quick just to see, does it crash?

@pepisg
Copy link
Contributor

pepisg commented May 16, 2024

Hey! Sorry for the delay, will do next week when I can easily access a robot and let you know

@SteveMacenski
Copy link
Member

Thanks! Your time is appreciated :-)

@pepisg
Copy link
Contributor

pepisg commented May 17, 2024 •
edited

Hey! actually I had time do check it out today. With the params file provided by @jimdinunzio and installing ros-iron-nav2-mppi-controller, the controller did not crash launching nav2_bringup's navigation.launch.py:

[INFO] [launch]: All log files can be found below /root/.ros/log/2024-05-17-15-42-13-898346-0690693-2326
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [controller_server-1]: process started with pid [2329]
[INFO] [smoother_server-2]: process started with pid [2331]
[INFO] [planner_server-3]: process started with pid [2333]
[INFO] [behavior_server-4]: process started with pid [2335]
[INFO] [bt_navigator-5]: process started with pid [2337]
[INFO] [waypoint_follower-6]: process started with pid [2339]
[INFO] [velocity_smoother-7]: process started with pid [2341]
[INFO] [collision_monitor-8]: process started with pid [2343]
[INFO] [lifecycle_manager-9]: process started with pid [2345]
[behavior_server-4] [1715960537.355056146][INFO][printLifecycleNodeNotification][122]: 
[behavior_server-4]     behavior_server lifecycle node launched. 
[behavior_server-4]     Waiting on external lifecycle transitions to activate
[behavior_server-4]     See https://design.ros2.org/articles/node_lifecycle.html for more information.
[collision_monitor-8] [1715960537.358836709][INFO][printLifecycleNodeNotification][122]: 
[collision_monitor-8]   collision_monitor lifecycle node launched. 
[collision_monitor-8]   Waiting on external lifecycle transitions to activate
[collision_monitor-8]   See https://design.ros2.org/articles/node_lifecycle.html for more information.
[waypoint_follower-6] [1715960537.379527286][INFO][printLifecycleNodeNotification][122]: 
[waypoint_follower-6]   waypoint_follower lifecycle node launched. 
[waypoint_follower-6]   Waiting on external lifecycle transitions to activate
[waypoint_follower-6]   See https://design.ros2.org/articles/node_lifecycle.html for more information.
[waypoint_follower-6] [1715960537.414301536][INFO][WaypointFollower][35]: Creating
[velocity_smoother-7] [1715960537.389353030][INFO][printLifecycleNodeNotification][122]: 
[velocity_smoother-7]   velocity_smoother lifecycle node launched. 
[velocity_smoother-7]   Waiting on external lifecycle transitions to activate
[velocity_smoother-7]   See https://design.ros2.org/articles/node_lifecycle.html for more information.
[bt_navigator-5] [1715960537.609612103][INFO][printLifecycleNodeNotification][122]: 
[bt_navigator-5]        bt_navigator lifecycle node launched. 
[bt_navigator-5]        Waiting on external lifecycle transitions to activate
[bt_navigator-5]        See https://design.ros2.org/articles/node_lifecycle.html for more information.
[lifecycle_manager-9] [1715960537.633238787][INFO][LifecycleManager][38]: Creating
[bt_navigator-5] [1715960537.673909123][INFO][BtNavigator][37]: Creating
[planner_server-3] [1715960537.705070851][INFO][printLifecycleNodeNotification][122]: 
[planner_server-3]      planner_server lifecycle node launched. 
[planner_server-3]      Waiting on external lifecycle transitions to activate
[planner_server-3]      See https://design.ros2.org/articles/node_lifecycle.html for more information.
[smoother_server-2] [1715960537.722996114][INFO][printLifecycleNodeNotification][122]: 
[smoother_server-2]     smoother_server lifecycle node launched. 
[smoother_server-2]     Waiting on external lifecycle transitions to activate
[smoother_server-2]     See https://design.ros2.org/articles/node_lifecycle.html for more information.
[planner_server-3] [1715960537.772954921][INFO][PlannerServer][50]: Creating
[controller_server-1] [1715960537.784534124][INFO][printLifecycleNodeNotification][122]: 
[controller_server-1]   controller_server lifecycle node launched. 
[controller_server-1]   Waiting on external lifecycle transitions to activate
[controller_server-1]   See https://design.ros2.org/articles/node_lifecycle.html for more information.
[smoother_server-2] [1715960537.821870767][INFO][SmootherServer][41]: Creating smoother server
[lifecycle_manager-9] [1715960537.829985518][INFO][message][534]: Creating and initializing lifecycle service clients
[planner_server-3] [1715960537.887772631][INFO][printLifecycleNodeNotification][122]: 
[planner_server-3]      global_costmap lifecycle node launched. 
[planner_server-3]      Waiting on external lifecycle transitions to activate
[planner_server-3]      See https://design.ros2.org/articles/node_lifecycle.html for more information.
[planner_server-3] [1715960537.939146257][INFO][init][109]: Creating Costmap
[lifecycle_manager-9] [1715960537.939794831][INFO][message][534]: Starting managed nodes bringup...
[lifecycle_manager-9] [1715960537.940532786][INFO][message][534]: Configuring controller_server
[controller_server-1] [1715960538.008648355][INFO][ControllerServer][49]: Creating controller server
[controller_server-1] [1715960538.094687522][INFO][printLifecycleNodeNotification][122]: 
[controller_server-1]   local_costmap lifecycle node launched. 
[controller_server-1]   Waiting on external lifecycle transitions to activate
[controller_server-1]   See https://design.ros2.org/articles/node_lifecycle.html for more information.
[controller_server-1] [1715960538.122432672][INFO][init][109]: Creating Costmap
[controller_server-1] [1715960538.136501785][INFO][on_configure][86]: Configuring controller interface
[controller_server-1] [1715960538.136751844][INFO][on_configure][88]: getting progress checker plugins..
[controller_server-1] [1715960538.139765651][INFO][on_configure][112]: getting goal checker plugins..
[controller_server-1] [1715960538.141420801][INFO][on_configure][139]: Controller frequency set to 30.0000Hz
[controller_server-1] [1715960538.141811827][INFO][on_configure][147]: Configuring
[controller_server-1] [1715960538.172688325][INFO][on_configure][180]: Using plugin "obstacle_layer"
[controller_server-1] [1715960538.260837864][INFO][onInitialize][106]: Subscribed to Topics: scan base_scan
[controller_server-1] [1715960538.305039600][INFO][on_configure][196]: Initialized plugin "obstacle_layer"
[controller_server-1] [1715960538.306367150][INFO][on_configure][180]: Using plugin "voxel_layer"
[controller_server-1] [1715960538.315401081][INFO][onInitialize][106]: Subscribed to Topics: pointcloud
[controller_server-1] [1715960538.377192895][INFO][on_configure][196]: Initialized plugin "voxel_layer"
[controller_server-1] [1715960538.377808796][INFO][on_configure][180]: Using plugin "inflation_layer"
[controller_server-1] [1715960538.387542664][INFO][on_configure][196]: Initialized plugin "inflation_layer"
[controller_server-1] [1715960538.479715912][INFO][on_configure][156]: Created progress_checker : progress_checker of type nav2_controller::SimpleProgressChecker
[controller_server-1] [1715960538.482385030][INFO][on_configure][173]: Controller Server has progress_checker  progress checkers available.
[controller_server-1] [1715960538.491339053][INFO][on_configure][182]: Created goal checker : goal_checker of type nav2_controller::SimpleGoalChecker
[controller_server-1] [1715960538.496058764][INFO][on_configure][199]: Controller Server has goal_checker  goal checkers available.
[controller_server-1] [1715960538.518912675][INFO][on_configure][208]: Created controller : FollowPath of type nav2_mppi_controller::MPPIController
[controller_server-1] [1715960538.530795380][WARN][setOffset][102]: Controller period is less then model dt, consider setting it equal
[controller_server-1] [1715960538.567617759][INFO][initialize][27]: ConstraintCritic instantiated with 1 power and 4.000000 weight.
[controller_server-1] [1715960538.568267006][INFO][loadCritics][58]: Critic loaded : mppi::critics::ConstraintCritic
[controller_server-1] [1715960538.575425616][INFO][initialize][49]: InflationCostCritic instantiated with 1 power and 300.000000 / 0.015000 weights. Critic will collision check based on footprint cost.
[controller_server-1] [1715960538.575890598][INFO][loadCritics][58]: Critic loaded : mppi::critics::CostCritic
[controller_server-1] [1715960538.578275319][INFO][initialize][31]: GoalCritic instantiated with 1 power and 5.000000 weight.
[controller_server-1] [1715960538.578512034][INFO][loadCritics][58]: Critic loaded : mppi::critics::GoalCritic
[controller_server-1] [1715960538.580270901][INFO][initialize][29]: GoalAngleCritic instantiated with 1 power, 3.000000 weight, and 0.500000 angular threshold.
[controller_server-1] [1715960538.580490943][INFO][loadCritics][58]: Critic loaded : mppi::critics::GoalAngleCritic
[controller_server-1] [1715960538.584613922][INFO][initialize][40]: ReferenceTrajectoryCritic instantiated with 1 power and 14.000000 weight
[controller_server-1] [1715960538.584861934][INFO][loadCritics][58]: Critic loaded : mppi::critics::PathAlignCritic
[controller_server-1] [1715960538.590863849][INFO][loadCritics][58]: Critic loaded : mppi::critics::PathFollowCritic
[controller_server-1] [1715960538.595903415][INFO][initialize][58]: PathAngleCritic instantiated with 1 power and 2.000000 weight. Mode set to: Forward Preference
[controller_server-1] [1715960538.596222118][INFO][loadCritics][58]: Critic loaded : mppi::critics::PathAngleCritic
[controller_server-1] [1715960538.598789567][INFO][initialize][29]: PreferForwardCritic instantiated with 1 power and 5.000000 weight.
[controller_server-1] [1715960538.599027723][INFO][loadCritics][58]: Critic loaded : mppi::critics::PreferForwardCritic
[controller_server-1] [1715960538.653226954][INFO][reset][132]: Optimizer reset
[controller_server-1] [1715960538.686179902][INFO][configure][48]: Configured MPPI Controller: FollowPath
[controller_server-1] [1715960538.686549648][INFO][on_configure][227]: Controller Server has FollowPath  controllers available.
[lifecycle_manager-9] [1715960538.740616937][INFO][message][534]: Configuring smoother_server
[smoother_server-2] [1715960538.743557108][INFO][on_configure][68]: Configuring smoother server
[smoother_server-2] [1715960538.845636456][INFO][loadSmootherPlugins][138]: Created smoother : simple_smoother of type nav2_smoother::SimpleSmoother
[smoother_server-2] [1715960538.858830935][INFO][loadSmootherPlugins][157]: Smoother Server has simple_smoother  smoothers available.
[lifecycle_manager-9] [1715960538.902555272][INFO][message][534]: Configuring planner_server
[planner_server-3] [1715960538.906049165][INFO][on_configure][83]: Configuring
[planner_server-3] [1715960538.906844371][INFO][on_configure][147]: Configuring
[planner_server-3] [1715960538.968234150][INFO][on_configure][180]: Using plugin "static_layer"
[planner_server-3] [1715960539.009075502][INFO][onInitialize][82]: Subscribing to the map topic (/map) with transient local durability
[planner_server-3] [1715960539.019316146][INFO][onInitialize][98]: Subscribing to updates
[planner_server-3] [1715960539.038165420][INFO][on_configure][196]: Initialized plugin "static_layer"
[planner_server-3] [1715960539.038691973][INFO][on_configure][180]: Using plugin "obstacle_layer"
[planner_server-3] [1715960539.042855242][INFO][onInitialize][106]: Subscribed to Topics: scan base_scan
[planner_server-3] [1715960539.076480253][INFO][on_configure][196]: Initialized plugin "obstacle_layer"
[planner_server-3] [1715960539.077013815][INFO][on_configure][180]: Using plugin "voxel_layer"
[planner_server-3] [1715960539.082363411][INFO][onInitialize][106]: Subscribed to Topics: pointcloud
[planner_server-3] [1715960539.132521428][INFO][on_configure][196]: Initialized plugin "voxel_layer"
[planner_server-3] [1715960539.133096175][INFO][on_configure][180]: Using plugin "inflation_layer"
[planner_server-3] [1715960539.139622371][INFO][on_configure][196]: Initialized plugin "inflation_layer"
[planner_server-3] [1715960539.329178075][INFO][on_configure][113]: Created global planner plugin GridBased of type nav2_smac_planner/SmacPlannerHybrid
[planner_server-3] [1715960539.330330609][INFO][configure][64]: Configuring GridBased of type SmacPlannerHybrid
[planner_server-3] [1715960539.345973780][INFO][configure][182]: Even sized heuristic lookup table size set 400.000000, increasing size by 1 to make odd
[planner_server-3] [1715960591.450833158][INFO][configure][230]: Configured plugin GridBased of type SmacPlannerHybrid with maximum iterations 1000000, max on approach iterations 1000, and not allowing unknown traversal. Tolerance 0.50.Using motion model: Reeds-Shepp.
[planner_server-3] [1715960591.451509030][INFO][on_configure][130]: Planner Server has GridBased  planners available.
[lifecycle_manager-9] [1715960591.513998669][INFO][message][534]: Configuring behavior_server
[behavior_server-4] [1715960591.516804497][INFO][on_configure][83]: Configuring
[behavior_server-4] [1715960591.575364319][INFO][loadBehaviorPlugins][110]: Creating behavior plugin spin of type nav2_behaviors/Spin
[behavior_server-4] [1715960591.597374638][INFO][loadBehaviorPlugins][110]: Creating behavior plugin backup of type nav2_behaviors/BackUp
[behavior_server-4] [1715960591.610348627][INFO][loadBehaviorPlugins][110]: Creating behavior plugin drive_on_heading of type nav2_behaviors/DriveOnHeading
[behavior_server-4] [1715960591.622207907][INFO][loadBehaviorPlugins][110]: Creating behavior plugin wait of type nav2_behaviors/Wait
[behavior_server-4] [1715960591.678112563][INFO][configure][125]: Configuring spin
[behavior_server-4] [1715960591.723199716][INFO][configure][125]: Configuring backup
[behavior_server-4] [1715960591.752374342][INFO][configure][125]: Configuring drive_on_heading
[behavior_server-4] [1715960591.774570142][INFO][configure][125]: Configuring wait
[lifecycle_manager-9] [1715960591.798680016][INFO][message][534]: Configuring velocity_smoother
[velocity_smoother-7] [1715960591.801610651][INFO][on_configure][49]: Configuring velocity smoother
[lifecycle_manager-9] [1715960591.840233435][INFO][message][534]: Configuring collision_monitor
[collision_monitor-8] [1715960591.843738240][INFO][on_configure][47]: Configuring
[collision_monitor-8] [1715960591.910985512][ERROR][configureSources][374]: Error while getting parameters: parameter 'observation_sources' is not initialized
[lifecycle_manager-9] [1715960591.916989827][ERROR][changeStateForNode][245]: Failed to change state for node: collision_monitor
[lifecycle_manager-9] [1715960591.918109848][ERROR][startup][312]: Failed to bring up all requested nodes. Aborting bringup.

The collision monitor didn't activate because there don't seem to be CM params in the params file, but otherwise it looked OK.

I'd be harder for me to test on humble though.

@SteveMacenski
Copy link
Member

@jimdinunzio can you try iron and let me know if you run into an issue? This is a really good datapoint @pepisg, thanks!

@SteveMacenski
Copy link
Member

SteveMacenski commented May 23, 2024 •
edited

@jimdinunzio Any update? Is this still a problem? Have you had a chance to test?

@SteveMacenski
Copy link
Member

Closing due to lack of response. Happy to open if someone can reproduce.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@SteveMacenski @jimdinunzio @pepisg