Using Action Graphs for agent Actions

[MaltheT]

Hi,

I have a setup where I use an Action Graph with the Ackermann Steering node to drive my robot. However, it doesn't articulate the joints in my orbit sim when I use a preconfigured 3.0 speed. I have tested that it works in regular Isaac Sim. How does orbit work with Action Graphs, and can I use my Action Graph's interface for the Actions of my RL agent (i.e. speed and steeringAngle fields)?

[Ltesfaye]

I believe you'll need to implement your action graphs as standalone python script using omni graph accounting for the specific ros version your system is running. This is not really a well documented feature so you might need to actually open up the isaac sim code base in vs-code and read through the ogn.rst which can be found in the exts/omni.isaac.ros_bridge and ext/omni/isaac_ros2_bridge. This will have all the omnigraph nodes along with their input and outputs documented. You then pretty much manually specify all the wire connections between your nodes.

If your workflow in orbit involves directly updating primitive transformation then you might need to disable fabric and run orbit simulation on your CPU. I personally found that simply turning off fabric is not enough you need to apply the following configuration as well.

        sim: SimulationCfg = SimulationCfg()
        sim.device = "cpu"
        sim.use_gpu_pipeline = False
        sim.physx = sim_utils.PhysxCfg(use_gpu=False)
        sim.use_fabric = False

Please note if you're running a large number of robot instances this might cause a significant slow down.

[MaltheT]

Hi, thank you for your reply. Right now I'm not using the nodes coming from ext/omni/isaac_ros2_bridge in my setup, as I just want to get some training going and worry about the ROS2 communication later. The ackermann_sterring node that I'm using is part of omni.isaac.sim if I'm not mistaken (It has nothing to do with ROS2 on its own).

I'm curious if using this ackermann_steering node to define my action space in orbit is possible. Right now I have a setup in orbit like this:

F1TENTH_CFG = ArticulationCfg(
    spawn=sim_utils.UsdFileCfg(
        usd_path=f"{F1TNEHT_PROJET_DIR}/robot/instanceable/f1tenth.usd",
    ),
    init_state=ArticulationCfg.InitialStateCfg(
        pos=(0.0, 0.0, 1.0)
    ),
    actuators={
        "wheels": ImplicitActuatorCfg(
            joint_names_expr=['wheel_.*'],
            effort_limit=None,
            velocity_limit=None,
            stiffness=None,
            damping=None
        ),
        "rotators": ImplicitActuatorCfg(
            joint_names_expr=["rotator_.*"],
            effort_limit=None,
            velocity_limit=None,
            stiffness=None,
            damping=None
        ),
    },
)

And for the actions:

@configclass
class ActionsCfg:
    """Action specifications for the MDP."""
    steering = mdp.JointPositionActionCfg(asset_name="robot", joint_names=["rotator_left", "rotator_right"], scale=1.0)
    drive = mdp.JointVelocityActionCfg(asset_name="robot", joint_names=["wheel_back_left", "wheel_back_right", "wheel_front_left", "wheel_front_right"], scale=1.0)

However, this means that my agent thinks that it can control each joint independently, which is not the case for the physical system. Previously in my simulation, I used the ackermann_steering node to solve this issue, to implement the correct control behaviour. All of the examples I have looked at in the orbit repo perform control directly on the joints. However, I'm curious if there is a way to define the action space using the action graphs as they are used to control a lot of wheeled robots in Isaac Sim, i.e. something like this:

@configclass
class ActionsCfg:
    """Action specifications for the MDP."""
    # example:
    ackermann = mdp.OmniGraphCfg(asset_name="robot", node_name=["ackermann_drive"], scale=1.0) #(This code does not work)

I know that the code I have provided does not work, but I hope that the desired behavior is clear. Is there a way to achieve this through the current actions or commands APIs?

[Ltesfaye]

Oh okay, so you're just trying to directly use the Ackremann steering inside your orbit repo. I think a modified version of this example below would work. You'd need to create probably write a custom actions method that handles this for your wheels.

Look at: "../isaac_sim-2023.1.1/extsPhysics/omni.physx.vehicle-105.1.12-5.1/omni/physxvehicle/scripts/samples/AckermannSteeringHelper.py"

Full transparency: I have not tested this but read through steering helper. I believe you should be impliment a version of this in your project. I hope this helps!

import math

from pxr import Gf, UsdGeom, PhysxSchema

import omni.physxvehicle

from ..helpers import Factory
from .VehicleSampleBase import VehicleSampleBase
from . import Stepper

import omni.physxdemos as demo


class AckermannSteeringDemo(VehicleSampleBase):
    title = "Ackermann steering"
    category = demo.Categories.VEHICLES
    short_description = "Usage of Ackermann steering helper method"
    description = "Demo showing how to use the Ackermann steering helper method."

    def create(self, stage):
        super().create(stage)

        self._physxVehicleInterface = omni.physxvehicle.get_physx_vehicle_interface()

        rootPath = str(stage.GetDefaultPrim().GetPath())
        cameraPath = rootPath + "/Camera"
        camera = UsdGeom.Camera.Define(stage, cameraPath)
        camera.AddTranslateOp().Set(Gf.Vec3d(-5, 35, 0))
        camera.AddRotateYXZOp(precision=UsdGeom.XformOp.PrecisionDouble).Set(Gf.Vec3d(90, 180, 0))
        self.demo_camera = cameraPath

        create(stage, self._physxVehicleInterface)

    def on_shutdown(self):
        self._physxVehicleInterface = None


class AckermannSteeringHelperScenario(Stepper.Scenario):
    def __init__(self, stage, physxVehicleInterface, wheelAttachmentPaths, wheelSteerAngles, axleSeparation, axleWidth,
        timeStepsPerSecond):
        secondsToRun = 4.0
        super().__init__(secondsToRun, 1.0 / timeStepsPerSecond)
        self._stage = stage
        self._physxVehicleInterface = physxVehicleInterface
        self._wheelAttachmentPaths = wheelAttachmentPaths
        self._wheelSteerAngles = wheelSteerAngles
        self._axleSeparation = axleSeparation
        self._axleWidth = axleWidth

        self._steerStop = secondsToRun
        self._steerCycle = self._steerStop

    def on_start(self):
        for i in range(len(self._wheelAttachmentPaths)):
            for j in range(len(self._wheelAttachmentPaths[i])):
                prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[i][j])
                wheelControllerAPI = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)
                wheelControllerAPI.GetSteerAngleAttr().Set(0)

    def on_end(self):
        return

    def on_step(self, deltaTime, totalTime):
        if totalTime < self._steerStop:
            fullSteerCycleCount = math.trunc(totalTime / self._steerCycle)
            currentCycle = totalTime - (fullSteerCycleCount * self._steerCycle)
            ratio = currentCycle / self._steerCycle

            steerAngleInnerWheel = ratio * self._wheelSteerAngles[0]
            steerAngleOuterWheel = self._physxVehicleInterface.compute_ackermann_steering_angle(
                steerAngleInnerWheel, self._axleSeparation, self._axleWidth
            )

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[0][Factory.WHEEL_FRONT_LEFT])
            wheelControllerAPI_FL = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[0][Factory.WHEEL_FRONT_RIGHT])
            wheelControllerAPI_FR = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            if steerAngleInnerWheel >= 0:
                wheelControllerAPI_FL.GetSteerAngleAttr().Set(steerAngleInnerWheel)
                wheelControllerAPI_FR.GetSteerAngleAttr().Set(steerAngleOuterWheel)
            else:
                wheelControllerAPI_FL.GetSteerAngleAttr().Set(steerAngleOuterWheel)
                wheelControllerAPI_FR.GetSteerAngleAttr().Set(steerAngleInnerWheel)

            # ---

            # the helper can of course also be used to set steering angles for rear wheels.
            # In that case the angles just have to be applied in the opposite order.

            steerAngleInnerWheel = ratio * self._wheelSteerAngles[1]
            steerAngleOuterWheel = self._physxVehicleInterface.compute_ackermann_steering_angle(
                steerAngleInnerWheel, self._axleSeparation, self._axleWidth
            )

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[1][Factory.WHEEL_REAR_LEFT])
            wheelControllerAPI_RL = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[1][Factory.WHEEL_REAR_RIGHT])
            wheelControllerAPI_RR = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            if steerAngleInnerWheel >= 0:
                wheelControllerAPI_RL.GetSteerAngleAttr().Set(steerAngleOuterWheel)
                wheelControllerAPI_RR.GetSteerAngleAttr().Set(steerAngleInnerWheel)
            else:
                wheelControllerAPI_RL.GetSteerAngleAttr().Set(steerAngleInnerWheel)
                wheelControllerAPI_RR.GetSteerAngleAttr().Set(steerAngleOuterWheel)

            # ---

            # or all four wheels can be set such that they share the same center of the turning circle.

            steerAngleInnerWheel = ratio * self._wheelSteerAngles[2]
            steerAngleOuterWheel = self._physxVehicleInterface.compute_ackermann_steering_angle(
                steerAngleInnerWheel, self._axleSeparation / 2, self._axleWidth
            )

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[2][Factory.WHEEL_FRONT_LEFT])
            wheelControllerAPI_FL = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[2][Factory.WHEEL_FRONT_RIGHT])
            wheelControllerAPI_FR = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[2][Factory.WHEEL_REAR_LEFT])
            wheelControllerAPI_RL = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            prim = self._stage.GetPrimAtPath(self._wheelAttachmentPaths[2][Factory.WHEEL_REAR_RIGHT])
            wheelControllerAPI_RR = PhysxSchema.PhysxVehicleWheelControllerAPI(prim)

            if steerAngleInnerWheel >= 0:
                wheelControllerAPI_FL.GetSteerAngleAttr().Set(steerAngleInnerWheel)
                wheelControllerAPI_FR.GetSteerAngleAttr().Set(steerAngleOuterWheel)
                wheelControllerAPI_RL.GetSteerAngleAttr().Set(-steerAngleInnerWheel)
                wheelControllerAPI_RR.GetSteerAngleAttr().Set(-steerAngleOuterWheel)
            else:
                wheelControllerAPI_FL.GetSteerAngleAttr().Set(steerAngleOuterWheel)
                wheelControllerAPI_FR.GetSteerAngleAttr().Set(steerAngleInnerWheel)
                wheelControllerAPI_RL.GetSteerAngleAttr().Set(-steerAngleOuterWheel)
                wheelControllerAPI_RR.GetSteerAngleAttr().Set(-steerAngleInnerWheel)


def create(stage, physxVehicleInterface):
    vehicleCount = 3
    vehiclePaths = []
    wheelAttachmentPaths = []
    timeStepsPerSec = 60

    Factory.create4WheeledCarsScenario(
        stage,
        1.0,
        vehicleCount,
        createCollisionShapesForWheels=True,
        driveMode=Factory.DRIVE_NONE,
        vehiclePathsOut=vehiclePaths,
        wheelAttachmentPathsOut=wheelAttachmentPaths,
        vehicleDelta=[-5, 0, 0],
        addChassisRenderMesh=False,
        timeStepsPerSecond=timeStepsPerSec
    )

    wheelSteerAngles = [(45 * math.pi) / 180, (-45 * math.pi) / 180, (45 * math.pi) / 180]

    prim = stage.GetPrimAtPath(wheelAttachmentPaths[0][Factory.WHEEL_FRONT_LEFT])
    posFL = prim.GetAttribute("xformOp:translate").Get()

    prim = stage.GetPrimAtPath(wheelAttachmentPaths[0][Factory.WHEEL_FRONT_RIGHT])
    posFR = prim.GetAttribute("xformOp:translate").Get()

    prim = stage.GetPrimAtPath(wheelAttachmentPaths[0][Factory.WHEEL_REAR_LEFT])
    posRL = prim.GetAttribute("xformOp:translate").Get()

    axleSeparation = posFL[2] - posRL[2]
    axleWidth = posFL[0] - posFR[0]

    scenario = AckermannSteeringHelperScenario(
        stage, physxVehicleInterface, wheelAttachmentPaths, wheelSteerAngles, axleSeparation, axleWidth,
        timeStepsPerSec
    )
    Stepper.run_scenario(scenario, resetAtEnd=False)

[Mayankm96]

Hi,

That is an interesting question. I believe if you want to make full use of parallelization, going through OmniGraphs may not work as expected. You can write your custom action term for Ackermann steering.

We did a sample implementation for differential steering here: https://isaac-orbit.github.io/orbit/source/api/orbit/omni.isaac.orbit.envs.mdp.html#omni.isaac.orbit.envs.mdp.actions.actions_cfg.NonHolonomicActionCfg

[MaltheT]

Thank you for your answers, they were very helpful! I was able to implement a custom AckermannAction with inspiration from the NonHolonomicAction and some of the equations from AckermannSteeringHelper.py.

Thanks for clairfying about the OmniGraphs not working as expected. Going into this project I thought they would be the main interface for interacting with the robot, as the Isaac Sim tutorials spend a fair amount of time explaining these.