Applying external forces at specific times during RL training

[alvarobelmontebaeza]

Hi all!

I have an environment where I want to apply external forces to some links of a legged robot at certain times while the simulation is running for training. For example, if a foot gets in contact with some objects, i want to apply a force towards that object (think of it as a magnetic force for instance).

I managed to do this within the OmniIsaacGymEnvs framework using the 'pre_physics_step()' method and the omni.isaac.core.RigidPrimView.apply_forces() method. However, I want to migrate my environment to the ORBIT framework since it allows more customization, but I can't figure out where should I add the piece of code that handles this.

One idea is to create an observation term that reads from the contact sensors in the feet, and then applies this logic, but doesn't seem the best way to manage this.

Thanks in advance for your help!

[Mayankm96]

You can make a randomization term (the name isn't ideal but should do the job). We have a basic implementation for applying external forces. Using the randomization manager you can set different "modes" that decides when this term is applied:

• at only resets: https://github.com/NVIDIA-Omniverse/orbit/blob/main/source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks/locomotion/velocity/velocity_env_cfg.py#L173-L182
• at fixed intervals: https://github.com/NVIDIA-Omniverse/orbit/blob/main/source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks/locomotion/velocity/velocity_env_cfg.py#L209-L215

In your case, you can use fixed intervals and set the interval range to match your environment time-step, s.t. it applies that term in every environment step.

If you want to apply the term at every "simulation" step, then you'll need to customize things a bit inside the RLTaskEnv.step method to add another mode for the randomization manager to trigger every physics step (similar to how internval mode is handled).

[alvarobelmontebaeza]

Thanks a lot for the reply! I will check at how can I condition the randomization to be applied at every step only when the objects are in contact.

I took a look at the implementation for applying the external forces. Tracking it down, it ends up applying the RigidBodyView.apply_forces_and_torques_at_pos() method from the PhysX API. I have some concerns regarding this because it doesnt seem to me that the forces applied are in Newtons, although the documentation says so. For example, I'm applying a force of 2 kN on a robot feet directed to the ground, but the robot is still able to lift the foot, which shouldn't be possible taking into account that the actuators maximum torque is limited.

Do you have any insight on this as well since you use the method in your implementation? Thank you!

[Mayankm96]

Which frame are you trying to apply the force in? By default, we have it hard-coded in the local body frame. But maybe you need to adapt this to global frame?

https://isaac-orbit.github.io/orbit/source/api/orbit/omni.isaac.orbit.assets.html#omni.isaac.orbit.assets.RigidObject.write_data_to_sim

We have some simple tests that check that the "force" is valid (i.e. counteracts gravity). But we haven't tested for something more complex. You can check the test here:

https://github.com/NVIDIA-Omniverse/orbit/blob/main/source/extensions/omni.isaac.orbit/test/assets/test_rigid_object.py#L103-L159

[alvarobelmontebaeza]

I would implement it in local frame since I try to simulate an "adhesion" to the surface. I'm founding it tricky because the simulator technically reports contact impulses, not forces. This is more than enough to track just contacts, but for my goal of simulating adhesion I need to apply a force that "sticks" the leg to the surface, but finding a suitable adhesion force is not straightforward reading impulses.

Thanks for sharing the links. I have one more question: You use physx.RigidBodyView class, which is different to omni.isaac.core.RigidPrimView. Is there a reason for this? Does it have any benefit?

[Mayankm96]

The Isaac Sim RigidPrimView class wraps around the physx.RigidBodyView as well. It provides some additional functions for cases where users don't want to do physics (i.e. they want to do animation). However, we found this hardly useful for most practical purposes in robotics. Thus, we directly interface with the PhysX wrappers. It also allowed us to optimize certain operations and have some speed-ups (more details: 6f4cc59).

The only downside: the PhysX classes are not that nicely documented. But if you check the omni.isaac.core.RigidPrimView you'll notice it very thinly wraps the PhysX class.

[alvarobelmontebaeza]

Thanks for the insight! I will try and make that randomization term then to try to avoid modify the RLTaskEnv.step method and reducing the non-standard behavior of the framework.

And thanks for your work with Orbit! It makes working with Isaac Sim much more straightforwar. Keep pushing!