HeteroMorpheus

This is the code for the paper [HeteroMorpheus: Universal Control Based on Morphological Heterogeneity Modeling] at IJCNN 2024.

In the field of robotic control, designing individual controllers for each robot leads to high computational costs. Universal control policies, applicable across diverse robot morphologies, promise to mitigate this challenge. Predominantly, models based on Graph Neural Networks (GNN) and Transformers are employed, owing to their effectiveness in capturing relational dynamics across a robot's limbs. However, these models typically employ homogeneous graph structures that overlook the functional diversity of different limbs. To bridge this gap, we introduce HeteroMorpheus, a novel method based on heterogeneous graph Transformer. This method uniquely addresses limb heterogeneity, fostering better representation of robot dynamics of various morphologies. Through extensive experiments we demonstrate the superiority of HeteroMorpheus against state-of-the-art methods in the capability of policy generalization, including zero-shot generalization and sample-efficient transfer to unfamiliar robot morphologies.

Our work builds upon MetaMorph, a SOTA transformer-based method for universal morphology control, i.e., learning a universal policy that can generalize across different morphologies. We further propose to use heterogeneous graph Transformer(HGT) and the morphology-based message passing scheme to better model the complex dependence between robot morphology and control policy. See our paper for more details.

We choose Evolution Gym, a specialized simulation platform for voxel-based soft robots (VSRs), as our evaluation environment. Unlike rigid-body simulations, VSRs are normally arranged in grid structure, offering expansive design possibilities compared to rigid robots. Our experiments encompass five tasks in Evogym, comprising two locomotion tasks (Walker-v0, UpStepper-v0) and three manipulation tasks (Pusher-v0, Carrier-v0, Catcher-v0).

The overall architecture of HeteroMorpheus is shown in the figure below：

Installation

Clone the repo and submodules:

git clone --recurse-submodules https://github.com/hyf1530588058/HeteroMorpheus.git

Requirements

• Python 3.7
• Linux

sudo apt-get install xorg-dev libglu1-mesa-dev

Install Python dependencies with pip:

pip install -r requirements.txt

Running the code

For multi-robot training, run the following commands to train with different methods.

AMORPHEUS:

python examples/run_amorpheus.py --env-name "Walker-v0" --algo ppo --use-gae --lr 0.0001 --clip-param 0.1 --value-loss-coef 0.5 --num-processes 16 --num-steps 128 --num-mini-batch 12 --log-interva  100 --use-linear-lr-decay --entropy-coef 0.01 --eval-interval 50

MetaMorph:

python examples/run_metamorph.py --env-name "Walker-v0" --algo ppo --use-gae --lr 0.0001 --clip-param 0.1 --value-loss-coef 0.5 --num-processes 16 --num-steps 128 --num-mini-batch 12 --log-interva  100 --use-linear-lr-decay --entropy-coef 0.01 --eval-interval 50

HeteroMorpheus-D:

python examples/run_heteromorpheus_D.py --env-name "Walker-v0" --algo ppo --use-gae --lr 0.0001 --clip-param 0.1 --value-loss-coef 0.5 --num-processes 16 --num-steps 128 --num-mini-batch 12 --log-interva  100 --use-linear-lr-decay --entropy-coef 0.01 --eval-interval 50

HeteroMorpheus-N:

python examples/run_heteromorpheus_N.py --env-name "Walker-v0" --algo ppo --use-gae --lr 0.0001 --clip-param 0.1 --value-loss-coef 0.5 --num-processes 16 --num-steps 128 --num-mini-batch 12 --log-interva  100 --use-linear-lr-decay --entropy-coef 0.01 --eval-interval 50

Hyperparameter

The meaning and values of the hyperparameters required for experimental running can be found in examples/ppo/arguments.py

Training Result

This is the training curves of different methods in each environment. We evaluate on 5 different seeds and plot the mean of average returns over all morphologies. Shaded regions denotes standard deviation.

Zero-shot and Transfer Learning Result

The graph presents the results of both zero-shot learning and fine-tuning, obtained through five repeated experiments. Shaded regions denotes standard deviation

Citation

If you find this code useful, please consider citing:

@inproceedings{
    hao2024HeteroMorpheus,
    title={HeteroMorpheus: Universal Control Based on Morphological Heterogeneity Modeling},
    author={YiFan Hao, Yang Yang and Junru Song, Wei Peng and Weien Zhou, Tingsong Jiang and Wen Yao},
    journal={International Joint Conference on Neural Networks},
    year={2024},
}