Skip to content

Our VMAgent is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks.

License

Notifications You must be signed in to change notification settings

mail-ecnu/VMAgent

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

99 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

VMAgent LOGO

GitHub license Read the Docs GitHub issues GitHub forks GitHub stars

VMAgent is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks. It is developed by the Multi-Agent Artificial Intelligence Lab (MAIL) in East China Normal University and Algorithm Innovation Lab in HUAWEI Cloud. VMAgent is constructed based on one month real VM scheduling dataset called Huawei-East-1 from HUAWEI Cloud and it contains multiple practicle VM scheduling scenarios (such as Fading, Rcovering, etc). These scenarios also correspond to the challanges in the RL. Exploiting the design of RL methods in these secenarios help both the RL and VM scheduling communities. To emphasis, more details about VMAgent can be found in our paper VMAgent: Scheduling Simulator for Reinforcement Learning. Our another paper Learning to Schedule Multi-NUMA Virtual Machines via Reinforcement Learning has employed this VMAgent simultor to design RL-based VM scheduling algorithms.

Key Components of VMAgent:

  • SchedGym (Simulator): it provides many practical scenarios and flexible configurations to define custom scenarios.
  • SchedAgent (Algorithms): it provides many popular RL methods as the baselines.
  • SchedVis (Visulization): it provides the visualization of schedlueing dynamics on many metrics.

Scenarios and Baselines

The VMAgent provides multiple practical scenarios:

Scenario Allow-Deletion Allow-Expansion Server Num
Fading False False Small
Recovering True False Small
Expanding True True Small
Recovering-L True False Large

Researchers can also flexibly customized their scenarios in the vmagent/config/ folder.

Besides, we provides many baselines for quick startups. It includes FirstFit, BestFit, DQN, PPO, A2C and SAC. More baselines is coming.

Installation

git clone https://github.com/mail-ecnu/VMAgent.git
cd VMAgent
conda env create -f conda_env.yml
conda activate VMAgent-dev
python3 setup.py develop

To use runx module and .cv2 module, you may need to do next instructs:

pip install runx
sudo apt update
sudo apt install libgl1-mesa-glx

Quick Examples

In this quick example, we show how to train a dqn agent in a fading scenario. For more examples and the configurations' concrete definitions, we refer readers to our docs.

config/fading.yaml:

N: 5
cpu: 40 
mem: 90
allow_release: False

config/algs/dqn.yaml:

mac: 'vectormac'
learner: 'q_learner'
agent: 'DQNAgent'

Then

python vmagent/train.py --env=fading --alg=dqn

It provides the first VM scheudling simulator based on the one month east china data in HUAWEI Cloud. It includes three scenarios in practical cloud: Recovering, Fading and Expansion. Our video is at video. Some demonstrations are listed:

Docs

For more information of our VMAgent, we refer the readers to the document. It describes the detail of SchedGym, SchedAgent and SchedVis.

Data

We collect one month scheduling data in east china region of huawei cloud. The format and the stastical analysis of the data are presented in the docs. one month east china data in huawei cloud.

Visualization

For visualization, see the schedvis directory in detail.

References

  • Junjie Sheng, Shengliang Cai, Haochuan Cui, Wenhao Li, Yun Hua, Bo Jin, Wenli Zhou, Yiqiu Hu, Lei Zhu, Qian Peng, Hongyuan Zha and Xiangfeng Wang, VMAgent: Scheduling Simulator for Reinforcement Learning. IJCAI Demos, 2022.
  • Junjie Sheng, Yiqiu Hu, Wenli Zhou, Lei Zhu, Bo Jin, Jun Wang and Xiangfeng Wang, Learning to Schedule Multi-NUMA Virtual Machines via Reinforcement Learning, Pattern Recognition, 121, 2021, pp.108254.

License

Licensed under the MIT License.

About

Our VMAgent is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages