Skip to content

[NeurIPS 2023] We use large language models as commonsense world model and heuristic policy within Monte-Carlo Tree Search, enabling better-reasoned decision-making for daily task planning problems.

License

Notifications You must be signed in to change notification settings

1989Ryan/llm-mcts

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

llm-mcts

This repository contains the code for NeurIPS'23 paper: Large language models as commonsense knowledge for large-scale task planning.

We use Large Language Models as both the commonsense world model and the heuristic policy within Monte Carlo Tree Search. LLM's world model provides with MCTS a commonsense prior belief of states for reasoned decision-making. The LLM's heuristic policy guides the search to relevant parts of the tree, substantially reducing the search complexity.

Figure

Updates

  • [25 Feb 2024] We have updated the code to use the latest version of the OpenAI API.

Cite

@inproceedings{
  zhao2023large,
  title={Large Language Models as Commonsense Knowledge for Large-Scale Task Planning},
  author={Zirui Zhao and Wee Sun Lee and David Hsu},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
  year={2023},
  url={https://openreview.net/forum?id=Wjp1AYB8lH}
}

Install

Install the repo:

git clone --recurse-submodules https://github.com/1989Ryan/llm-mcts.git

You need to first install virtual home. Please follow with the link at here as well as the official repository at here to install.

To intall the dependencies in our method, run

pip install -r requirement.txt

Generate Data

We use the code from here to generate the data. You can also use the script at here to generate the data.

To generate data, you need to generate the goal of a domain first, using the command

python vh/data_gene/gen_data/vh_init.py \
    --port "{Port Number}" \
    --task {choose your task} \
    --mode {choose one difficulty} \
    --usage {training or testing} \
    --num-per-apartment {a number} 

Then, to generate expert data, you need to use

python vh/data_gene/testing_agents/gene_data.py \
    --mode {difficulty} \
    --dataset_path {the path to the file generated in the previous step}\
    --base-port {port number}

After that, we need to pre-process the expert data

python mcts/virtualhome/expert_data.py

Run

Add your openai api key in both ./mcts/virtualhome/llm_model.py and ./mcts/virtualhome/llm_policy.py.

Generate the world model by LLM:

python mcts/virtualhome/llm_model.py

To run the code for LLM-MCTS, use

python mcts/virtualhome/mcts_agent.py \
    --exploration_constant 24 \
    --max_episode_len 50 \
    --max_depth 20 \
    --round 0 \
    --simulation_per_act 2 \
    --simulation_num 100 \
    --discount_factor 0.95  \
    --uct_type PUCT \
    --mode simple \
    --seen_item \
    --seen_apartment\
    --model gpt-3.5-turbo-0125 \
    --seen_comp

Acknowledge

This repository is built upon a number of prior opensource works.

About

[NeurIPS 2023] We use large language models as commonsense world model and heuristic policy within Monte-Carlo Tree Search, enabling better-reasoned decision-making for daily task planning problems.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published