Code for "SVRG for Policy Evaluation with Fewer Gradient Evaluations"

Dependencies

  - blas=1.0=mkl
  - ca-certificates=2020.10.14=0
  - certifi=2020.11.8=py36hecd8cb5_0
  - cffi=1.14.4=py36h2125817_0
  - intel-openmp=2019.4=233
  - libcxx=10.0.0=1
  - libedit=3.1.20191231=h1de35cc_1
  - libffi=3.3=hb1e8313_2
  - mkl=2019.4=233
  - mkl-service=2.3.0=py36h9ed2024_0
  - mkl_fft=1.2.0=py36hc64f4ea_0
  - mkl_random=1.1.1=py36h959d312_0
  - ncurses=6.2=h0a44026_1
  - ninja=1.10.2=py36hf7b0b51_0
  - numpy=1.19.2=py36h456fd55_0
  - numpy-base=1.19.2=py36hcfb5961_0
  - openssl=1.1.1h=haf1e3a3_0
  - pandas=1.1.3=py36hb1e8313_0
  - pip=20.3=py36hecd8cb5_0
  - pycparser=2.20=py_2
  - python=3.6.12=h26836e1_2
  - python-dateutil=2.8.1=py_0
  - pytorch=1.3.1=cpu_py36h0c87eb2_0
  - pytz=2020.4=pyhd3eb1b0_0
  - readline=8.0=h1de35cc_0
  - setuptools=50.3.2=py36hecd8cb5_2
  - six=1.15.0=py36hecd8cb5_0
  - sqlite=3.33.0=hffcf06c_0
  - tk=8.6.10=hb0a8c7a_0
  - wheel=0.36.0=pyhd3eb1b0_0
  - xz=5.2.5=h1de35cc_0
  - zlib=1.2.11=h1de35cc_3
  - pip:
    - cached-property==1.5.2
    - cloudpickle==1.6.0
    - future==0.18.2
    - gym==0.17.3
    - h5py==3.1.0
    - progressbar==2.5
    - pyglet==1.5.0
    - scipy==1.5.4
    - tqdm==4.54.1

To get the results:

For figure 1, run python figure_1.py

For table 3, run python table_3.py

For figure 2, run python figure_2.py

Links to our paper and my thesis:

Arxiv paper

My Msc thesis