Merge pull request #116 from Exabyte-io/feature/SOF-7291

Feature/SOF-7291 adjust API-examples to work
Exabyte-io · Apr 3, 2024 · 59f1b9b · 59f1b9b
2 parents 40f831c + b205cdd
commit 59f1b9b
Show file tree

Hide file tree

Showing 27 changed files with 592 additions and 317 deletions.
diff --git a/README.md b/README.md
@@ -11,7 +11,7 @@ Below, we list the contents of this repository, in roughly the order that a user
 | [Examples/Workflow](examples/workflow/) | [Quantum Espresso Workflow and Job](examples/workflow/qe_scf_calculation.ipynb)  | Create Quantum Espresso workflow starting from QE input file; create and submit job; after the job is finished, download output file, and finally perform postprocessing analysis.
 | [Examples/Material](examples/material/) | [Get Materials by Formula](examples/material/get_materials_by_formula.ipynb)                | Shows how [queries](https://docs.mat3ra.com/rest-api/query-structure/) can be made to search for materials stored on your account by their formula. In this example, we search for a system containing Si.
 | [Examples/Material](examples/material/) | [Create Material](examples/material/create_material.ipynb)                         | Gives an overview of how materials can be generated in [JSON format](https://docs.mat3ra.com/materials/data/) and uploaded to your user account. In this example, we create an FCC Si crystal and upload it.
-| [Examples/Material](examples/material/) | [Import Materials from Materials Project](examples/material/import_materials_from_materialsproject.ipynb) | Demonstrates how materials can be imported from [Materials Project](https://materialsproject.org/about), if their Materials Project ID is known. In this example, we import monoclinic and hexagonal SiGe cells.
+| [Examples/Material](examples/material/) | [Import Materials from Materials Project](other/materialsproject/import_materials_from_materialsproject.ipynb) | Demonstrates how materials can be imported from [Materials Project](https://materialsproject.org/about), if their Materials Project ID is known. In this example, we import monoclinic and hexagonal SiGe cells.
 | [Examples/Material](examples/material/) | [Import Materials from Poscar](examples/material/upload_materials_from_file_poscar.ipynb)            | Provides an example of how materials can be imported directly from Poscar files (a common chemical file format best-known [for its use in VASP](https://www.vasp.at/wiki/index.php/Input)). In this example, we import the unit cell of SiGe.
 | [Examples/Job](examples/job/)            | [Create and Submit Job](examples/job/create_and_submit_job.ipynb)                   | Shows how to use the Mat3ra API to [create jobs](https://docs.mat3ra.com/jobs/data/) and run them on our cluster. In this example, we run a DFT calculation to get the total energy of an FCC Si unit cell using Quantum Espresso.
 | [Examples/Job](examples/job/)            | [Get File from Job](examples/job/get-file-from-job.ipynb) | Uses the Mat3ra API to query for a list of files produced by a job, describes the metadata assigned to each file, and ends by demonstrating how to download any remote file generated by a job to the local disk.

diff --git a/other/materials_designer/config.yml → config.yml b/other/materials_designer/config.yml → config.yml
@@ -6,6 +6,7 @@
 # The default packages are installed for all the notebooks.
 default:
   packages_common:
+    - pandas==1.5.3
     - ase==3.22.1
   packages_python:
     - pymatgen==2023.9.10
@@ -21,18 +22,17 @@ default:
     - tabulate==0.9.0
     - sympy==1.12
     - uncertainties==3.1.6
+    - jinja2
 notebooks:
   - name: create_interface_with_min_strain_zsl.ipynb
     packages_common:
-      - pandas==1.5.3
       - ipywidgets
       - plotly==5.18
       - nbformat>=4.2.0
     packages_python:
     packages_pyodide:
   - name: import_material_from_jarvis_db_entry.ipynb
     packages_common:
-      - pandas==1.5.3
       - express-py==2024.2.2.post2
       - mat3ra-esse
       - jarvis-tools
@@ -44,7 +44,6 @@ notebooks:
       - https://files.mat3ra.com:44318/uploads/watchdog-2.3.1-py3-none-any.whl
   - name: import_materials_from_files.ipynb
     packages_common:
-      - pandas==1.5.3
       - express-py==2024.2.2.post2
       - mat3ra-esse
       - munch

diff --git a/examples/job/create_and_submit_job.ipynb b/examples/job/create_and_submit_job.ipynb
@@ -66,6 +66,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -77,7 +78,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/job/get-file-from-job.ipynb b/examples/job/get-file-from-job.ipynb
@@ -88,6 +88,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -99,7 +100,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/job/ml-train-model-predict-properties.ipynb b/examples/job/ml-train-model-predict-properties.ipynb
@@ -88,6 +88,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -99,7 +100,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/job/run-simulations-and-extract-properties.ipynb b/examples/job/run-simulations-and-extract-properties.ipynb
@@ -95,6 +95,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -106,7 +107,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/material/create_material.ipynb b/examples/material/create_material.ipynb
@@ -61,6 +61,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -72,7 +73,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/material/get_materials_by_formula.ipynb b/examples/material/get_materials_by_formula.ipynb
@@ -62,6 +62,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -73,7 +74,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/material/upload_materials_from_file_poscar.ipynb b/examples/material/upload_materials_from_file_poscar.ipynb
@@ -62,6 +62,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -76,7 +77,27 @@
     "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; export IS_USING_GIT_LFS=true; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
     "    from examples.utils.notebook import get_notebook_info\n",
     "\n",
-    "    os.chdir(os.path.join(\"api-examples\", os.path.dirname(get_notebook_info()[\"notebook_path\"])))"
+    "    os.chdir(os.path.join(\"api-examples\", os.path.dirname(get_notebook_info()[\"notebook_path\"])))\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/system/get_authentication_params.ipynb b/examples/system/get_authentication_params.ipynb
@@ -59,9 +59,29 @@
     "PASSWORD = getpass(\"Please enter password: \")\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {

diff --git a/examples/workflow/get_workflows.ipynb b/examples/workflow/get_workflows.ipynb
@@ -66,6 +66,7 @@
     "ORGANIZATION_ID = \"ORGANIZATION_ID\"  # @param {type:\"string\"}\n",
     "\n",
     "import os\n",
+    "import sys\n",
     "\n",
     "if \"COLAB_JUPYTER_IP\" in os.environ:\n",
     "    os.environ.update(\n",
@@ -77,7 +78,27 @@
     "        )\n",
     "    )\n",
     "\n",
-    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash"
+    "    !GIT_BRANCH=\"dev\"; export GIT_BRANCH; curl -s \"https://raw.githubusercontent.com/Exabyte-io/api-examples/${GIT_BRANCH}/scripts/env.sh\" | bash\n",
+    "\n",
+    "if sys.platform == \"emscripten\":\n",
+    "    apiConfig = data_from_host.get(\"apiConfig\")\n",
+    "    os.environ.update(data_from_host.get(\"environ\", {}))\n",
+    "    os.environ.update(\n",
+    "        dict(\n",
+    "            ACCOUNT_ID=apiConfig.get(\"accountId\"),\n",
+    "            AUTH_TOKEN=apiConfig.get(\"authToken\"),\n",
+    "            MATERIALS_PROJECT_API_KEY=MATERIALS_PROJECT_API_KEY,\n",
+    "            ORGANIZATION_ID=apiConfig.get(\"organizationId\") or \"\",\n",
+    "        )\n",
+    "    )\n",
+    "\n",
+    "    import micropip\n",
+    "\n",
+    "    await micropip.install(\"mat3ra-api-examples\", deps=False)\n",
+    "    await micropip.install(\"exabyte-api-client\")\n",
+    "    from utils.jupyterlite import install_packages\n",
+    "\n",
+    "    await install_packages(\"\", \"../../config.yml\")"
    ]
   },
   {