RestAPI_v1_3E.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6bd72b8b-78b6-4385-8205-96cc2b1ab0fe",
   "metadata": {},
   "source": [
    "# Using Python to call 3E's Rest API\n",
    "**NOTE:** \n",
    "- The following assumes decent usage of Python. \n",
    "- This is an exploratory workbook as a way to discover how to use the 3e API\n",
    "- Use anything here at your own risk\n",
    "\n",
    "At time of writing there isn't much documentation on how to use the Rest API for 3e. Below are a few samples of some calls to create, retrieve and update. The list of examples will grow within this notebook as time permits.\n",
    "\n",
    "There are a couple of articles in the Thomson Reuters Knowledge base that cover a few of the basics. Specifically:\n",
    "- TR-19784 : How to obtain access to 3E APIs \n",
    "- TR-19969 : ASG - 3E OData - Troubleshooting Guide (E) \n",
    "- TR-19847 : 3E API & OData - A Primer for Developers \n",
    "- TR-19363 : Using Postman to Demo 3E API Connections \n",
    "\n",
    "For this sample, we'll use NTLM authentication, simply because that's all that is available currently. If you've got a cloud instance, you'd go through the authentication to get your token. See the section in TR-19847 for more information on this.\n",
    "\n",
    "Because this will be saved to the internet, credentials and instance urls are tucked into another Python file in the same folder as this one. Use whatever method you like to accomplish this hiding of credentials, whether that be environment variables, calls out to third party password managers, or read from a database. For simplicity, I'm using a service account with limited permissons. The `environment.py` file looks a little like the following:\n",
    "\n",
    "```python\n",
    "class Credentials:\n",
    "    \n",
    "    def __init__(self, wapi=None, instance=None, run_as=None, run_as_password=None):\n",
    "        self.wapi = wapi\n",
    "        self.instance = instance\n",
    "        self.run_as = run_as\n",
    "        self.run_as_password = run_as_password\n",
    "        \n",
    "    def load_defaults(self):\n",
    "        self.wapi = 'your_internal_url.local'\n",
    "        self.instance = 'te_3e_dbname'\n",
    "        self.run_as = 'domain\\some_user_name'\n",
    "        self.run_as_password = 'some_password'\n",
    "    \n",
    "    def base_url(self, version=1):\n",
    "        return f\"https://{self.wapi}/{self.instance}/web/api/v{version}\"\n",
    "```\n",
    "\n",
    "#### Requirements\n",
    "`Requests` is the only 3rd party library used here. The rest is Python standard library.\n",
    "\n",
    "The following uses `requests_ntlm` which can be installed in your environment with `pip install requests-ntlm`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89bf9c27-1d81-497f-b2c7-ad292dcbc2bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from operator import attrgetter  # allows us to use a dotted notation for the API responses\n",
    "import requests\n",
    "from requests_ntlm import HttpNtlmAuth\n",
    "import json\n",
    "import warnings\n",
    "from types import SimpleNamespace\n",
    "            \n",
    "from environment import Credentials"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7eda820a-eab0-4191-bb0c-b76fb8b6150c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_value(dotted, obj, default=None):\n",
    "    \"\"\" \n",
    "    Convenience function to return a default value when AttributeError is encountered\n",
    "    \"\"\"\n",
    "    try:\n",
    "        return attrgetter(dotted)(obj)\n",
    "    except AttributeError:\n",
    "        return default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6cd695e-091b-418d-943a-35ac2caa205d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load up the default connection information\n",
    "env = Credentials()\n",
    "env.load_defaults()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80006c9b-bcea-4e06-b6db-2ea5bdbba8dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fire up a NTLM session with the Wapi\n",
    "# working with a self signed certificate here that can be problematic\n",
    "# for now, turning off the ssl verification for local 3e endpoint\n",
    "\n",
    "warnings.filterwarnings(\"ignore\", message=\"Unverified HTTPS request\")\n",
    "\n",
    "session = requests.Session()\n",
    "session.auth = HttpNtlmAuth(env.run_as, env.run_as_password)\n",
    "session.verify = False\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94ad22c3-2a2d-4176-8182-72aae62271a6",
   "metadata": {},
   "source": [
    "## Get details about a Timekeeper\n",
    "This one is an easy place to start. This endpoint `/api/v1/timekeeper` will per the documentation, \"Gets Timekeepers and returns a TimekeeperGetResponse.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8924e99b-3a38-4565-8124-69d7ffc21ad9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# specify the url of the REST API endpoint\n",
    "timekeeper_url = f\"{env.base_url()}/timekeeper\"\n",
    "\n",
    "# issue a GET request with the query parameters and store the response\n",
    "query_params = {'Number': \"3171\", }\n",
    "response = session.get(timekeeper_url, params=query_params)\n",
    "\n",
    "# decode the response data from JSON to a Python dotted notation object\n",
    "# the reponse has 'timekeepers' as top level key\n",
    "# because by definition, everything coming back is JSON serialized so \n",
    "# the object_hook with a SimpleNamespace can give us dot notation\n",
    "\n",
    "# it is also possible to keep it as a normal dictionary by removing the object_hook\n",
    "\n",
    "timekeepers = response.json(object_hook=lambda d: SimpleNamespace(**d)).timekeepers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4deaaba-29bc-4785-bfd6-9dcea2f4f8f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# if you want to see what the shape of timekeepers 'looks like' uncomment the following\n",
    "# timekeepers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2d5e257-0117-4c26-ac58-1a09c43715ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "# couple of ways to get the value\n",
    "for timekeeper in timekeepers:\n",
    "    print(timekeeper.id)\n",
    "    print(timekeeper.attributes.Number.value)\n",
    "    print(timekeeper.attributes.DisplayName.value)\n",
    "    print(attrgetter(\"attributes.DisplayName.value\")(timekeeper))\n",
    "    # demonstrate why get_value may be of some use\n",
    "    print(get_value(\"attributes.DisplayName.value.some_non_existant\", timekeeper, \"Unknown\"))\n",
    "    print(get_value(\"attributes.DisplayName.value.some_non_existant\", timekeeper))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8448a2b-3992-496b-933f-a9f84fc864d7",
   "metadata": {},
   "source": [
    "### Advanced Filtering\n",
    "It looks like there is some ability to specify the attributes and child objects to return. On a large list, I suspect this would improve performance. It feels a little bit like using GraphQL queries, but not as easy.\n",
    "\n",
    "I've not yet discovered how to return a `TimekeeperGetResponse` without any child objects. Passing an empty list, empty string, and `None` all don't perform as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa169833-8d85-4670-af58-86a5a53668bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "query_params = {'Number': \"3171\",\n",
    "                'AdvancedFilter.AttributesToInclude': [\"Number\", \"DisplayName\"],\n",
    "                'AdvancedFilter.ChildObjectsToInclude': [\"TkprSchool\"],}\n",
    "response = session.get(timekeeper_url, params=query_params)\n",
    "timekeepers = response.json(object_hook=lambda d: SimpleNamespace(**d)).timekeepers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1849f486-2879-4442-b32e-f2036ea30e48",
   "metadata": {},
   "outputs": [],
   "source": [
    "timekeepers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2de379f5-10a0-46aa-aa00-ffecd8667bbf",
   "metadata": {},
   "source": [
    "## Updating the timekeeper record\n",
    "To update a record, the `Patch` method is used. The tricky part here was trying to figure out what the body of the request should look like. There doesn't appear to be any documentation available at the moment for how to do this, but with some trial and error and attempts to decipher PostMan's suggestions we end up with the following. It would be good as with most API's that the Swagger documentation would more clearly state this format. The Postman suggestions are filled with `<Error: Too many levels of nesting to fake this schema>` which is unfortunate. \n",
    "\n",
    "It is not clear if the reponse can be tailored to only return specific fields. The response is quite a bit different than the others in that looks like the following.\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"success\": true,\n",
    "    \"dataCollection\": {\n",
    "        \"id\": \"Timekeeper\",\n",
    "        \"objectId\": \"Timekeeper\",\n",
    "        \"actualRowCount\": 2,\n",
    "        \"rows\": [\n",
    "            {\n",
    "                ...\n",
    "            }\n",
    "        ]\n",
    "     }\n",
    "}            \n",
    "```\n",
    "\n",
    "If one of the records is locked by another user, you'll get a `202 Multi-Status` response with the following in the `rows` array/list.\n",
    "```json\n",
    "\"index\": 1,\n",
    "\"isLocked\": true,\n",
    "\"lockedMessage\": \"Timekeeper record was modified by _named_user_ since it was opened.  Reopen the record to save changes.\",\n",
    "\"subclassId\": \"Timekeeper\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97406434-c617-4788-b01f-111a956223ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "# in practice, this would be built up programatically, but for illustrative purposes, this is \n",
    "# what the body of the request will contain. The following updates two different timekeeper records in one request.\n",
    "# the id's will need to be changed to reflect your own instance. The id's can be found in the previous step, or\n",
    "# by examining the `Timekeeper.TimekeeperID` field using SQL. It is unclear if other identifiers (like TkprIndex, \n",
    "# or Number) can be used in place of \"id\"\n",
    "\n",
    "request_body = \"\"\"\n",
    "{\n",
    "    \"DataCollection\": {\n",
    "        \"rows\": [\n",
    "            {\n",
    "                \"id\": \"7a0c32275e384c2b887b3af476bd1eb5\",\n",
    "                \"attributes\": {\n",
    "                    \"displayName\": {\n",
    "                        \"value\": \"A MacGillivary \"\n",
    "                    },\n",
    "                    \"SortName\": {\n",
    "                        \"value\": \"MacGillivary, A\"\n",
    "                    }\n",
    "                }\n",
    "            },\n",
    "            {\n",
    "                \"id\": \"13e3ee7b60694c7583b36ca37eb8ea80\",\n",
    "                \"attributes\": {\n",
    "                    \"SortName\": {\n",
    "                        \"value\": \"Costanza, G\"\n",
    "                    }\n",
    "\n",
    "                }\n",
    "            }\n",
    "       ]\n",
    "    }\n",
    "}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a01293c-4bb5-473d-ac0f-17440a541f6a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# in this example, that request_body is a string, it needs to be converted to json when passed to the json argument - use `json.loads` for that\n",
    "response = session.patch(timekeeper_url, json=json.loads(request_body))\n",
    "timekeepers_updated = response.json(object_hook=lambda d: SimpleNamespace(**d))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6da65ad2-d274-43f9-b61a-3065f19a476f",
   "metadata": {},
   "outputs": [],
   "source": [
    "timekeepers_updated"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a8cd842-ea01-4836-a7fe-129ffff45502",
   "metadata": {},
   "source": [
    "## Creating a record\n",
    "We'll use a timecard pending record as a way to illustrate how to use the `POST` method to create a new record into the `TimecardPending` table. Similarly to the update of a timekeeper, we'll use a string represention of what we want to insert, but typically it would be constructed programmatically and not need the conversion from string to json (using the `json.loads` call)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73b08002-0095-4801-b35d-efab86d787fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# substitute in an appropriate matter and timekeeper\n",
    "# for these the aliasValue can be used (matter.number, timekeeper.number etc), instead of the index or id\n",
    "# it is unclear at this time if the order of the attributes matters as it sometimes does when sending a similar \n",
    "# request through the SOAP transaction services\n",
    "\n",
    "request_body = \"\"\"\n",
    "{\n",
    "    \"DataCollection\": {\n",
    "        \"Id\": \"TimeCardPending\",\n",
    "        \"Rows\": [\n",
    "            {\n",
    "                \"Attributes\": {\n",
    "                    \"Matter\": {\n",
    "                        \"aliasValue\": \"186772\"\n",
    "                    },\n",
    "                    \"WorkDate\": {\n",
    "                        \"Value\": \"2022-08-08\"\n",
    "                    },\n",
    "                    \"Timekeeper\": {\n",
    "                        \"aliasValue\": \"3171\"\n",
    "                    },\n",
    "                    \"WorkHrs\": {\n",
    "                        \"Value\": \"2.5\"\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    "}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "457118b8-ab51-4606-b078-3a607c1bd66f",
   "metadata": {},
   "outputs": [],
   "source": [
    "timecard_pending_url = f\"{env.base_url()}/time/pending\"\n",
    "response = session.post(timecard_pending_url, json=json.loads(request_body))\n",
    "timecard_pending = response.json(object_hook=lambda d: SimpleNamespace(**d))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d4cb5a0-9132-418b-94a3-2c2126c3a90b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# to see what timecard_pending 'holds'\n",
    "timecard_pending"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d69314e1-1565-4e11-8e66-3e0c1bd9646b",
   "metadata": {},
   "outputs": [],
   "source": [
    "for timecard in timecard_pending.dataCollection.rows:\n",
    "    print(timecard.attributes.TimePendIndex.value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "609214f9-1821-49e0-b14b-7d0d6b4719ce",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}