atomsci/ddm/examples/tutorials/04_Explore_Data_DTC_Curate.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/ATOMScience-org/AMPL/blob/master/atomsci/ddm/examples/tutorials/04_Explore_Data_DTC_Curate.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0V2ybLgAH-0V"
   },
   "source": [
    "# Exploring HTR3A protein target activity data from Drug Target Commons\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5wzPlqzmrOOn"
   },
   "source": [
    "# Scope of the tutorial\n",
    "*   Input data from Excape dataset for HTR3A protein target\n",
    "*   AMPL will be used to accomplish the following steps: \n",
    "  *   Standardize SMILES string\n",
    "  *   Clean the data (look for duplicates, average the assay data, cluster the compounds etc.)\n",
    "  *   Curate the data\n",
    "  *   Carry out some Exploratory Data Analysis (Chemical space exploration; PCA etc.) \n",
    "  *   Save the final curated dataset for modeling\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zxDMhFSoIL15"
   },
   "source": [
    "Diseases associated with HTR3A include Irritable Bowel Syndrome and Motion Sickness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "giQ9v_J_rQH-"
   },
   "source": [
    "# Time on COLAB-Pro ( ~ 3 minutes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_lnDSsrkrWlE"
   },
   "source": [
    "# Protein target (HTR3A) information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wtw4W4ngsq9B"
   },
   "source": [
    "The HTR3A target binding data was downloaded from DTC database, https://drugtargetcommons.fimm.fi.\n",
    "\n",
    "Please refer to the following publication,https://pubmed.ncbi.nlm.nih.gov/29276046/, for details about the database\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7J41ky8ULQdh"
   },
   "source": [
    "## Additional information about HTR3A gene:\n",
    "\n",
    "**Gene location:** Chromosome 11\n",
    "**Exon count:** 10\n",
    "\n",
    "mRNA and protein information for its three transcripts:\n",
    "\n",
    "*   NM_000869.6 → NP_000860.3 \n",
    "*   NM_001161772.3 → NP_001155244.1\n",
    "*   NM_213621.4 → NP_998786.3 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "od_Bn9W5gCh9"
   },
   "source": [
    "## Before you begin, make sure you close all other COLAB notebooks. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2DWqm4Cxm5h_"
   },
   "source": [
    "# Change Runtime settings\n",
    "If you have access to COLAB-Pro (commercial/not-free), please change your runtime settings to use GPU and high-memory,\n",
    "\n",
    "```Runtime --> Change Runtime Type --> GPU with high-RAM```\n",
    "\n",
    "If you are not a paid COLAB-Pro customer, you can still choose GPU, with standard-RAM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rlYz7j65MDcb",
    "outputId": "02273f0c-b477-491e-b7bb-1497cf042475"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fri Sep 17 11:54:43 UTC 2021\n"
     ]
    }
   ],
   "source": [
    "!date # starting time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5qGO2T0cIvIm"
   },
   "source": [
    "## Install AMPL  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TW1L0cVrIqxm"
   },
   "outputs": [],
   "source": [
    "! pip install rdkit-pypi\n",
    "! pip install deepchem\n",
    "\n",
    "import deepchem\n",
    "# print(deepchem.__version__)\n",
    "! pip install umap\n",
    "! pip install -U --ignore-installed numba\n",
    "! pip install umap-learn\n",
    "! pip install molvs\n",
    "! pip install bravado"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8AzLwTe2q2_P"
   },
   "outputs": [],
   "source": [
    "import deepchem as dc\n",
    "\n",
    "# get the Install AMPL_GPU_test.sh\n",
    "!wget 'https://raw.githubusercontent.com/ATOMScience-org/AMPL/master/atomsci/ddm/examples/tutorials/config/install_AMPL_GPU_test.sh'\n",
    "\n",
    "# run the script to install AMPL\n",
    "! chmod u+x install_AMPL_GPU_test.sh\n",
    "! ./install_AMPL_GPU_test.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4qtjXXtuWZLQ"
   },
   "source": [
    "## Exploring target activity data from selected database\n",
    "## Data source = ExcapeDB\n",
    "## Target activity for the receptor HTR3A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "F-c9OaSoJHmG"
   },
   "outputs": [],
   "source": [
    "# We temporarily disable warnings for demonstration.\n",
    "# FutureWarnings and DeprecationWarnings are present from some of the AMPL \n",
    "# dependency modules.\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import json\n",
    "import os\n",
    "import requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "6r_-HG0aHwsE"
   },
   "outputs": [],
   "source": [
    "#\n",
    "# Import AMPL libraries\n",
    "# \n",
    "import atomsci.ddm.utils.data_curation_functions as dcf\n",
    "import atomsci.ddm.utils.curate_data as curate_data\n",
    "import atomsci.ddm.pipeline.diversity_plots as dp\n",
    "import atomsci.ddm.pipeline.chem_diversity as cd\n",
    "# Additional python libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import getpass,os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1X7qQCYVHwsG"
   },
   "source": [
    "## Select a target to work with \n",
    "### (e.g. PDE2A, KCNH2, SCNA5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "tkweVTMSHwsG"
   },
   "outputs": [],
   "source": [
    "target_name='HTR3A'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yCA0PykcHwsH"
   },
   "source": [
    "# Define data locations "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "1SVhXwHgJiZN"
   },
   "outputs": [],
   "source": [
    "ofile=target_name+'_dtc_smiles.csv'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f1RXnxhVVcxt"
   },
   "source": [
    "### Note the file `HTR3A_dtc_smiles.csv` is the output from\n",
    "https://github.com/ATOMScience-org/AMPL/blob/master/atomsci/ddm/examples/tutorials/03_Explore_Data_DTC.ipynb \n",
    "notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "bN1KN4sGJjjM"
   },
   "outputs": [],
   "source": [
    "import io\n",
    "url = 'https://raw.githubusercontent.com/ATOMScience-org/AMPL/master/atomsci/ddm/examples/tutorials/datasets/HTR3A_dtc_smiles.csv'\n",
    "download = requests.get(url).content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "XBDmluXMKD38"
   },
   "outputs": [],
   "source": [
    "# Reading the downloaded content and turning it into a pandas dataframe\n",
    "save_smiles_df = pd.read_csv(io.StringIO(download.decode('utf-8')), sep=',', header=0 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "fRw8B47gqtpA"
   },
   "outputs": [],
   "source": [
    "# save_smiles_df.rename( columns={ \"PIC50\" : \"PXC50\" }, inplace=True) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SWCSjlwyq_HZ"
   },
   "source": [
    "##Check for assay replicates with high measurement variation and flag them "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "inaUPauW-BFE",
    "outputId": "1c7c2b61-393b-404a-c3b4-f8164ca01bdb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bad duplicates removed from dataset\n",
      "Dataframe size (197, 52)\n",
      "List of 'bad' duplicates removed\n",
      "              standard_inchi_key     PIC50  ...   Perc_Var  VALUE_NUM_std\n",
      "172  FZGWDCDUMFWUEO-UHFFFAOYSA-N  8.159894  ...  11.975455       1.234152\n",
      "173  FZGWDCDUMFWUEO-UHFFFAOYSA-N  6.414539  ...  11.975455       1.234152\n",
      "67   KQKPFRSPSRPDEB-UHFFFAOYSA-N  5.000000  ...  23.262963       2.143606\n",
      "68   KQKPFRSPSRPDEB-UHFFFAOYSA-N  8.031517  ...  23.262963       2.143606\n",
      "\n",
      "[4 rows x 5 columns]\n",
      "\n",
      "Dataset de-duplicated\n",
      "Dataframe size (170, 52)\n",
      "New column created with averaged values:  VALUE_NUM_mean\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "# From our dataframe, we are working with the PIC50 column \n",
    "data=save_smiles_df\n",
    "\n",
    "# Here we are defining our response column, which will be PIC50\n",
    "column = 'PIC50'\n",
    "\n",
    "# tolerance: Percentage of individual respsonse values allowed to different from \n",
    "# the average to be included in averaging\n",
    "tolerance = 10\n",
    "\n",
    "# list_bad_duplicates: Print structures with bad duplicates\n",
    "list_bad_duplicates = 'Yes'\n",
    "\n",
    "# max_std: Maximum allowed standard deviation for computed average response value\n",
    "# NOTE: In this example, we set this value very high to disable this feature\n",
    "max_std = 1\n",
    "\n",
    "# compound_id: Compound ID column\n",
    "compound_id = 'standard_inchi_key'\n",
    "\n",
    "# smiles_col: SMILES column\n",
    "smiles_col = 'rdkit_smiles'\n",
    "\n",
    "# Here we are creating a new dataframe, called check_df\n",
    "check_df = curate_data.average_and_remove_duplicates(column, tolerance, \n",
    "                                                       list_bad_duplicates, \n",
    "                                                       data, max_std, \n",
    "                                                       compound_id=compound_id, \n",
    "                                                       smiles_col=smiles_col)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wwWNUc9nsjdT"
   },
   "source": [
    "## Examine the rejected compounds and examine them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 558
    },
    "id": "iimss0y6soSA",
    "outputId": "7f512eef-1109-4a2c-ea66-92a369726839"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>compound_id</th>\n",
       "      <th>uniprot_id</th>\n",
       "      <th>compound_name</th>\n",
       "      <th>standard_inchi_key</th>\n",
       "      <th>max_phase</th>\n",
       "      <th>target_pref_name</th>\n",
       "      <th>gene_names</th>\n",
       "      <th>target_class</th>\n",
       "      <th>wildtype_or_mutant</th>\n",
       "      <th>mutation_information</th>\n",
       "      <th>pubmed_id</th>\n",
       "      <th>standard_type</th>\n",
       "      <th>standard_relation</th>\n",
       "      <th>standard_value</th>\n",
       "      <th>standard_units</th>\n",
       "      <th>mode_of_action</th>\n",
       "      <th>assay_format</th>\n",
       "      <th>assay_type</th>\n",
       "      <th>assay_sub_type</th>\n",
       "      <th>inhibitor_type</th>\n",
       "      <th>detection_technology</th>\n",
       "      <th>compound_concentration_value</th>\n",
       "      <th>compound_concentration_value_units</th>\n",
       "      <th>substrate_type</th>\n",
       "      <th>substrate_type_standard_relation</th>\n",
       "      <th>substrate_type_standard_value</th>\n",
       "      <th>substrate_type_standard_units</th>\n",
       "      <th>assay_cell_line</th>\n",
       "      <th>assay_description</th>\n",
       "      <th>activity_comments</th>\n",
       "      <th>title</th>\n",
       "      <th>journal</th>\n",
       "      <th>year</th>\n",
       "      <th>volume</th>\n",
       "      <th>issue</th>\n",
       "      <th>authors</th>\n",
       "      <th>annotation_comments</th>\n",
       "      <th>assay_id</th>\n",
       "      <th>dtc_tid</th>\n",
       "      <th>dtc_activity_id</th>\n",
       "      <th>dtc_molregno</th>\n",
       "      <th>record_id</th>\n",
       "      <th>dtc_document_id</th>\n",
       "      <th>PIC50</th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>CID</th>\n",
       "      <th>smiles</th>\n",
       "      <th>rdkit_smiles</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>CHEMBL128</td>\n",
       "      <td>P46098</td>\n",
       "      <td>SUMATRIPTAN</td>\n",
       "      <td>KQKPFRSPSRPDEB-UHFFFAOYSA-N</td>\n",
       "      <td>4</td>\n",
       "      <td>SEROTONIN 3A (5-HT3A) RECEPTOR</td>\n",
       "      <td>HTR3A</td>\n",
       "      <td>Ion channel</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7752204.0</td>\n",
       "      <td>IC50</td>\n",
       "      <td>&gt;</td>\n",
       "      <td>10000.00</td>\n",
       "      <td>NM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Binding activity radioligand.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Synthesis and serotonergic activity of N,N-dim...</td>\n",
       "      <td>J. Med. Chem.</td>\n",
       "      <td>1995.0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>Street LJ, Baker R, Davey WB, Guiblin AR, Jell...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3126.0</td>\n",
       "      <td>DTCT0023221</td>\n",
       "      <td>520894</td>\n",
       "      <td>DTCC00154636</td>\n",
       "      <td>658692</td>\n",
       "      <td>13017</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>110</td>\n",
       "      <td>5358</td>\n",
       "      <td>CNS(=O)(=O)CC1=CC2=C(C=C1)NC=C2CCN(C)C</td>\n",
       "      <td>CNS(=O)(=O)Cc1ccc2[nH]cc(CCN(C)C)c2c1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>CHEMBL128</td>\n",
       "      <td>P46098</td>\n",
       "      <td>SUMATRIPTAN</td>\n",
       "      <td>KQKPFRSPSRPDEB-UHFFFAOYSA-N</td>\n",
       "      <td>4</td>\n",
       "      <td>SEROTONIN 3A (5-HT3A) RECEPTOR</td>\n",
       "      <td>HTR3A</td>\n",
       "      <td>Ion channel</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9357515.0</td>\n",
       "      <td>IC50</td>\n",
       "      <td>=</td>\n",
       "      <td>9.30</td>\n",
       "      <td>NM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Compound was evaluated for the affinity at 5-h...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Selective, orally active 5-HT1D receptor agoni...</td>\n",
       "      <td>J. Med. Chem.</td>\n",
       "      <td>1997.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>MacLeod AM, Street LJ, Reeve AJ, Jelley RA, St...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3459.0</td>\n",
       "      <td>DTCT0023221</td>\n",
       "      <td>712596</td>\n",
       "      <td>DTCC00154636</td>\n",
       "      <td>659675</td>\n",
       "      <td>14385</td>\n",
       "      <td>8.031517</td>\n",
       "      <td>110</td>\n",
       "      <td>5358</td>\n",
       "      <td>CNS(=O)(=O)CC1=CC2=C(C=C1)NC=C2CCN(C)C</td>\n",
       "      <td>CNS(=O)(=O)Cc1ccc2[nH]cc(CCN(C)C)c2c1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>CHEMBL470432</td>\n",
       "      <td>P46098</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FZGWDCDUMFWUEO-UHFFFAOYSA-N</td>\n",
       "      <td>0</td>\n",
       "      <td>SEROTONIN 3A (5-HT3A) RECEPTOR</td>\n",
       "      <td>HTR3A</td>\n",
       "      <td>Ion channel</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19261477.0</td>\n",
       "      <td>IC50</td>\n",
       "      <td>=</td>\n",
       "      <td>6.92</td>\n",
       "      <td>NM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Displacement of [3H]granisetron from human rec...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Novel antagonists of serotonin-4 receptors: sy...</td>\n",
       "      <td>Bioorg. Med. Chem.</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Lemaître S, Lepailleur A, Bureau R, Butt-Gueul...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>564583.0</td>\n",
       "      <td>DTCT0023221</td>\n",
       "      <td>2613037</td>\n",
       "      <td>DTCC00570224</td>\n",
       "      <td>1082935</td>\n",
       "      <td>33265</td>\n",
       "      <td>8.159894</td>\n",
       "      <td>723</td>\n",
       "      <td>42636941</td>\n",
       "      <td>CCCN1CCC(CC1)COC2=NC3=C(C(=CS3)C)N4C2=CC=C4</td>\n",
       "      <td>CCCN1CCC(COc2nc3scc(C)c3n3cccc23)CC1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>CHEMBL470432</td>\n",
       "      <td>P46098</td>\n",
       "      <td>NaN</td>\n",
       "      <td>FZGWDCDUMFWUEO-UHFFFAOYSA-N</td>\n",
       "      <td>0</td>\n",
       "      <td>SEROTONIN 3A (5-HT3A) RECEPTOR</td>\n",
       "      <td>HTR3A</td>\n",
       "      <td>Ion channel</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19261477.0</td>\n",
       "      <td>IC50</td>\n",
       "      <td>=</td>\n",
       "      <td>385.00</td>\n",
       "      <td>NM</td>\n",
       "      <td>NaN</td>\n",
       "      <td>cell_based</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NG108-15</td>\n",
       "      <td>Antagonist activity at 5HT3 receptor in hybrid...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Novel antagonists of serotonin-4 receptors: sy...</td>\n",
       "      <td>Bioorg. Med. Chem.</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Lemaître S, Lepailleur A, Bureau R, Butt-Gueul...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>564575.0</td>\n",
       "      <td>DTCT0023221</td>\n",
       "      <td>2613019</td>\n",
       "      <td>DTCC00570224</td>\n",
       "      <td>1082935</td>\n",
       "      <td>33265</td>\n",
       "      <td>6.414539</td>\n",
       "      <td>723</td>\n",
       "      <td>42636941</td>\n",
       "      <td>CCCN1CCC(CC1)COC2=NC3=C(C(=CS3)C)N4C2=CC=C4</td>\n",
       "      <td>CCCN1CCC(COc2nc3scc(C)c3n3cccc23)CC1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      compound_id  ...                           rdkit_smiles\n",
       "67      CHEMBL128  ...  CNS(=O)(=O)Cc1ccc2[nH]cc(CCN(C)C)c2c1\n",
       "68      CHEMBL128  ...  CNS(=O)(=O)Cc1ccc2[nH]cc(CCN(C)C)c2c1\n",
       "172  CHEMBL470432  ...   CCCN1CCC(COc2nc3scc(C)c3n3cccc23)CC1\n",
       "173  CHEMBL470432  ...   CCCN1CCC(COc2nc3scc(C)c3n3cccc23)CC1\n",
       "\n",
       "[4 rows x 48 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "old_compound_id='rdkit_smiles'\n",
    "new_compound_id='rdkit_smiles'\n",
    "\n",
    "# Takes all the compounds that aren't part of the curated data frame and prints them\n",
    "reject=data[~data[old_compound_id].isin(check_df[new_compound_id])]\n",
    "reject"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EIOxXzq8s8vg",
    "outputId": "d95a80d9-092b-4e85-ddf8-ce65b32c154e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of discarded compounds 2\n"
     ]
    }
   ],
   "source": [
    "# Prints number of unique rejected compound names and rows\n",
    "print(\"Number of discarded compounds\",reject[compound_id].nunique())\n",
    "\n",
    "# Apecifies directory for csv file to send rejected data to, saving it as ofile \n",
    "ofile = target_name+'.reject_check'\n",
    "\n",
    "# Sends the rejected compound dataframe to a csv file with the path (ofile) specified above\n",
    "reject.to_csv(ofile,index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XgF6g71ItUhf",
    "outputId": "e6555b92-a8f4-4181-9edc-759247615315"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Inhibitory concentration required against 5-HT3 receptor in bovine area postrema using [3H]GR-65630',\n",
       "       'Binding affinity to human recombinant serotonin 5HT3 receptor',\n",
       "       'Agonist activity at human 5HT3A receptor expressed in HEK293 cells by FLIPR assay',\n",
       "       'Displacement of [3H]granisetron from human recombinant 5-HT3 receptor expressed in HEK293 cells',\n",
       "       nan,\n",
       "       'Antagonist activity at human wild type 5-HT3A receptor expressed in HEK293 cells assessed as inhibition of 5-HT-induced response after 45 mins by FLIPR assay',\n",
       "       'Compound was tested for its affinity towards 5-hydroxytryptamine 3 receptor',\n",
       "       'Displacement of [3H]BRL 43694 from human recombinant 5HT3 receptor expressed in CHO cells',\n",
       "       'Antagonist activity at human 5HT3 receptor',\n",
       "       'Displacement of [3H]BRL-43694 from human 5HT3 receptor expressed in HEK293 cells by liquid scintillation counting',\n",
       "       'Displacement of [3H]BRL-43694 from human recombinant 5HT3 receptor',\n",
       "       'Binding activity radioligand.',\n",
       "       'Compound was evaluated for the affinity at 5-hydroxytryptamine 3 receptor',\n",
       "       'Displacement of [3H]BRL43694 from human recombinant 5-HT3 receptor expressed in CHO cells',\n",
       "       'Inhibition of GR-65630 binding to 5-hydroxytryptamine 3 receptor',\n",
       "       'Binding affinity to 5-HT3 receptor',\n",
       "       'Inhibitory activity against 5-hydroxytryptamine 3 receptor expressed in NG 108 cells',\n",
       "       'Antagonist activity at human 5HT3A receptor expressed in HEK293 cells by FMP assay',\n",
       "       'Antagonist activity at human 5HT3A receptor expressed in HEK293 cells assessed as inhibition of m-chlorophenylbiguanide-induced calcium influx measured every 1 sec for 1 min followed by every 3 secs for 3 mins by FLIPR assay',\n",
       "       'Binding affinity to 5HT3 receptor',\n",
       "       'Agonist activity at human 5HT3R expressed in HEK293 cells assessed as stimulation of calcium flux by FLIPR',\n",
       "       'Antagonist activity at human 5HT3A receptor expressed in xenopus oocytes assessed as inhibition of 5HT-induced effect by electrophysiological method',\n",
       "       'Antagonist activity at 5HT3A receptor expressed in Xenopus oocyte',\n",
       "       'Inhibition of 5HT3 receptor (unknown origin)',\n",
       "       'Displacement of [3H]BRL43694 from 5HT3 receptor',\n",
       "       'Displacement of radioligand from 5HT3 receptor by liquid scintillation spectrometry',\n",
       "       'Inhibition of 5HT3 receptor',\n",
       "       'Antagonist activity at human 5HT3 receptor by calcium influx assay',\n",
       "       'Displacement of [3H]granisetron from human recombinant 5HT3 receptor',\n",
       "       'Antagonist activity at 5HT3 receptor in hybrid NG108-15 cells assessed as inhibition of 5HT-induced current by electrophysiology',\n",
       "       'Displacement of [3H]granisetron form 5HT3 receptor in hybrid NG180-15 cells',\n",
       "       'Displacement of [3H]LY278584 from human 5HT3 receptor expressed in HEK293 cells',\n",
       "       'Agonist activity at human 5HT3 receptor expressed in human skin epithelial cells assessed as stimulation of calcium flux by FLIPR assay'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Prints all the results of the unique assay tests run on this target\n",
    "save_smiles_df['assay_description'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IVeRrPCNtitM",
    "outputId": "2bb3a6d6-f323-4f24-d397-a5ca3b221e66"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(197, 48)\n"
     ]
    }
   ],
   "source": [
    "# Collects all the non-rejected data into a new frame called data\n",
    "data=save_smiles_df[~save_smiles_df[compound_id].isin(reject[compound_id])]\n",
    "\n",
    "# Prints the rows & columns (in that order) of the dataset from above (i.e. all the matching data)\n",
    "# Shape is an array that can check how large the dataframe is \n",
    "# It will print out (x, y), the x is the rows, or lines, and y is the number of columns \n",
    "print(data.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "aBtqLk9XuZ4k",
    "outputId": "f0cce72e-b55b-4d0d-caf2-30001c4b8609"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>standard_inchi_key</th>\n",
       "      <th>rdkit_smiles</th>\n",
       "      <th>standard_relation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>OSVMSQACZRYAPT-UHFFFAOYSA-N</td>\n",
       "      <td>COc1cccc(CN2CCN(C3CCC(O)(c4ccc5c(c4)OCO5)CC3)C...</td>\n",
       "      <td>&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>CKLPLPZSUQEDRT-WPCRTTGESA-N</td>\n",
       "      <td>CC1Cc2c([nH]c3cc(Cl)c(F)cc23)C2(N1)C(=O)Nc1ccc...</td>\n",
       "      <td>&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>YOCUJVXRRWEPDN-UHFFFAOYSA-N</td>\n",
       "      <td>O=C(NCCCCN1CCCCC1)Nc1ccc(-c2ccccc2F)nc1</td>\n",
       "      <td>&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>LMGUUVWYFJIKCD-UHFFFAOYSA-N</td>\n",
       "      <td>O=C(NCCCCN1CCCCC1)Nc1ccc(-c2ccc(F)cc2)nc1</td>\n",
       "      <td>&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DFBQTBNRGSIKHN-UHFFFAOYSA-N</td>\n",
       "      <td>O=C(NCCCCN1CCCCC1)Nc1ccc(-c2ccc(F)nc2)nc1</td>\n",
       "      <td>&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            standard_inchi_key  ... standard_relation\n",
       "0  OSVMSQACZRYAPT-UHFFFAOYSA-N  ...                 >\n",
       "1  CKLPLPZSUQEDRT-WPCRTTGESA-N  ...                 >\n",
       "2  YOCUJVXRRWEPDN-UHFFFAOYSA-N  ...                 >\n",
       "3  LMGUUVWYFJIKCD-UHFFFAOYSA-N  ...                 >\n",
       "4  DFBQTBNRGSIKHN-UHFFFAOYSA-N  ...                 >\n",
       "\n",
       "[5 rows x 3 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['standard_inchi_key','rdkit_smiles','standard_relation']].head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bZWJcTGwuEo9",
    "outputId": "1e7f50a5-dfb4-4a84-e913-691bb58389f8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 entries in input table are missing SMILES strings\n",
      "170 unique SMILES strings are reduced to 170 unique base SMILES strings\n"
     ]
    }
   ],
   "source": [
    "#specifies the column we want (response value) and also tells it to include the duplicates\n",
    "column='PIC50'; #'standard_value'\n",
    "list_bad_duplicates='Yes'\n",
    "\n",
    "# Actually compiles all the data with the specifies column and tells what columns to include \n",
    "# (inchi_key, smiles, and relation)\n",
    "temp_df=curate_data.aggregate_assay_data(data, \n",
    "                                         value_col=column, \n",
    "                                         output_value_col=None,\n",
    "                                         label_actives=True,\n",
    "                                         active_thresh=6,\n",
    "                                         id_col='standard_inchi_key', \n",
    "                                         smiles_col='rdkit_smiles', \n",
    "                                         relation_col='standard_relation')\n",
    "\n",
    "#Removes all duplicates from the data frame and puts it in the final, curated dataframe\n",
    "curated_df = temp_df[~temp_df.isin([np.inf]).any(1)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IghTbhQ_vNKS"
   },
   "source": [
    "## Save data to a file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 223
    },
    "id": "oELdfonJvMYc",
    "outputId": "318da815-eee3-460e-fb86-e4e81f997728"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "before 197 after 170\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>compound_id</th>\n",
       "      <th>base_rdkit_smiles</th>\n",
       "      <th>relation</th>\n",
       "      <th>PIC50</th>\n",
       "      <th>active</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>XYQXSKXZFRFXOW-UHFFFAOYSA-N</td>\n",
       "      <td>Nn1c(CCN2CCN(c3ccc4ccccc4n3)CC2)nc2c(c1=O)CCCC2</td>\n",
       "      <td>&gt;</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>KELGFUJORMNBDM-UHFFFAOYSA-N</td>\n",
       "      <td>COc1ccc(-c2cc(NC(=O)C(C)CCCN3CCCN(C(C)=O)CC3)n...</td>\n",
       "      <td>&gt;</td>\n",
       "      <td>4.522879</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>OXIAXPOOKVTTDN-UHFFFAOYSA-N</td>\n",
       "      <td>O=C(NCCCCN1CCCCC1)Nc1ccc(-c2ccccc2F)cc1</td>\n",
       "      <td></td>\n",
       "      <td>5.244125</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FTPAKCBSUUBRFU-UHFFFAOYSA-N</td>\n",
       "      <td>O=C(NCCCCN1CCCCC1)Nc1ccc(-c2ccc(F)cc2)cc1</td>\n",
       "      <td></td>\n",
       "      <td>5.236572</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>UKZPDTRKIBHQLP-JKBBQUHLSA-L</td>\n",
       "      <td>CC(=O)OCC[N+]12CCC3C4C5C(CC31)C(=CCOC5N1c3cccc...</td>\n",
       "      <td>&gt;</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   compound_id  ... active\n",
       "0  XYQXSKXZFRFXOW-UHFFFAOYSA-N  ...      1\n",
       "1  KELGFUJORMNBDM-UHFFFAOYSA-N  ...      0\n",
       "2  OXIAXPOOKVTTDN-UHFFFAOYSA-N  ...      0\n",
       "3  FTPAKCBSUUBRFU-UHFFFAOYSA-N  ...      0\n",
       "4  UKZPDTRKIBHQLP-JKBBQUHLSA-L  ...      0\n",
       "\n",
       "[5 rows x 5 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prints the number of rows in the before and after curation \n",
    "print(\"before\",data.shape[0],\"after\",curated_df.shape[0])\n",
    "\n",
    "#prints the first few rows of the \"after\" (after curation) dataframes\n",
    "curated_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "j-lhDvMwu_NS",
    "outputId": "9976fc8c-1f7f-44a9-902c-36f589280188"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of discarded compounds 2\n"
     ]
    }
   ],
   "source": [
    "# Prints number of unique resjected compound names/rows\n",
    "print(\"Number of discarded compounds\",reject[compound_id].nunique())\n",
    "\n",
    "# Apecifies directory for csv file to send rejected data to, saving it as ofile \n",
    "ofile = target_name+'.reject_check'\n",
    "\n",
    "# Sends the rejected compound dataframe to a csv file with the path (ofile) specified above\n",
    "reject.to_csv(ofile,index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "HPZ8bb4qvXtQ"
   },
   "outputs": [],
   "source": [
    "# specifies file path to save the curated data to\n",
    "ofile = target_name+'_dtc_smiles_curated.csv'\n",
    "\n",
    "# sends data from the curated_df to csv with specified path above\n",
    "curated_df.to_csv(ofile,index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 302
    },
    "id": "kFz8VF17vmvn",
    "outputId": "aa68cfab-1c18-4748-839e-a5609295a8ca"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.0, 12.0)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEMCAYAAADK231MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAYlklEQVR4nO3df5BU5Z3v8fdHII4yMxuBkVghgJIIBAo1jsHEoKaMkljRa4Sty4XERGMRQyTGXbc2uRGc9UeyW8n1UhLUYKnxBsk1eIHErLFcNqAShWSIP8qRURajgKvXgWtGBhwF871/nB5sW2F6enq6h4fPq6rL6eecPud7cOYzzzz99HMUEZiZWboOq3YBZmbWtxz0ZmaJc9CbmSXOQW9mljgHvZlZ4gZW46TDhg2L0aNHV+PUZmYHrQ0bNmyPiIaevq4qQT969Giam5urcWozs4OWpBdLeZ2HbszMEuegNzNLnIPezCxx3Y7RS+ooaDoCuDki5ua2nwUsAkYC64GvRURJ40hmdujas2cP27Zto7Ozs9qlVF1NTQ0jRoxg0KBBZTlet0EfEbVdX0uqBV4BluWeDwOWA5cC9wHXAfcAp5alOjM7ZGzbto26ujpGjx6NpGqXUzURwY4dO9i2bRvHHntsWY7Z06GbacCrwCO55xcCLRGxLCI6gSbgBEnjylKdmR0yOjs7GTp06CEd8gCSGDp0aFn/sulp0H8V+F/xzpKXE4AnuzZGxC5gc679XSTNltQsqbmtra3Ues0sYYd6yHcp979D0UEvaRRwBnBXXnMt0F6waztQV/j6iFgcEY0R0djQ0OP5/mZmVqKefGDqK8DaiPhzXlsHUF+wXz2ws7eFmdmhben6LWU93szJI7vd54UXXuCLX/wiTz/9dFHHbG1tZcaMGUji3nvvZcyYMb0ts0/0JOgvAv65oK2FbDgHAEmDgTG5dktUuX8Au1PMD6hZNaxcuZLp06dz9dVXF7V/RBARHHZYZWe2F3U2SZ8GPkxutk2eFcBESdMk1QDzgaciorW8ZZqZVcbevXuZNWsW48ePZ/r06ezevZsNGzZwxhlncPLJJzN16lRefvll7r//fhYsWMAtt9zCZz/7WQBuvPFGJk6cyMSJE1mwYAGQ/ZUwduxYLrroIiZOnMjWrVv50Y9+xCmnnMKkSZO45ppr+vyaiv218lVgeUS8a0gmItrIZuLcALwGTAZmlLVCM7MKevbZZ5kzZw4bN26kvr6eRYsWMXfuXO699142bNjAJZdcwve//33OPfdcLrvsMq688kpWr17Nhg0buPPOO1m/fj3r1q3jtttu4/HHHwdg06ZNzJkzh5aWFp599lk2bdrEH/7wB5544gk2bNjAww8/3KfXVNTQTUR84wDbVgGeTmlmSfjIRz7CaaedBsCXv/xlfvCDH/D0009z9tlnA/D2229zzDHHvOd1a9eu5Utf+hKDBw8G4MILL+SRRx7h/PPPZ9SoUZx6avbxogcffJAHH3yQk046CYCOjg42bdrE6aef3mfXVJXVK83M+qvCqY11dXVMmDCBxx57rORjdoU/ZOP03/ve9/jGN/bbfy47r3VjZpZny5Yt+0J96dKlnHrqqbS1te1r27NnDy0t751vMmXKFFauXMnu3bvZtWsXK1asYMqUKe/Zb+rUqdxxxx10dGSry7z00ku8+uqrfXhF7tGbWT9VrdlWY8eOZdGiRVxyySV8/OMfZ+7cuUydOpVvf/vbtLe3s3fvXr7zne8wYcK7Pxf6iU98gq997Wt88pOfBODSSy/lpJNO4oUXXnjXfueccw4bN27kU5/6FAC1tbUsWbKEo48+us+uSe98yLVyGhsbwzceOXh5eqX1hY0bNzJ+/Phql9FvvN+/h6QNEdHY02N56MbMLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxHkevZn1T813lvd4jReX/NJly5Yxf/58PvShD3HNNdfwgQ98gE9/+tNlLK5vuUdvZtaN22+/ndtuu43Vq1ezZs0aHn300R69fu/evX1UWXHcozczy3PBBRewdetWOjs7ueKKK3jllVdYu3YtX//615k0aRKPPPIIAwYMYMmSJSxcuJBx48Zx2WWXsWVL9kHCBQsWcNppp9HU1MTmzZt5/vnnGTlyJL/4xS+qdk0OejOzPHfccQdDhgzhjTfe4JRTTuGhhx7id7/7HT/+8Y9pbGykqamJ2tparrrqKgBmzpzJlVdeyWc+8xm2bNnC1KlT2bhxIwDPPPMMa9eu5YgjjqjmJTnozczy3XTTTaxYsQKArVu3smnTpgPuv2rVKp555pl9z19//fV9C5adf/75VQ95cNCbme2zZs0aVq1axWOPPcaRRx7JmWeeSWdn5wFf89e//pV169ZRU1Pznm35yxNXk9+MNTPLaW9v56ijjuLII4+ktbWVdevWvWefuro6du5852Z755xzDgsXLtz3/IknnqhIrT3hHr2Z9U+9mA5Zqs9//vPceuutjB8/nrFjx+67K1S+8847j+nTp/OrX/2KhQsXctNNN/Gtb32LSZMmsXfvXk4//XRuvfXWitd+IF6m2HrMyxRbX/Ayxe/mZYrNzKxoRQe9pBmSNkraJWmzpCm59rMktUraLWm1pFF9V66ZmfVUUUEv6WzgX4CLgTrgdOB5ScOA5cA8YAjQDNzTN6WaWeqqMZTcH5X736HYHv0/AddGxLqI+GtEvBQRLwEXAi0RsSwiOoEm4ARJ48papZklr6amhh07dhzyYR8R7Nix432na5aq21k3kgYAjcCvJf0HUAOsBP4BmAA8mVfgLkmbc+2tBceZDcwGGDnSb66Z2buNGDGCbdu20dbWVu1Sqq6mpoYRI0aU7XjFTK8cDgwCpgNTgD3Ar4CrgVqg8P9KO9nwzrtExGJgMWSzbkov2cxSNGjQII499thql5GkYoZu3sj9d2FEvBwR24EbgXOBDqC+YP96YCdmZtYvdBv0EfEasA3I74V3fd0CnNDVKGkwMCbXbmZm/UCxb8beCcyVdLSko4Argd8AK4CJkqZJqgHmA09FROsBjmVmZhVUbNBfB/wReA7YCDwO3BARbcA04AbgNWAyMKMP6jQzsxIVtdZNROwB5uQehdtWAZ5OaWbWT3kJBDOzxDnozcwS52WKrd/zaplmveMevZlZ4hz0ZmaJc9CbmSXOQW9mljgHvZlZ4jzrxnpszJZlFT3f5pF/W9HzmaXGPXozs8Q56M3MEuegNzNLnIPezCxxDnozs8Q56M3MEuegNzNLnIPezCxxDnozs8Q56M3MEuegNzNLXFFBL2mNpE5JHbnHs3nbZkp6UdIuSSslDem7cs3MrKd60qO/PCJqc4+xAJImAD8FvgIMB3YDN5e/TDMzK1VvV6+cBdwXEQ8DSJoHbJRUFxE7e12dmZn1Wk969D+UtF3S7yWdmWubADzZtUNEbAbeAo4vfLGk2ZKaJTW3tbX1pmYzM+uBYoP+H4HjgA8Di4H7JI0BaoH2gn3bgbrCA0TE4ohojIjGhoaGXpRsZmY9UVTQR8T6iNgZEW9GxF3A74FzgQ6gvmD3esDDNmZm/USpY/QBCGgBTuhqlHQccDjwXO9LMzs0LF2/paLnmzl5ZEXPZ9XXbdBL+iAwGXgI2Av8V+B04ApgEPCYpCnAn4BrgeV+I9bMrP8opkc/CLgeGAe8DbQCF0TEcwCSLgPuBoYCq4CL+6ZUMzMrRbdBHxFtwCkH2L4UWFrOoszMrHy8BIKZWeIc9GZmievtJ2PN+tyYLcsqe8LJf1/R06V+fVZ97tGbmSXOQW9mljgHvZlZ4hz0ZmaJc9CbmSXOQW9mljgHvZlZ4hz0ZmaJc9CbmSXOQW9mljgHvZlZ4hz0ZmaJc9CbmSXOQW9mljgHvZlZ4hz0ZmaJc9CbmSWuR0Ev6WOSOiUtyWubKelFSbskrZQ0pPxlmplZqXrao18E/LHriaQJwE+BrwDDgd3AzWWrzszMeq3oe8ZKmgH8BXgU+GiueRZwX0Q8nNtnHrBRUl1E7Cx3sWZm1nNF9egl1QPXAn9XsGkC8GTXk4jYDLwFHP8+x5gtqVlSc1tbW+kVm5lZjxQ7dHMdcHtEbCtorwXaC9ragbrCA0TE4ohojIjGhoaGnldqZmYl6XboRtKJwOeAk95ncwdQX9BWD3jYxsysnyhmjP5MYDSwRRJkvfgBkj4OPACc0LWjpOOAw4Hnyl2omZmVppigXwz877znV5EF/zeBo4HHJE0B/kQ2jr/cb8SamfUf3QZ9ROwmmzYJgKQOoDMi2oA2SZcBdwNDgVXAxX1Uq5mZlaDo6ZVdIqKp4PlSYGm5CjIzs/LyEghmZolz0JuZJc5Bb2aWOAe9mVniHPRmZolz0JuZJc5Bb2aWOAe9mVniHPRmZolz0JuZJc5Bb2aWOAe9mVniHPRmZolz0JuZJc5Bb2aWOAe9mVniHPRmZolz0JuZJc5Bb2aWOAe9mVniigp6SUskvSzpdUnPSbo0b9tZklol7Za0WtKovivXzMx6qtge/Q+B0RFRD5wPXC/pZEnDgOXAPGAI0Azc0yeVmplZSQYWs1NEtOQ/zT3GACcDLRGxDEBSE7Bd0riIaC1zrWZmVoKix+gl3SxpN9AKvAzcD0wAnuzaJyJ2AZtz7YWvny2pWVJzW1tbrws3M7PiFB30ETEHqAOmkA3XvAnUAu0Fu7bn9it8/eKIaIyIxoaGhtIrNjOzHunRrJuIeDsi1gIjgG8CHUB9wW71wM7ylGdmZr1V6vTKgWRj9C3ACV2NkgbntZuZWT/QbdBLOlrSDEm1kgZImgr8N+DfgRXAREnTJNUA84Gn/EasmVn/UUyPPsiGabYBrwE/Br4TEb+OiDZgGnBDbttkYEYf1WpmZiXodnplLszPOMD2VcC4chZlZmbl4yUQzMwS56A3M0ucg97MLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxHUb9JIOl3S7pBcl7ZT0hKQv5G0/S1KrpN2SVksa1bclm5lZTxTTox8IbAXOAP4GuBr4paTRkoYBy4F5wBCgGbinj2o1M7MSDOxuh4jYBTTlNf1G0p+Bk4GhQEtELAOQ1ARslzQuIlrLX66ZmfVUj8foJQ0HjgdagAnAk13bcr8UNufaC183W1KzpOa2trbSKzYzsx7pUdBLGgTcDdyV67HXAu0Fu7UDdYWvjYjFEdEYEY0NDQ2l1mtmZj1UdNBLOgz4OfAWcHmuuQOoL9i1HthZlurMzKzXigp6SQJuB4YD0yJiT25TC3BC3n6DgTG5djMz6weK7dHfAowHzouIN/LaVwATJU2TVAPMB57yG7FmZv1HMfPoRwHfAE4EXpHUkXvMiog2YBpwA/AaMBmY0ZcFm5lZzxQzvfJFQAfYvgoYV86izMysfLwEgplZ4hz0ZmaJc9CbmSXOQW9mljgHvZlZ4hz0ZmaJc9CbmSXOQW9mlrhuPzBldqhZun5LRc83pqJns0ORe/RmZolz0JuZJc5Bb2aWOAe9mVniHPRmZonzrBuzAmO2LKt2CWZl5R69mVniHPRmZolz0JuZJc5Bb2aWOAe9mVniigp6SZdLapb0pqSfFWw7S1KrpN2SVksa1SeVmplZSYrt0f8ncD1wR36jpGHAcmAeMARoBu4pZ4FmZtY7Rc2jj4jlAJIagRF5my4EWiJiWW57E7Bd0riIaC1zrWZmVoLejtFPAJ7sehIRu4DNufZ3kTQ7N/zT3NbW1svTmplZsXob9LVAe0FbO1BXuGNELI6IxohobGho6OVpzcysWL0N+g6gvqCtHtjZy+OamVmZ9DboW4ATup5IGkx2w5yWXh7XzMzKpNjplQMl1QADgAGSaiQNBFYAEyVNy22fDzzlN2LNzPqPYnv0VwNvAN8Fvpz7+uqIaAOmATcArwGTgRl9UKeZmZWo2OmVTUDTfratAsaVryQz60uVvPn5zMkjK3Yu2z8vgWBmljgHvZlZ4hz0ZmaJc9CbmSXOQW9mljgHvZlZ4hz0ZmaJc9CbmSWuqA9MmVk6xmxZVrmTTf77yp3L9ss9ejOzxDnozcwS56A3M0ucg97MLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxDnozcwS56A3M0tcWda6kTQEuB04B9gOfC8ilpbj2Na99cv+R7VLMHtfqX9vTv7bg2Mtn3ItarYIeAsYDpwI/KukJyOipUzHNzOzEvV66EbSYGAaMC8iOiJiLfBr4Cu9PbaZmfVeOXr0xwN7I+K5vLYngTPyd5I0G5ide/qmpKfLcO7+ahjZEFaqfH0Ht5Svr8LXdlXlTpUZW8qLyhH0tcDrBW3tQF1+Q0QsBhYDSGqOiMYynLtf8vUd3Hx9B6+Urw2y6yvldeWYddMB1Be01QM7y3BsMzPrpXIE/XPAQEkfy2s7AfAbsWZm/UCvgz4idgHLgWslDZZ0GvBfgJ8f4GWLe3vefs7Xd3Dz9R28Ur42KPH6FBG9PnNuHv0dwNnADuC7nkdvZtY/lCXozcys//ISCGZmiXPQm5klrqJBL2mIpBWSdkl6UdLMSp6/L0k6XNLtuevaKekJSV+odl19QdLHJHVKWlLtWspN0gxJG3Pfo5slTal2TeUiabSk+yW9JukVST+RVK5lUCpK0uWSmiW9KelnBdvOktQqabek1ZJGVanMku3v+iSdKunfJP0/SW2Slkk6prvjVbpHn78mzizgFkkTKlxDXxkIbCX7RPDfAFcDv5Q0uoo19ZVFwB+rXUS5STob+BfgYrIP/J0OPF/VosrrZuBV4BiyNanOAOZUtaLS/SdwPdkkkH0kDSObBTgPGAI0A/dUvLree9/rA44im3kzGhhF9nmlO7s7WMV+m+etiTMxIjqAtZK61sT5bqXq6Cu5aaZNeU2/kfRn4GTghWrU1BckzQD+AjwKfLTK5ZTbPwHXRsS63POXqllMHzgW+ElEdAKvSHoAOCg7WhGxHEBSIzAib9OFQEtELMttbwK2SxoXEa0VL7RE+7u+iPht/n6SfgI81N3xKtmj39+aOAflN1p3JA0nu+ZkPjgmqR64Fvi7atdSbpIGAI1Ag6T/kLQtN7RxRLVrK6MFwAxJR0r6MPAF4IEq11RuE8hyBdjXAdtMojlD9ldntxlTyaAvak2cFEgaBNwN3HUw9SKKcB1we0Rsq3YhfWA4MAiYDkwhG9o4iWwILhUPkwXe68A2smGNlVWtqPxqyXIlX6o5MwmYD/xDd/tWMugPiTVxJB1G9qngt4DLq1xO2Ug6Efgc8D+rXUsfeSP334UR8XJEbAduBM6tYk1lk/u+fIBs/How2SqPR5G9J5GSQyVnPgr8FrgiIh7pbv9KBn3ya+JIEtmdtoYD0yJiT5VLKqczyd4A2iLpFbL1WadJ+lM1iyqXiHiNrJeb/wnClD5NOAQYSTZG/2ZE7CB7Ey+JX2R5WshyBdj33uAY0sqZUcAq4LqIONBSM/tULOhLXBPnYHMLMB44LyLe6G7ng8xish+YE3OPW4F/BaZWs6gyuxOYK+loSUcBVwK/qXJNZZH7C+XPwDclDZT0QeCrwFPVraw0uWuoAQYAAyTV5KaKrgAmSpqW2z4feOpgG0Ld3/Xl3lv5Hdkv7FuLPmBEVOxB1qtYCewCtgAzK3n+Pr62UWQ9wE6yPx+7HrOqXVsfXW8TsKTadZT5mgaRTUH8C/AKcBNQU+26ynh9JwJrgNfIbs7xS2B4tesq8Vqacj9v+Y+m3LbPAa1kw3FrgNHVrrdc1wdck/s6P2M6ujue17oxM0ucl0AwM0ucg97MLHEOejOzxDnozcwS56A3M0ucg97MLHEOejOzxDnoLTmSXpD0hqQOSf9X0s8k1UpaI+nSvP3qJS2QtCW37+bc82G57WtyN1jpyD2eLTjPzNyNZnZJWilpSKWv1awYDnpL1XkRUQt8gmz54XetQinpA8C/k63m+Hmyha8+BewAPpm36+URUZt7jM17/QTgp2T3UxgO7Cb7VK1Zv3NQ3kbMrFgR8ZKk3wITCzZdRLbI12cjuxEOZHdfuq7IQ88C7ouIhwEkzQM2SqqLiKRWSrSDn3v0ljRJHyFbofHxgk2fAx7IC/n9+aGk7ZJ+L+nMvPbCG1xsJlua+vjeV21WXg56S9VKSX8B1pLdau0HBduHAi93c4x/BI4DPky2eud9ksbkth0yN7iwg5+D3lJ1QUR8MCJGRcSceO+y0TvIbpK9XxGxPiJ2RrZ++13A73ln/fZD4gYXlgYHvR2qVgFTczemKFYAyn1deIOL44DDyW6wY9avOOjtUPVzYCvwfySNk3SYpKGS/rukcyV9UNLUvBs+zCK7EXPXzbTvBs6TNCX3y+JaYLnfiLX+yEFvh6SIeJN3blDxb2Q3zP4D2b1U15PdhOR6oI3sJh1zyYaDnsu9vgW4jCzwXyUbm59T2aswK45vPGJmljj36M3MEuegNzNLnIPezCxxDnozs8Q56M3MEuegNzNLnIPezCxxDnozs8T9f+dyktfmr8jeAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# load graphics libraries matplotlib and seaborn (sns)\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# plt.subplots is a function that creates a full grid of subplots in a single array, and returning \n",
    "# them in a NumPy array\n",
    "# A NumPy array is a grid of values, all of the same type, and is indexed by a tubule of nonnegative integers \n",
    "# We are saving this subplots we're creating as fig, ax variables\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "# compiles the data frames before and after all the curation with the specified column value (response values)\n",
    "# If we remember or look up further in the notebook, we created two dataframes: data and curated_df \n",
    "# Our data dataframe consists of ofile (target_name+'_excape_curated.csv.') and we will be using the PIC50 column of our dataframe\n",
    "# This will be our before variable\n",
    "before=save_smiles_df[\"PIC50\"]\n",
    "\n",
    "# Our after variable is comprised of our curated_df dataframe \n",
    "# Our curated_df was configured earlier to discard any out of range values\n",
    "after=curated_df[\"PIC50\"]\n",
    "\n",
    "# plots the values of the response value column of both the before and after dataframes from above\n",
    "# sns.distplot is a Seaborn distplot, which allows you to show a histogram with a line on it\n",
    "# A distplot plots a univariate distribution of observations \n",
    "# Below, we are separating our data set into a before section and an after section \n",
    "# To break down the sns.distplot functions: \n",
    "# bins=range(0, 12, 1) is used to bin our range of values, meaning to divide the entire range of values into a series of intervals\n",
    "# ax=ax \n",
    "# kde is a kernel density estimate plot that represents the data using a continuous probability density curve\n",
    "# kde=False will remove the density line on the histogram\n",
    "# The label= function is used to label different sections of our plot \n",
    "sns.distplot(before, bins=range(0, 12, 1), ax=ax, kde=False, label='before')\n",
    "sns.distplot(after, bins=range(0, 12, 1), ax=ax, kde=False, label='after')\n",
    "\n",
    "#configures plot by adding a legend and setting the x axis limits for readability\n",
    "# plt.legend() adds the legend to the plot\n",
    "plt.legend()\n",
    "\n",
    "# ax.set_xlim is setting the x-axis limits of the graph, setting it to be from (0,12)\n",
    "ax.set_xlim([0, 12])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "101z-_vAhdlp",
    "outputId": "be0f1d68-625a-4a16-e102-87bda805e4e8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fri Sep 17 11:56:12 UTC 2021\n"
     ]
    }
   ],
   "source": [
    "!date # ending time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "aQmT5Lg9TCle"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "machine_shape": "hm",
   "name": "04_Explore_Data_DTC_Curate.ipynb",
   "provenance": []
  },
  "extensions": {
   "jupyter_dashboards": {
    "activeView": "report_default",
    "version": 1,
    "views": {
     "grid_default": {
      "cellMargin": 10,
      "defaultCellHeight": 20,
      "maxColumns": 12,
      "name": "grid",
      "type": "grid"
     },
     "report_default": {
      "name": "report",
      "type": "report"
     }
    }
   }
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}