notebooks/dataprep/01h-PublicationLink.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Link Dataset to Publications\n",
    "**[Work in progress]**\n",
    "\n",
    "This notebook downloads and standardizes accession numbers from life science and biological databases textmined from PubMed Central (PMC) full text articles and preprints (PPR) by [Europe PMC](https://europepmc.org/) for ingestion into a Knowledge Graph. In addition, it downloads PubMed to PDB mappings.\n",
    "\n",
    "Data sources: [Europe PMC](ftp://ftp.ebi.ac.uk/pub/databases/pmc/TextMinedTerms/), [PMC](https://www.ncbi.nlm.nih.gov/pmc/)\n",
    "\n",
    "Author: Peter Rose (pwrose@ucsd.edu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import dateutil\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.options.display.max_rows = None  # display all rows\n",
    "pd.options.display.max_columns = None  # display all columsns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "ftp = 'ftp://ftp.ebi.ac.uk/pub/databases/pmc/TextMinedTerms/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/peter/Library/Application Support/com.Neo4j.Relate/data/dbmss/dbms-8bf637fc-0d20-4d9f-9c6f-f7e72e92a4da/import\n"
     ]
    }
   ],
   "source": [
    "NEO4J_IMPORT = Path(os.getenv('NEO4J_IMPORT'))\n",
    "print(NEO4J_IMPORT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Assign unique identifiers for interoperabilitiy\n",
    "A [CURIE](https://en.wikipedia.org/wiki/CURIE) (Compact URI) is a compact abbreviation for Uniform Resource Identifiers (URIs). CURIEs consist of registered prefix and an accession number (prefix:accession). They provide a name space for identifiers to enable uniqueness of identifiers and interoperability among data resources.\n",
    "\n",
    "[Identifiers.org](http://identifiers.org/) provides a registry and resolution service for life science CURIEs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def assign_publication_id(row):\n",
    "    if row['PMCID'] != '':\n",
    "        # CURIE: pmc (PubMed Central, PMC)\n",
    "        return 'pmc:' + str(row['PMCID'])\n",
    "    elif row['SOURCE'] == 'PPR':\n",
    "        # no CURIE available, use URI for preprints\n",
    "        return 'https://europepmc.org/article/PPR/' + row['EXTID']\n",
    "    else:\n",
    "        return ''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### UniProt\n",
    "**accession**: CURIE: [uniprot](https://registry.identifiers.org/registry/uniprot) ( UniProt Knowledgebase, UniProtKB)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "uniprot = pd.read_csv(ftp + \"uniprot.csv\", dtype=str)\n",
    "uniprot.fillna('', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "uniprot['id'] = uniprot.apply(assign_publication_id, axis=1)\n",
    "uniprot['accession'] = 'uniprot:' + uniprot['uniprot']\n",
    "uniprot = uniprot[['id','accession']]\n",
    "uniprot.query(\"id != ''\", inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(412910, 2)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "uniprot.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NCBI Reference Sequences\n",
    "**accession**: CURIE: [refseq](https://registry.identifiers.org/registry/refseq) (NCBI Reference Sequences, Refseq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>refseq</th>\n",
       "      <th>PMCID</th>\n",
       "      <th>EXTID</th>\n",
       "      <th>SOURCE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NM_015973</td>\n",
       "      <td>PMC7512552</td>\n",
       "      <td>32963006</td>\n",
       "      <td>MED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NM_001034</td>\n",
       "      <td>PMC7512552</td>\n",
       "      <td>32963006</td>\n",
       "      <td>MED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NM_001789</td>\n",
       "      <td>PMC7512552</td>\n",
       "      <td>32963006</td>\n",
       "      <td>MED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NM_001008708</td>\n",
       "      <td>PMC7512552</td>\n",
       "      <td>32963006</td>\n",
       "      <td>MED</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NM_203467</td>\n",
       "      <td>PMC7512552</td>\n",
       "      <td>32963006</td>\n",
       "      <td>MED</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         refseq       PMCID     EXTID SOURCE\n",
       "0     NM_015973  PMC7512552  32963006    MED\n",
       "1     NM_001034  PMC7512552  32963006    MED\n",
       "2     NM_001789  PMC7512552  32963006    MED\n",
       "3  NM_001008708  PMC7512552  32963006    MED\n",
       "4     NM_203467  PMC7512552  32963006    MED"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "refseq = pd.read_csv(ftp + \"refseq.csv\", dtype=str)\n",
    "refseq.fillna('', inplace=True)\n",
    "refseq.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "refseq['id'] = refseq.apply(assign_publication_id, axis=1)\n",
    "# Remove version number from refseq to match to the latest version\n",
    "refseq['accession'] = 'refseq:' + refseq['refseq'].str.split('.', expand=True)[0]\n",
    "refseq = refseq[['id','accession']]\n",
    "refseq.query(\"id != ''\", inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(363994, 2)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "refseq.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GISAID Genome Sequences\n",
    "**id**: CURIE: [pmc](https://registry.identifiers.org/registry/pmc) (PubMed Central, PMC)\n",
    "\n",
    "**accession**: URI: [https://www.gisaid.org/](https://www.gisaid.org/help/publish-with-gisaid-references) (Global Initiative on Sharing All Influenza Data, GISAID)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "gisaid = pd.read_csv(ftp + \"gisaid.csv\", dtype=str)\n",
    "gisaid.fillna('', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "gisaid['id'] = gisaid.apply(assign_publication_id, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "gisaid['accession'] = 'https://www.gisaid.org/' + gisaid['gisaid']\n",
    "gisaid = gisaid[['id','accession']]\n",
    "gisaid.query(\"id != ''\", inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10170, 2)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gisaid.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Protein Data Bank\n",
    "**accession**: CURIE: [pdb](https://registry.identifiers.org/registry/pdb) (Protein Data Bank, PDB)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "pdb = pd.read_csv(ftp + \"pdb.csv\", dtype=str)\n",
    "pdb.fillna('', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "pdb['id'] = pdb.apply(assign_publication_id, axis=1)\n",
    "pdb['accession'] = 'pdb:' + pdb['pdb']\n",
    "pdb = pdb[['id','accession']]\n",
    "pdb.query(\"id != ''\", inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(557281, 2)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pdb.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Match Dataset Mentions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Match by UniProt accessions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>uniprot:P0DTC6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>uniprot:P0DTD3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>uniprot:P0DTC2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>uniprot:P0DTC7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>uniprot:P0DTD8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        accession\n",
       "0  uniprot:P0DTC6\n",
       "1  uniprot:P0DTD3\n",
       "2  uniprot:P0DTC2\n",
       "7  uniprot:P0DTC7\n",
       "9  uniprot:P0DTD8"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ref1 = pd.read_csv(NEO4J_IMPORT / \"01a-UniProtProtein.csv\")\n",
    "ref1 = ref1[['accession']]\n",
    "ref1 = ref1.drop_duplicates()\n",
    "ref1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_uniprot = pd.merge(uniprot, ref1, on='accession')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_uniprot.to_csv(NEO4J_IMPORT / \"01h-PMC-UniProtProtein.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "UniProt mentions: 225702\n"
     ]
    }
   ],
   "source": [
    "print('UniProt mentions:', pmc_uniprot.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "      <td>uniprot:P24385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC5761900</td>\n",
       "      <td>uniprot:P24385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC3275796</td>\n",
       "      <td>uniprot:P24385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC4823807</td>\n",
       "      <td>uniprot:P24385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC5474285</td>\n",
       "      <td>uniprot:P24385</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               id       accession\n",
       "0  pmc:PMC6713643  uniprot:P24385\n",
       "1  pmc:PMC5761900  uniprot:P24385\n",
       "2  pmc:PMC3275796  uniprot:P24385\n",
       "3  pmc:PMC4823807  uniprot:P24385\n",
       "4  pmc:PMC5474285  uniprot:P24385"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_uniprot.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Match Strains by NCBI refSeq and GISAID accessions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/miniconda3/envs/covid-19-community/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3166: DtypeWarning: Columns (10,15) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  interactivity=interactivity, compiler=compiler, result=result)\n"
     ]
    }
   ],
   "source": [
    "ref2 = pd.read_csv(NEO4J_IMPORT / \"01c-CNCBStrain.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>primaryAccession</th>\n",
       "      <th>secondaryAccession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_458142</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_458142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_458142</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_524788</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_524788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_524788</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_524796</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_524796</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        primaryAccession  \\\n",
       "0  https://www.gisaid.org/EPI_ISL_458142   \n",
       "0  https://www.gisaid.org/EPI_ISL_458142   \n",
       "1  https://www.gisaid.org/EPI_ISL_524788   \n",
       "1  https://www.gisaid.org/EPI_ISL_524788   \n",
       "2  https://www.gisaid.org/EPI_ISL_524796   \n",
       "\n",
       "                      secondaryAccession  \n",
       "0  https://www.gisaid.org/EPI_ISL_458142  \n",
       "0                                         \n",
       "1  https://www.gisaid.org/EPI_ISL_524788  \n",
       "1                                         \n",
       "2  https://www.gisaid.org/EPI_ISL_524796  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ref2['secondaryAccession'] = ref2['accessions'].str.split(';')\n",
    "ref2 = ref2.explode('secondaryAccession')\n",
    "ref2.rename(columns={'id': 'primaryAccession'}, inplace=True)\n",
    "ref2 = ref2[['primaryAccession', 'secondaryAccession']]\n",
    "ref2 = ref2.drop_duplicates()\n",
    "ref2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb1 = pd.merge(refseq, ref2, left_on='accession', right_on='secondaryAccession')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "      <th>primaryAccession</th>\n",
       "      <th>secondaryAccession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC7681180</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC7596387</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC7754929</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC8016547</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC7290700</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "      <td>refseq:NC_045512</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               id         accession primaryAccession secondaryAccession\n",
       "0  pmc:PMC7681180  refseq:NC_045512   insdc:MN908947   refseq:NC_045512\n",
       "1  pmc:PMC7596387  refseq:NC_045512   insdc:MN908947   refseq:NC_045512\n",
       "2  pmc:PMC7754929  refseq:NC_045512   insdc:MN908947   refseq:NC_045512\n",
       "3  pmc:PMC8016547  refseq:NC_045512   insdc:MN908947   refseq:NC_045512\n",
       "4  pmc:PMC7290700  refseq:NC_045512   insdc:MN908947   refseq:NC_045512"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_cncb1.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use primary accession to establish linkage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb1['accession'] = pmc_cncb1['primaryAccession']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb1 = pmc_cncb1[['id', 'accession']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "regseq mentions: 1315\n"
     ]
    }
   ],
   "source": [
    "print('regseq mentions:', pmc_cncb1.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC7681180</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC7596387</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC7754929</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC8016547</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC7290700</td>\n",
       "      <td>insdc:MN908947</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               id       accession\n",
       "0  pmc:PMC7681180  insdc:MN908947\n",
       "1  pmc:PMC7596387  insdc:MN908947\n",
       "2  pmc:PMC7754929  insdc:MN908947\n",
       "3  pmc:PMC8016547  insdc:MN908947\n",
       "4  pmc:PMC7290700  insdc:MN908947"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_cncb1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb2 = pd.merge(gisaid, ref2, left_on='accession', right_on='secondaryAccession')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "      <th>primaryAccession</th>\n",
       "      <th>secondaryAccession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR167663</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC7166309</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC7720937</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR266787</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC7786107</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "      <td>https://www.gisaid.org/EPI_ISL_402131</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            id  \\\n",
       "0  https://europepmc.org/article/PPR/PPR167663   \n",
       "1                               pmc:PMC7166309   \n",
       "2                               pmc:PMC7720937   \n",
       "3  https://europepmc.org/article/PPR/PPR266787   \n",
       "4                               pmc:PMC7786107   \n",
       "\n",
       "                               accession primaryAccession  \\\n",
       "0  https://www.gisaid.org/EPI_ISL_402131  GWHABKP00000001   \n",
       "1  https://www.gisaid.org/EPI_ISL_402131  GWHABKP00000001   \n",
       "2  https://www.gisaid.org/EPI_ISL_402131  GWHABKP00000001   \n",
       "3  https://www.gisaid.org/EPI_ISL_402131  GWHABKP00000001   \n",
       "4  https://www.gisaid.org/EPI_ISL_402131  GWHABKP00000001   \n",
       "\n",
       "                      secondaryAccession  \n",
       "0  https://www.gisaid.org/EPI_ISL_402131  \n",
       "1  https://www.gisaid.org/EPI_ISL_402131  \n",
       "2  https://www.gisaid.org/EPI_ISL_402131  \n",
       "3  https://www.gisaid.org/EPI_ISL_402131  \n",
       "4  https://www.gisaid.org/EPI_ISL_402131  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_cncb2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb2['accession'] = pmc_cncb2['primaryAccession']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb2 = pmc_cncb2[['id', 'accession']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GISAID mentions: 5177\n"
     ]
    }
   ],
   "source": [
    "print('GISAID mentions:', pmc_cncb2.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR167663</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC7166309</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC7720937</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR266787</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC7786107</td>\n",
       "      <td>GWHABKP00000001</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            id        accession\n",
       "0  https://europepmc.org/article/PPR/PPR167663  GWHABKP00000001\n",
       "1                               pmc:PMC7166309  GWHABKP00000001\n",
       "2                               pmc:PMC7720937  GWHABKP00000001\n",
       "3  https://europepmc.org/article/PPR/PPR266787  GWHABKP00000001\n",
       "4                               pmc:PMC7786107  GWHABKP00000001"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_cncb2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb = pd.concat([pmc_cncb1, pmc_cncb2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_cncb.to_csv(NEO4J_IMPORT / \"01h-PMC-CNCBStrain.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Match Genomes by refseq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>refseq:NC_045512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>refseq:NC_038294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>refseq:NC_004718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>refseq:NC_006577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>refseq:NC_006213</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          accession\n",
       "0  refseq:NC_045512\n",
       "1  refseq:NC_038294\n",
       "2  refseq:NC_004718\n",
       "3  refseq:NC_006577\n",
       "4  refseq:NC_006213"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ref3 = pd.read_csv(NEO4J_IMPORT / \"Genome.csv\")\n",
    "ref3 = ref3[['refSeq']]\n",
    "ref3 = ref3.drop_duplicates()\n",
    "ref3.rename(columns={'refSeq': 'accession'}, inplace=True)\n",
    "ref3.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_genome = pd.merge(refseq, ref3, on='accession')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "regseq mentions: 4889\n"
     ]
    }
   ],
   "source": [
    "print('regseq mentions:', pmc_genome.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC2684143</td>\n",
       "      <td>refseq:NC_002645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC6390631</td>\n",
       "      <td>refseq:NC_002645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC3812135</td>\n",
       "      <td>refseq:NC_002645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC7121196</td>\n",
       "      <td>refseq:NC_002645</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC3911626</td>\n",
       "      <td>refseq:NC_002645</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               id         accession\n",
       "0  pmc:PMC2684143  refseq:NC_002645\n",
       "1  pmc:PMC6390631  refseq:NC_002645\n",
       "2  pmc:PMC3812135  refseq:NC_002645\n",
       "3  pmc:PMC7121196  refseq:NC_002645\n",
       "4  pmc:PMC3911626  refseq:NC_002645"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_genome.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_genome.to_csv(NEO4J_IMPORT / \"01h-PMC-Genome.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Match Protein Structures by PDB ID"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pdb:3HSD</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pdb:3HSK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pdb:1NRN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pdb:3HT1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pdb:3HSQ</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  accession\n",
       "0  pdb:3HSD\n",
       "1  pdb:3HSK\n",
       "2  pdb:1NRN\n",
       "3  pdb:3HT1\n",
       "4  pdb:3HSQ"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ref4 = pd.read_csv(NEO4J_IMPORT / \"01f-PDBStructure.csv\")\n",
    "ref4 = ref4[['pdbId']]\n",
    "ref4.rename(columns={'pdbId': 'accession'}, inplace=True)\n",
    "ref4 = ref4.drop_duplicates()\n",
    "ref4.head()   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_pdb = pd.merge(pdb, ref4, on='accession')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PDB mentions: 439811\n"
     ]
    }
   ],
   "source": [
    "print('PDB mentions:', pmc_pdb.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>accession</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC4309170</td>\n",
       "      <td>pdb:1A0J</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC4190110</td>\n",
       "      <td>pdb:1A0J</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC3458898</td>\n",
       "      <td>pdb:1A0J</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC3057020</td>\n",
       "      <td>pdb:1A0J</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC2974730</td>\n",
       "      <td>pdb:1A0J</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               id accession\n",
       "0  pmc:PMC4309170  pdb:1A0J\n",
       "1  pmc:PMC4190110  pdb:1A0J\n",
       "2  pmc:PMC3458898  pdb:1A0J\n",
       "3  pmc:PMC3057020  pdb:1A0J\n",
       "4  pmc:PMC2974730  pdb:1A0J"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc_pdb.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_pdb.to_csv(NEO4J_IMPORT / \"01h-PMC-PDBStructure.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Concatenate all ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_ids = pd.concat([pmc_uniprot, pmc_cncb, pmc_genome, pmc_pdb])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a list of preprints (anything that is not a pmcId)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "preprints = pmc_ids[~(pmc_ids['id'].str.startswith('pmc'))][['id']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR173016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR176767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR252942</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>270</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR190474</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>328</th>\n",
       "      <td>https://europepmc.org/article/PPR/PPR260154</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              id\n",
       "109  https://europepmc.org/article/PPR/PPR173016\n",
       "163  https://europepmc.org/article/PPR/PPR176767\n",
       "172  https://europepmc.org/article/PPR/PPR252942\n",
       "270  https://europepmc.org/article/PPR/PPR190474\n",
       "328  https://europepmc.org/article/PPR/PPR260154"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprints.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "preprints.to_csv(NEO4J_IMPORT / \"01h-PPR-Ids.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_ids.rename(columns={'id': 'pmcId'}, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_ids = pmc_ids[['pmcId']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of matched ids 676894\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pmcId</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC5761900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC3275796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC4823807</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC5474285</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            pmcId\n",
       "0  pmc:PMC6713643\n",
       "1  pmc:PMC5761900\n",
       "2  pmc:PMC3275796\n",
       "3  pmc:PMC4823807\n",
       "4  pmc:PMC5474285"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('Number of matched ids', pmc_ids.shape[0])\n",
    "pmc_ids.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download PMID to PMCID mappings and metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc = pd.read_csv(\"https://ftp.ncbi.nlm.nih.gov/pub/pmc/PMC-ids.csv.gz\", \n",
    "                 usecols=['PMID', 'PMCID', 'DOI', 'Journal Title', 'Year', 'Volume', 'Issue', 'Page'], dtype='str')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc.rename(columns={'PMID': 'pmId'}, inplace=True)\n",
    "pmc.rename(columns={'PMCID': 'pmcId'}, inplace=True)\n",
    "pmc.rename(columns={'DOI': 'doi'}, inplace=True)\n",
    "pmc.rename(columns={'Journal Title': 'journal'}, inplace=True)\n",
    "pmc.rename(columns={'Year': 'year'}, inplace=True)\n",
    "pmc.rename(columns={'Volume': 'volume'}, inplace=True)\n",
    "pmc.rename(columns={'Issue': 'issue'}, inplace=True)\n",
    "pmc.rename(columns={'Page': 'page'}, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc.fillna('', inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assign unique identifier (CURIEs) resolvable by [Identifiers.org](https://identifiers.org)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pmc['id'] = 'pubmed:' + pmc['pmId']\n",
    "pmc['pubmedId'] = 'pubmed:' + pmc['pmId']\n",
    "pmc['pmcId'] = 'pmc:' + pmc['pmcId']\n",
    "pmc['doi'] = 'doi:' + pmc['doi']\n",
    "pmc['id'] = pmc['doi']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>journal</th>\n",
       "      <th>year</th>\n",
       "      <th>volume</th>\n",
       "      <th>issue</th>\n",
       "      <th>page</th>\n",
       "      <th>doi</th>\n",
       "      <th>pmcId</th>\n",
       "      <th>pmId</th>\n",
       "      <th>pubmedId</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Breast Cancer Res</td>\n",
       "      <td>2000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>55</td>\n",
       "      <td>doi:10.1186/bcr271</td>\n",
       "      <td>pmc:PMC13900</td>\n",
       "      <td>11250746</td>\n",
       "      <td>pubmed:11250746</td>\n",
       "      <td>doi:10.1186/bcr271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Breast Cancer Res</td>\n",
       "      <td>2000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>61</td>\n",
       "      <td>doi:10.1186/bcr272</td>\n",
       "      <td>pmc:PMC13901</td>\n",
       "      <td>11250747</td>\n",
       "      <td>pubmed:11250747</td>\n",
       "      <td>doi:10.1186/bcr272</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Breast Cancer Res</td>\n",
       "      <td>2000</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>66</td>\n",
       "      <td>doi:10.1186/bcr273</td>\n",
       "      <td>pmc:PMC13902</td>\n",
       "      <td>11250748</td>\n",
       "      <td>pubmed:11250748</td>\n",
       "      <td>doi:10.1186/bcr273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Breast Cancer Res</td>\n",
       "      <td>1999</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>59</td>\n",
       "      <td>doi:10.1186/bcr29</td>\n",
       "      <td>pmc:PMC13911</td>\n",
       "      <td>11056684</td>\n",
       "      <td>pubmed:11056684</td>\n",
       "      <td>doi:10.1186/bcr29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Breast Cancer Res</td>\n",
       "      <td>1999</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>64</td>\n",
       "      <td>doi:10.1186/bcr30</td>\n",
       "      <td>pmc:PMC13912</td>\n",
       "      <td>11400682</td>\n",
       "      <td>pubmed:11400682</td>\n",
       "      <td>doi:10.1186/bcr30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             journal  year volume issue page                 doi  \\\n",
       "0  Breast Cancer Res  2000      3     1   55  doi:10.1186/bcr271   \n",
       "1  Breast Cancer Res  2000      3     1   61  doi:10.1186/bcr272   \n",
       "2  Breast Cancer Res  2000      3     1   66  doi:10.1186/bcr273   \n",
       "3  Breast Cancer Res  1999      2     1   59   doi:10.1186/bcr29   \n",
       "4  Breast Cancer Res  1999      2     1   64   doi:10.1186/bcr30   \n",
       "\n",
       "          pmcId      pmId         pubmedId                  id  \n",
       "0  pmc:PMC13900  11250746  pubmed:11250746  doi:10.1186/bcr271  \n",
       "1  pmc:PMC13901  11250747  pubmed:11250747  doi:10.1186/bcr272  \n",
       "2  pmc:PMC13902  11250748  pubmed:11250748  doi:10.1186/bcr273  \n",
       "3  pmc:PMC13911  11056684  pubmed:11056684   doi:10.1186/bcr29  \n",
       "4  pmc:PMC13912  11400682  pubmed:11400682   doi:10.1186/bcr30  "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pmc.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_ids = pmc_ids.merge(pmc, on='pmcId')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of PMC matches: 663383\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pmcId</th>\n",
       "      <th>journal</th>\n",
       "      <th>year</th>\n",
       "      <th>volume</th>\n",
       "      <th>issue</th>\n",
       "      <th>page</th>\n",
       "      <th>doi</th>\n",
       "      <th>pmId</th>\n",
       "      <th>pubmedId</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "      <td>Cancer Biol Med</td>\n",
       "      <td>2019</td>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>377</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "      <td>31516757</td>\n",
       "      <td>pubmed:31516757</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "      <td>Cancer Biol Med</td>\n",
       "      <td>2019</td>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>377</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "      <td>31516757</td>\n",
       "      <td>pubmed:31516757</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "      <td>Cancer Biol Med</td>\n",
       "      <td>2019</td>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>377</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "      <td>31516757</td>\n",
       "      <td>pubmed:31516757</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "      <td>Cancer Biol Med</td>\n",
       "      <td>2019</td>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>377</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "      <td>31516757</td>\n",
       "      <td>pubmed:31516757</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pmc:PMC6713643</td>\n",
       "      <td>Cancer Biol Med</td>\n",
       "      <td>2019</td>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>377</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "      <td>31516757</td>\n",
       "      <td>pubmed:31516757</td>\n",
       "      <td>doi:10.20892/j.issn.2095-3941.2018.0386</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            pmcId          journal  year volume issue page  \\\n",
       "0  pmc:PMC6713643  Cancer Biol Med  2019     16     2  377   \n",
       "1  pmc:PMC6713643  Cancer Biol Med  2019     16     2  377   \n",
       "2  pmc:PMC6713643  Cancer Biol Med  2019     16     2  377   \n",
       "3  pmc:PMC6713643  Cancer Biol Med  2019     16     2  377   \n",
       "4  pmc:PMC6713643  Cancer Biol Med  2019     16     2  377   \n",
       "\n",
       "                                       doi      pmId         pubmedId  \\\n",
       "0  doi:10.20892/j.issn.2095-3941.2018.0386  31516757  pubmed:31516757   \n",
       "1  doi:10.20892/j.issn.2095-3941.2018.0386  31516757  pubmed:31516757   \n",
       "2  doi:10.20892/j.issn.2095-3941.2018.0386  31516757  pubmed:31516757   \n",
       "3  doi:10.20892/j.issn.2095-3941.2018.0386  31516757  pubmed:31516757   \n",
       "4  doi:10.20892/j.issn.2095-3941.2018.0386  31516757  pubmed:31516757   \n",
       "\n",
       "                                        id  \n",
       "0  doi:10.20892/j.issn.2095-3941.2018.0386  \n",
       "1  doi:10.20892/j.issn.2095-3941.2018.0386  \n",
       "2  doi:10.20892/j.issn.2095-3941.2018.0386  \n",
       "3  doi:10.20892/j.issn.2095-3941.2018.0386  \n",
       "4  doi:10.20892/j.issn.2095-3941.2018.0386  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('Number of PMC matches:', pmc_ids.shape[0])\n",
    "pmc_ids.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "pmc_ids.to_csv(NEO4J_IMPORT / \"01h-PMC-Ids.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download PDB - PubMed mappings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "sifts_url = 'http://ftp.ebi.ac.uk/pub/databases/msd/sifts/flatfiles/tsv/pdb_pubmed.tsv.gz'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PDB</th>\n",
       "      <th>PUBMED_ID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100d</td>\n",
       "      <td>7816639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>101d</td>\n",
       "      <td>7711020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>102d</td>\n",
       "      <td>7608897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>102l</td>\n",
       "      <td>8429913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>103d</td>\n",
       "      <td>7966337</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    PDB PUBMED_ID\n",
       "0  100d   7816639\n",
       "1  101d   7711020\n",
       "2  102d   7608897\n",
       "3  102l   8429913\n",
       "4  103d   7966337"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pm_pdb = pd.read_csv(sifts_url, usecols=['PDB', 'PUBMED_ID'], sep='\\t', skiprows=1, dtype=str)\n",
    "pm_pdb.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm_pdb.rename(columns={'PDB': 'accession'}, inplace=True)\n",
    "pm_pdb.rename(columns={'PUBMED_ID': 'id'}, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm_pdb['accession'] = 'pdb:' + pm_pdb['accession'].str.upper()\n",
    "pm_pdb['id'] = 'pubmed:' + pm_pdb['id']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of matches: 172335\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>accession</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pdb:100D</td>\n",
       "      <td>pubmed:7816639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>pdb:101D</td>\n",
       "      <td>pubmed:7711020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>pdb:102D</td>\n",
       "      <td>pubmed:7608897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>pdb:102L</td>\n",
       "      <td>pubmed:8429913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>pdb:103D</td>\n",
       "      <td>pubmed:7966337</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  accession              id\n",
       "0  pdb:100D  pubmed:7816639\n",
       "1  pdb:101D  pubmed:7711020\n",
       "2  pdb:102D  pubmed:7608897\n",
       "3  pdb:102L  pubmed:8429913\n",
       "4  pdb:103D  pubmed:7966337"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('Number of matches:', pm_pdb.shape[0])\n",
    "pm_pdb.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "pm_pdb.to_csv(NEO4J_IMPORT / \"01h-PM-PDBStructure.csv\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}