Text_summarization_demo_using_BERT.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Text_summarization_using_BERT.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/praveenjune17/BERT_text_summarisation/blob/master/Text_summarization_demo_using_BERT.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iw3nLLDAvjdg",
        "colab_type": "text"
      },
      "source": [
        "## **Text summarization using BERT.**\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7RlXUGWVMXFP",
        "colab_type": "text"
      },
      "source": [
        "########################################################################################################################\n",
        "## Instructions for running the script\n",
        "  a) Select  Open in Google colab \n",
        "  b) Select  Runtime. 'change runtime type' to GPU\n",
        "      c) Select --> Run all (top left corner)\n",
        "  \n",
        "  d) Takes few mins to load the dependencies in the end you will be prompted to enter a Document. Summary will be created for this\n",
        "########################################################################################################################"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3ImPYXO4hu5g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!git clone https://github.com/praveenjune17/BERT_text_summarisation\n",
        "!pip install bunch\n",
        "!pip install rouge==0.3.2\n",
        "!pip install bert_score \n",
        "!pip install --upgrade grpcio\n",
        "!pip install tensorflow==2.1\n",
        "!pip install tensorflow_addons"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Nkjum2hyB5D",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "outputId": "920fecd7-e880-480b-8186-bf4dc8911b40"
      },
      "source": [
        "import tensorflow as tf\n",
        "import time\n",
        "import shutil\n",
        "import os\n",
        "from google_drive_downloader import GoogleDriveDownloader as gdd\n",
        "import sys\n",
        "sys.path.insert(0, '/content/BERT_text_summarisation/scripts')\n",
        "from metrics import convert_wordpiece_to_words\n",
        "from decode_text import *"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "setting default value to last_recorded_value\n",
            "INFO:tensorflow:Extracting pretrained word embeddings weights from BERT\n",
            "INFO:tensorflow:Embedding matrix shape '(30522, 768)'\n",
            "INFO:tensorflow:Loading Pre-trained BERT model for BERT SCORE calculation\n",
            "INFO:tensorflow:Extracting pretrained word embeddings weights from BERT\n",
            "INFO:tensorflow:Embedding matrix shape '(30522, 768)'\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O8hO24pT897S",
        "colab_type": "text"
      },
      "source": [
        "Download checkpoints from Gdrive"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pJ398mdT88sF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gdd.download_file_from_google_drive(file_id='1-_eJj-7FLYc-xEVBu9IMyf38eZO8E43C',\n",
        "                                      dest_path='/root/ckpt_dir/ckpt.zip',\n",
        "                                      unzip=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v1WlO6iFyQKa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        },
        "outputId": "472650d5-d052-4cd3-efb7-553391e00e76"
      },
      "source": [
        "def summarize(infer_ckpt):\n",
        "  ckpt = tf.train.Checkpoint(\n",
        "                             model=model\n",
        "                            )\n",
        "  ckpt.restore('/root/ckpt_dir/content/drive/My Drive/Text_summarization/BERT_text_summarisation/Summarization_inference_ckps/ckpt-'+infer_ckpt).expect_partial()\n",
        "  start = time.time()\n",
        "  ip_ids = tokenizer.encode(input('Enter a news article to be summarized --> '))\n",
        "  preds_draft_summary, preds_refined_summary, refine_attention_dist = predict_using_beam_search(tf.convert_to_tensor([ip_ids]),\n",
        "                                                                                               refine_decoder_sampling_type='topktopp',\n",
        "                                                                                               k=7,\n",
        "                                                                                               p=0.8)\n",
        "  sum_hyp = tokenizer.convert_ids_to_tokens([i for i in tf.squeeze(preds_refined_summary) if i not in [CLS_ID, SEP_ID, 0]])\n",
        "  sum_hyp = convert_wordpiece_to_words(sum_hyp)\n",
        "  print(f'the summarized output is --> {sum_hyp if sum_hyp else \"EMPTY\"}')\n",
        "  print(f'Time to process {round(time.time() - start)} seconds')\n",
        "\n",
        "if __name__ == '__main__':\n",
        "  infer_ckpt = '75'\n",
        "  summarize(infer_ckpt)  "
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Enter a news article to be summarized --> A wiki is run using wiki software, otherwise known as a wiki engine. A wiki engine is a type of content management system, but it differs from most other such systems, including blog software, in that the content is created without any defined owner or leader, and wikis have little inherent structure, allowing structure to emerge according to the needs of the users.[2] There are dozens of different wiki engines in use, both standalone and part of other software, such as bug tracking systems. Some wiki engines are open source, whereas others are proprietary. Some permit control over different functions (levels of access); for example, editing rights may permit changing, adding, or removing material. Others may permit access without enforcing access control. Other rules may be imposed to organize content.\n",
            "INFO:tensorflow:Building: 'Draft beam search decoder'\n",
            "INFO:tensorflow:Building: 'Refined topktopp decoder'\n",
            "the summarized output is --> wiki software allows state of use level control rules . some rules include allowing access to content . others have restrictions . comes from software computers , however . some may be anonymous . software computers , some have power . some have . click graphic systems , others allow other software . software files like specific software or others have limited leadership . some\n",
            "Time to process 30 seconds\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}