ocr.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OCR Experiments\n",
    "\n",
    "## Problem\n",
    "\n",
    "Given images containing text, we need to write code that can extract data from them.\n",
    "\n",
    "### Images\n",
    "\n",
    "All the images are put in the folder `img`. Each of them contains tabular or plain text data. Some images are good quality (i.e. higher resolution) while some are not. We need to come up with a strategy to identify what pre-processing we need to do to be able to parse the images and convert them into text.\n",
    "\n",
    "### Libraries Used\n",
    "We can use OpenCV to load the image and preprocess it. And we are using Tesseract library to parse text from images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing\n",
    "\n",
    "Almost all the images we have need to be preprocessed in one or more ways because they have several orthogonal abnormalities than what a perfect image for processing would be. To name a few:\n",
    "\n",
    "- Resolution is not very high\n",
    "- Use of Indian languages\n",
    "- Tables\n",
    "- Highly incoherent fonts, weights, colors and sizes of text\n",
    "\n",
    "Some things to try are:\n",
    "- Strip the vertical and horizontal lines fro images \n",
    "- Sharpen images\n",
    "- Apply thresholding or blurs\n",
    "- Use grayscale (remove the color factor as long as we don't care about colors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Up until now, there hasn't been a use case for learning about color, so lets just get rid of colors. So lets load the image we want to work with and create a gray scale out of it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image = cv2.imread('./img/raj.jpeg')\n",
    "gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
    "    \n",
    "plt.imshow(gray, cmap='Greys_r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]\n",
    "plt.imshow(thresh)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_gray(image):\n",
    "    return cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
    "\n",
    "def make_thresh(image):\n",
    "    gray = make_gray(image)\n",
    "    return cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets use the threshold as the base image and use it to remove the vertical and horizontal lines from it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_kernel(img, kernel, iterations=2):\n",
    "    detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=iterations)\n",
    "    cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n",
    "    cnts = cnts[0] if len(cnts) == 2 else cnts[1]\n",
    "    for c in cnts:\n",
    "        cv2.drawContours(img, [c], -1, (255, 255, 255), 2)\n",
    "\n",
    "# Remove horizontal\n",
    "h_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (30, 1))\n",
    "run_kernel(image, h_kernel)\n",
    "\n",
    "# Remove horizontal\n",
    "v_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 30))\n",
    "run_kernel(image, v_kernel)\n",
    "\n",
    "img = make_thresh(image)\n",
    "plt.imshow(image)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pytesseract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "custom_config = r'--oem 3 --psm 6'\n",
    "d = pytesseract.image_to_string(image, config=custom_config)\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}