scrape_bi_jobs_2019.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BI Career Skills in Belgium, according to Indeed\n",
    "\n",
    "### Ettore Rizza\n",
    "### 2019-12-09"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quelles technologies apparaissent le plus souvent dans les offres d'emploi BI en Belgique ? \n",
    "\n",
    "Largement inspiré de https://github.com/discdiver/2019-in-demand-ds-tech-skills"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import des modules\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from bs4 import BeautifulSoup as bs\n",
    "import requests\n",
    "import time\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scraping des données\n",
    "\n",
    "L'idée est de récupérer le nombre de résultats renvoyés par une série de recherches sur des sites d'offres d'emploi. Pour l'heure, je me limiterai à Indeed.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "header = {\n",
    "  \"User-Agent\": \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36\",\n",
    "  \"X-Requested-With\": \"XMLHttpRequest\"\n",
    "}\n",
    "\n",
    "# Liste des technologies dont nous voulons connaître la popularité dans les offres d'emploi BI \n",
    "search_terms = [\n",
    "\"\",\n",
    "\"Access\",\n",
    "\"Airflow\",\n",
    "\"Alteryx\",\n",
    "\"AWS\",\n",
    "\"Azure\",\n",
    "\"Bash\",\n",
    "\"BigQuery\",\n",
    "\"Cassandra\",\n",
    "\"Cognos\",\n",
    "\"Crystal Reports\",\n",
    "\"DB2\",\n",
    "\"D3\",\n",
    "\"D3.js\",\n",
    "\"Databricks\",\n",
    "\"Dataiku\",\n",
    "\"Dax\",\n",
    "\"Docker\",\n",
    "\"ElasticSearch\",\n",
    "\"Excel\",\n",
    "\"Git\",\n",
    "\"Google Cloud\",\n",
    "\"Google Data Studio\",\n",
    "\"Hadoop\",\n",
    "\"Hbase\",\n",
    "\"Hive\",\n",
    "\"Java\",\n",
    "\"Javascript\",\n",
    "\"Jupyter\",\n",
    "\"Kafka\",\n",
    "\"Keras\",\n",
    "\"Knime\",\n",
    "\"Kubernetes\",\n",
    "\"Linux\",\n",
    "\"Matlab\",\n",
    "\"MDX\",\n",
    "\"MicroStrategy\",\n",
    "\"MongoDB\",\n",
    "\"MySQL\",\n",
    "\"Neo4J\",\n",
    "\"NoSQL\",\n",
    "\"NumPy\",\n",
    "\"Pandas\",\n",
    "\"Pentaho\",\n",
    "\"Perl\",\n",
    "\"Pig\",\n",
    "\"PL/SQL\",\n",
    "\"postgresql\",\n",
    "\"Power BI\",\n",
    "\"Power Pivot\",\n",
    "\"Power Query\",\n",
    "\"PowerBI\",\n",
    "\"Powershell scripting\",\n",
    "\"Python\",\n",
    "\"PyTorch\",\n",
    "\"QGIS\",\n",
    "\"Qlik\",\n",
    "\"QlikView\",\n",
    "\"QlikSense\",\n",
    "\"R\",\n",
    "\"RapidMiner\",\n",
    "\"SAP\",\n",
    "\"SAS\",\n",
    "\"Scala\",\n",
    "\"Scikit-learn\",\n",
    "\"scrum\",\n",
    "\"Shell scripting\",\n",
    "\"Shiny\",\n",
    "\"Spark\",\n",
    "\"SPARQL\",\n",
    "\"SPSS\",\n",
    "\"SQL\",\n",
    "\"SSAS\",\n",
    "\"SSIS\",\n",
    "\"SSRS\",\n",
    "\"Stata\",\n",
    "\"T-SQL\",\n",
    "\"Tableau\",\n",
    "\"Talend\",\n",
    "\"TensorFlow\",\n",
    "\"Teradata\",\n",
    "\"Tidyverse\",\n",
    "\"VBA\",\n",
    "\"Weka\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Indeed\n",
    "\n",
    "Scraping du nombre de résultats renvoyés par la recherche data + \"business intelligence\" ou \"BI\", dans un rayon de 100 km de Bruxelles, accompagnée d'un nom de technologie."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://be.indeed.com/jobs?q=data+%22%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Access%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Airflow%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Alteryx%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22AWS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Azure%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Bash%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22BigQuery%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Cassandra%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Cognos%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Crystal+Reports%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22DB2%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22D3%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22D3.js%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Databricks%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Dataiku%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Dax%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Docker%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22ElasticSearch%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Excel%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Git%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Google+Cloud%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Google+Data+Studio%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Hadoop%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Hbase%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Hive%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Java%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Javascript%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Jupyter%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Kafka%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Keras%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Knime%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Kubernetes%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Linux%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Matlab%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22MDX%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22MicroStrategy%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22MongoDB%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22MySQL%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Neo4J%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22NoSQL%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22NumPy%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Pandas%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Pentaho%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Perl%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Pig%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22PL/SQL%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22postgresql%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Power+BI%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Power+Pivot%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Power+Query%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22PowerBI%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Powershell+scripting%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Python%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22PyTorch%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22QGIS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Qlik%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22QlikView%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22QlikSense%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22R%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22RapidMiner%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SAP%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SAS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Scala%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Scikit-learn%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22scrum%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Shell+scripting%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Shiny%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Spark%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SPARQL%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SPSS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SQL%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SSAS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SSIS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22SSRS%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://be.indeed.com/jobs?q=data+%22Stata%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22T-SQL%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Tableau%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Talend%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22TensorFlow%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Teradata%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Tidyverse%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22VBA%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n",
      "https://be.indeed.com/jobs?q=data+%22Weka%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100\n"
     ]
    }
   ],
   "source": [
    "indeed_list = []\n",
    "\n",
    "for term in search_terms:\n",
    "    term = term.replace(\" \", \"+\")\n",
    "    url = f'https://be.indeed.com/jobs?q=data+%22{term}%22+%28%22business+intelligence%22+or+BI%29&l=Brussels&radius=100'\n",
    "    print(url)\n",
    "    \n",
    "    try:\n",
    "        r = requests.get(url, headers=header, timeout=10)\n",
    "        soup = bs(r.text, 'html.parser')\n",
    "        count_str = soup.find('div', id=\"searchCountPages\").get_text()\n",
    "        numb = count_str.split()\n",
    "        indeed_list.append(numb[-2])\n",
    "    except Exception as e:\n",
    "        indeed_list.append('NA')\n",
    "    finally:\n",
    "        time.sleep(0.5)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(indeed_list, index=search_terms, columns=['Indeed'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Indeed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>BI</td>\n",
       "      <td>699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Access</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Airflow</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Alteryx</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>AWS</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>TensorFlow</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Teradata</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Tidyverse</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>VBA</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Weka</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>84 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           Indeed\n",
       "BI            699\n",
       "Access         57\n",
       "Airflow         2\n",
       "Alteryx         9\n",
       "AWS            30\n",
       "...           ...\n",
       "TensorFlow      7\n",
       "Teradata       12\n",
       "Tidyverse      NA\n",
       "VBA            23\n",
       "Weka           NA\n",
       "\n",
       "[84 rows x 1 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.rename(index={'':'BI'},inplace=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Graphique"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "        <script type=\"text/javascript\">\n",
       "        window.PlotlyConfig = {MathJaxConfig: 'local'};\n",
       "        if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}\n",
       "        if (typeof require !== 'undefined') {\n",
       "        require.undef(\"plotly\");\n",
       "        requirejs.config({\n",
       "            paths: {\n",
       "                'plotly': ['https://cdn.plot.ly/plotly-latest.min']\n",
       "            }\n",
       "        });\n",
       "        require(['plotly'], function(Plotly) {\n",
       "            window._Plotly = Plotly;\n",
       "        });\n",
       "        }\n",
       "        </script>\n",
       "        "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import collections\n",
    "\n",
    "from IPython.core.display import display, HTML\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "    \n",
    "# import plotly \n",
    "import plotly\n",
    "import plotly.figure_factory as ff\n",
    "import plotly.graph_objs as go\n",
    "import plotly.offline as py\n",
    "import plotly.tools as tls\n",
    "\n",
    "# for color scales in plotly\n",
    "import colorlover as cl \n",
    "\n",
    "\n",
    "pd.options.display.float_format = '{:,.2f}'.format  \n",
    "pd.options.display.max_columns = 999\n",
    "\n",
    "py.init_notebook_mode(connected=True)\n",
    "\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Tout ce qui ne peut pas être transformé en nombre deviendra NA et sera effacé\n",
    "df['Indeed'] = pd.to_numeric(df['Indeed'], errors='coerce')\n",
    "df.dropna(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[699.0]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Servira a calculer les pourcentages\n",
    "total_bi_jobs = list(df.iloc[0])\n",
    "total_bi_jobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "\"['BI'] not found in axis\"",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-17-8da53e5db499>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'BI'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/pandas/core/frame.py\u001b[0m in \u001b[0;36mdrop\u001b[0;34m(self, labels, axis, index, columns, level, inplace, errors)\u001b[0m\n\u001b[1;32m   4100\u001b[0m             \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   4101\u001b[0m             \u001b[0minplace\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minplace\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 4102\u001b[0;31m             \u001b[0merrors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   4103\u001b[0m         )\n\u001b[1;32m   4104\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/pandas/core/generic.py\u001b[0m in \u001b[0;36mdrop\u001b[0;34m(self, labels, axis, index, columns, level, inplace, errors)\u001b[0m\n\u001b[1;32m   3912\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabels\u001b[0m \u001b[0;32min\u001b[0m \u001b[0maxes\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3913\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mlabels\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3914\u001b[0;31m                 \u001b[0mobj\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mobj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_drop_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   3915\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3916\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0minplace\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/pandas/core/generic.py\u001b[0m in \u001b[0;36m_drop_axis\u001b[0;34m(self, labels, axis, level, errors)\u001b[0m\n\u001b[1;32m   3944\u001b[0m                 \u001b[0mnew_axis\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3945\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3946\u001b[0;31m                 \u001b[0mnew_axis\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   3947\u001b[0m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreindex\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0maxis_name\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mnew_axis\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3948\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mdrop\u001b[0;34m(self, labels, errors)\u001b[0m\n\u001b[1;32m   5338\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mmask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0many\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5339\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0merrors\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0;34m\"ignore\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5340\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"{} not found in axis\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mmask\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   5341\u001b[0m             \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mindexer\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mmask\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5342\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdelete\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindexer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyError\u001b[0m: \"['BI'] not found in axis\""
     ]
    }
   ],
   "source": [
    "#Plus besoin de cette ligne\n",
    "df = df.drop('BI')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Percentage'] = (df['Indeed'] / total_bi_jobs[0]) * 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sauvegarde\n",
    "\n",
    "On ajoute la date du jour au nom de fichier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "today = datetime.date.today()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(f'bi_tech_skills_{today}.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Top 20 Barchart in Plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "top_20 = df.nlargest(20, 'Percentage')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "linkText": "Export to plot.ly",
        "plotlyServerURL": "https://plot.ly",
        "showLink": false
       },
       "data": [
        {
         "marker": {
          "color": [
           50,
           60.714285714285715,
           71.42857142857143,
           82.14285714285714,
           92.85714285714286,
           103.57142857142857,
           114.28571428571428,
           125,
           135.71428571428572,
           146.42857142857142,
           157.14285714285714,
           167.85714285714283,
           178.57142857142856,
           189.28571428571428,
           200
          ],
          "colorscale": [
           [
            0,
            "rgb(0,0,131)"
           ],
           [
            0.2,
            "rgb(0,60,170)"
           ],
           [
            0.4,
            "rgb(5,255,255)"
           ],
           [
            0.6,
            "rgb(255,255,0)"
           ],
           [
            0.8,
            "rgb(250,0,0)"
           ],
           [
            1,
            "rgb(128,0,0)"
           ]
          ]
         },
         "type": "bar",
         "x": [
          "SQL",
          "Power BI",
          "SAP",
          "Python",
          "Excel",
          "Tableau",
          "R",
          "SSIS",
          "Azure",
          "Access",
          "PowerBI",
          "SAS",
          "scrum",
          "SSRS",
          "Hadoop",
          "Qlik",
          "Java",
          "SSAS",
          "Spark",
          "Cognos"
         ],
         "y": [
          40.2002861230329,
          19.170243204577968,
          17.8826895565093,
          16.30901287553648,
          15.164520743919885,
          14.735336194563661,
          11.158798283261802,
          9.15593705293276,
          8.726752503576538,
          8.15450643776824,
          8.15450643776824,
          8.011444921316166,
          7.296137339055794,
          7.296137339055794,
          6.723891273247497,
          6.723891273247497,
          6.580829756795422,
          6.151645207439199,
          5.436337625178827,
          4.86409155937053
         ]
        }
       ],
       "layout": {
        "height": 500,
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Skills in Data analyst/BI Job in 2019 (Indeed)",
         "x": 0.5
        },
        "xaxis": {
         "tickmode": "linear",
         "title": {
          "text": "Skills"
         }
        },
        "yaxis": {
         "title": {
          "text": "% of results"
         }
        }
       }
      },
      "text/html": [
       "<div>\n",
       "        \n",
       "        \n",
       "            <div id=\"b76d76bb-d904-4868-9204-67e02dae2575\" class=\"plotly-graph-div\" style=\"height:500px; width:100%;\"></div>\n",
       "            <script type=\"text/javascript\">\n",
       "                require([\"plotly\"], function(Plotly) {\n",
       "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
       "                    \n",
       "                if (document.getElementById(\"b76d76bb-d904-4868-9204-67e02dae2575\")) {\n",
       "                    Plotly.newPlot(\n",
       "                        'b76d76bb-d904-4868-9204-67e02dae2575',\n",
       "                        [{\"marker\": {\"color\": [50.0, 60.714285714285715, 71.42857142857143, 82.14285714285714, 92.85714285714286, 103.57142857142857, 114.28571428571428, 125.0, 135.71428571428572, 146.42857142857142, 157.14285714285714, 167.85714285714283, 178.57142857142856, 189.28571428571428, 200.0], \"colorscale\": [[0.0, \"rgb(0,0,131)\"], [0.2, \"rgb(0,60,170)\"], [0.4, \"rgb(5,255,255)\"], [0.6, \"rgb(255,255,0)\"], [0.8, \"rgb(250,0,0)\"], [1.0, \"rgb(128,0,0)\"]]}, \"type\": \"bar\", \"x\": [\"SQL\", \"Power BI\", \"SAP\", \"Python\", \"Excel\", \"Tableau\", \"R\", \"SSIS\", \"Azure\", \"Access\", \"PowerBI\", \"SAS\", \"scrum\", \"SSRS\", \"Hadoop\", \"Qlik\", \"Java\", \"SSAS\", \"Spark\", \"Cognos\"], \"y\": [40.2002861230329, 19.170243204577968, 17.8826895565093, 16.30901287553648, 15.164520743919885, 14.735336194563661, 11.158798283261802, 9.15593705293276, 8.726752503576538, 8.15450643776824, 8.15450643776824, 8.011444921316166, 7.296137339055794, 7.296137339055794, 6.723891273247497, 6.723891273247497, 6.580829756795422, 6.151645207439199, 5.436337625178827, 4.86409155937053]}],\n",
       "                        {\"height\": 500, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Skills in Data analyst/BI Job in 2019 (Indeed)\", \"x\": 0.5}, \"xaxis\": {\"tickmode\": \"linear\", \"title\": {\"text\": \"Skills\"}}, \"yaxis\": {\"title\": {\"text\": \"% of results\"}}},\n",
       "                        {\"responsive\": true}\n",
       "                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('b76d76bb-d904-4868-9204-67e02dae2575');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })\n",
       "                };\n",
       "                });\n",
       "            </script>\n",
       "        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cmax=200\n",
    "cmin=50\n",
    "color_s = np.linspace(cmin, cmax, 15)\n",
    "\n",
    "data = [\n",
    "    go.Bar(\n",
    "        x=top_20.index,          \n",
    "        y=top_20['Percentage'],\n",
    "        marker=dict(\n",
    "            colorscale='Jet',\n",
    "            color=color_s\n",
    "        ),\n",
    "    )\n",
    "]\n",
    "\n",
    "layout = {\n",
    "     'title': 'Skills in Data analyst/BI Job in 2019 (Indeed)',\n",
    "    'yaxis': {'title': '% of results', },\n",
    "    'xaxis': {'title': \"Skills\", 'tickmode': 'linear'},\n",
    "    'title_x':0.5,\n",
    "    'height': 500\n",
    "    \n",
    "}\n",
    "\n",
    "fig = go.Figure(data=data, layout=layout)\n",
    "py.iplot(fig)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}