Introduction to Topic Modeling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Topic Modeling\n",
    "\n",
    "Ilya Akdemir, 12/03/2019\n",
    "\n",
    "* *What is Topic Modeling?*\n",
    "    * A statistical model used to uncover latent/abstract topic structure within a corpus of documents. The output of these models provides a means of interpreting and/or representing each document within a corpus as a collection of *k* topics. \n",
    "    \n",
    "    \n",
    "* *Why Topic Modeling?*   \n",
    "    * Intuitively, topic modeling provides a \"sum of parts (topics)\" representation of documents in a given corpus. These representations of documents can further be used for other tasks of interest, such as search, close reading, labeling, supervised machine learning, etc. In this workshop, we will focus on two topic modeling approaches, namely LDA and NMF.\n",
    "    \n",
    "    \n",
    "* *LDA*:\n",
    "    * **Latent Dirichlet Allocation** [(Blei, Ng and Jordan; 2003)](http://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf) - A generative probabilistic model for implementation of topic modeling that assumes a Dirichlet prior. \n",
    "    \n",
    "    \n",
    "* *NMF*:\n",
    "    * **Non-Negative Matrix Factorization** [(Lee and Seung, 1999)](https://www.nature.com/articles/44565) - Matrix decomposition method with an imposed non-negativity constraint. \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I. The Data\n",
    "The superbly well-organized and high-quality data used for this workshop comes from the wonderful people working on the [case.law](https://case.law/) project at the [Harvard Library Innovation Lab](https://lil.law.harvard.edu/). \n",
    "\n",
    "We will be implementing topic modeling on New Mexico's court decisions. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lzma\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# specify file path\n",
    "file_path = \"/Users/path/to/your.../data/data.jsonl.xz\" \n",
    "\n",
    "# open and read \n",
    "with lzma.open(file_path) as f, open(file_path[:-3], 'wb') as fout:\n",
    "    file_content = f.read() \n",
    "    fout.write(file_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading data: Option 1 - if you don't have enough memory and/or space\n",
    "The uncompressed New Mexico dataset is about ~350 MB and contains 18,338 cases going back as far as 1852. If you don't have a strong computer, you should go for this option.\n",
    "\n",
    "In addition, for the purposes of this workshop, it is advised that you stick with Option 1 as building actual topic models on a larger corpus will take some time.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# specify the maximum amount of records\n",
    "max_records = 1000\n",
    "data = []\n",
    "\n",
    "with open(file_path[:-3]) as f:\n",
    "    for i, line in enumerate(f):\n",
    "        data.append(json.loads(line))\n",
    "        if i >= max_records - 1:\n",
    "            break\n",
    "\n",
    "data = pd.DataFrame(data)     \n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading Data: Option 2 - if you have enough memory/space\n",
    "If you have good computer, you can use pandas' built-in *read_json()* function to load all 18,338 cases from New Mexico. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# since this data is jsonl, make sure lines is set to True\n",
    "#data = pd.read_json(file_path[:-3], lines = True)\n",
    "\n",
    "#view the dataframe\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select the columns we want\n",
    "data  = data[['decision_date','name_abbreviation', 'court', 'casebody']]\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II. Preprocessing \n",
    "Preprocessing methods such as stemming, lemmatization, stopword removal/down-weighting, etc. are commonly used in natural language processing research and applications. However, it is important to point out that fundamentally, using a certain preprocessing method(s) is a *choice*, and whenever we make choices as researchers, we must be able to justify our choices in the context of our domain and/or research objective. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim.parsing.preprocessing import remove_stopwords, strip_non_alphanum, strip_numeric\n",
    "\n",
    "opinion_texts = [] # create empty list to store text of opinions\n",
    "\n",
    "for i in range(len(data)):\n",
    "    if data.iloc[i]['casebody']['data']['opinions']: \n",
    "        text = data.iloc[i]['casebody']['data']['opinions'][0]['text'].lower() # lowercase \n",
    "        text = strip_non_alphanum(text) # remove non-alphanumeric characters like #,@,¶ etc\n",
    "        text = strip_numeric(text)      # remove numbers\n",
    "        text = remove_stopwords(text)   # remove stopwords\n",
    "        opinion_texts.append(' '.join(text.split()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "opinion_texts[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# putting the text back into the dataframe, in case you need to save it\n",
    "data['text'] = opinion_texts\n",
    "#data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## III. Latent Dirichlet Allocation (LDA)\n",
    "### A. Fit a Topic Model using sklearn's LDA\n",
    "\n",
    "[Latent Dirichlet Allocation](https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation) is a generative model and is probably the most popular topic modeling approach in research and other applications. We will be using sklearn's LatentDirichletAllocation function. See [here](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.LatentDirichletAllocation.html) for more information about this function. \n",
    "\n",
    "This part of the workshop borrows heavily from [Laura Nelson's](https://github.com/lknelson) course on computational text analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.decomposition import LatentDirichletAllocation\n",
    "\n",
    "##This is a function to print out the top words for each topic in a pretty way.\n",
    "#Don't worry too much about understanding every line of this code.\n",
    "def print_top_words(model, feature_names, n_top_words):\n",
    "    for topic_idx, topic in enumerate(model.components_):\n",
    "        print(\"\\nTopic #%d:\" % topic_idx)\n",
    "        print(\" \".join([feature_names[i]\n",
    "                        for i in topic.argsort()[:-n_top_words - 1:-1]]))\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Vectorize our text using CountVectorizer\n",
    "print(\"Extracting tf features for LDA...\")\n",
    "tf_vectorizer = CountVectorizer(max_df = 0.60, # ignore terms that appear in more than 80% of documents, ie corpus-specific stopwords\n",
    "                                min_df = 50,   # ignore terms that appear in less than 50 of documents, ie remove very rare terms\n",
    "                                max_features = 10000, # consider only 10k top words by frequency\n",
    "                                stop_words='english' # remove stopwords\n",
    "                                )\n",
    "\n",
    "tf = tf_vectorizer.fit_transform(opinion_texts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# vizualize the document term matrix \n",
    "tf_matrix = tf.todense()\n",
    "tf_matrix = pd.DataFrame(tf_matrix)\n",
    "tf_matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_samples = len(data)\n",
    "n_topics = 10\n",
    "n_top_words = 20\n",
    "\n",
    "print(\"Fitting LDA models with tf features, \"\n",
    "      \"n_samples=%d and n_topics=%d...\"\n",
    "      % (n_samples, n_topics))\n",
    "\n",
    "\n",
    "#define the lda function, with desired options\n",
    "#Check the documentation, linked above, to look through the options\n",
    "lda = LatentDirichletAllocation(n_components = n_topics, # how many topics we want \n",
    "                                max_iter = 20, # maximum learning iterations \n",
    "                                learning_method = 'online',\n",
    "                                learning_offset = 80., \n",
    "                                total_samples = n_samples,\n",
    "                                random_state = 0)\n",
    "#fit the model\n",
    "lda.fit(tf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#print the top words per topic, using the function defined above.\n",
    "\n",
    "print(\"\\nTopics in LDA model:\")\n",
    "tf_feature_names = tf_vectorizer.get_feature_names()\n",
    "print_top_words(lda, tf_feature_names, n_top_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Document by Topic Distribution\n",
    "\n",
    "One thing we may want to do with the output is find the most representative texts for each topic. A simple way to do this is to merge the topic distribution back into the Pandas dataframe.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the distribution array\n",
    "topic_dist = lda.transform(tf)\n",
    "#topic_dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Merge back in with the original dataframe.\n",
    "pd.options.display.max_colwidth = 100\n",
    "\n",
    "topic_dist_df = pd.DataFrame(topic_dist)\n",
    "df_w_topics = topic_dist_df.join(data)\n",
    "df_w_topics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can sort the dataframe for the topic of interest, and view the top documents for the topics.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "topic_of_interest = 8\n",
    "\n",
    "df_w_topics[['name_abbreviation', \n",
    "             'decision_date', \n",
    "             'text',\n",
    "              topic_of_interest]].sort_values(by=[topic_of_interest], ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "topic_of_interest = 1\n",
    "\n",
    "df_w_topics[['name_abbreviation', \n",
    "             'decision_date', \n",
    "             'text',\n",
    "              topic_of_interest]].sort_values(by=[topic_of_interest], ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "!{sys.executable} -m pip install pyldavis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import pyLDAvis\n",
    "import pyLDAvis.sklearn\n",
    "pyLDAvis.enable_notebook()\n",
    "\n",
    "lda_display = pyLDAvis.sklearn.prepare(lda, \n",
    "                                       tf, \n",
    "                                       tf_vectorizer)\n",
    "\n",
    "pyLDAvis.save_html(lda_display, 'lda_visualization.html')\n",
    "# See lda_visualization.html to explore the LDA based topics\n",
    "\n",
    "lda_display # smaller lambda shows more unique/rare terms for the topic "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Fitting an LDA Topic Model using gensim library\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim.utils import simple_preprocess\n",
    "from gensim import corpora\n",
    "\n",
    "## Create DTM using Gensim\n",
    "# Tokenize the documents\n",
    "tokenized_list = [simple_preprocess(doc) for doc in opinion_texts] # tokenize\n",
    "\n",
    "# Create a Dictionary, that is a mapping between words and their integer ids.\n",
    "dictionary = corpora.Dictionary(tokenized_list)\n",
    "\n",
    "# Convert a document in a corpus, into the bag-of-words (BoW) format = list of (token_intiger_id, token_count) tuples.\n",
    "corpus = [dictionary.doc2bow(line) for line in tokenized_list]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dictionary.token2id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim.models import LdaModel, LdaMulticore\n",
    "\n",
    "n_topics = 10\n",
    "lda_model = LdaMulticore(corpus = corpus,\n",
    "                         id2word = dictionary,\n",
    "                         random_state = 100,\n",
    "                         num_topics = n_topics,\n",
    "                         passes=2, # Number of passes through the corpus during training.\n",
    "                         per_word_topics=True)\n",
    "\n",
    "\n",
    "# See the topics\n",
    "lda_model.print_topics(-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "top_words = 10\n",
    "topics = lda_model.show_topics(formatted = False,  \n",
    "                               num_topics = -1,\n",
    "                               num_words = top_words)\n",
    "\n",
    "for t in range(len(topics)):\n",
    "    print(\"Topic {}, top {} words:\".format( t+1, top_words))\n",
    "    print(\", \".join([w[0] for w in topics[t][1]]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### takes a bit longer due to passes in lda_model\n",
    "#import pyLDAvis.gensim \n",
    "\n",
    "#pyLDAvis.enable_notebook()\n",
    "\n",
    "#vis = pyLDAvis.gensim.prepare(lda_model, \n",
    "#                              corpus, \n",
    "#                              dictionary=lda_model.id2word)\n",
    "#vis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IV. Non-negative Matrix Factorization (NMF)\n",
    "[Non-Negative Matrix Factorization](https://en.wikipedia.org/wiki/Non-negative_matrix_factorization) is another approach to topic modeling. It is a matrix decomposition method and does not assume a prior probability distribution. NMF provides a simple, deterministic method which seems to give highly interpretable results with minimal tweaking/hyperparameter-tuning. NMF topic models are also extremely fast and memory optimized and are fit on TFIDF normalized DTMs.\n",
    "\n",
    "This part of the workshop borrows heavily from [Derek Greene's](https://github.com/derekgreene) tutorial on topic modeling using NMF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "# Create a TFIDF vectorizer\n",
    "vectorizer = TfidfVectorizer(min_df = 2)\n",
    "\n",
    "V = vectorizer.fit_transform(opinion_texts)\n",
    "V.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get terms/features in our new matrix\n",
    "features = vectorizer.get_feature_names()\n",
    "len(features)\n",
    "features[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_topics = 10\n",
    "top_words = 10\n",
    "\n",
    "# create NMF model\n",
    "from sklearn import decomposition\n",
    "model = decomposition.NMF(init = \"nndsvd\", \n",
    "                          n_components = n_topics)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# apply the model and extract the two W and H matrices -> V ~= W*H \n",
    "W = model.fit_transform(V)\n",
    "H = model.components_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define get_descriptor function which will show top words for a given topic\n",
    "def get_descriptor( features, H, topic_index, top ):\n",
    "    top_indices = np.argsort( H[topic_index,:] )[::-1]\n",
    "    top_terms = []\n",
    "    for term_index in top_indices[0:top]:\n",
    "        top_terms.append( features[term_index] )\n",
    "    return top_terms\n",
    "\n",
    "# define get_top_documents function which will show us top cases associated with topics\n",
    "def get_top_documents( cases, W, topic_index, top ):\n",
    "    top_indices = np.argsort( W[:,topic_index] )[::-1]\n",
    "    top_documents = []\n",
    "    for doc_index in top_indices[0:top]:\n",
    "        top_documents.append(cases[doc_index])\n",
    "    return top_documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show topics and words in those topics\n",
    "descriptors = []\n",
    "for topic_index in range( n_topics ):\n",
    "    descriptors.append( get_descriptor( features, H, topic_index, top_words) )  # Top 10 words\n",
    "    str_descriptor = \", \".join( descriptors[topic_index] )\n",
    "    print(\"Topic %02d: %s\" % ( topic_index+1, str_descriptor ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "case_names = data.name_abbreviation.tolist()\n",
    "\n",
    "topic_of_interest = 2\n",
    "n_docs = 10\n",
    "\n",
    "#Print top documents for a given topic\n",
    "topic_documents = get_top_documents(case_names, W, topic_of_interest, n_docs) \n",
    "for i, doc in enumerate(topic_documents):\n",
    "    print(\"%02d. %s\" % ((i+1), doc))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NMF with gensim\n",
    "To tokenize, we use the same code as we used for LDA for gensim. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gensim.models import nmf\n",
    "from gensim.models import TfidfModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Create DTM using Gensim\n",
    "# Tokenize the documents\n",
    "tokenized_list = [simple_preprocess(doc) for doc in opinion_texts] # tokenize\n",
    "\n",
    "# Create a Dictionary, that is a mapping between words and their integer ids.\n",
    "dictionary = corpora.Dictionary(tokenized_list)\n",
    "\n",
    "# Convert a document in a corpus, into the bag-of-words (BoW) format = list of (token_intiger_id, token_count) tuples.\n",
    "corpus = [dictionary.doc2bow(line) for line in tokenized_list]\n",
    "\n",
    "# An important benefit of NMF is its use of TFIDF document term matrix out of the box \n",
    "model_tfidf = TfidfModel(corpus) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_topics = 10\n",
    "nmf_model = nmf.Nmf(model_tfidf[corpus], \n",
    "                    id2word = dictionary,\n",
    "                    num_topics = 10,\n",
    "                    passes = 20,\n",
    "                    random_state = 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nmf_model.show_topics(num_topics=10, \n",
    "                      num_words=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "top_words = 10\n",
    "topics_nmf = nmf_model.show_topics(formatted = False,  \n",
    "                               num_topics = -1,\n",
    "                               num_words = top_words)\n",
    "\n",
    "for t in range(len(topics_nmf)):\n",
    "    print(\"Topic {}, top {} words:\".format( t+1, top_words))\n",
    "    print(\", \".join([w[0] for w in topics_nmf[t][1]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V. Conclusion\n",
    "Topic modeling is an important part of NLP research and there are many applications where representing documents as topics is extremely useful. This notebook introduces sklearn and gensim based LDA and NMF topic modeling using [case.law](https://case.law) data. \n",
    "\n",
    "A key practical insight to be gained from this is the fundamental importance of the underlying data, domain knowledge, and the centrality of text vectorization for many NLP tasks. Another practical consideration worth keeping in mind is that the topics will be somewhat different for NMF and LDA models depending on hyperparameters and the underlying library used (sklearn vs gensim). \n",
    "\n",
    "There are a number of questions that this workshop does not address, largely due to its introductory nature. Firstly, the question which constantly comes up in topic modeling research and applications relates to the determination of the number of topics **k** to be used in these algorithms - that is, how do we really know that a corpus has **k** number of topics? Secondly, how coherent are the words lists for a given topic, and how do we make sure that our judgement on the interpretability of top-n words in a topic is not entirely subjective? These question are outside the scope of this introductory workshop, but nevertheless, one should keep them in mind when working with topic modeling. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## VI. Acknowledgements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I would like to thank the (1) organizers of TextXD, (2) the wonderful people at the D-Lab who have constantly encouraged and supported me throughout my NLP journey, (3) the NLP community at UC Berkeley, namely D-Lab's CTAWG and I-School's NLP community\n",
    "\n",
    "In addition to my own code, this notebook utilizes some (modified) code and insights from the following repositories:\n",
    "\n",
    "* [CAP's example code notebooks for the case.law dataset](https://github.com/harvard-lil)\n",
    "\n",
    "* [Laura Nelson computational text analysis course](https://github.com/lknelson)\n",
    "\n",
    "* [Derek Greene topic modeling tutorial](https://github.com/derekgreene/)\n",
    "\n",
    "* [Geoffrey Boushey work on case.law dataset](https://github.com/gboushey/)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}