cwpk-15-using-notebooks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CWPK \\#15: Using Notebooks for CWPK Documentation\n",
    "=================================================\n",
    "\n",
    "Recipes for Jupyter Notebooks Going Forward \n",
    "-------------------------------------------\n",
    "\n",
    "<div style=\"float: left; width: 305px; margin-right: 10px;\">\n",
    "\n",
    "<img src=\"http://kbpedia.org/cwpk-files/cooking-with-kbpedia-305.png\" title=\"Cooking with KBpedia\" width=\"305\" />\n",
    "\n",
    "</div>\n",
    "\n",
    "In the last installment of the [*Cooking with Python and\n",
    "KBpedia*](https://www.mkbergman.com/cooking-with-python-and-kbpedia/) series, we began to learn\n",
    "about weaving code and narrative in a [Jupyter\n",
    "Notebook](https://en.wikipedia.org/wiki/Project_Jupyter) page. We also\n",
    "saw that we can generate narratives to accompany our code with the\n",
    "[Markdown](https://en.wikipedia.org/wiki/Markdown) mark-up language, though it is not designed (in my view) for\n",
    "efficient document creation. Short explanations between code snippets\n",
    "are fine in Jupyter Notebook, but longer narratives or ones where\n",
    "formatting or decorating are required are fairly difficult. (For an update, see the **NB** box at the conclusion of this installment.) Further, we\n",
    "also want to publish Web pages independent of our environment. What I\n",
    "describe in this **CWPK** installment is how I combine standard Web page\n",
    "editing and publishing with Jupyter, as well as the starting parts to my\n",
    "standard workflow.\n",
    "\n",
    "Having a repeatable and fairly efficient workflow for formulating a\n",
    "lesson or question, then scoping it out framed with introduction and\n",
    "working parts, and then skeletonizing it such that good working\n",
    "templates can be put in place is important when one contemplates\n",
    "progressing through all of the stages of discovering, addressing, and\n",
    "documenting a project. In the case of this **CWPK** series, this is not a\n",
    "trifling consideration. I am anticipating literally dozens of\n",
    "installments in this series; heck, we are already at installment \\#15\n",
    "and we haven't begun yet to code anything in\n",
    "[Python](https://en.wikipedia.org/wiki/Python_%28programming_language%29)!\n",
    "We could stitch together more direct methods of doing a given task, but\n",
    "that will not necessarily arm us to do a broader set of tasks.\n",
    "\n",
    "Not everyone prefers my style of trying to get systems and game plan in\n",
    "place before tackling a big task, in which case I suggest you skip to\n",
    "the end where we conclude with a discussion of directory organization.\n",
    "For this initial part, however, I will assume that you want to sometimes\n",
    "rely on an interacting coding environment and other times want to\n",
    "generate narratives efficiently. In this use case, the ability to\n",
    "'[round-trip](https://en.wikipedia.org/wiki/Round-trip_format_conversion)'\n",
    "between HTML editing and Jupyter is an important consideration.\n",
    "Efficiency and document size are relevant considerations, too.\n",
    "\n",
    "Recall in our last installment that we pointed to two ways to get HTML\n",
    "pages from a Jupyter Notebook: 1) either from a download, or 2) from\n",
    "invoking the `nbconvert` service from a command window. We could not\n",
    "invoke `nbconvert` from within a notebook page because it is a Jupyter\n",
    "service. This next frame shows the file created from the article herein\n",
    "using the download method. You invoke the cell by entering <code>shift+enter</code> to call up the file, and then, once inspected, use Cell → All Output → Clear to clear and collapse the view area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('files/cwpk-15-using-notebooks-download.txt', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"center-image\">\n",
    "\n",
    "<iframe src=\"files/cwpk-15-using-notebooks-download.txt\" allowfullscreen=\"\" style=\"border: 1px solid rgb(238, 238, 238);\" height=\"150\" width=\"800\">Info here.</iframe>\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I should mention that both the `nbconvert` and download methods produce\n",
    "similarly bloated files. Go ahead, scroll through it. While the\n",
    "generated file renders very well, it is about <span\n",
    "style=\"font-weight: bold; font-style: italic;\">10x larger</span> than\n",
    "the original HTML file that captures its narrative (13,644 v 397 lines; 298 K v 30 K). This bloat in file\n",
    "size is due to the fact that all of the style information (`*.css`)\n",
    "contained in the original document gets re-expressed in this version,\n",
    "along with much other styling information not directly related to this\n",
    "page. Thus, while the generation of the page is super easy, and renders\n",
    "beautifully, it is an overweight pig. We could spend some time whittling\n",
    "down this monster to size with some of the built-in functionality of\n",
    "`nbconvert`, but why not deal that problem using\n",
    "[Pandoc](https://en.wikipedia.org/wiki/Pandoc) directly upon which\n",
    "`nbconvert` is based?  \n",
    "\n",
    "So, in testing the cycle from HTML to notebooks and back again, we find\n",
    "that certain aspects of generating project documentation present\n",
    "challenges. In working through the documentation for this series I have\n",
    "found these types of problem areas for round-tripping:\n",
    "\n",
    "-   Use of a standard, formatted header (with logo)\n",
    "-   Use of standard footers (notification boxes in our case)\n",
    "-   Centering images\n",
    "-   Centering text\n",
    "-   Tables, and\n",
    "-   Loss of the interactive functionality in the notebook in the HTML.\n",
    "\n",
    "Only the last consideration is essential to create useful project and\n",
    "code documentation. However, if one likes professional, well-formatted\n",
    "pages with loads of images and other pretty aspects, it is worth some\n",
    "time to work out productive ways to handle these aspects. In broad\n",
    "terms, for me, that means to be able to move between Web page authoring\n",
    "and interactive code development, testing, and documentation. I also\n",
    "decided to devote some time to these questions as a way to better\n",
    "understand the flexibilities and power of the tools we have chosen. We\n",
    "will always encounter gaps in knowledge when working new problems. I'd\n",
    "like to find the practical balance between the <span\n",
    "style=\"font-style: italic;\">de minimus</span> path to get something done\n",
    "with learning enough to be able to travel similar paths in the future,\n",
    "perhaps even in a production mode.\n",
    "\n",
    "Since Markdown is a superset of HTML it is not possible to round-trip\n",
    "using Markdown alone within Jupyter Notebook. Fortunately, many Markdown\n",
    "interpreters, including Jupyter, accept some limited HTML in documents.\n",
    "There are two ways that may happen. The first is to use one of the\n",
    "so-called ['magic'\n",
    "terms](https://ipython.readthedocs.io/en/stable/interactive/magics.html)\n",
    "in [iPython](https://en.wikipedia.org/wiki/IPython), the command shell\n",
    "underneath Jupyter Notebook. By placing the magic term `%%html` at the\n",
    "start of a notebook Markdown cell, we instruct the system to render that\n",
    "entire cell as HTML. Since it is easy to stop a cell and add a new one\n",
    "below it, we can 'fence' such aspects easily in our notebook code bases.\n",
    "I encourage you to study other 'magic' terms from the [prior\n",
    "link](https://ipython.readthedocs.io/en/stable/interactive/magics.html)\n",
    "that are shortcuts to some desired notebook capabilities.\n",
    "\n",
    "A second way to use HTML in notebooks is to embed HTML tags. This way is\n",
    "trickier since the various Markdown evaluation engines -- due to\n",
    "Markdown's diversity of implementations -- may recognize different tags\n",
    "or, when recognized, treat them differently. One of the reasons to\n",
    "embrace Pandoc, introduced in the last installment, is to accept its\n",
    "standard way of handling languages, markups, formats, and functions.\n",
    "\n",
    "Boiled down to its essence, then, we have two functional challenges in\n",
    "round-tripping:\n",
    "\n",
    "1.  Loss of HTML tags and styling with Markdown\n",
    "2.  Loss of notebook functionality in HTML.\n",
    "\n",
    "One of Pandoc's attractions is that both `<div>` and `<span>` can be\n",
    "flagged to be skipped in the conversions, which means we can isolate our\n",
    "HTML changes to these tag types, with `divs` giving us block 'fencing'\n",
    "capabilities and `spans` inline 'fencing' capabilities. (There are also\n",
    "[Lua](https://en.wikipedia.org/wiki/Lua_%28programming_language%29)\n",
    "filter capabilities with Pandoc to provide essentially unlimited control\n",
    "over conversions, but we will leave that complexity outside of our\n",
    "scope.) Another observation we make is that many of the difficult tags\n",
    "that do not round-trip well deal with styling or HTML\n",
    "tags that can be captured via\n",
    "[CSS](https://en.wikipedia.org/wiki/Cascading_Style_Sheets) styling.\n",
    "\n",
    "Another challenge that must be deciphered are the many flavors of\n",
    "Markdown that appear in the wild. Pandoc handles many flavors of\n",
    "Markdown, including the specified readers of `markdown`,\n",
    "`markdown_strict`, `markdown_mmd`, `markdown_phpextra`, and `gfm`\n",
    "(GitHub-flavored Markdown). One can actually ingest any of these flavors\n",
    "in Pandoc and express any of the others. As I noted in the last\n",
    "installment, Pandoc presently has 33 different format readers from Word\n",
    "docs to rich text and can write out nearly twice that many in different\n",
    "formats. For our purposes, however, it is best to choose Pandoc's\n",
    "canonical internal form of `markdown`. However, besides translation\n",
    "purposes, the `gfm` option likely has the broadest external\n",
    "applicability.\n",
    "\n",
    "OK, so it appears that Pandoc's own flavor of Markdown is the best\n",
    "conversion target and that we will try to move problem transfer areas to\n",
    "`div` and `span`. As for the loss of notebook functionality in HTML,\n",
    "there is no direct answer. However, because an interactive notebook page\n",
    "is organized in a sequence of cells, we can segregate activity areas\n",
    "from interactive areas in our documents. That does not give us complete\n",
    "document convertibility, but we can do it in sections if need be after\n",
    "initial drafting. With this basic approach decided, we begin to work\n",
    "through the issues.\n",
    "\n",
    "After testing inline styling, we see that we can find recipes that move\n",
    "the CSS and related HTML (such as `<center>` or `<i>` or `<italics>`)\n",
    "between our HTML and notebook environments without loss. Once we go\n",
    "beyond proofs of concept, however, we want to be able to capture our CSS\n",
    "information in styling class and ID designations so that we not need to\n",
    "duplicate lengthy styling code. However, handling and referencing CSS\n",
    "[stylesheets](https://en.wikipedia.org/wiki/Style_sheet_%28web_development%29)\n",
    "is not straightforward with the complexity of applications and\n",
    "configuration files of an\n",
    "[Anaconda](https://en.wikipedia.org/wiki/Anaconda_%28Python_distribution%29)\n",
    "Python distribution and environment. For a very useful discussion of CSS\n",
    "in this context see [Jack Northrup's notebook on customizing\n",
    "CSS](https://jacknorthrup.com/jupyter-notebooks/customizing-jupyter-notebooks-appearance-css.html)\n",
    "\n",
    "Now, the tools at both ends of this process, Jupyter Notebook and\n",
    "Pandoc, both recognize users will want their own `custom.css` files to\n",
    "guide these matters. But, of course, each tool has a different location\n",
    "and mechanism for specifying this. There is surprisingly little\n",
    "documentation or guidance on the Web for how to handle these things.\n",
    "Most of the references I encountered on these matters were incorrect. We\n",
    "have two fundamental challenges in this quest: 1) how do we define and\n",
    "where do we locate our `custom.css` file on disk?; and 2) what are our\n",
    "command-line instructions to best guide the two round-trip conversion\n",
    "steps? We will use Pandoc and proper stylesheet locations to guide both\n",
    "questions.\n",
    "\n",
    "Let's take the first trip of moving from HTML draft into an operating\n",
    "shell for Jupyter Notebook. First, as we draft material with an HTML\n",
    "editor, we are going to want to store our `custom.css` information that\n",
    "we need to segregate into some pre-defined, understood location. One way\n",
    "to do that is through a relative location in relation to where our\n",
    "authored HTML document resides. An easy choice is to create a\n",
    "sub-directory of files that is placed immediately below where our HTML\n",
    "document resides. If we follow this location, we may always find the\n",
    "stylesheet (CSS) in the relative location of '`files/custom.css`'. (You\n",
    "may name the subdirectory something different, but it is probably best\n",
    "to retain the name of '`custom.css`' since that is expected by Jupyter.)\n",
    "However, those same CSS specifications need to be available to Jupyter\n",
    "Notebook, which follows different file look-up conventions. One way to\n",
    "discover where Jupyter Notebook expects to find its supplementary CSS\n",
    "files is to open the relevant notebook page and save it as HTML\n",
    "(download or `nbconvert` or Pandoc methods). When you open the HTML file\n",
    "with an editor, look for the reference to '`custom.css`'. That will give\n",
    "you the file location in relation to your document's root. In my case,\n",
    "this location is\n",
    "`C:\\1-PythonProjects\\Python\\Lib\\site-packages\\notebook\\static\\custom`.\n",
    "For your own circumstance, it may also be under the `\\user\\user\\`\n",
    "profile area depending on whether you first installed Anaconda for an\n",
    "individual user. At any rate, look for the `\\Lib\\` directory and then\n",
    "follow the directory cascade under your main Python location.\n",
    "\n",
    "<div style=\"background-color:#ffc; border:1px dotted #ff0; vertical-align:middle; margin:15px 60px; padding:8px;\"><strong>NB</strong>: Unfortunately, should you later update your Jupyter Notebook package, you may find that your <code>custom.css</code> is overwritten with the standard, blank placeholder. You may again need to copy your active version into this location.</div> \n",
    "\n",
    "Once you have your desired CSS both where Pandoc will look for it\n",
    "(again, relative is best) and where Jupyter Notebook will separately\n",
    "look for it, we can concentrate on getting our additional styles into\n",
    "[`custom.css`](files/custom.css), which you may click on to see its\n",
    "contents for this current page. Once we have populated `custom.css`, it\n",
    "is now time to figure out the conversion parameters. There is much\n",
    "flexibility in Pandoc for all aspects of instructing the application at\n",
    "the command line. I present one hard-earned configuration below, but for\n",
    "your own purposes, I strongly recommend you inspect the PDF version of\n",
    "the [Pandoc User Guide](https://pandoc.org/MANUAL.pdf) should you want\n",
    "to pursue your own modifications. At any rate, here is the basic HTML →\n",
    "Notebook initial conversion, using this current page as the example:\n",
    "\n",
    "    $ pandoc -f html -t ipynb+native_divs cwpk-15-using-notebooks.html -o cwpk-15-using-notebooks.ipynb\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is what these command-line options and switches mean:\n",
    "\n",
    "-   `-f html` - `-f` (also `--from`) is the source or from switch, with\n",
    "    `html` indicating the source format type. Multiple types are\n",
    "    possible, but only one may be specified at a time\n",
    "-   `-t ipynb` - `-t` (also `--to`) is the target of the conversion,\n",
    "    with `ipynb` in this case indicating a notebook document. Multiple\n",
    "    types are possible, but only one may be specificed here\n",
    "-   `+native_divs` - this is a conversion switch that tells Pandoc to\n",
    "    retain the content within a native HTML div in the source  \n",
    "-   `cwpk-15-using-notebooks.html` - this is the source file\n",
    "    specification. There are defaults within Pandoc that allow `-f html`\n",
    "    to not be specified, for example and for other formats, once this\n",
    "    input file type is specified\n",
    "-   `-o cwpk-15-using-notebooks.ipynb` - this is the output (`-o`) file\n",
    "    name; if left unspecified, the default is to write the original file\n",
    "    name with the new `.ipynb` extension (or whatever target format was\n",
    "    specified).\n",
    "\n",
    "These commands and switches require the Windows command window or\n",
    "PowerShell to be opened in the same directory as the `*.html` document\n",
    "you are converting when you instruct at the command line. Upon entering\n",
    "this command, the appearance of the prompt tells you the conversion\n",
    "proceeded to completion.\n",
    "\n",
    "This command will now establish a new notebook file (`*.ipynb`) in the\n",
    "same directory. Please make sure this directory location is under the\n",
    "root you established when you installed Jupyter Notebook (see [**CWPK \\#10**](https://www.mkbergman.com/2337/cwpk-10-installing-a-project-notebook/)\n",
    "if you need to refresh that or change locations).\n",
    "\n",
    "When you invoke Jupyter Notebook and call up the new `*.ipynb` file, it\n",
    "will open as a single Markdown cell. If you need to split that input\n",
    "into multiple parts in order to interweave interactive parts,\n",
    "double-click to edit, cut the sections you need to move, Run the cell,\n",
    "add a cell below, and paste the split section into the new cell. In this\n",
    "way, you can skeletonize your active portions with existing narrative.\n",
    "\n",
    "Upon completing your activities and additions and code tests within\n",
    "Notebook, you may now save out your results to HTML for publishing\n",
    "elsewhere. Again, you could Download or use `nbconvert`, but to keep our\n",
    "file sizes manageable and to give ourselves the requisite control we\n",
    "will again do this conversion with Pandoc. After saving your work and\n",
    "exiting to the command window, and while still in the current working\n",
    "directory where the `*.ipynb` resides, go ahead and issue this command\n",
    "at the command window prompt:\n",
    "\n",
    "    $ pandoc -s -f ipynb -t html -c files/custom.css --highlight-style=kate cwpk-15-using-notebooks.ipynb -o cwpk-15-using-notebooks-test.html\n",
    "\n",
    "There we have it! We now have our recipes to move from HTML to <code>*.ipynb</code> and the reverse!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is what the new command-line options and switches mean:\n",
    "\n",
    "We have now reversed the `-f` and `-t` switches since we are now\n",
    "exporting as HTML; again, multiple format options may be substituted\n",
    "here (though specific options may change depending on format)\n",
    "\n",
    "-   `-s` means to process the export as standalone, which will bring in\n",
    "    the HTML information outside of the `<body>` tags\n",
    "-   `-c` (or `--css=`) tells the writer where to find the supplementary,\n",
    "    external CSS file. This example is the `files` subdirectory under\n",
    "    the current notebook; the file could be `whatever.css`, but we keep\n",
    "    the `custom.css` name to be consistent with the required name for\n",
    "    Jupyter (even though in a different location)\n",
    "-   `--highlight-style=kate` is one of the language syntax highlighting\n",
    "    options available in Pandoc; there are many others available and you\n",
    "    may also create your own\n",
    "-   `-o cwpk-15-using-notebooks-test.html` - is an optional output only\n",
    "    if we want to change the base name from the input name; during\n",
    "    drafting it is recommended to use another name (`-test`) to prevent\n",
    "    inadvertent overwriting of good files.\n",
    "\n",
    "Upon executing this command, you will get a successful export, but a\n",
    "message indicating you did not provide a title for the project and it\n",
    "will default to the file name as shown by <span\n",
    "style=\"font-style: italic;\">Figure 1:</span>\n",
    "\n",
    "<div style=\"margin: 10px auto; display: table;\">\n",
    "\n",
    "<img src=\"files/using-notebooks-1.png\" title=\"Message at HTML Export\" width=\"800\" alt=\"Message at HTML Export\" />\n",
    "\n",
    "</div>\n",
    "\n",
    "<div style=\"margin: 10px auto; display: table; font-style: italic;\">\n",
    "\n",
    "Figure 1: Message at HTML Export\n",
    "\n",
    "</div>\n",
    "\n",
    "\n",
    "There are metadata options you may assign at the command line, plus, of\n",
    "course, many other configuration options. Again, the best consolidated\n",
    "source for learning about these options is in the PDF [Pandoc Users\n",
    "Guide](https://pandoc.org/MANUAL.pdf). This document is kept current\n",
    "with the many revisions that occur frequently for Pandoc.\n",
    "\n",
    "The next panel shows the HTML generated by this export. Note this\n",
    "document is much smaller (10x) than the version that comes from the\n",
    "download or `nbconvert` methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('files/cwpk-15-using-notebooks-html.txt', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"center-image\">\n",
    "\n",
    "<iframe src=\"files/cwpk-15-using-notebooks-html.txt\" allowfullscreen=\"\" style=\"border: 1px solid rgb(238, 238, 238);\" height=\"150\" width=\"800\">Info here.</iframe>\n",
    "\n",
    "</div>\n",
    "\n",
    "This HTML export is good for publication purposes, but lacks the\n",
    "interactivity of its interactive parent. You should thus refrain from\n",
    "such exports until development is largely complete. In any case, we still see\n",
    "static sections for the interactive portions of the notebook. These\n",
    "were styled according to `custom.css`.  \n",
    "\n",
    "<div style=\"background-color:#ffc; border:1px dotted #ff0; vertical-align:middle; margin:15px 60px; padding:8px;\"><strong>NB</strong>: The Web pages that appear on my AI3 blog are the HTML conversions of these interactive notebook pages. The information box at the bottom of each installment page instructs as to where you may obtain the fully interactive versions.</div>\n",
    "\n",
    "Using similar commands you can also produce outputs in other formats,\n",
    "such as this one for the GitHub flavor of Markdown using this command\n",
    "line instruction:\n",
    "\n",
    "    $ pandoc -s -f ipynb -t gfm -c files/custom.css --highlight-style=kate cwpk-15-using-notebooks.ipynb\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note we have changed the `-t` to option to `gfm` and have removed the\n",
    "`-o` output option because we will use the same notebook file name. Here\n",
    "is the output from that conversion:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('files/cwpk-15-using-notebooks-md.txt', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"center-image\">\n",
    "\n",
    "<iframe src=\"files/cwpk-15-using-notebooks-md.txt\" allowfullscreen=\"\" style=\"border: 1px solid rgb(238, 238, 238);\" height=\"150\" width=\"800\">Info here.</iframe>\n",
    "\n",
    "</div>\n",
    "\n",
    "You can see that headers are more cleanly shown by `#` symbols, and that\n",
    "`gfm` is a generally clean design. It is becoming the <span\n",
    "style=\"font-style: italic;\">de facto</span> standard for shared\n",
    "Markdown.\n",
    "\n",
    "Of course, no export is necessary for the actual notebook file since\n",
    "they are plain text. As noted in earlier installments, Jupyter Notebook\n",
    "files are natively expressed in JavaScript Object Notation\n",
    "([JSON](https://en.wikipedia.org/wiki/JSON)). This is the only file\n",
    "representation that contains transferrable instructions for the\n",
    "interactive code cells in the notebook page. This JSON file is for the\n",
    "same content expressed in the files above:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('files/cwpk-15-using-notebooks-ipynb.txt', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"center-image\">\n",
    "\n",
    "<iframe src=\"files/cwpk-15-using-notebooks-ipynb.txt\" allowfullscreen=\"\" style=\"border: 1px solid rgb(238, 238, 238);\" height=\"150\" width=\"800\">Info here.</iframe>\n",
    "\n",
    "</div>\n",
    "\n",
    "Mastery of these tools is a career in itself. I'm sure there are better\n",
    "ways to write these commands or even how to approach the workflow. As\n",
    "the examples presently stand there are a few minor glitches, for\n",
    "example, that keep this round-tripping from being completely automatic.\n",
    "Relative file locations get re-written with an '`attachment:`' prefix\n",
    "during the round-trip, which must be removed from the HTML code to get\n",
    "images to display. For some strange reason, images also need to have a\n",
    "width entry (<span style=\"font-style: italic;\">e.g.</span>,\n",
    "`width=\"800\"`) in order not to be converted to Markdown format. Also, in\n",
    "some instances HTML code within a `div` gets converted to Markdown\n",
    "syntax, which then can not be recognized when later writing to HTML. The\n",
    "Pandoc system is full-featured and difficult to master without, I am\n",
    "sure, much use.\n",
    "\n",
    "In working with these tools, here is what I have discovered to be a good\n",
    "starting workflow:\n",
    "\n",
    "1.  Author initial skeleton in HTML. Write intro, get references and\n",
    "    links, set footer. Try to use familiar editing tools to apply\n",
    "    desired formatting and styles\n",
    "2.  Add blocks and major steps, including some thought for actual\n",
    "    interactive pieces; name this working file with an `-edit` name\n",
    "    extension to help prevent overwriting it\n",
    "3.  Convert to Notebook format and transfer to notebook\n",
    "4.  Work on interaction steps within Jupyter Notebook, one by one. Add\n",
    "    narrative lead in and following commentary to each given step. If\n",
    "    the narrative is too long or too involved to readily handle in the\n",
    "    notebook with Markdown, save, revert to the HTML version for the\n",
    "    interstitial narrative Markdown cell\n",
    "5.  Generate Markdown for the connecting cell, copy back into the\n",
    "    working notebook page\n",
    "6.  Repeat as necessary to work through the interaction steps\n",
    "7.  Save, and generate the new notebook\n",
    "8.  Export to multiple publication platforms.\n",
    "\n",
    "### Directory and File Structure\n",
    "These conversion efforts have also helped refine some of the directory\n",
    "refinements useful to this workflow. I first began laying out a broad\n",
    "directory structure for this project in [**CWPK \\#9**](https://www.mkbergman.com/2336/cwpk-9-installing-python/). We can now add a major\n",
    "branch under our project for notebooks, with a major sub-branch being\n",
    "this one for the **CWPK** series. I personally use CamelCase for naming my\n",
    "upper directory levels, those I know will likely last for a year or\n",
    "more. Lower levels I have tended to lower case and hyphen separate more\n",
    "akin to a consistent treatment on Linux boxes.\n",
    "\n",
    "Here is how I am setting up my directory structure:\n",
    "\n",
    "    |-- PythonProject                             # directory first introduced in CWPK #9 \n",
    "         |-- Python                                               \n",
    "               |-- [Anaconda3 distribution]                        \n",
    "         |-- Notebooks                            # see next directory expansion\n",
    "              |-- CWPKNotebook                                    \n",
    "         |-- TBA                                  # We'll add to this directory structure as we move on\n",
    "     |-- TBA\n",
    "\n",
    "Individual notebooks should live in their own directory along side any\n",
    "ancillary files related to them. For example:\n",
    "\n",
    "    Notebooks/\n",
    "    |-- CWPKNotebook\n",
    "    |    |-- cwpk-1-installment                   # one folder/notebook per installment\n",
    "    |        |-- .ipyn_checkpoints                # created automatically as notebooks are saved and checkpointed\n",
    "    |            +-- cwpk-1-installment.ipynb     # backup file from checkpoint, same name as current\n",
    "    |        +-- cwpk-1-installment.ipynb         # current active notebook file \n",
    "    |        +-- cwpk-1-installment-edit.html     # initial drafting file named differently to prevent overwriting \n",
    "    |        +-- cwpk-1-installment.html\n",
    "    |        |-- files                            # a sub-directory for all supporting files for that installment\n",
    "    |            +-- custom.css                   # same across installments, + one in the Jupyter Notebook settings \n",
    "    |            +-- image-1.png\n",
    "    |            +-- image-2.jpg\n",
    "    |            +-- attachment.txt\n",
    "    |    |-- cwpk-2-installment\n",
    "    |        +-- etc.\n",
    "    |    |-- cwpk-3-etc.\n",
    "\n",
    "Note that Save and Checkpoint within Jupyter Notebook automatically creates a\n",
    "`.ipynb_checkpoints` subdirectory and populates it with the current\n",
    "version of the `*.ipynb` file. (So, don't mix up the current file in the\n",
    "parent directory with this backup one.) Further, it is perhaps better to\n",
    "create a more streamlined version of this directory structure that would\n",
    "place all notebook files (`*.ipynb`) in a single directory with a single\n",
    "location for the `custom.css`. That approach requires more logic in the\n",
    "application and is harder to include in a lesson. One advantage of the\n",
    "somewhat duplicative structure herein is that we are able to treat each\n",
    "notebook installment as a standalone unit.\n",
    "\n",
    "<div style=\"background-color:#ffc; border:1px dotted #ff0; vertical-align:middle; margin:15px 60px; padding:8px;\"><strong>NB</strong>: From the perspective of <strong>CWPK #60</strong> looking back, I find that my assumptions of how I would use Jupyter Notebook did not prove to be exactly accurate. In fact, I have found using Notebooks to be tremendously helpful and productive for all drafting activities. I like the way that the flow of cells, either code or Markdown, can lead to productive drafting. My actual experience going forward is that I completely ceased using HTML or Web pages for any drafting. The interactive notebook environment has proven to be a real favorite with me. True, I now do more drafting directly using Markdown, but even that has proven to be quick and productive.</div>\n",
    "\n",
    "This installment completes our first major section on set-up and\n",
    "configuration of our working environment. In our next installment we\n",
    "switch gears to working with Python and lay out our game plan for doing\n",
    "so.\n",
    "\n",
    " <div style=\"background-color:#efefff; border:1px dotted #ceceff; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "  <span style=\"font-weight: bold;\">NOTE:</span> This article is part of the <a href=\"https://www.mkbergman.com/cooking-with-python-and-kbpedia/\" style=\"font-style: italic;\">Cooking with Python and KBpedia</a> series. See the <a href=\"https://www.mkbergman.com/cooking-with-python-and-kbpedia/\"><strong>CWPK</strong> listing</a> for other articles in the series. <a href=\"http://kbpedia.org/\">KBpedia</a> has its own Web site.\n",
    "  </div>\n",
    "\n",
    "<div style=\"background-color:#ebf8e2; border:1px dotted #71c837; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "\n",
    "<span style=\"font-weight: bold;\">NOTE:</span> This <strong>CWPK \n",
    "installment</strong> is available both as an online interactive\n",
    "file <a href=\"https://mybinder.org/v2/gh/Cognonto/CWPK/master\" ><img src=\"https://mybinder.org/badge_logo.svg\" style=\"display:inline-block; vertical-align: middle;\" /></a> or as a <a href=\"https://github.com/Cognonto/CWPK\" title=\"CWPK notebook\" alt=\"CWPK notebook\">direct download</a> to use locally. Make sure and pick the correct installment number. For the online interactive option, pick the <code>*.ipynb</code> file. It may take a bit of time for the interactive option to load.</div>\n",
    "\n",
    "<div style=\"background-color:#feeedc; border:1px dotted #f7941d; vertical-align:middle; margin:15px 60px; padding:8px;\"> \n",
    "<div style=\"float: left; margin-right: 5px;\"><img src=\"http://kbpedia.org/cwpk-files/warning.png\" title=\"Caution!\" width=\"32\" /></div>I am at best an amateur with Python. There are likely more efficient methods for coding these steps than what I provide. I encourage you to experiment -- which is part of the fun of Python -- and to <a href=\"mailto:mike@mkbergman.com\">notify me</a> should you make improvements.    \n",
    "\n",
    "</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}