julia.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Julia @ IHI Code Club"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## The REPL\n",
    "\n",
    "Julia comes with a\n",
    "[REPL](https://docs.julialang.org/en/v1/stdlib/REPL/#The-Julia-REPL-1)\n",
    "(Read-Eval-Print-Loop) where you can interactively run commands."
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4"
     },
     "metadata": {},
     "execution_count": 1
    }
   ],
   "cell_type": "code",
   "source": [
    "2 + 2"
   ],
   "metadata": {},
   "execution_count": 1
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "0.0015926529164868282"
     },
     "metadata": {},
     "execution_count": 2
    }
   ],
   "cell_type": "code",
   "source": [
    "sin(3.14)"
   ],
   "metadata": {},
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can define simple functions with the one-line syntax"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "f (generic function with 1 method)"
     },
     "metadata": {},
     "execution_count": 3
    }
   ],
   "cell_type": "code",
   "source": [
    "f(x) = 4 * x - 2.5"
   ],
   "metadata": {},
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "source": [
    "or with the longer, equivalent, syntax"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "g (generic function with 1 method)"
     },
     "metadata": {},
     "execution_count": 4
    }
   ],
   "cell_type": "code",
   "source": [
    "function g(x)\n",
    "    return 4 * x - 2.5\n",
    "end"
   ],
   "metadata": {},
   "execution_count": 4
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.5430512678345503"
     },
     "metadata": {},
     "execution_count": 5
    }
   ],
   "cell_type": "code",
   "source": [
    "f(log(57) / 4)"
   ],
   "metadata": {},
   "execution_count": 5
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.5430512678345503"
     },
     "metadata": {},
     "execution_count": 6
    }
   ],
   "cell_type": "code",
   "source": [
    "g(log(57) / 4)"
   ],
   "metadata": {},
   "execution_count": 6
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can assign a value to a variable, like the speed of light in vacuum in m/s\n",
    "(underscores are ignored, but make the number more readable)"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "299792458"
     },
     "metadata": {},
     "execution_count": 7
    }
   ],
   "cell_type": "code",
   "source": [
    "c = 299_792_458"
   ],
   "metadata": {},
   "execution_count": 7
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.1991698295e9"
     },
     "metadata": {},
     "execution_count": 8
    }
   ],
   "cell_type": "code",
   "source": [
    "f(c)"
   ],
   "metadata": {},
   "execution_count": 8
  },
  {
   "cell_type": "markdown",
   "source": [
    "Use the semicolon `;` at the end of the line to hide the output of the result"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "c ^ 4;"
   ],
   "metadata": {},
   "execution_count": 9
  },
  {
   "cell_type": "markdown",
   "source": [
    "Do you note something strange?  Can you guess what happened?  Read the\n",
    "[Frequently Asked\n",
    "Questions](https://docs.julialang.org/en/v1/manual/faq/#faq-integer-arithmetic-1)\n",
    "in the documentation to know more.\n",
    "\n",
    "Julia has full support for Unicode"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "人人生而自由,在尊严和权利上一律平等。他们赋有理性和良心,并应以兄弟关系的精神相对待。\n"
     ]
    }
   ],
   "cell_type": "code",
   "source": [
    "println(\"人人生而自由,在尊严和权利上一律平等。他们赋有理性和良心,并应以兄弟关系的精神相对待。\")"
   ],
   "metadata": {},
   "execution_count": 10
  },
  {
   "cell_type": "markdown",
   "source": [
    "including in identifiers (tip: you can use LaTeX-like commands `\\alpha + TAB`\n",
    "and `\\beta + TAB` to get these symbols)"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "0.35294117647058826"
     },
     "metadata": {},
     "execution_count": 11
    }
   ],
   "cell_type": "code",
   "source": [
    "α = 1.2\n",
    "β = 3.4\n",
    "α / β"
   ],
   "metadata": {},
   "execution_count": 11
  },
  {
   "cell_type": "markdown",
   "source": [
    "or even emoji, if your font supports them (tip: use `\\:apple: + TAB` and\n",
    "`\\:tangerine: + TAB` for these).  Who said you can't sum apples and oranges?"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "12"
     },
     "metadata": {},
     "execution_count": 12
    }
   ],
   "cell_type": "code",
   "source": [
    "🍎 = 5\n",
    "🍊 = 7\n",
    "🍎 + 🍊"
   ],
   "metadata": {},
   "execution_count": 12
  },
  {
   "cell_type": "markdown",
   "source": [
    "Julia has by default other REPL modes: the package manager mode (that you can\n",
    "enter with `]`), the shell-mode (enter with `;`) and the help mode (enter with\n",
    "`?`)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Arrays and statistics\n",
    "\n",
    "Julia has native support for arrays"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4-element Array{Float64,1}:\n 7.43\n 5.05\n 1.7\n 6.68"
     },
     "metadata": {},
     "execution_count": 13
    }
   ],
   "cell_type": "code",
   "source": [
    "v = [7.43, 5.05, 1.7, 6.68]"
   ],
   "metadata": {},
   "execution_count": 13
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4"
     },
     "metadata": {},
     "execution_count": 14
    }
   ],
   "cell_type": "code",
   "source": [
    "length(v)"
   ],
   "metadata": {},
   "execution_count": 14
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "7.43"
     },
     "metadata": {},
     "execution_count": 15
    }
   ],
   "cell_type": "code",
   "source": [
    "v[1]"
   ],
   "metadata": {},
   "execution_count": 15
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.7"
     },
     "metadata": {},
     "execution_count": 16
    }
   ],
   "cell_type": "code",
   "source": [
    "v[3]"
   ],
   "metadata": {},
   "execution_count": 16
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can apply functions or operators with the broadcasting syntax: the dot:"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4-element Array{Float64,1}:\n 27.22\n 17.7\n  4.3\n 24.22"
     },
     "metadata": {},
     "execution_count": 17
    }
   ],
   "cell_type": "code",
   "source": [
    "4 .* v .- 2.5"
   ],
   "metadata": {},
   "execution_count": 17
  },
  {
   "cell_type": "markdown",
   "source": [
    "Broadcasting is completely generic and works with user-defined functions:"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4-element Array{Float64,1}:\n 27.22\n 17.7\n  4.3\n 24.22"
     },
     "metadata": {},
     "execution_count": 18
    }
   ],
   "cell_type": "code",
   "source": [
    "f.(v)"
   ],
   "metadata": {},
   "execution_count": 18
  },
  {
   "cell_type": "markdown",
   "source": [
    "Nested broadcasted expressions are fused into a single loop.  To know more\n",
    "about broadcasting, read the blog post [More Dots: Syntactic Loop Fusion in\n",
    "Julia](https://julialang.org/blog/2017/01/moredots/).\n",
    "\n",
    "We can define a simple function implementing a naive summation algorithm:"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "20.86"
     },
     "metadata": {},
     "execution_count": 19
    }
   ],
   "cell_type": "code",
   "source": [
    "function mysum(vector)\n",
    "    result = 0.0\n",
    "    for x in vector\n",
    "        result = result + x # or result += x\n",
    "    end\n",
    "    return result\n",
    "end\n",
    "mysum(v)"
   ],
   "metadata": {},
   "execution_count": 19
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can generate random numbers with\n",
    "[`rand`](https://docs.julialang.org/en/v1/stdlib/Random/#Base.rand) and\n",
    "[`randn`](https://docs.julialang.org/en/v1/stdlib/Random/#Base.randn):"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "0.22142191789160015"
     },
     "metadata": {},
     "execution_count": 20
    }
   ],
   "cell_type": "code",
   "source": [
    "rand()"
   ],
   "metadata": {},
   "execution_count": 20
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.5040760697280728"
     },
     "metadata": {},
     "execution_count": 21
    }
   ],
   "cell_type": "code",
   "source": [
    "randn()"
   ],
   "metadata": {},
   "execution_count": 21
  },
  {
   "cell_type": "markdown",
   "source": [
    "or random arrays by specifying the dimension:"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4×4 Array{Float64,2}:\n 0.909987   0.891994  0.280367  0.879094\n 0.0194884  0.621215  0.30542   0.167984\n 0.0603712  0.414248  0.214522  0.316397\n 0.179692   0.631837  0.900932  0.373998"
     },
     "metadata": {},
     "execution_count": 22
    }
   ],
   "cell_type": "code",
   "source": [
    "r = randn(1000)\n",
    "A = rand(4, 4)"
   ],
   "metadata": {},
   "execution_count": 22
  },
  {
   "cell_type": "markdown",
   "source": [
    "Statistical functions are defined in the\n",
    "[`Statistics.jl`](https://docs.julialang.org/en/v1/stdlib/Statistics/)\n",
    "standard library"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "-0.05037708944441693"
     },
     "metadata": {},
     "execution_count": 23
    }
   ],
   "cell_type": "code",
   "source": [
    "using Statistics\n",
    "\n",
    "mean(r)"
   ],
   "metadata": {},
   "execution_count": 23
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "-0.022756614328553082"
     },
     "metadata": {},
     "execution_count": 24
    }
   ],
   "cell_type": "code",
   "source": [
    "median(r)"
   ],
   "metadata": {},
   "execution_count": 24
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.0079731976488129"
     },
     "metadata": {},
     "execution_count": 25
    }
   ],
   "cell_type": "code",
   "source": [
    "std(r)"
   ],
   "metadata": {},
   "execution_count": 25
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "-2.4473617111205455"
     },
     "metadata": {},
     "execution_count": 26
    }
   ],
   "cell_type": "code",
   "source": [
    "quantile(r, 0.01)"
   ],
   "metadata": {},
   "execution_count": 26
  },
  {
   "cell_type": "markdown",
   "source": [
    "More advanced statistical functions are provided in third-party packages, like\n",
    "[`StatsBase.jl`](https://github.com/JuliaStats/StatsBase.jl)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Rock-paper-scissors: playing with multiple dispatch\n",
    "\n",
    "We can write a simple implementatio of the popular hand game\n",
    "rock-paper-scissors in less than 10 lines of code in Julia.  This\n",
    "implementation will give use the opportunity to have a closer look to one of\n",
    "Julia's main features: [multiple\n",
    "dispatch](https://docs.julialang.org/en/v1/manual/methods/#Methods-1).\n",
    "\n",
    "*Note: this section is based on the blog post [Rock–paper–scissors game in\n",
    "less than 10 lines of\n",
    "code](https://giordano.github.io/blog/2017-11-03-rock-paper-scissors/)*.\n",
    "\n",
    "### The game\n",
    "\n",
    "Here we go:"
   ],
   "metadata": {}
  },
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "abstract type Shape end\n",
    "struct Rock     <: Shape end\n",
    "struct Paper    <: Shape end\n",
    "struct Scissors <: Shape end\n",
    "play(::Type{Paper}, ::Type{Rock}) = \"Paper wins\"\n",
    "play(::Type{Paper}, ::Type{Scissors}) = \"Scissors wins\"\n",
    "play(::Type{Rock}, ::Type{Scissors}) = \"Rock wins\"\n",
    "play(::Type{T}, ::Type{T}) where {T<: Shape} = \"Tie, try again\"\n",
    "play(a::Type{<:Shape}, b::Type{<:Shape}) = play(b, a); # Commutativity"
   ],
   "metadata": {},
   "execution_count": 27
  },
  {
   "cell_type": "markdown",
   "source": [
    "That's all.  Nine lines of code, as promised.\n",
    "\n",
    "### Play the game\n",
    "\n",
    "Now that we’ve implemented the game we can play it in the Julia REPL:"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "\"Scissors wins\""
     },
     "metadata": {},
     "execution_count": 28
    }
   ],
   "cell_type": "code",
   "source": [
    "play(Paper, Scissors)"
   ],
   "metadata": {},
   "execution_count": 28
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "\"Tie, try again\""
     },
     "metadata": {},
     "execution_count": 29
    }
   ],
   "cell_type": "code",
   "source": [
    "play(Rock, Rock)"
   ],
   "metadata": {},
   "execution_count": 29
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "\"Paper wins\""
     },
     "metadata": {},
     "execution_count": 30
    }
   ],
   "cell_type": "code",
   "source": [
    "play(Rock, Paper)"
   ],
   "metadata": {},
   "execution_count": 30
  },
  {
   "cell_type": "markdown",
   "source": [
    "There was no explicit method for the combination of arguments `Rock`-`Paper`,\n",
    "but the commutative rule has been used here."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Data science with Julia\n",
    "\n",
    "### Dataframes\n",
    "\n",
    "[`DataFrames.jl`](https://github.com/JuliaData/DataFrames.jl) provides the\n",
    "tools to work with tabular data.  This section is largely based on [its\n",
    "documentation](http://juliadata.github.io/DataFrames.jl/v0.21/)"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4×2 DataFrames.DataFrame\n│ Row │ A     │ B      │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n├─────┼───────┼────────┤\n│ 1   │ 1     │ M      │\n│ 2   │ 2     │ F      │\n│ 3   │ 3     │ F      │\n│ 4   │ 4     │ M      │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>4 rows × 2 columns</p><tr><th>1</th><td>1</td><td>M</td></tr><tr><th>2</th><td>2</td><td>F</td></tr><tr><th>3</th><td>3</td><td>F</td></tr><tr><th>4</th><td>4</td><td>M</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 31
    }
   ],
   "cell_type": "code",
   "source": [
    "using DataFrames\n",
    "df = DataFrame(A = 1:4, B = [\"M\", \"F\", \"F\", \"M\"])"
   ],
   "metadata": {},
   "execution_count": 31
  },
  {
   "cell_type": "markdown",
   "source": [
    "Columns can be accessed via `df.col`, `df.\"col\"`, `df[:, \"col\"]` and and a few\n",
    "more options not mentioned here for brevity."
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4-element Array{Int64,1}:\n 1\n 2\n 3\n 4"
     },
     "metadata": {},
     "execution_count": 32
    }
   ],
   "cell_type": "code",
   "source": [
    "df.A"
   ],
   "metadata": {},
   "execution_count": 32
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "4-element Array{Int64,1}:\n 1\n 2\n 3\n 4"
     },
     "metadata": {},
     "execution_count": 33
    }
   ],
   "cell_type": "code",
   "source": [
    "df[:, \"A\"]"
   ],
   "metadata": {},
   "execution_count": 33
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "2-element Array{String,1}:\n \"A\"\n \"B\""
     },
     "metadata": {},
     "execution_count": 34
    }
   ],
   "cell_type": "code",
   "source": [
    "names(df)"
   ],
   "metadata": {},
   "execution_count": 34
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### Constructing from another table type\n",
    "\n",
    "`DataFrames.jl` can also interact with other packages:\n",
    "\n",
    "```julia\n",
    "df = DataFrame(a=[1, 2, 3], b=[:a, :b, :c])\n",
    "\n",
    "# write DataFrame out to CSV file\n",
    "CSV.write(\"dataframe.csv\", df)\n",
    "\n",
    "# store DataFrame in an SQLite database table\n",
    "SQLite.load!(df, db, \"dataframe_table\")\n",
    "\n",
    "# transform a DataFrame through Query.jl package\n",
    "df = df |> @map({a=_.a + 1, _.b}) |> DataFrame\n",
    "```\n",
    "\n",
    "#### Working with dataframes"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "20×3 DataFrames.DataFrame\n│ Row │ A     │ B     │ C     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┼───────┤\n│ 1   │ 1     │ 1     │ 1     │\n│ 2   │ 3     │ 1     │ 2     │\n│ 3   │ 5     │ 1     │ 3     │\n│ 4   │ 7     │ 1     │ 4     │\n│ 5   │ 9     │ 1     │ 5     │\n│ 6   │ 11    │ 2     │ 6     │\n│ 7   │ 13    │ 2     │ 7     │\n⋮\n│ 13  │ 25    │ 3     │ 13    │\n│ 14  │ 27    │ 3     │ 14    │\n│ 15  │ 29    │ 3     │ 15    │\n│ 16  │ 31    │ 4     │ 16    │\n│ 17  │ 33    │ 4     │ 17    │\n│ 18  │ 35    │ 4     │ 18    │\n│ 19  │ 37    │ 4     │ 19    │\n│ 20  │ 39    │ 4     │ 20    │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>20 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>1</td></tr><tr><th>2</th><td>3</td><td>1</td><td>2</td></tr><tr><th>3</th><td>5</td><td>1</td><td>3</td></tr><tr><th>4</th><td>7</td><td>1</td><td>4</td></tr><tr><th>5</th><td>9</td><td>1</td><td>5</td></tr><tr><th>6</th><td>11</td><td>2</td><td>6</td></tr><tr><th>7</th><td>13</td><td>2</td><td>7</td></tr><tr><th>8</th><td>15</td><td>2</td><td>8</td></tr><tr><th>9</th><td>17</td><td>2</td><td>9</td></tr><tr><th>10</th><td>19</td><td>2</td><td>10</td></tr><tr><th>11</th><td>21</td><td>3</td><td>11</td></tr><tr><th>12</th><td>23</td><td>3</td><td>12</td></tr><tr><th>13</th><td>25</td><td>3</td><td>13</td></tr><tr><th>14</th><td>27</td><td>3</td><td>14</td></tr><tr><th>15</th><td>29</td><td>3</td><td>15</td></tr><tr><th>16</th><td>31</td><td>4</td><td>16</td></tr><tr><th>17</th><td>33</td><td>4</td><td>17</td></tr><tr><th>18</th><td>35</td><td>4</td><td>18</td></tr><tr><th>19</th><td>37</td><td>4</td><td>19</td></tr><tr><th>20</th><td>39</td><td>4</td><td>20</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 35
    }
   ],
   "cell_type": "code",
   "source": [
    "df = DataFrame(A = 1:2:40, B = repeat(1:4, inner=5), C = 1:20)"
   ],
   "metadata": {},
   "execution_count": 35
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "3×8 DataFrames.DataFrame. Omitted printing of 1 columns\n│ Row │ variable │ mean    │ min   │ median  │ max   │ nunique │ nmissing │\n│     │ \u001b[90mSymbol\u001b[39m   │ \u001b[90mFloat64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mNothing\u001b[39m │ \u001b[90mNothing\u001b[39m  │\n├─────┼──────────┼─────────┼───────┼─────────┼───────┼─────────┼──────────┤\n│ 1   │ A        │ 20.0    │ 1     │ 20.0    │ 39    │         │          │\n│ 2   │ B        │ 2.5     │ 1     │ 2.5     │ 4     │         │          │\n│ 3   │ C        │ 10.5    │ 1     │ 10.5    │ 20    │         │          │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>variable</th><th>mean</th><th>min</th><th>median</th><th>max</th><th>nunique</th><th>nmissing</th><th>eltype</th></tr><tr><th></th><th>Symbol</th><th>Float64</th><th>Int64</th><th>Float64</th><th>Int64</th><th>Nothing</th><th>Nothing</th><th>DataType</th></tr></thead><tbody><p>3 rows × 8 columns</p><tr><th>1</th><td>A</td><td>20.0</td><td>1</td><td>20.0</td><td>39</td><td></td><td></td><td>Int64</td></tr><tr><th>2</th><td>B</td><td>2.5</td><td>1</td><td>2.5</td><td>4</td><td></td><td></td><td>Int64</td></tr><tr><th>3</th><td>C</td><td>10.5</td><td>1</td><td>10.5</td><td>20</td><td></td><td></td><td>Int64</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 36
    }
   ],
   "cell_type": "code",
   "source": [
    "describe(df)"
   ],
   "metadata": {},
   "execution_count": 36
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "6×3 DataFrames.DataFrame\n│ Row │ A     │ B     │ C     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┼───────┤\n│ 1   │ 13    │ 2     │ 7     │\n│ 2   │ 15    │ 2     │ 8     │\n│ 3   │ 17    │ 2     │ 9     │\n│ 4   │ 19    │ 2     │ 10    │\n│ 5   │ 21    │ 3     │ 11    │\n│ 6   │ 23    │ 3     │ 12    │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>6 rows × 3 columns</p><tr><th>1</th><td>13</td><td>2</td><td>7</td></tr><tr><th>2</th><td>15</td><td>2</td><td>8</td></tr><tr><th>3</th><td>17</td><td>2</td><td>9</td></tr><tr><th>4</th><td>19</td><td>2</td><td>10</td></tr><tr><th>5</th><td>21</td><td>3</td><td>11</td></tr><tr><th>6</th><td>23</td><td>3</td><td>12</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 37
    }
   ],
   "cell_type": "code",
   "source": [
    "df[7:12, :]"
   ],
   "metadata": {},
   "execution_count": 37
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "20×2 DataFrames.DataFrame\n│ Row │ A     │ B     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┤\n│ 1   │ 1     │ 1     │\n│ 2   │ 3     │ 1     │\n│ 3   │ 5     │ 1     │\n│ 4   │ 7     │ 1     │\n│ 5   │ 9     │ 1     │\n│ 6   │ 11    │ 2     │\n│ 7   │ 13    │ 2     │\n⋮\n│ 13  │ 25    │ 3     │\n│ 14  │ 27    │ 3     │\n│ 15  │ 29    │ 3     │\n│ 16  │ 31    │ 4     │\n│ 17  │ 33    │ 4     │\n│ 18  │ 35    │ 4     │\n│ 19  │ 37    │ 4     │\n│ 20  │ 39    │ 4     │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>20 rows × 2 columns</p><tr><th>1</th><td>1</td><td>1</td></tr><tr><th>2</th><td>3</td><td>1</td></tr><tr><th>3</th><td>5</td><td>1</td></tr><tr><th>4</th><td>7</td><td>1</td></tr><tr><th>5</th><td>9</td><td>1</td></tr><tr><th>6</th><td>11</td><td>2</td></tr><tr><th>7</th><td>13</td><td>2</td></tr><tr><th>8</th><td>15</td><td>2</td></tr><tr><th>9</th><td>17</td><td>2</td></tr><tr><th>10</th><td>19</td><td>2</td></tr><tr><th>11</th><td>21</td><td>3</td></tr><tr><th>12</th><td>23</td><td>3</td></tr><tr><th>13</th><td>25</td><td>3</td></tr><tr><th>14</th><td>27</td><td>3</td></tr><tr><th>15</th><td>29</td><td>3</td></tr><tr><th>16</th><td>31</td><td>4</td></tr><tr><th>17</th><td>33</td><td>4</td></tr><tr><th>18</th><td>35</td><td>4</td></tr><tr><th>19</th><td>37</td><td>4</td></tr><tr><th>20</th><td>39</td><td>4</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 38
    }
   ],
   "cell_type": "code",
   "source": [
    "df[:, [\"A\", \"B\"]]"
   ],
   "metadata": {},
   "execution_count": 38
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "5×2 DataFrames.DataFrame\n│ Row │ A     │ C     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┤\n│ 1   │ 11    │ 6     │\n│ 2   │ 13    │ 7     │\n│ 3   │ 15    │ 8     │\n│ 4   │ 17    │ 9     │\n│ 5   │ 19    │ 10    │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>5 rows × 2 columns</p><tr><th>1</th><td>11</td><td>6</td></tr><tr><th>2</th><td>13</td><td>7</td></tr><tr><th>3</th><td>15</td><td>8</td></tr><tr><th>4</th><td>17</td><td>9</td></tr><tr><th>5</th><td>19</td><td>10</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 39
    }
   ],
   "cell_type": "code",
   "source": [
    "df[6:10, [\"A\", \"C\"]]"
   ],
   "metadata": {},
   "execution_count": 39
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "10×3 DataFrames.DataFrame\n│ Row │ A     │ B     │ C     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┼───────┤\n│ 1   │ 11    │ 2     │ 6     │\n│ 2   │ 13    │ 2     │ 7     │\n│ 3   │ 15    │ 2     │ 8     │\n│ 4   │ 17    │ 2     │ 9     │\n│ 5   │ 19    │ 2     │ 10    │\n│ 6   │ 31    │ 4     │ 16    │\n│ 7   │ 33    │ 4     │ 17    │\n│ 8   │ 35    │ 4     │ 18    │\n│ 9   │ 37    │ 4     │ 19    │\n│ 10  │ 39    │ 4     │ 20    │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>10 rows × 3 columns</p><tr><th>1</th><td>11</td><td>2</td><td>6</td></tr><tr><th>2</th><td>13</td><td>2</td><td>7</td></tr><tr><th>3</th><td>15</td><td>2</td><td>8</td></tr><tr><th>4</th><td>17</td><td>2</td><td>9</td></tr><tr><th>5</th><td>19</td><td>2</td><td>10</td></tr><tr><th>6</th><td>31</td><td>4</td><td>16</td></tr><tr><th>7</th><td>33</td><td>4</td><td>17</td></tr><tr><th>8</th><td>35</td><td>4</td><td>18</td></tr><tr><th>9</th><td>37</td><td>4</td><td>19</td></tr><tr><th>10</th><td>39</td><td>4</td><td>20</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 40
    }
   ],
   "cell_type": "code",
   "source": [
    "df[iseven.(df.B), :]"
   ],
   "metadata": {},
   "execution_count": 40
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "10×3 DataFrames.DataFrame\n│ Row │ A     │ B     │ C     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┼───────┤\n│ 1   │ 21    │ 3     │ 11    │\n│ 2   │ 23    │ 3     │ 12    │\n│ 3   │ 25    │ 3     │ 13    │\n│ 4   │ 27    │ 3     │ 14    │\n│ 5   │ 29    │ 3     │ 15    │\n│ 6   │ 31    │ 4     │ 16    │\n│ 7   │ 33    │ 4     │ 17    │\n│ 8   │ 35    │ 4     │ 18    │\n│ 9   │ 37    │ 4     │ 19    │\n│ 10  │ 39    │ 4     │ 20    │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>10 rows × 3 columns</p><tr><th>1</th><td>21</td><td>3</td><td>11</td></tr><tr><th>2</th><td>23</td><td>3</td><td>12</td></tr><tr><th>3</th><td>25</td><td>3</td><td>13</td></tr><tr><th>4</th><td>27</td><td>3</td><td>14</td></tr><tr><th>5</th><td>29</td><td>3</td><td>15</td></tr><tr><th>6</th><td>31</td><td>4</td><td>16</td></tr><tr><th>7</th><td>33</td><td>4</td><td>17</td></tr><tr><th>8</th><td>35</td><td>4</td><td>18</td></tr><tr><th>9</th><td>37</td><td>4</td><td>19</td></tr><tr><th>10</th><td>39</td><td>4</td><td>20</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 41
    }
   ],
   "cell_type": "code",
   "source": [
    "df[df.A .> 20, :]"
   ],
   "metadata": {},
   "execution_count": 41
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "5×3 DataFrames.DataFrame\n│ Row │ A     │ B     │ C     │\n│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n├─────┼───────┼───────┼───────┤\n│ 1   │ 31    │ 4     │ 16    │\n│ 2   │ 33    │ 4     │ 17    │\n│ 3   │ 35    │ 4     │ 18    │\n│ 4   │ 37    │ 4     │ 19    │\n│ 5   │ 39    │ 4     │ 20    │",
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>5 rows × 3 columns</p><tr><th>1</th><td>31</td><td>4</td><td>16</td></tr><tr><th>2</th><td>33</td><td>4</td><td>17</td></tr><tr><th>3</th><td>35</td><td>4</td><td>18</td></tr><tr><th>4</th><td>37</td><td>4</td><td>19</td></tr><tr><th>5</th><td>39</td><td>4</td><td>20</td></tr></tbody></table>"
      ]
     },
     "metadata": {},
     "execution_count": 42
    }
   ],
   "cell_type": "code",
   "source": [
    "df[(df.A .> 20) .& (iseven.(df.B)), :]"
   ],
   "metadata": {},
   "execution_count": 42
  },
  {
   "cell_type": "markdown",
   "source": [
    "This is only the tip of what you can do with `DataFrames.jl`.  For a more\n",
    "in-depth guide, besides the official documentation of the package check out\n",
    "this [tutorial on\n",
    "DataFrames](https://github.com/bkamins/Julia-DataFrames-Tutorial)\n",
    "\n",
    "An alternative stack to deal with tabular data is the\n",
    "[Queryverse](https://www.queryverse.org/).\n",
    "\n",
    "#### Dataframes and CSV files\n",
    "\n",
    "For reading and writing tabular data from CSV and other delimited text files,\n",
    "use the [`CSV.jl`](https://github.com/JuliaData/CSV.jl) package.\n",
    "\n",
    "```julia\n",
    "using CSV\n",
    "# Read a dataframe from a CSV file\n",
    "df = DataFrame(CSV.File(input))\n",
    "# Create a dataframe to be written to a CSV file\n",
    "df = DataFrame(x = 1, y = 2)\n",
    "CSV.write(output, df)\n",
    "```\n",
    "\n",
    "### Databases\n",
    "\n",
    "There are a few packages to access different flavours of databases:\n",
    "\n",
    "* [`MySQL.jl`](https://github.com/JuliaDatabases/MySQL.jl),\n",
    "* [`ODBC.jl`](https://github.com/JuliaDatabases/ODBC.jl),\n",
    "* [`SQLite.jl`](https://github.com/JuliaDatabases/SQLite.jl),\n",
    "* [`JDBC.jl`](https://github.com/JuliaDatabases/JDBC.jl),\n",
    "* [`LibPQ.jl`](https://github.com/invenia/LibPQ.jl)."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Coming from other languages?\n",
    "\n",
    "Julia has got you covered!  It has a good interoperability with other\n",
    "programming languages, which makes it easier to move to Julia from other\n",
    "languages but still keep using your current toolbox.\n",
    "\n",
    "*Note: all languages are different from each other and of course Julia is no\n",
    "special, so make sure to read the [noteworthy differences between Julia and\n",
    "the other\n",
    "languages](https://docs.julialang.org/en/v1/manual/noteworthy-differences/)*.\n",
    "\n",
    "### C/Fortran\n",
    "\n",
    "If you are C or Fortran programmer, calling function in a C/Fortran libraries\n",
    "(in particular, the libraries should be [\"shared\n",
    "libraries\"](https://en.wikipedia.org/wiki/Library_(computing)#Shared_libraries))\n",
    "is as easy as running the\n",
    "[`ccall`](https://docs.julialang.org/en/v1/base/c/#ccall) function, without\n",
    "writing wrapper code.  For example, with this command we can run the function\n",
    "`getenv` in the standard C library"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Cstring(0x00000000013d5376)"
     },
     "metadata": {},
     "execution_count": 43
    }
   ],
   "cell_type": "code",
   "source": [
    "path = ccall(:getenv, Cstring, (Cstring,), \"SHELL\")"
   ],
   "metadata": {},
   "execution_count": 43
  },
  {
   "cell_type": "markdown",
   "source": [
    "As you may expect, dealing with C is rather low-level.  The output is quite\n",
    "unreadable for a human being, but can be converted to a standard string with"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "\"/bin/bash\""
     },
     "metadata": {},
     "execution_count": 44
    }
   ],
   "cell_type": "code",
   "source": [
    "unsafe_string(path)"
   ],
   "metadata": {},
   "execution_count": 44
  },
  {
   "cell_type": "markdown",
   "source": [
    "I will not indulge more into interfacing C libraries, but those interested can\n",
    "read more in the [Calling C and Fortran\n",
    "Code](https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/)\n",
    "section of the manual.\n",
    "\n",
    "### Python and R\n",
    "\n",
    "Calling Python libraries is made simple by the\n",
    "[`PyCall.jl`](https://github.com/JuliaPy/PyCall.jl) package.  Simple packages\n",
    "requires little or no effort, The classes in the Python standard library or in\n",
    "the popular NumPy package are naturally translated to their equivalent Julia\n",
    "data types, to make the experience as seamless as possible:"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "0.0"
     },
     "metadata": {},
     "execution_count": 45
    }
   ],
   "cell_type": "code",
   "source": [
    "using PyCall\n",
    "\n",
    "const math = pyimport(\"math\")\n",
    "const np = pyimport(\"numpy\")\n",
    "\n",
    "math.sin(pi) - sin(math.pi)"
   ],
   "metadata": {},
   "execution_count": 45
  },
  {
   "cell_type": "markdown",
   "source": [
    "Here we have mixed Python and Julia functions and variables: Python `math.sin`\n",
    "and `math.pi` with `sin` and `pi` from Julia.  The syntax used by PyCall\n",
    "closely resembles that of Python.\n",
    "\n",
    "Non-standard Python packages can also be called with PyCall and they mostly\n",
    "work without extra work, but you may encounter some quirks.  For example, we\n",
    "can play a bit with [`pandas`](https://pandas.pydata.org/):"
   ],
   "metadata": {}
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "PyObject    age   name\n0   27  James\n1   29   Jill\n2   27   Jake",
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>27</td>\n",
       "      <td>James</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>29</td>\n",
       "      <td>Jill</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>27</td>\n",
       "      <td>Jake</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ]
     },
     "metadata": {},
     "execution_count": 46
    }
   ],
   "cell_type": "code",
   "source": [
    "const pd = pyimport(\"pandas\")\n",
    "\n",
    "df = pd.DataFrame(Dict(:age=>[27, 29, 27], :name=>[\"James\", \"Jill\", \"Jake\"]))"
   ],
   "metadata": {},
   "execution_count": 46
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "PyObject              age\ncount   3.000000\nmean   27.666667\nstd     1.154701\nmin    27.000000\n25%    27.000000\n50%    27.000000\n75%    28.000000\nmax    29.000000",
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>27.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.154701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>27.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>27.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>27.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>28.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>29.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ]
     },
     "metadata": {},
     "execution_count": 47
    }
   ],
   "cell_type": "code",
   "source": [
    "df.describe()"
   ],
   "metadata": {},
   "execution_count": 47
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "PyObject 0    27\n1    29\n2    27\nName: age, dtype: int64"
     },
     "metadata": {},
     "execution_count": 48
    }
   ],
   "cell_type": "code",
   "source": [
    "df.age"
   ],
   "metadata": {},
   "execution_count": 48
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can note in this case that the custom Python class used by `pandas` shows\n",
    "here an annoying `PyObject` text in the output.  If you want to use a more\n",
    "integrated interface to `pandas`, consider the package\n",
    "[`Pandas.jl`](https://github.com/JuliaPy/Pandas.jl).\n",
    "\n",
    "To call R package, check out\n",
    "[`RCall.jl`](https://github.com/JuliaInterop/RCall.jl).\n",
    "\n",
    "![RCall example](https://github.com/JuliaInterop/RCall.jl/raw/c5d8de4145e2b3b3cf2a35502c0a80590209254e/ggplot.png \"RCall example\")"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## What about plotting?\n",
    "\n",
    "That's a great question!  Plotting is not a field where Julia shines\n",
    "currently.  There are several nice plotting packages, with many different\n",
    "features, but a common problem is compilation latency, leading to coining the\n",
    "term \"time to first plot\".  I will list here some of the options:\n",
    "\n",
    "* [`PyPlot.jl`](https://github.com/JuliaPy/PyPlot.jl): interface to Python's\n",
    "  Matplotlib, this is reasonably quick\n",
    "* [`Plots.jl`](https://github.com/JuliaPlots/Plots.jl): this is one of the\n",
    "  most popular plotting packages in Julia, as it can use to many different\n",
    "  plotting backends always using the same interface.  It is also extensible:\n",
    "  it allows to define \"recipes\" for custom Julia types.  On the other end, it\n",
    "  has a large compilation latency.  After loading the package, the first plot\n",
    "  in a session can take ~20-30 seconds, but this is improving!\n",
    "* [`Makie.jl`](https://github.com/JuliaPlots/Makie.jl): it aims to replace\n",
    "  `Plots.jl` in the future, adding many more interactive and 3D features, but\n",
    "  it has similar issues in terms of initial plot\n",
    "* [`Gadfly.jl`](https://github.com/GiovineItalia/Gadfly.jl): heavily inspired\n",
    "  by \"The Grammar of Graphics\" book.  Kind of quick\n",
    "* [`UnicodePlots.jl`](https://github.com/Evizero/UnicodePlots.jl): simple\n",
    "  plots directly in terminal.  Super fast!"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## More advanced topics\n",
    "\n",
    "So far we have relatively simple applications of Julia, but if you are\n",
    "interested you can have a look to more advanced ones.  Even though the\n",
    "environment is young when compared with established languages like Python and\n",
    "R, many high-quality packages are being used to do avanced scientific\n",
    "research.  Here is a summary of some of the most popular ones, classified by\n",
    "topic:\n",
    "\n",
    "* Automatic differentiation:\n",
    "  [`ChainRules.jl`](https://github.com/JuliaDiff/ChainRules.jl),\n",
    "  [`ForwardDiff.jl`](https://github.com/JuliaDiff/ForwardDiff.jl),\n",
    "  [`ReverseDiff.jl`](https://github.com/JuliaDiff/ReverseDiff.jl),\n",
    "  [`Zygote.jl`](https://github.com/FluxML/Zygote.jl)\n",
    "* Differential equations:\n",
    "  [`DifferentialEquations.jl`](https://github.com/SciML/DifferentialEquations.jl)\n",
    "  and many other packages in the [`SciML`](https://github.com/SciML)\n",
    "  organisation on GitHub\n",
    "* GPU programming: [`CUDA.jl`](https://github.com/JuliaGPU/CUDA.jl) for NVidia\n",
    "  GPUs, [`AMDGPUnative.jl`](https://github.com/JuliaGPU/AMDGPUnative.jl) for\n",
    "  AMD ones\n",
    "* Machine learning: [`Flux.jl`](https://github.com/FluxML/Flux.jl),\n",
    "  [`Knet.jl`](https://github.com/denizyuret/Knet.jl),\n",
    "  [`MLJ.jl`](https://github.com/alan-turing-institute/MLJ.jl)\n",
    "* Mathematical optimisation: [`JuMP.jl`](https://github.com/jump-dev/JuMP.jl)\n",
    "* Probabilistic programming: [`Gen.jl`](https://github.com/probcomp/Gen.jl),\n",
    "  [`Turing.jl`](https://github.com/TuringLang/Turing.jl)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Beyond scientific computing\n",
    "\n",
    "Julia is mainly employed for scientific computing, but more and more users are\n",
    "realising that its syntax and packages are equally good for general\n",
    "programming, too. The [Administrative Scripting with\n",
    "Julia](https://github.com/ninjaaron/administrative-scripting-with-julia#id4)\n",
    "repository is a guide about how to use Julia for system administration.  This\n",
    "also rightly points out the shortcomings of Julia in this field: its\n",
    "JIT-compilation model favours launching a single long-running script, rather\n",
    "than many small ones: the latter case can be exceedingly slow as compilation\n",
    "of each script may take *much* longer than actually running them.  Many ways\n",
    "to tackle the issue of the compilation latency are currently being worked out."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Environments and reproducibility\n",
    "\n",
    "![Reproducibility](./figures/replication.jpg \"Reproducibility\")\n",
    "\n",
    "A great attention is dedicated to reproducibility in the Julia community.  The\n",
    "built-in package manager, `Pkg.jl`, allows you to create environments, which\n",
    "are controlled by two files:\n",
    "\n",
    "* `Project.toml`: it records the list of the packages required in the\n",
    "  environment — the direct dependencies — and you can also optioanlly specify\n",
    "  the compatibility bounds of these packages, according to [Semantic\n",
    "  Versioning](https://semver.org/)\n",
    "* `Manifest.toml`: it takes a snapshot al *all* packages in the environment,\n",
    "  with the exact version number of all packages — both direct and indirect\n",
    "  dependencies of the environment.\n",
    "\n",
    "When compatibility bounds are properly used in the `Project.toml` file, it\n",
    "lets you instantiate at any point in the future a \"compatible\" environment —\n",
    "\"compatible\" in the SemVer sense.  The `Manifest.toml`, instead, lets you\n",
    "recreate exactly the same environment.\n",
    "\n",
    "In its top-level, this repository provides an example of\n",
    "[`Project.toml`](./Project.toml) with the full specification of compatible\n",
    "versions for all non-standard libraries.  The accompanying\n",
    "[`Manifest.toml`](./Manifest.toml) lists all packages in the environment.\n",
    "This, for example, allows [binder](https://mybinder.org/) to generate a\n",
    "container with the same set of packages as me.\n",
    "\n",
    "You can read more about [working with\n",
    "environments](https://julialang.github.io/Pkg.jl/v1/environments/) in the\n",
    "documentation of `Pkg.jl`.\n",
    "\n",
    "Also binary package or data can become a \"package\" with the [artifact\n",
    "system](https://julialang.org/blog/2019/11/artifacts/): you can add an\n",
    "\"artifact\", with a version, as dependency to your package or environment to\n",
    "ensure full reproducibility."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Notebooks and literate programming\n",
    "\n",
    "Julia is fully supported by the popular Jupyter notebooks.  In fact, the \"Ju\"\n",
    "in its names stands for \"Julia\", alongside with Python and R.  Julia kernels\n",
    "for Jupyter notebooks are provided by the\n",
    "[`IJulia.jl`](https://github.com/JuliaLang/IJulia.jl) package.\n",
    "\n",
    "[`Pluto.jl`](https://github.com/fonsp/Pluto.jl) is a lightweight alternative\n",
    "to Jupyter notebooks.  Pluto notebooks are reactive, which means that the\n",
    "values of the cells don't depend on their order of execution, which is a\n",
    "common gripe in Jupyter notebooks.\n",
    "\n",
    "![Pluto notebook](https://raw.githubusercontent.com/fonsp/Pluto.jl/580ab811f13d565cc81ebfa70ed36c84b125f55d/demo/plutodemo.gif \"Pluto notebook\")\n",
    "\n",
    "There are different tools for [literate\n",
    "programming](https://en.wikipedia.org/wiki/Literate_programming) in Julia.\n",
    "This tutorial was created with\n",
    "[`Literate.jl`](https://github.com/fredrikekre/Literate.jl), which allows you\n",
    "to generate a Markdown file and a Jupyter noetbook from a simple commented\n",
    "Julia script.  Another option is\n",
    "[`Weave.jl`](https://github.com/JunoLab/Weave.jl).  Similar to Pweave, knitr,\n",
    "R Markdown, and Sweave, `Weave.jl` supports multiple input and output formats,\n",
    "including HTML and PDF."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Keep in touch\n",
    "\n",
    "There are several ways to engage the Julia community online:\n",
    "\n",
    "* [Julia Discourse](https://discourse.julialang.org/), the discussion forum\n",
    "* [Slack workspace](https://slackinvite.julialang.org/)\n",
    "* [Zulip](https://julialang.zulipchat.com/register/)\n",
    "* [`#JuliaLang`](https://twitter.com/hashtag/JuliaLang) hashtag on Twitter\n",
    "* [JuliaLang](https://www.youtube.com/user/JuliaLanguage) channel on YouTube,\n",
    "  where many tutorials and most JuliaCon talks are collected\n",
    "* Register for free to watch online [JuliaCon\n",
    "  2020](https://juliacon.org/2020/)!\n",
    "\n",
    "You can find more resources in the [community\n",
    "page](https://julialang.org/community/) of the official website."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "---\n",
    "\n",
    "*This notebook was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*"
   ],
   "metadata": {}
  }
 ],
 "nbformat_minor": 3,
 "metadata": {
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.4.2"
  },
  "kernelspec": {
   "name": "julia-1.4",
   "display_name": "Julia 1.4.2",
   "language": "julia"
  }
 },
 "nbformat": 4
}