13 Julia-Python - 01 Accelerating Python with PyJulia.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Binder](https://mybinder.org/badge_logo.svg)](https://notebooks.gesis.org/binder/v2/gh/jolin-io/fall-in-love-with-julia/main?filepath=13%20Julia-Python20-%2001%20Accelerating%20Python%20with%20PyJulia.ipynb)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://www.jolin.io\" target=\"_blank\" rel=\"noreferrer noopener\">\n",
    "<img src=\"https://www.jolin.io/assets/Jolin/Jolin-Banner-Website-v1.3-darkmode.webp\">\n",
    "</a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fall-in-love-with-Julia: Accelerating Python 101\n",
    "\n",
    "For Julia Python interactions there are two packages:\n",
    "\n",
    "|    | [PyCall.jl](https://github.com/JuliaPy/PyCall.jl) | [PythonCall.jl](https://github.com/cjdoris/PythonCall.jl) |\n",
    "| -: | --------- | ------------- |\n",
    "| pypi | with [`PyJulia`](https://github.com/JuliaPy/pyjulia) python package (simply called `julia` on pypi) | with [`JuliaCall`](https://pypi.org/project/juliacall/) python package |\n",
    "| conversions | automatically converts between native types | no auto-conversion, just wrapping |\n",
    "| dependencies | Global package management via `Conda.jl` | Project-separated package management via `CondaPkg.jl` |\n",
    "| run python | use `py\"...\"` | use `@pyexec \"...\"` |\n",
    "\n",
    "Outline\n",
    "- **[01](https://notebooks.gesis.org/binder/v2/gh/jolin-io/fall-in-love-with-julia/main?filepath=13%20Julia-Python20-%2001%20Accelerating%20Python%20with%20PyJulia.ipynb) first notebook shows how to use PyJulia**\n",
    "- [02](https://notebooks.gesis.org/binder/v2/gh/jolin-io/fall-in-love-with-julia/main?filepath=13%20Julia-Python20-%2002%20Accelerating%20Python%20with%20JuliaCall.ipynb) second notebook is about JuliaCall\n",
    "- [03](https://notebooks.gesis.org/binder/v2/gh/jolin-io/fall-in-love-with-julia/main?filepath=13%20Julia-Python20-%2003%20Use%20Python%20with%20PyCall%20and%20PythonCall.ipynb) using Python from Julia with both PyCall and PythonCall\n",
    "- [special extra](https://mybinder.org/v2/gh/jolin-io/workshop-accelerate-Python-with-Julia/main?filepath=03-example-cython-vs-cpp-vs-julia.ipynb) - Julia vs C++"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyJulia\n",
    "\n",
    "Let's look at the basics how to call Julia from Python with PyJulia. Note that this Jupyter has a **Python kernel**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from julia.api import Julia\n",
    "_jl = Julia(compiled_modules=False)\n",
    "%load_ext julia.magic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "array = np.arange(10)\n",
    "array"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`PyJulia` does autoconversions for you"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%julia\n",
    "println(typeof($array))  # caution! @show does not work because of the special handling of interpolation $\n",
    "result = isodd.($array)\n",
    "println(typeof(result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = %julia isodd.($array)\n",
    "type(result)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is especially handy when working with `True`/`False`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mycondition = (array == 40).any()\n",
    "mycondition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%julia\n",
    "if $mycondition\n",
    "    println(\"yes! works\")\n",
    "else\n",
    "    println(\"also works :)! (we only want to test that julia actually gets a julia Bool value)\")\n",
    "end"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DataFrames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = {\n",
    "  \"calories\": [420, 380, 390],\n",
    "  \"duration\": [50, 40, 45]\n",
    "}\n",
    "df = pd.DataFrame(data, index = [\"day1\", \"day2\", \"day3\"])\n",
    "df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For PyCall.jl there is not much default support for converting a Pandas Dataframe to a Julia Dataframe. Thankfully there is a separate package, compatible with PyCall, which wraps Pandas within Julia."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%julia\n",
    "import Pandas\n",
    "using DataFrames\n",
    "\n",
    "DataFrame(Pandas.DataFrame($df))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The other way around works analogously"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%julia\n",
    "df2 = DataFrame(grp=repeat(1:2, 3), x=6:-1:1, y=4:9, z=[3:7; missing], id='a':'f')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "jdf2 = %julia df2\n",
    "df2 = %julia Pandas.DataFrame(df2)\n",
    "type(jdf2), type(df2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df2"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more comparisons between Pandas and Julia Dataframe see https://dataframes.juliadata.org/stable/man/comparisons/"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to do it without %julia magic?\n",
    "\n",
    "The julia's Main module allows you to access almost everything from Julia programmatically. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from julia import Main as jl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "jl.eval(\"\"\"\n",
    "broadcast_isodd(a) = isodd.(a)\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "jl.broadcast_isodd(array)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Caution! However be cautious that autoconversion, while handy, can also become tricky.\n",
    "\n",
    "While this works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%julia DataFrame(Pandas.DataFrame($df))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This does not work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from julia import Pandas\n",
    "jl.DataFrame(Pandas.DataFrame(df))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a fallback you always have to define a custom julia function which you then can pass python objects to.\n",
    "\n",
    "(Unfortunately Python does not have an interpolation syntax outside Jupyter %julia magics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "jl.eval(\"\"\"\n",
    "pdf2jdf(df) = DataFrame(Pandas.DataFrame(df))\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "jl.pdf2jdf(df)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next: [02 JuliaCall](https://notebooks.gesis.org/binder/v2/gh/jolin-io/fall-in-love-with-julia/main?filepath=13%20Julia-Python20-%2002%20Accelerating%20Python%20with%20JuliaCall.ipynb)\n",
    "\n",
    "\n",
    "For questions or suggestions please contact me at stephan.sahm@jolin.io\n",
    "\n",
    "<a href=\"https://www.jolin.io\" target=\"_blank\" rel=\"noreferrer noopener\">\n",
    "<img src=\"https://www.jolin.io/assets/Jolin/Jolin-Banner-Website-v1.3-darkmode.webp\">\n",
    "</a>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}