gps-art.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>From Image to GPS Art</h1>\n",
    "\n",
    "This jupyter notebook demonstrates how to take an image and recreate it as a GPS route for use in navigating GPS art. \n",
    "\n",
    "I feel I ought to start out by stating that this is by no means a tutorial on the best practices of coding and my knowledge of all the components used vary - I am relatively new to several of the elements used here. Instead, treat this notebook as the (slightly) tidied up scribblings and workings as I set about the problem.\n",
    "\n",
    "Not all parts are fully automated yet, but the intention is to revist this problem in my free time to better scale, translate and rotate the image to best suit the location.\n",
    "\n",
    "<h2>Setup and prerequisites</h2>\n",
    "\n",
    "From here on in, I am assuming you are competent enough in the python language and understand how to get all the required libaries etc. to make this notebook function.\n",
    "\n",
    "However, if you don't think you are - you can still play with the code: just click in the boxes with the code and push `shift` + `enter` and the code in that box should run (you will need to do it in the sequence it is written).\n",
    "\n",
    "Firstly, the image that you wish to recreate needs to be selected. Here, I have chosen the GRGR 2020 logo as shown in the figure below (this whole notebook came about because of The Ginger Runner's Global Run (GRGR - https://www.rungrgr.com)). \n",
    "\n",
    "<img src=\"./img/GRGR2020_ColorLogo.png\" alt=\"GRGR Logo\" width=\"200\"/>\n",
    "\n",
    "If you want some back story here it is (if not skip to the next paragraph!): I ran the GRGR '19 logo as my  <a href=\"https://www.strava.com/activities/2786773404\">attempt</a> for the GPS art award (which was a 2 hour event). To create the logo, I hand drew the logo in Suunto's Movescount map - it was a one-shot draw and then serveral iterations as I compared the screenshots to a semitransparent copy of the logo and the tweaked accordingly. It was labourious. When GRGR2020 was announced as a week long event full of surprise challenges I knew I needed to have another go at GPS art and I needed a better way to create it! And that, my friends, is how this notebook came in to existance. Right, back to the workings... \n",
    "\n",
    "It is important to note that the image should be fairly easy to recreate as a continuous-line drawing and therefore will be easy to transfer to a GPS route without loosing the image's detail. If you wish to recreate a more complex image (for example, a cartoon or a photo) then you will need to dig deep into your creative pocket to work out how much detail you want to recreate as lines.\n",
    "\n",
    "To make the continous-line image, the original image was opened in Inkscape (an open-source graphics editing tool https://inkscape.org/) and a route was traced over the top of the image (using the 'Draw Bezier curves and straight lines' tool) so that all the lines could be linked continuously - note: only use straight lines (as this is needed for the next step. In this image's case it required several repeated sections, which ideally need to be minimised for ease of running (and also the distance will be minimised!). The resulting SVG file can be seen below.\n",
    "\n",
    "<img src=\"./img/grgr2020.svg\" alt=\"GRGR route\" width=\"200\"/>\n",
    "\n",
    "From here, the SVG was saved as a DXF (a file format associated with drawing and CAD) using Inkscape. Some key points should be observed here:\n",
    " <ol>\n",
    "  <li>The width of the line drawing should be set to a sensible value for scaling (I resized the line drawings to be 100 mm wide throughout all my designs)</li>\n",
    "  <li>Remove all layers so that the SVG consists of just a path (this should guarantee the LWPolyline will be in Layer 0). This can be done by firstly cutting the line-drawing and deleting everything else in the document, then in the 'Objects' tab removing any layers (highlight it and click the '-' sign at the bottom) and then paste and centre your drawing.</li>\n",
    "    <li>The document should be resized to the path object (this allow the centre of the drawing to found in the code).</li>\n",
    "</ol>\n",
    "\n",
    "Once this is done and you have your saved DXF file you are good to start working through this notebook.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# From image to GPS route\n",
    "\n",
    "## The Code\n",
    "\n",
    "### Import the DXF\n",
    "\n",
    "From here on in I will walk you through my code and attempt to give an explaination of what is happening. We start by importing all the required python libaries for this task (some might be obselete!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import geopandas as gpd\n",
    "import numpy as np\n",
    "import math\n",
    "import dxfgrabber\n",
    "import folium\n",
    "import cmath as cm\n",
    "from shapely.geometry import LineString\n",
    "from geopandas import GeoDataFrame\n",
    "from operator import itemgetter\n",
    "from pyproj import CRS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next up is reading in the DXF file (either the example I have provided or one you have just created) and adding some variables we are going to call upon later.\n",
    "\n",
    "Now, this single code box is the section you want to play with if you want to change the size, location or even route. Everything else from here just processes it (and displays it as we go)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "########### FOR YOU TO PLAY WITH ################\n",
    "\n",
    "# Read the DXF file from it's location in the repository:\n",
    "grgr_dxf = dxfgrabber.readfile(\"./img/grgr2020.dxf\") # another example provided is \"./img/donkey.dxf\"\n",
    "\n",
    "# set a target output file and filename:\n",
    "grgr_gpx = \"./out/grgr2020.gpx\" # if you change the input file, don't forget to change the output name\n",
    "\n",
    "# set the width of the GPS drawing (scale * width):\n",
    "scale = 10 # use this value to change the size of the route\n",
    "# desired lat long centre point:\n",
    "centre_latlon = [50.9615, -1.6034] # be sure to change this to the centre of your local canvas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we manipulate the data to collect the vertices of the route:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get vertices from dxf file:\n",
    "layer = [entity for entity in grgr_dxf.entities if entity.layer =='0']\n",
    "grgr_vertices = [entity.points for entity in layer if entity.dxftype == 'LWPOLYLINE']\n",
    "\n",
    "# get x and y offset values to find the centre of the drawing:\n",
    "offset_x = max(grgr_vertices[0],key=itemgetter(0))[0] / 2.0\n",
    "offset_y = max(grgr_vertices[0],key=itemgetter(1))[1] / 2.0\n",
    "\n",
    "#create array to collect scaled and centred vertices:\n",
    "grgr_vertices_centred = []\n",
    "for i in range(len(grgr_vertices[0])):\n",
    "    grgr_vertices_centred.append(tuple(np.multiply(np.subtract(grgr_vertices[0][i], (offset_x, offset_y, 0)), scale)))\n",
    "\n",
    "# Drop the z-value to be left with x & y values:\n",
    "grgr_xy = [(x, y) for x, y, z in grgr_vertices_centred]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualise the route in coordinate view\n",
    "\n",
    "To visualise what we have done we can plot the centred and scaled vertices using the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(offset_x/5, offset_y/5))\n",
    "plt.plot(*zip(*grgr_xy))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Translate to GPS coordinates\n",
    "\n",
    "Now we have the vertices in a useable form, we can convert them to latitude and longitude points using the following functions: `bearing_rad`, `getlonlat` which as the names suggest, gets a bearing and gets the longitude and latitude (using the great arc method - check this <a href=\"https://www.movable-type.co.uk/scripts/latlong.html#dest-point\">link</a> out for more details) respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bearing_rad(x, y):\n",
    "    \n",
    "    # Use imaginary numbers to get the bearing to the point:\n",
    "    z = x + y*1j\n",
    "    res = math.pi / 2.0 - cm.phase(z)\n",
    "    \n",
    "    # Return the resulting angle:\n",
    "    return res % (2.0 * math.pi)\n",
    "\n",
    "def getlonlat(lonlat1, xy1, xy2):\n",
    "    \n",
    "    # Radius of the Earth in meters:\n",
    "    R = 6371000\n",
    "    \n",
    "    # Start point:\n",
    "    lon1 = math.radians(lonlat1[0])\n",
    "    lat1 = math.radians(lonlat1[1])\n",
    "    \n",
    "    # Vertices of each point used to get the bearing:\n",
    "    x1 = xy1[0]\n",
    "    y1 = xy1[1]\n",
    "    \n",
    "    x2 = xy2[0]\n",
    "    y2 = xy2[1]\n",
    "    \n",
    "    # Get the bearing between the points:\n",
    "    bearing = bearing_rad((x2-x1), (y2-y1))\n",
    "    \n",
    "    # Get the distance between the points:\n",
    "    distance = math.sqrt(((x2-x1)**2 + (y2-y1)**2))\n",
    "    \n",
    "    # Calculate the latitude and longitude of the end point:\n",
    "    #φ2 = asin( sin φ1 ⋅ cos δ + cos φ1 ⋅ sin δ ⋅ cos θ )\n",
    "    #λ2 = λ1 + atan2( sin θ ⋅ sin δ ⋅ cos φ1, cos δ − sin φ1 ⋅ sin φ2 )\n",
    "    \n",
    "    lat2 = math.asin(math.sin(lat1)*math.cos(distance/R) + math.cos(lat1)*math.sin(distance/R)*math.cos(bearing))\n",
    "    \n",
    "    lon2 = lon1 + math.atan2(math.sin(bearing)*math.sin(distance/R)*math.cos(lat1),\n",
    "                             math.cos(distance/R) - math.sin(lat1)*math.sin(lat2))\n",
    "    \n",
    "    # Convert to the format needed for future use:\n",
    "    lonlat2 = (math.degrees(lon2), math.degrees(lat2))\n",
    "    \n",
    "    # Return point 2's longitude and latitude:\n",
    "    return lonlat2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we just use those new functions on the list of x, y coordinates (using the starting centre point we defined earlier to prime the first latitude and longitude):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create empty array to collect longitudes and latitudes:\n",
    "grgr_lonlat = []\n",
    "\n",
    "# Append the first point into the array:\n",
    "grgr_lonlat.append(getlonlat((centre_latlon[1], centre_latlon[0]), (0,0), grgr_xy[0]))\n",
    "\n",
    "# Cycle through the remaining points:\n",
    "for i in range(len(grgr_xy)-1):\n",
    "    grgr_lonlat.append(getlonlat(grgr_lonlat[i], grgr_xy[i], grgr_xy[i+1]))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this we can now form our geodataframe with a linestring (which is the needed format for creating a GPX route file):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a linestring from the longitude and latitude gathered above:\n",
    "grgr_line = LineString(grgr_lonlat)\n",
    "\n",
    "# Create a pandas dataframe:\n",
    "data = {'geometry': [grgr_line]}\n",
    "grgr_line_df = pd.DataFrame(data, columns = ['id', 'geometry'])\n",
    "\n",
    "# Create the GeoDataFrame from the above:\n",
    "grgr_route = GeoDataFrame(grgr_line_df, crs=\"epsg:4326\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualise in GPS coordinate view\n",
    "\n",
    "Lastly we can plot it on a map to see how it fits in the surrounding area (this is where having a large open space as your canvas is very useful!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "figg = folium.Figure(height = 500)\n",
    "mapp = folium.Map(centre_latlon, zoom_start = 15, tiles='OpenStreetMap')\n",
    "mapp.add_to(figg)\n",
    "\n",
    "folium.GeoJson(grgr_route).add_to(mapp)\n",
    "mapp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Output the GPS route file\n",
    "\n",
    "Finally, the last task is to just output to GPX file (to the file location set at the beginning):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grgr_route.to_file(grgr_gpx, driver=\"GPX\", FORCE_GPX_ROUTE=\"YES\", GPX_USE_EXTENSIONS=\"YES\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This should then have created a file in the `out` folder where this Jupyter Notebook is running.\n",
    "\n",
    "If you need a hand finding this then:\n",
    "\n",
    "<ul>\n",
    "    <li>\n",
    "        Go to the home page which should look like this\n",
    "        <img src=\"./how-to/play-home.PNG\" alt=\"Home page\" width=\"700\"/>\n",
    "    </li>\n",
    "    <li>\n",
    "        Click on the <code>out</code> folder and then click on the <code>.gpx</code> file that should be in there.\n",
    "    </li>\n",
    "    <li>\n",
    "        Then at the top of the page there should be a drop down menu for file with a download option, click this and save it.\n",
    "        <img src=\"./how-to/play-download.PNG\" alt=\"Download route\" width=\"700\"/>\n",
    "    </li>\n",
    "    <li>\n",
    "        You can now use this a navigation tool for your next bit of GPS art. I use it on my watch as a route to navigate and on my phone using a GPX viewer app.\n",
    "    </li>\n",
    "</ul>\n",
    "   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Further analysis\n",
    "\n",
    "Further analysis can be done with the data we have created. One thing that I found is useful look at is the route's route (if that makes sense!). With this we can also extract the distance between points to gauge the routes length.\n",
    "\n",
    "Firstly, we have to split up the routes single LineString into each leg of the route. This is done using the following function to pull out the information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_lines(gdf, df_out, i, geometry = 'geometry'):\n",
    "    \n",
    "    # Get the adjacent points:\n",
    "    geom0 = gdf[i]\n",
    "    geom1 = gdf[i + 1]\n",
    "    \n",
    "    # Organise them in a way that is easy to use and produce a linestring:\n",
    "    start, end = [(geom0[0], geom0[1]), (geom1[0], geom1[1])]\n",
    "    line = LineString([start, end])\n",
    "    \n",
    "    # Create a DataFrame to hold the record:\n",
    "    data = {'id': i,\n",
    "            'geometry': [line]}\n",
    "    df_line = pd.DataFrame(data, columns = ['id', 'geometry'])\n",
    "    \n",
    "    # Add the record DataFrame of compiled records (given as an input to the function):\n",
    "    df_out = pd.concat([df_out, df_line])\n",
    "    \n",
    "    # Return the updated DataFrame:\n",
    "    return df_out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we just make use of the function and split out the lines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initialize an output DataFrame\n",
    "df = pd.DataFrame(columns = ['id', 'geometry'])\n",
    "\n",
    "# Loop through each row of the input point GeoDataFrame\n",
    "x = 0\n",
    "while x < len(grgr_lonlat) - 1:\n",
    "    df = make_lines(grgr_lonlat, df, x)\n",
    "    x = x + 1\n",
    "\n",
    "df.set_index('id', inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grgr_line = GeoDataFrame(df, crs=\"epsg:4326\")\n",
    "\n",
    "ECKERT_IV_PROJ4_STRING = \"+proj=eck4 +lon_0=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs\"\n",
    "\n",
    "grgr_line = grgr_line.to_crs(ECKERT_IV_PROJ4_STRING)\n",
    "\n",
    "grgr_line['dist'] = grgr_line.length\n",
    "\n",
    "grgr_line['sum'] = 0\n",
    "\n",
    "for i in range(len(grgr_line)-1):\n",
    "    grgr_line.loc[i+1, 'sum'] = grgr_line.loc[i, 'sum'] + grgr_line.loc[i+1, 'dist'] / 1000\n",
    "\n",
    "grgr_line = grgr_line.to_crs(\"epsg:4326\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally we can plot it using a colourful colour map based on the distance (notice the scale in the top right for a gauge of distance):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import branca.colormap as bcm\n",
    "import math\n",
    "\n",
    "f2 = folium.Figure(height = 500)\n",
    "m2 = folium.Map(centre_latlon, zoom_start = 15, tiles='OpenStreetMap')\n",
    "\n",
    "total_distance = grgr_line['sum'].max()\n",
    "linear = bcm.linear.Set1_08.scale(0,total_distance)\n",
    "gps_lyr = folium.GeoJson(grgr_line,\n",
    "                         style_function = lambda feature: {\n",
    "                             'color': linear(feature['properties']['sum']),\n",
    "                             'weight': 5})       \n",
    "\n",
    "\n",
    "gps_lyr.add_child\n",
    "    \n",
    "m2.add_child(linear)\n",
    "m2.add_child(gps_lyr)\n",
    "m2.add_to(f2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Time for you to play\n",
    "\n",
    "Now you have seen the code at work, you can play with it! Here are some things to try.\n",
    "\n",
    "## Basic\n",
    "\n",
    "In the code box that has <code>########### FOR YOU TO PLAY WITH ################</code> at the top try changing:\n",
    "<ul>    \n",
    "    <li>\n",
    "        the scale by editing the line <code>scale = 10</code> to <code>scale = 20</code>\n",
    "    </li>\n",
    "    <li>\n",
    "        the centre location to near you by changing <code>centre_latlon = [50.9615, -1.6034]</code> to <code>centre_latlon = [ your_lat , your_lon ]</code> where <code> your_lat , your lon </code> are your coordinates\n",
    "    </li>\n",
    "    <li> the image to a donkey by changing <code>grgr_dxf = dxfgrabber.readfile(\"./img/grgr2020.dxf\")</code> to <code>grgr_dxf = dxfgrabber.readfile(\"./img/donkey.dxf\")</code>\n",
    "    </li>\n",
    "</ul>\n",
    "\n",
    "## Advanced\n",
    "\n",
    "<ul> \n",
    "    <li>\n",
    "        Try designing your own image by creating a DXF file as discussed at the top of the page. You can also check out my <code>grgr2020.svg</code> and <code>grgr2020.dfx</code> files in the <code>img</code> folder to set the details.\n",
    "    </li>\n",
    "    <li>\n",
    "        Then upload it by going to the <code>img</code> folder form the home screen and clicking upload.\n",
    "        <img src=\"./how-to/play-diy.PNG\" alt=\"Upload your own\" width=\"700\"/>\n",
    "    </li>\n",
    "    <li>\n",
    "        Then follow the instructions in Basic for changing the image file name (you might also want to change the output name too).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notes and useful resources\n",
    "\n",
    "Remember this it not a polished document and still can be improved. If you have any suggestion feel free to get in touch.\n",
    "\n",
    "Below are useful links that helped me throughout writing this:\n",
    "<ul>\n",
    "    <li> <a href\"http://ryan-m-cooper.com/blog/gps-points-to-line-segments.html\">http://ryan-m-cooper.com/blog/gps-points-to-line-segments.html</a> </li>\n",
    "    <li> <a href\"https://www.movable-type.co.uk/scripts/latlong.html#dest-point\">https://www.movable-type.co.uk/scripts/latlong.html#dest-point</a> </li>\n",
    "</ul>\n",
    "    "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}