mini_book/track_observations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Track observations\n",
    "\n",
    "Track observations are point observations for moving points. The index therefore consists of time, x- and y-positions. Altimetry data acquired from satellite is an example of track observations; data obtained from boat is another example. \n",
    "\n",
    "The track observation data may come from a file or web api. We will cover the following situations here: \n",
    "\n",
    "* File\n",
    "    - dfs0\n",
    "    - csv/excel\n",
    "* REST API\n",
    "\n",
    "modelskill has the class [TrackObservation](https://dhi.github.io/fmskill/api.html#modelskill.observation.TrackObservation) for working with track observations. \n",
    "\n",
    "Track observations consist of time-**position**-value (data) and *meta* data such as\n",
    "\n",
    "* data type (e.g. water level)\n",
    "* unit (e.g. meter)\n",
    "\n",
    "ModelSkill is agnostic to the coordinate reference system (CRS) and it is therefore the responsibility of the user to make sure that all data (observations and model) use the same CRS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import modelskill as ms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data from file\n",
    "\n",
    "The two first items in the file must be x- and y- coordinate values. \n",
    "\n",
    "**dfs0** files can be read directly by TrackObservation. \n",
    "\n",
    "**csv** files needs to be read by pandas first and passed to TrackObservation as a DataFrame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lon</th>\n",
       "      <th>lat</th>\n",
       "      <th>surface_elevation</th>\n",
       "      <th>significant_wave_height</th>\n",
       "      <th>wind_speed</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-10-26 04:37:37</th>\n",
       "      <td>8.757272</td>\n",
       "      <td>53.926136</td>\n",
       "      <td>1.6449</td>\n",
       "      <td>0.426</td>\n",
       "      <td>6.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-10-26 04:37:54</th>\n",
       "      <td>8.221631</td>\n",
       "      <td>54.948459</td>\n",
       "      <td>1.1200</td>\n",
       "      <td>1.634</td>\n",
       "      <td>9.030000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-10-26 04:37:55</th>\n",
       "      <td>8.189390</td>\n",
       "      <td>55.008547</td>\n",
       "      <td>1.0882</td>\n",
       "      <td>1.717</td>\n",
       "      <td>9.370000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-10-26 04:37:56</th>\n",
       "      <td>8.157065</td>\n",
       "      <td>55.068627</td>\n",
       "      <td>1.0309</td>\n",
       "      <td>1.869</td>\n",
       "      <td>9.559999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-10-26 04:37:58</th>\n",
       "      <td>8.124656</td>\n",
       "      <td>55.128700</td>\n",
       "      <td>1.0369</td>\n",
       "      <td>1.939</td>\n",
       "      <td>9.980000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          lon        lat  surface_elevation  \\\n",
       "date                                                          \n",
       "2017-10-26 04:37:37  8.757272  53.926136             1.6449   \n",
       "2017-10-26 04:37:54  8.221631  54.948459             1.1200   \n",
       "2017-10-26 04:37:55  8.189390  55.008547             1.0882   \n",
       "2017-10-26 04:37:56  8.157065  55.068627             1.0309   \n",
       "2017-10-26 04:37:58  8.124656  55.128700             1.0369   \n",
       "\n",
       "                     significant_wave_height  wind_speed  \n",
       "date                                                      \n",
       "2017-10-26 04:37:37                    0.426    6.100000  \n",
       "2017-10-26 04:37:54                    1.634    9.030000  \n",
       "2017-10-26 04:37:55                    1.717    9.370000  \n",
       "2017-10-26 04:37:56                    1.869    9.559999  \n",
       "2017-10-26 04:37:58                    1.939    9.980000  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fn = \"data/SW/altimetry_NorthSea_20171027.csv\"\n",
    "df = pd.read_csv(fn, index_col=0, parse_dates=True)  # step 1: create DataFrame\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jan/src/book-learn-mikeio-fmskill/.venv/lib/python3.10/site-packages/modelskill/timeseries/_track.py:135: UserWarning: Removed 22 duplicate timestamps with keep=first\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrackObservation: surface_elevation, n=1093"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "o1 = ms.TrackObservation(df, item=\"surface_elevation\")  # step 2: create TrackObservation\n",
    "o1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DHI Altimetry API\n",
    "\n",
    "You need to pip install \"watobs\" for this part!\n",
    "\n",
    "See [Altimetry_data.ipynb](https://nbviewer.jupyter.org/github/DHI/watobs/blob/main/notebooks/Altimetry_data.ipynb) example notebook.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from watobs import DHIAltimetryRepository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'DHI_ALTIMETRY_API_KEY'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[7], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m api_key \u001b[38;5;241m=\u001b[39m \u001b[43mos\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43menviron\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mDHI_ALTIMETRY_API_KEY\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m]\u001b[49m\n\u001b[1;32m      2\u001b[0m repo \u001b[38;5;241m=\u001b[39m DHIAltimetryRepository(api_key)\n",
      "File \u001b[0;32m/usr/lib/python3.10/os.py:680\u001b[0m, in \u001b[0;36m_Environ.__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m    677\u001b[0m     value \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_data[\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mencodekey(key)]\n\u001b[1;32m    678\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m:\n\u001b[1;32m    679\u001b[0m     \u001b[38;5;66;03m# raise KeyError with the original key value\u001b[39;00m\n\u001b[0;32m--> 680\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m(key) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    681\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdecodevalue(value)\n",
      "\u001b[0;31mKeyError\u001b[0m: 'DHI_ALTIMETRY_API_KEY'"
     ]
    }
   ],
   "source": [
    "api_key = os.environ[\"DHI_ALTIMETRY_API_KEY\"]\n",
    "repo = DHIAltimetryRepository(api_key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = repo.get_altimetry_data(area=\"lon=2.9&lat=53.9&radius=100\", start_time=\"2019-10-1\", \n",
    "end_time=\"2019-10-8\")\n",
    "data.df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.plot_map()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'data' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[8], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m o1 \u001b[38;5;241m=\u001b[39m ms\u001b[38;5;241m.\u001b[39mTrackObservation(\u001b[43mdata\u001b[49m\u001b[38;5;241m.\u001b[39mdf, item\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124msignificant_wave_height\u001b[39m\u001b[38;5;124m\"\u001b[39m, name\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mAlti_from_df\u001b[39m\u001b[38;5;124m'\u001b[39m, quantity\u001b[38;5;241m=\u001b[39mms\u001b[38;5;241m.\u001b[39mQuantity(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mSignificant wave height\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mm\u001b[39m\u001b[38;5;124m\"\u001b[39m))\n\u001b[1;32m      2\u001b[0m o1\n",
      "\u001b[0;31mNameError\u001b[0m: name 'data' is not defined"
     ]
    }
   ],
   "source": [
    "o1 = ms.TrackObservation(data.df, item=\"significant_wave_height\", name='Alti_from_df', quantity=ms.Quantity(\"Significant wave height\", \"m\"))\n",
    "o1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you can save the data to a dfs0 first..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.to_dfs0('alti_NS_20191001.dfs0')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "o2 = ms.TrackObservation('alti_NS_20191001.dfs0', item=\"Significant Wave Height\", quantity=ms.Quantity(\"Significant wave height\", \"m\")\n",
    "o2"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "fa576ebcd40e010bdc0ae86b06ce09151f3424f9e9aed6893ff04f39a9299d89"
  },
  "kernelspec": {
   "display_name": "Python 3.8.10 64-bit ('base': conda)",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}