Clean up and small fixes for release 0.2.2 (#36)

* Update README and delete TODO * Update `whats-new.rst` * Update pandas matplotlib converter registration * Updated requirements * set version to 0.2.2 in `setup.py` * updated pandas version number in travis.yml
pycomlink · Apr 17, 2018 · fb4fe64 · fb4fe64
1 parent c78a047
commit fb4fe64
Show file tree

Hide file tree

Showing 9 changed files with 70 additions and 95 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -5,7 +5,7 @@ matrix:
     - python: "2.7"
       env: DEPS="numpy
                  scipy
-                 pandas>=0.18
+                 pandas>=0.22
                  matplotlib
                  numba
                  folium
@@ -18,7 +18,7 @@ matrix:
     - python: "3.6"
       env: DEPS="numpy
                  scipy
-                 pandas>=0.18
+                 pandas>=0.22
                  matplotlib
                  numba
                  folium

diff --git a/README.md b/README.md
@@ -8,18 +8,28 @@ A python toolbox for deriving rainfall information from commerical microwave lin
 Installation
 ------------
 
-`pycomlink` works with Python 2.7 and can be installed via `pip`. However, since one of its dependencies, `numba` is easiest to install via the [Anaconda Python distribution](https://store.continuum.io/cshop/anaconda/), we recommend to install Anaconda Python first and then do
+`pycomlink` works with Python 2.7 and Python 3.6 and can be installed via `pip`.
 
-    $ conda install numba
     $ pip install pycomlink
 
-To run the example notebooks you will also need the [Jupyter Notebook](https://jupyter.org/) and `ipython`, both also available via `conda` or `pip`.
+However, for using scientific Python packages it is in general recommended to 
+install the [Anaconda Python distribution](https://store.continuum.io/cshop/anaconda/) and use
+its package manager `conda` for managing all Python packages. `pycomlink` is, however,
+not yet installable via the Anaconda community package channel [conda-forge](https://conda-forge.org/).
+Hence, it is recommended to install all `pycomlink` dependencies (listed in `requirements.txt`) 
+via `conda` and then use `pip` to install `pycomlink`. 
+
+To run the example notebooks you will also need the [Jupyter Notebook](https://jupyter.org/) 
+and `ipython`, both also available via `conda` or `pip`.
 
 Usage
 -----
 
- * Jupyter notebook on [how to get started with CML data from a CSV file](http://nbviewer.jupyter.org/github/pycomlink/pycomlink/blob/master/notebooks/Use%20CML%20data%20from%20CSV%20file.ipynb)
- * More examples to come...
+The following jupyter notebooks showcase some use cases of `pycomlink`
+
+ * [How to do baseline determination](http://nbviewer.jupyter.org/github/pycomlink/pycomlink/blob/master/notebooks/Baseline%20determination.ipynb)
+ * [How to do spatial interpolation of CML rainfall](http://nbviewer.jupyter.org/github/pycomlink/pycomlink/blob/master/notebooks/Spatial%20interpolation.ipynb)
+ * [How to get started with your CML data from a CSV file](http://nbviewer.jupyter.org/github/pycomlink/pycomlink/blob/master/notebooks/Use%20CML%20data%20from%20CSV%20file.ipynb)
 
 Features
 --------

diff --git a/TODO.md b/TODO.md
diff --git a/docs/whats-new.rst b/docs/whats-new.rst
@@ -9,15 +9,41 @@ Enhancements
 
 * Codebase is Python 3 now, keeping backwards compatibility to Python 2.7
   via using the `future` module.
+
 * min-max CML data can now be written to and read from cmlh5. Standard column
   names are `tx_min`, `tx_max`, `rx_min` and `rx_max`. When reading from cmlh5
   without specifying dedicated column names, the function tries out the
   standard column names for min-max and instantaneous. If it does not find any
   match it will print an error message.
+
 * Added example file with min-max data for 75 CMLs. This dataset is derived
   from the existing example dataset of 75 CMLs with instantaneous measurements.
+
 * Added example notebook comparing min-max and instantaneous CML data
 
+* Added TravisCI and Codecov and increased the test coverage a little
+
+* Extended functionality for `append_data`. A maximum length or maximum
+  allowed age for the data can be specified
+
+* More options for interpolation. Added option to pass `max_distance`
+  for IDW and Added option for resampling in `Interpolator`
+  (instead of just doing hourly means of variable `R`)
+
+* Interpolated fields are now always transformed into an `xarray.Dataset`.
+  The `Dataset` is also stored as attribute if the `Interpolator` object
+
+* Improved grid intersection calculation in validator
+
+Bug fixes
+~~~~~~~~~
+
+* `t_start` and `t_stop` have not been taken into account
+  in the main interpolation loop
+
+* Fix: Catching `LinAlgError` in Kriging interpolation
+
+
 v0.2.1
 ------
 

diff --git a/notebooks/Spatial interpolation.ipynb b/notebooks/Spatial interpolation.ipynb
@@ -23,21 +23,12 @@
    "cell_type": "code",
    "execution_count": 2,
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "/Users/chwala-c/code/pycomlink/pycomlink/core/comlink.py:36: FutureWarning: 'pandas.tseries.converter.register' has been moved and renamed to 'pandas.plotting.register_matplotlib_converters'. \n",
-      "  converter.register()\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "import pycomlink as pycml\n",
     "\n",
     "import matplotlib.pyplot as plt\n",
-    "from tqdm import tqdm_notebook"
+    "from tqdm import tqdm"
    ]
   },
   {
@@ -100,44 +91,15 @@
    "metadata": {},
    "outputs": [
     {
-     "data": {
-      "application/vnd.jupyter.widget-view+json": {
-       "model_id": "0a448a3368aa4a308ffd36b222118027",
-       "version_major": 2,
-       "version_minor": 0
-      },
-      "text/html": [
-       "<p>Failed to display Jupyter Widget of type <code>HBox</code>.</p>\n",
-       "<p>\n",
-       "  If you're reading this message in the Jupyter Notebook or JupyterLab Notebook, it may mean\n",
-       "  that the widgets JavaScript is still loading. If this message persists, it\n",
-       "  likely means that the widgets JavaScript library is either not installed or\n",
-       "  not enabled. See the <a href=\"https://ipywidgets.readthedocs.io/en/stable/user_install.html\">Jupyter\n",
-       "  Widgets Documentation</a> for setup instructions.\n",
-       "</p>\n",
-       "<p>\n",
-       "  If you're reading this message in another frontend (for example, a static\n",
-       "  rendering on GitHub or <a href=\"https://nbviewer.jupyter.org/\">NBViewer</a>),\n",
-       "  it may mean that your frontend doesn't currently support widgets.\n",
-       "</p>\n"
-      ],
-      "text/plain": [
-       "HBox(children=(IntProgress(value=0, max=75), HTML(value=u'')))"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    },
-    {
-     "name": "stdout",
+     "name": "stderr",
      "output_type": "stream",
      "text": [
-      "\n"
+      "100%|██████████| 75/75 [00:01<00:00, 43.09it/s]\n"
      ]
     }
    ],
    "source": [
-    "for cml in tqdm_notebook(cml_list):\n",
+    "for cml in tqdm(cml_list):\n",
     "    window_length = 60\n",
     "    threshold = 1.0\n",
     "    cml.process.wet_dry.std_dev(window_length=window_length, threshold=threshold)\n",
@@ -196,7 +158,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "100%|██████████| 48/48 [00:00<00:00, 50.49it/s]\n"
+      "100%|██████████| 48/48 [00:01<00:00, 47.07it/s]\n"
      ]
     }
    ],
@@ -295,8 +257,8 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Building grid points: 100%|██████████| 7526/7526 [00:00<00:00, 108339.11it/s]\n",
-      "Checking intersections: 100%|██████████| 7526/7526 [00:06<00:00, 1113.68it/s]\n"
+      "Building grid points: 100%|██████████| 7526/7526 [00:00<00:00, 103954.59it/s]\n",
+      "Checking intersections: 100%|██████████| 7526/7526 [00:06<00:00, 1112.04it/s]\n"
      ]
     },
     {
@@ -339,8 +301,8 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Building grid points: 100%|██████████| 7526/7526 [00:00<00:00, 106273.93it/s]\n",
-      "Checking intersections: 100%|██████████| 7526/7526 [00:02<00:00, 2851.82it/s]\n"
+      "Building grid points: 100%|██████████| 7526/7526 [00:00<00:00, 92139.99it/s]\n",
+      "Checking intersections: 100%|██████████| 7526/7526 [00:02<00:00, 2860.53it/s]\n"
      ]
     },
     {

diff --git a/pycomlink/core/comlink.py b/pycomlink/core/comlink.py
@@ -29,11 +29,13 @@
 from ..spatial.helper import distance
 
 
-# Intermediate fix for pandas 0.21
-# https://github.com/pandas-dev/pandas/issues/18283
-# TODO: Remove this when solved via pandas 0.21.1
-from pandas.tseries import converter
-converter.register()
+# Assure that the pandas matplotlib converters are registered,
+# as long as a new matplotlib release does not handle pandas
+# time data (or np.datetime64) automatically
+# TODO: Remove this when solved via new matplotlib, maybe in 2.2.something...
+# here: https://github.com/matplotlib/matplotlib/pull/9779
+import pandas.plotting
+pandas.plotting.register_matplotlib_converters()
 
 Coords = namedtuple('coords', ['lon_a', 'lon_b', 'lat_a', 'lat_b'])
 

diff --git a/pycomlink/util/temporal.py b/pycomlink/util/temporal.py
@@ -65,7 +65,8 @@ def aggregate_df_onto_DatetimeIndex(df, new_index, method, label='right',
     df_reindexed = df_temp.groupby('new_time_ix').agg(method)
     # Update name and timezone of new index
     df_reindexed.index.name = df_temp.index.name
-    df_reindexed.index = df_reindexed.index.tz_localize('UTC').tz_convert(
-        df_temp.index.tzinfo)
+    if not df_reindexed.index.tzinfo:
+        df_reindexed.index = df_reindexed.index.tz_localize('UTC').tz_convert(
+            df_temp.index.tzinfo)
 
     return df_reindexed
diff --git a/requirements.txt b/requirements.txt
@@ -1,6 +1,6 @@
 numpy
 scipy
-pandas>=0.18
+pandas>=0.22
 matplotlib
 numba
 folium
@@ -9,4 +9,5 @@ xarray
 shapely
 pyproj
 tqdm
-pykrige
+pykrige
+future
diff --git a/setup.py b/setup.py
@@ -17,15 +17,15 @@ def read(fname):
 
 setup(
     name = "pycomlink",
-    version = "0.2.1",
+    version = "0.2.2",
     author = "Christian Chwala",
     author_email = "christian.chwala@kit.edu",
     description = ("Python tools for MW link data processing"),
     license = "BSD",
     keywords = "microwave links precipitation radar",
     url = "https://github.com/pycomlink/pycomlink",
     download_url = (
-        "https://github.com/pycomlink/pycomlink/archive/0.2.1.tar.gz"),
+        "https://github.com/pycomlink/pycomlink/archive/0.2.2.tar.gz"),
     packages=find_packages(exclude=['test']),
     include_package_data=True,
     long_description=read('README.md'),
@@ -34,13 +34,14 @@ def read(fname):
         "Topic :: Scientific/Engineering :: Atmospheric Science",
         "License :: OSI Approved :: BSD License",
         'Programming Language :: Python :: 2.7',
+        'Programming Language :: Python :: 3.6'
     ],
     # A list of all available classifiers can be found at 
     # https://pypi.python.org/pypi?%3Aaction=list_classifiers
     install_requires=[
         'numpy',
         'scipy',
-        'pandas>=0.18',
+        'pandas>=0.22',
         'matplotlib',
         'numba',
         'folium',