imp warning when starting cellxgene

[bkmartinjr]

Lately, when you start cellxgene CLI, you see deprecation warnings from the cloudpickle module, regarding use of a deprecated imp module:

# cellxgene scanpy ./example-dataset/
/mnt/c/Users/bruce/projects/cellxgene/venv/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
 * Running on http://127.0.0.1:5005/ (Press CTRL+C to quit)
 * Restarting with stat
/mnt/c/Users/bruce/projects/cellxgene/venv/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
 * Debugger is active!
 * Debugger PIN: 188-773-702

Root cause: ScanPy has a dependency on scikit-learn, which is using cloudpickle, which uses imp.

The cloudpickle fix has been made in this PR - see this bug for a bit more info. The version with the fix is 0.6.1 or later.

Unfortunately, scikit-learn embeds a copy of cloudpickle in their module (rather than having a dependency). So, we need to wait for scikit-learn to be released with a fix. The updated dependency will need to be fixed in scikit-learn, after which ScanPy will need to be updated.

As noted below, scikit-learn should release a fix shortly reference

[bkmartinjr]

More info on the imp warning - forcing a newer cloudpickle doens't solve it due to the way scikit-learn embeds cloudpickle. However, it looks like they plan to fix it shortly, so we should just wait for the fix, then update our dependencies.

[csweaver]

I think we should remove the debug flag from flask entirely. I don't think even the power users want to debug the flask code, that is more for if you get an error on the front end you can get the stack trace directly in the web browser.

I've been following the imp error, actually. I was going to add some suppression to warnings when I added the --debug flag (so that import library warnings are suppressed by default and only shown when debug is selected)