Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reducing number of dependencies #95

Open
dtoubelis opened this issue Sep 18, 2017 · 23 comments
Open

Reducing number of dependencies #95

dtoubelis opened this issue Sep 18, 2017 · 23 comments
Milestone

Comments

@dtoubelis
Copy link

We are using asciimatics with an embedded device and it requires lots of dependencies. The main offender is Pillow which also requires 3-6 other dependencies and some of them do not compile very well for anything other than basic i386/amd64 platforms. I also noticed that sub dependencies are some sort of imaging libraries for jpeg and png and I can hardly imagine they have anything to do with TUI library. So, would it be possible to remove dependency on Pillow and revisit other dependencies as well.

@peterbrittain
Copy link
Owner

The Pillow dependency is for image to text conversion, as per this (https://asciinema.org/a/19920) demo.

If you avoid the ImageFile and ColourImageFile renderers, it should be safe to run without Pillow. Of course, you then can't import any renderers without fixing the source to avoid the import...

I'm open to suggestions for how to package this more neatly, but don't want to remove the feature entirely as it is required for the animatics aspect of the package (which is where it all started).

@peterbrittain
Copy link
Owner

Was looking at this last night. What I'd really like to have is an "opt out" option on the installation so that most people will just get the whole solution. In fact, what I'd like is this: pypa/setuptools#163 - which isn't implemented yet. Sigh.

@dtoubelis
Copy link
Author

dtoubelis commented Sep 20, 2017

That would work. We were chosen asciimatics as more modern alternative to urwid and we just expected it to use ncurses and everything else to be optional. So, if it is possible to turn off the optional features it would be awesome.

@peterbrittain
Copy link
Owner

Hmmm... Opt outs will be hard and need more time than I can invest into setuptools...

After more thinking, maybe I can get what I need with a "default specifier" setting that gets used if you don't specify any specifier for extras on the install. This way, I could define a default which picks all the extras, but allows you to reduce that set by selecting a different (reduced) set of options if you so desire.

@dtoubelis
Copy link
Author

I think this in an excellent idea. Realistically, if we can have some sensible defaults for majority of uses and be able to overwrite them if needed that would be the perfect solution.

@peterbrittain
Copy link
Owner

Looks like this is going to take a while to get agreed. Do you have a suitable workaround while that happens? If not, I think that a simple one would be to:

  • clone this repo
  • remove asciimatics/renderers.py
  • remove the Pillow dependency from setup.py
  • build your own distribution
  • use that as a dependency for your TUI application.

I've had a quick test and this can run the contact_list demo, but obviously cannot run any of the others that use a Renderer.

@dtoubelis
Copy link
Author

That sounds like it may be a reasonable workaround. We build our packages with yocto, so it should be rather straightforward to create a patch and apply it to the base release at build time. Let me try that.

@brycepg
Copy link

brycepg commented Oct 18, 2017

I ended up stripping asciimatics of dependencies as well (pyfiglet, Pillow, and future), but now I'm stuck on an older version. Definitely would like to see Pillow as optional.

@peterbrittain
Copy link
Owner

Given that I'm just aiming at supporting pip installations, maybe the option in pypa/setuptools#1139 will work. The theory would be having everything installed if no extras are specified and skip Pillow and pyfiglet for a tui extra.

I could just then add some import protection in renderers to handle the missing dependencies a little more gracefully.

@peterbrittain
Copy link
Owner

peterbrittain commented Nov 2, 2017

OK, so I've just tried using the pip only solution I found above... I think the code in https://github.com/peterbrittain/asciimatics/tree/slim_tui now allows you to pip install asciimatics[tui] to avoid figlet and Pillow dependencies.

It also prevents you from trying to use any renderers until you have a full installation using the normal pip install.

Does this do the trick for your environments?

@dtoubelis
Copy link
Author

Yes, that sounds like a neat idea. Let me try it in our environment. However, one of the problems we were facing was that our developers could not use pip in our yocto builds and they were using python setup.py install instead but I will check where we stand on this.

@peterbrittain
Copy link
Owner

Any news?

@brycepg
Copy link

brycepg commented Nov 22, 2017

Thanks for working on this Peter.

With latest version of setuptools I'm getting an exception when i run the slim_tui branch with python setup.py install, pip install '.[tui]', or pip install . However when I use setuptools==20.4 I do not get an exception.

It looks like this issue is supposed to be fixed?: pypa/setuptools#523

Here's the traceback from setuptools 37.0

(pm5-dev) bguinta  [1]➜  asciimatics git:(slim_tui) ✗ python setup.py install         
running install
running bdist_egg
running egg_info
writing asciimatics.egg-info/PKG-INFO
writing dependency_links to asciimatics.egg-info/dependency_links.txt
writing requirements to asciimatics.egg-info/requires.txt
writing top-level names to asciimatics.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
writing manifest file 'asciimatics.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying asciimatics/version.py -> build/lib/asciimatics
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/__init__.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/effects.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/event.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/particles.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/paths.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/scene.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/sprites.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/utilities.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/exceptions.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/renderers.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/screen.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/widgets.py -> build/bdist.linux-x86_64/egg/asciimatics
copying build/lib/asciimatics/version.py -> build/bdist.linux-x86_64/egg/asciimatics
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/effects.py to effects.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/event.py to event.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/particles.py to particles.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/paths.py to paths.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/scene.py to scene.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/sprites.py to sprites.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/utilities.py to utilities.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/exceptions.py to exceptions.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/renderers.py to renderers.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/screen.py to screen.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/widgets.py to widgets.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/asciimatics/version.py to version.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying asciimatics.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying asciimatics.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying asciimatics.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying asciimatics.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying asciimatics.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/asciimatics-1.8.1.dev56+g8aea4e5-py3.6.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing asciimatics-1.8.1.dev56+g8aea4e5-py3.6.egg
Removing /home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/asciimatics-1.8.1.dev56+g8aea4e5-py3.6.egg
Copying asciimatics-1.8.1.dev56+g8aea4e5-py3.6.egg to /home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages
asciimatics 1.8.1.dev56+g8aea4e5 is already the active version in easy-install.pth

Installed /home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/asciimatics-1.8.1.dev56+g8aea4e5-py3.6.egg
Processing dependencies for asciimatics==1.8.1.dev56+g8aea4e5
Traceback (most recent call last):
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2650, in _dep_map
    return self.__dep_map
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2725, in __getattr__
    raise AttributeError(attr)
AttributeError: _Distribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 78, in <module>
    test_suite='nose.collector',
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run()
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 411, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 653, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 700, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 745, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 878, in resolve
    new_requirements = dist.requires(req.extras)[::-1]
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2669, in requires
    dm = self._dep_map
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2658, in _dep_map
    if invalid_marker(marker):
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1441, in invalid_marker
    evaluate_marker(text)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1459, in evaluate_marker
    return marker.evaluate()
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/markers.py", line 301, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/markers.py", line 226, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/home/bguinta/miniconda3/envs/pm5-dev/lib/python3.6/site-packages/pkg_resources/_vendor/packaging/markers.py", line 208, in _get_env
    "{0!r} does not exist in evaluation environment.".format(name)
pkg_resources.extern.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.

@peterbrittain
Copy link
Owner

OK - I've reproduced your issue exactly. I agree that it is setuptools issue 523, but have found a later open issue that is tracking this. Have followed up there and am hoping that we'll get a response at some point.

@peterbrittain
Copy link
Owner

I've been looking at this again. I now think it was a regression introduced by release 36.2 of setuptools. Plenty of other versions before that, including 36.1, work fine. I'll see if I can find the cause...

@peterbrittain
Copy link
Owner

Getting there. Have tried to repro the bug in setuptools unit tests and think I'm there. Trying to get an official patch in pypa/setuptools#1139

@brycepg
Copy link

brycepg commented Dec 21, 2017

Sweet. I'm making a conda package for asciimatics (since I use the conda toolchain, which also requires me to make a pyfiglet conda package). When you get this branch into a release, I'll update the package to make pyfiglet optional.

@peterbrittain
Copy link
Owner

Some kind person has submitted the PR for me. See pypa/setuptools#1503 for progress.

@peterbrittain
Copy link
Owner

Hmmm... Looks like that PR has been dropped. Maybe the suggestion of specifying a blank extras definition will do the trick here?

@pawamoy
Copy link
Contributor

pawamoy commented Nov 16, 2019

Voting in favor of the slim_tui branch 👍

@peterbrittain
Copy link
Owner

I would dearly love to merge that change, but setuptools doesn't work with that use of extras. The best I can do is make the default install of asciimatics not include Pillow and pyfiglet, then define an all option that users must specify when using the non-tui features.

I've been reluctant to make that default change up to now as this will break existing package definitions. I suppose I could move to a new major version and handle it that way.

@pawamoy
Copy link
Contributor

pawamoy commented Nov 16, 2019

Oh I see, I thought the change in the slim_tui branch was functional. That's too bad! Then I'm totally OK with waiting for a working solution that will not break existing code (or for a new major version) 🙂

@peterbrittain peterbrittain added this to the V2.0 milestone Nov 17, 2019
@peterbrittain
Copy link
Owner

Ok - let's make that the plan of record. Will convert the install when I bump the major version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants