[Packaging] Error prone tests and docs modules included

[qbey]

The tests (and docs) modules are packaged, despite it's not good practice it may alors break projects having their own tests module.

Environment details

• OS type and version:
  Linux mistigri 5.8.0-53-generic #60-Ubuntu SMP Thu May 6 07:46:32 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

• Python version: python --version :
  Python 3.8.6

• pip version: pip --version:
  pip 20.1.1

• google-cloud-recpatchaenterprise version: pip show google-cloud-recaptcha-enterprise

  Name: google-cloud-recaptcha-enterprise
  Version: 0.3.1
  Summary: UNKNOWN
  Home-page: https://github.com/googleapis/python-recaptcha-enterprise
  Author: Google LLC
  Author-email: googleapis-packages@google.com
  License: Apache 2.0
  Location: /home/qbey/dev_perso/venv/test_project/lib/python3.8/site-packages
  Requires: packaging, google-api-core, proto-plus
  Required-by:
  

Steps to reproduce

1. new virtual env : virtualenv -p /usr/bin/python3.8 ../venv/test_project
2. python -c "import tests;print(tests.__file__)"
/home/qbey/dev_perso/venv/test_project/lib/python3.8/site-packages/tests/__init__.py

Code example

No real code example, this causes crash on one of our Django library that implements its own tests module and cannot be loaded through the Django magic mod = importlib.import_module(self.SETTINGS_MODULE) where self.SETTINGS_MODULE is tests.projects.settings.

Stack trace

Traceback (most recent call last):
  File "/home/jenkins/venv/2/bin/btr", line 8, in <module>
    sys.exit(run())
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/bluetils/pytest/cli.py", line 138, in run
    return_code = pytest.main()
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/opt/python-3.8.5/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/opt/python-3.8.5/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/bluetils/pytest/fixtures.py", line 8, in <module>
    from bluetils.test import helpers
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/bluetils/test/__init__.py", line 16, in <module>
    from .test import BluesysTestCaseMixin
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/bluetils/test/test.py", line 30, in <module>
    from bluetils.conf import settings
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/bluetils/conf.py", line 15, in <module>
    CONFIG = settings.CONFIG
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/django/conf/__init__.py", line 76, in __getattr__
    self._setup(name)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/django/conf/__init__.py", line 63, in _setup
    self._wrapped = Settings(settings_module)
  File "/home/jenkins/venv/2/lib/python3.8/site-packages/django/conf/__init__.py", line 142, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/opt/python-3.8.5/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'tests.project'

I will draft a pull request to remove tests and docs from the Python package.

[parthea]

Hi @qbey,

Thanks for opening this feature request and proposing a fix! I'll get back to you next week after I gather feedback from team members to determine if there is consensus to proceed. If we decide as an organization (googleapis) to move forward with this, we would want to apply this change broadly, across most / all the googleapis repos.

cc @busunkim96
cc @tseaver

[qbey]

Hello @parthea,

Thank you for your quick feedback :)

Actually in my PR I initially changed the owlbot.py file which is not required after second thought, only the setup.py's change is required. I updated the commit accordingly (I only saw your answer afterward).

[qbey]

Thank you for the fix :) I will wait for the next release to start using it.

Have a good day!