Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pytest plugin #448

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Pytest plugin #448

wants to merge 1 commit into from

Conversation

AlexeySanko
Copy link
Contributor

@AlexeySanko AlexeySanko commented Dec 27, 2016 •
edited

Integration into core plugins external pytest plugin
https://github.com/AlexeySanko/pybuilder_pytest

@coveralls
Copy link

coveralls commented Dec 27, 2016 •
edited

Coverage Status

Coverage decreased (-1.4%) to 72.092% when pulling eea1889 on AlexeySanko:feature/pytest_plugin into 4d943c8 on pybuilder:master.

@AlexeySanko AlexeySanko mentioned this pull request Dec 27, 2016
Open
@AlexeySanko
Copy link
Contributor Author

It's strange.

[DEBUG] Module coverage report: {'branches_missing': 0, 'sum_lines': 26, 'lines_not_covered': [], 'coverage': 100, 'branches': 4, 'branches_partial': 0, 'lines': [17, 18, 20, 21, 22, 24, 26, 29, 32, 33, 34, 37, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54], 'sum_lines_not_covered': 0, 'module': 'pybuilder.plugins.python.pytest_plugin'}

but merge decreased total coverage. o.O

arcivanov
arcivanov reviewed Jan 11, 2017
View reviewed changes
build.py Outdated
@@ -91,6 +91,7 @@ def initialize(project):
project.build_depends_on("pygments")
if sys.version_info[0:2] == (2, 6):
project.build_depends_on("importlib") # for fluentmock
project.build_depends_on("pytest")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Building PyB does not depend on PyTest. This is supposed to be in pytest plugin as plugin_depends_on only.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Fixed.

src/main/python/pybuilder/plugins/python/pytest_plugin.py Outdated
# See the License for the specific language governing permissions and
# limitations under the License.

import pytest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail if pytest is not installed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Fixed (added assert_pytest_avaiable and import moved to inner method)

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.942% when pulling f95a636 on AlexeySanko:feature/pytest_plugin into 88a6f9c on pybuilder:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.942% when pulling f95a636 on AlexeySanko:feature/pytest_plugin into 88a6f9c on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.942% when pulling f95a636 on AlexeySanko:feature/pytest_plugin into 88a6f9c on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.942% when pulling f95a636 on AlexeySanko:feature/pytest_plugin into 88a6f9c on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.942% when pulling f95a636 on AlexeySanko:feature/pytest_plugin into 88a6f9c on pybuilder:master.

@AlexeySanko
Copy link
Contributor Author

Huh... Changes according "use_plugin" usage instead of "build_depends_on" were more deeper than expected, but build is greeen again.

arcivanov
arcivanov requested changes Jan 16, 2017
View reviewed changes
build.py Outdated
@@ -38,6 +38,7 @@
use_plugin("source_distribution")

use_plugin("python.unittest")
use_plugin("python.pytest")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I know only two ways to force pybuilder to install any package (in this case - pytest, because pytest_plugin tests use it):

  1. use build_depends_on('pytest')
  2. use plugin_depends_on('pytest') and call use_plugin into build.py (otherwise plugin will not be used and pytest will not be installed)
    Any other idea?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're forcing PyBuilder to use pytest for itself here. PyBuilder doesn't use a PyTest unless I'm missing something.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not? We test our project by two tools - perfect doublecheck.
On the other hand: if we want to get coverage with pytest we need to call it through import pytest. So we need installed pytest when we call tests for this plugin. As I understand we can do it by two ways:

  1. use build_depends_on('pytest')
  2. use_plugin + plugin_depends_on('pytest')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, guys.
Do You have any ideas how we can test pytest plugin without usage build_depends_on('pytest') or use_plugin + plugin_depends_on('pytest')?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need an integration test for the embedded plugin there are integration tests:

https://github.com/pybuilder/pybuilder/tree/master/src/integrationtest/python

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for idea @arcivanov
Moved pytest plugin tests to integration tests.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.957% when pulling e12508f on AlexeySanko:feature/pytest_plugin into 481ce6b on pybuilder:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.957% when pulling e12508f on AlexeySanko:feature/pytest_plugin into 481ce6b on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.957% when pulling e12508f on AlexeySanko:feature/pytest_plugin into 481ce6b on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.957% when pulling e12508f on AlexeySanko:feature/pytest_plugin into 481ce6b on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.001%) to 71.957% when pulling e12508f on AlexeySanko:feature/pytest_plugin into 481ce6b on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling 4c671f7 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling 4c671f7 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling 4c671f7 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling 4c671f7 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling 4c671f7 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

arcivanov
arcivanov reviewed Feb 6, 2017
View reviewed changes
src/main/python/pybuilder/plugins/python/coverage_plugin.py
if not _is_module_essential(module.__name__, sys_packages, sys_modules):
try:
# if we try to remove non-module object
module__name__ = module.__name__
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a "non-module" object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi.
sys.modules can contains specific objects indead of module. For example, pytest uses AliasModule for this purpose (I see the same behaviour in the couple projects. Unfortunately, I still do not understand reason of it)

print module
<AliasModule 'py.error' for 'py._error.error'>
print type(module)
<class 'py._apipkg.AliasModule'>

In this case module.__name__ will return None.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bug in PyTest, that violates PEP-0320 and PEP-0420 since Python 3.3. Namely:

https://docs.python.org/3/reference/import.html#import-related-module-attributes

The import machinery fills in these attributes on each module object during loading, based on the module’s spec, before the loader executes the module.

__name__
The __name__ attribute **must be set to the fully-qualified name of the module**. This name is used to uniquely identify the module in the import system.
``` (emphasis added)

Please file a bug with PyTest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is old as mammoth feces and Pytest doesn't use AliasModule anymore. I'll do changes next week and add >=3.2.3 version of Pytest to plugin dependsies.
Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this problem was fixed, but still isn't released: pytest-dev/py#73

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Asked about new release pytest-dev/py#153

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I'm not sure that py will be releases in the nearest future. I suggest to leave this changes and create new issue revert coverage plugin changes related to py issue. I'll periodically check py and do corresponding changes after py release.

build.py Outdated
@@ -38,6 +38,7 @@
use_plugin("source_distribution")

use_plugin("python.unittest")
use_plugin("python.pytest")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're forcing PyBuilder to use pytest for itself here. PyBuilder doesn't use a PyTest unless I'm missing something.

AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request Feb 7, 2017
pytest_plugin pybuilder#13(pybuilder#448)
fe1e8e4
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling fe1e8e4 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling fe1e8e4 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling fe1e8e4 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling fe1e8e4 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.002%) to 72.352% when pulling fe1e8e4 on AlexeySanko:feature/pytest_plugin into 7017a2a on pybuilder:master.

AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request May 19, 2017
pytest_plugin pybuilder#13(pybuilder#448)
64cfde8 
pytest tests where moved to integration tests
@AlexeySanko AlexeySanko mentioned this pull request Aug 28, 2017
Closed
@AlexeySanko AlexeySanko changed the title Feature/pytest plugin Pytest plugin Oct 12, 2017
@coveralls
Copy link

coveralls commented Oct 12, 2017 •
edited

Coverage Status

Coverage decreased (-0.5%) to 74.493% when pulling 5d8dfaa on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

@AlexeySanko
Copy link
Contributor Author

I still suggest to add pytest as build dependency. It will allow to cover 90% of pytest plugin with unit tests instead of current 35% (now pytest functionality is covered by integration tests).

AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request Oct 15, 2017
pytest_plugin pybuilder#13(pybuilder#448)
8dae883
AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request Oct 15, 2017
pytest_plugin pybuilder#13(pybuilder#448)
da14970
AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request Oct 15, 2017
pytest_plugin pybuilder#13(pybuilder#448)
fed57de
@coveralls
Copy link

coveralls commented Oct 15, 2017 •
edited

Coverage Status

Coverage decreased (-0.5%) to 74.476% when pulling fed57de on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request Oct 16, 2017
pytest_plugin pybuilder#13(pybuilder#448)
81f55a3
@coveralls
Copy link

coveralls commented Oct 16, 2017 •
edited

Coverage Status

Coverage decreased (-0.5%) to 74.476% when pulling 81f55a3 on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.5%) to 74.476% when pulling 81f55a3 on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.5%) to 74.476% when pulling 81f55a3 on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.5%) to 74.476% when pulling 81f55a3 on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.5%) to 74.476% when pulling 81f55a3 on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

AlexeySanko pushed a commit to AlexeySanko/pybuilder that referenced this pull request Oct 16, 2017
pytest_plugin pybuilder#13(pybuilder#448)
bbb04d0
@AlexeySanko
Copy link
Contributor Author

Added dir_source_pytest_python to python_plugin_helper.execute_tool_on_source_files. As result all checkers will parse dir_source_pytest_python if inculde test sources.

@coveralls
Copy link

coveralls commented Oct 16, 2017 •
edited

Coverage Status

Coverage decreased (-0.5%) to 74.505% when pulling 15f7990 on AlexeySanko:feature/pytest_plugin into d0f8f80 on pybuilder:master.

@esc
Copy link
Contributor

esc commented Oct 30, 2017

@arcivanov any news on this, are you happy now?

@AlexeySanko
Copy link
Contributor Author

Over a month. Marked as useless. Closed.

@arcivanov arcivanov reopened this Nov 18, 2017
@AlexeySanko
Copy link
Contributor Author

Should also affect python.integrationtest plugin.
Like

if project.get_property("integrationtest_framework") == 'pytest':
    command_and_arguments = ('pytest', test)
else:
    command_and_arguments = (sys.executable, test)

And assert_executable pytest into init if project.get_property("integrationtest_framework") == 'pytest'

@AlexeySanko
Copy link
Contributor Author

AlexeySanko commented Mar 6, 2018 •
edited

Workaround for PyTest usage for integration tests. Unfortunately, it's code copy-paste.

import inspect
import os
import sys

import pytest

from some_project import (
    some_function_for_testing
)


def test_some_function_for_testing():
    assert some_function_for_testing(1) == 'abc'


if __name__ == "__main__":
    ret = pytest.main([os.path.abspath((inspect.stack()[0])[1]), '-s', '-vv'])
    if ret:
        sys.exit(1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arcivanov arcivanov arcivanov requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@AlexeySanko @coveralls @esc @arcivanov