Multiple test failures with PyPy3.10 7.3.15

[mgorny]

When running the test suite with PyPy3.10, I'm getting a bunch of test failures:

tox output

$ tox -e pypy3
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: install_requires_for_build_wheel> python -I -m pip install wheel
.pkg: prepare_metadata_for_build_wheel> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
pypy3: install_package_deps> python -I -m pip install graphviz
pypy3: install_package> python -I -m pip install --force-reinstall --no-deps /tmp/objgraph/.tox/.tmp/package/1/objgraph-3.5.1.dev0.tar.gz
pypy3: commands[0]> python tests.py
..F........................s..........FF.s.......FF^[[5;7~FF./tmp/objgraph/objgraph.py:966: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='utf-8'>
  gc.collect()
.F/tmp/objgraph/objgraph.py:438: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='utf-8'>
  gc.collect()
F.
======================================================================
FAIL: test_new_garbage (__main__.ByTypeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/objgraph/tests.py", line 412, in test_new_garbage
    self.assertLessEqual(len(gc.get_referrers(res[0])), 2)
AssertionError: 3 not less than or equal to 2

======================================================================
FAIL: test_short_repr_mocked_instance_method_bound (__main__.StringRepresentationTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/objgraph/tests.py", line 474, in test_short_repr_mocked_instance_method_bound
    self.assertRegex(objgraph._short_repr(obj.my_method), '<Mock')
AssertionError: Regex didn't match: '<Mock' not found in '<bound method ? of <__main__.StringRepre'

======================================================================
FAIL: test_short_repr_mocked_instance_method_bound_with_mocked_name (__main__.StringRepresentationTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/objgraph/tests.py", line 497, in test_short_repr_mocked_instance_method_bound_with_mocked_name
    self.assertRegex(objgraph._short_repr(obj.my_method), '<Mock')
AssertionError: Regex didn't match: '<Mock' not found in '<bound method ? of <__main__.StringRepre'

======================================================================
FAIL: /tmp/objgraph/docs/extra-info.txt
Doctest: extra-info.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/doctest.py", line 2223, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for extra-info.txt
  File "/tmp/objgraph/docs/extra-info.txt", line 0

----------------------------------------------------------------------
File "/tmp/objgraph/docs/extra-info.txt", line 29, in extra-info.txt
Failed example:
    objgraph.at(id(a))
Expected nothing
Got:
    'a string'


======================================================================
FAIL: /tmp/objgraph/docs/generator-sample.txt
Doctest: generator-sample.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/doctest.py", line 2223, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for generator-sample.txt
  File "/tmp/objgraph/docs/generator-sample.txt", line 0

----------------------------------------------------------------------
File "/tmp/objgraph/docs/generator-sample.txt", line 43, in generator-sample.txt
Failed example:
    objgraph.show_chain(
        objgraph.find_backref_chain(objgraph.by_type('Canary')[0],
                                    objgraph.is_proper_module),
        filename='canary-chain.png')
Expected:
    Graph written to ....dot (11 nodes)
    Image generated as canary-chain.png
Got:
    Graph written to /tmp/test-objgraph-nwf3s8tc/objgraph-rtr5hst2.dot (9 nodes)
    Image generated as canary-chain.png


======================================================================
FAIL: /tmp/objgraph/docs/highlighting.txt
Doctest: highlighting.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/doctest.py", line 2223, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for highlighting.txt
  File "/tmp/objgraph/docs/highlighting.txt", line 0

----------------------------------------------------------------------
File "/tmp/objgraph/docs/highlighting.txt", line 16, in highlighting.txt
Failed example:
    objgraph.show_backrefs(a, max_depth=15,
        extra_ignore=[id(locals())],
        highlight=lambda x: isinstance(x, Node),
        filename='highlight.png')
Expected:
    Graph written to ....dot (12 nodes)
    Image generated as highlight.png
Got:
    Graph written to /tmp/test-objgraph-0qqpwuhn/objgraph-wjse_9g6.dot (100 nodes)
    Image generated as highlight.png


======================================================================
FAIL: /tmp/objgraph/docs/index.txt
Doctest: index.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/doctest.py", line 2223, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for index.txt
  File "/tmp/objgraph/docs/index.txt", line 0

----------------------------------------------------------------------
File "/tmp/objgraph/docs/index.txt", line 110, in index.txt
Failed example:
    objgraph.show_chain(
        objgraph.find_backref_chain(
            random.choice(objgraph.by_type('MyBigFatObject')),
            objgraph.is_proper_module),
        filename='chain.png')
Expected:
    Graph written to ...dot (13 nodes)
    Image generated as chain.png
Got:
    Graph written to /tmp/test-objgraph-3z1llwbv/objgraph-jk5ezpur.dot (10 nodes)
    Image generated as chain.png
----------------------------------------------------------------------
File "/tmp/objgraph/docs/index.txt", line 158, in index.txt
Failed example:
    objgraph.show_refs(roots[:3], refcounts=True, filename='roots.png')
    # doctest: +NODES_VARY
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/pypy3.10/doctest.py", line 1352, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest index.txt[16]>", line 1, in <module>
        objgraph.show_refs(roots[:3], refcounts=True, filename='roots.png')
      File "/tmp/objgraph/objgraph.py", line 805, in show_refs
        return _show_graph(objs, max_depth=max_depth, extra_ignore=extra_ignore,
      File "/tmp/objgraph/objgraph.py", line 976, in _show_graph
        _obj_label(target, extra_info,
      File "/tmp/objgraph/objgraph.py", line 1118, in _obj_label
        label[0] += ' [%d]' % (sys.getrefcount(obj) - 4)
    AttributeError: module 'sys' has no attribute 'getrefcount'


======================================================================
FAIL: /tmp/objgraph/docs/references.txt
Doctest: references.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/doctest.py", line 2223, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for references.txt
  File "/tmp/objgraph/docs/references.txt", line 0

----------------------------------------------------------------------
File "/tmp/objgraph/docs/references.txt", line 7, in references.txt
Failed example:
    objgraph.show_refs([list(range(7))], too_many=5, filename='too-many.png')
Expected:
    Graph written to ....dot (6 nodes)
    Image generated as too-many.png
Got:
    Graph written to /tmp/test-objgraph-3mjnayr1/objgraph-9o1bm5qw.dot (1 nodes)
    Image generated as too-many.png
----------------------------------------------------------------------
File "/tmp/objgraph/docs/references.txt", line 17, in references.txt
Failed example:
    objgraph.show_backrefs([moo], too_many=5, max_depth=1, filename='42.png')
Expected:
    Graph written to ....dot (6 nodes)
    Image generated as 42.png
Got:
    Graph written to /tmp/test-objgraph-3mjnayr1/objgraph-sp9hh556.dot (2 nodes)
    Image generated as 42.png
----------------------------------------------------------------------
File "/tmp/objgraph/docs/references.txt", line 34, in references.txt
Failed example:
    objgraph.show_backrefs([one_reference], refcounts=True,
        filename='refcounts.png') # doctest: +NODES_VARY
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/pypy3.10/doctest.py", line 1352, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest references.txt[7]>", line 1, in <module>
        objgraph.show_backrefs([one_reference], refcounts=True,
      File "/tmp/objgraph/objgraph.py", line 725, in show_backrefs
        return _show_graph(objs, max_depth=max_depth, extra_ignore=extra_ignore,
      File "/tmp/objgraph/objgraph.py", line 976, in _show_graph
        _obj_label(target, extra_info,
      File "/tmp/objgraph/objgraph.py", line 1118, in _obj_label
        label[0] += ' [%d]' % (sys.getrefcount(obj) - 4)
    AttributeError: module 'sys' has no attribute 'getrefcount'


======================================================================
FAIL: doctest_get_new_ids_prints (__main__)
Doctest: __main__.doctest_get_new_ids_prints
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pypy3.10/doctest.py", line 2223, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for __main__.doctest_get_new_ids_prints
  File "/tmp/objgraph/tests.py", line 376, in doctest_get_new_ids_prints

----------------------------------------------------------------------
File "/tmp/objgraph/tests.py", line 383, in __main__.doctest_get_new_ids_prints
Failed example:
    _ = objgraph.get_new_ids(limit=1)
    # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
Expected:
    ========================================================
    Type      Old_ids  Current_ids      New_ids Count_Deltas
    ========================================================
    list          ...          ...          ...           +2
    ========================================================
Got:
    =======================================================
    Type      Old_ids  Current_ids      New_ids Count_Deltas
    =======================================================
    int        15089        16956        +1867        +1867
    =======================================================


----------------------------------------------------------------------
Ran 58 tests in 88.947s

FAILED (failures=9, skipped=2)
pypy3: exit 1 (89.41 seconds) /tmp/objgraph> python tests.py pid=37859
.pkg: _exit> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  pypy3: FAIL code 1 (100.12=setup[10.71]+cmd[89.41] seconds)
  evaluation failed :( (100.22 seconds)

At least some of them seem to be caused by relying on aggressive CPython GC behavior.

[mgedmin]

objgraph's test suite has never passed on PyPy. The garbage collector is sufficiently different from CPython's.

This is why objgraph's classifiers do not claim support for PyPy.

(I wouldn't mind adding support for PyPy, but I'm not willing to spend effort on it myself.)