Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dtype issues on Windows #2012

Open
mataton opened this issue Apr 23, 2024 · 1 comment
Open

dtype issues on Windows #2012

mataton opened this issue Apr 23, 2024 · 1 comment

Comments

@mataton
Copy link
Collaborator

mataton commented Apr 23, 2024

In skbio/stats, the function subsample_counts and it's underlying Cython function _subsample_counts_without_replacement throw the following errors when run on Windows systems:

======================================================================
ERROR: test_subsample_counts_nonrandom (__main__.SubsampleCountsTests.test_subsample_counts_nonrandom)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\stats\tests\test_subsample.py", line 38, in test_subsample_counts_nonrandom
    npt.assert_equal(subsample_counts(a, 2), exp)
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\stats\_subsample.py", line 247, in subsample_counts
    result = _subsample_counts_without_replacement(counts, n, counts_sum)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\stats\\__subsample.pyx", line 13, in skbio.stats.__subsample._subsample_counts_without_replacement
    def _subsample_counts_without_replacement(
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_subsample_counts_without_replacement (__main__.SubsampleCountsTests.test_subsample_counts_without_replacement)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\stats\tests\test_subsample.py", line 54, in test_subsample_counts_without_replacement
    obs = subsample_counts(a, 2)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\stats\_subsample.py", line 247, in subsample_counts
    result = _subsample_counts_without_replacement(counts, n, counts_sum)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\stats\\__subsample.pyx", line 13, in skbio.stats.__subsample._subsample_counts_without_replacement
    def _subsample_counts_without_replacement(
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

----------------------------------------------------------------------

It seems that there is a difference in the underlying data types of integers on Windows vs UNIX, as well as something regarding Cython. See here, here, here, and here. I have tried casting things to np.int64 to no avail.

EDIT: The issues surrounding subsample have been resolved with PR #2016. The other issues here still remain.

@mataton mataton changed the title Subsampling dtype issues on Windows dtype issues on Windows Apr 25, 2024
@mataton
Copy link
Collaborator Author

mataton commented Apr 25, 2024

There are very similar, but slightly different errors occurring in the _tip_distances function in diversity/beta/_unifrac.py and the associated Cython code.

======================================================================
ERROR: test_weighted_normalized_root_not_observed (__main__.UnifracTests.test_weighted_normalized_root_not_observed)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\tests\test_unifrac.py", line 160, in test_weighted_normalized_root_not_observed
    actual = weighted_unifrac([1, 0, 0, 0], [1, 1, 0, 0],
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\_unifrac.py", line 317, in weighted_unifrac
    node_to_root_distances = _tip_distances(branch_lengths, tree, tip_indices)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\diversity\\_phylogenetic.pyx", line 17, in skbio.diversity._phylogenetic._tip_distances
    @cython.boundscheck(False)
ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'long'

======================================================================
ERROR: test_weighted_unifrac_identity_normalized (__main__.UnifracTests.test_weighted_unifrac_identity_normalized)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\tests\test_unifrac.py", line 517, in test_weighted_unifrac_identity_normalized
    actual = weighted_unifrac(
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\_unifrac.py", line 317, in weighted_unifrac
    node_to_root_distances = _tip_distances(branch_lengths, tree, tip_indices)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\diversity\\_phylogenetic.pyx", line 17, in skbio.diversity._phylogenetic._tip_distances
    @cython.boundscheck(False)
ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'long'

======================================================================
ERROR: test_weighted_unifrac_non_overlapping_normalized (__main__.UnifracTests.test_weighted_unifrac_non_overlapping_normalized)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\tests\test_unifrac.py", line 535, in test_weighted_unifrac_non_overlapping_normalized
    actual = weighted_unifrac(
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\_unifrac.py", line 317, in weighted_unifrac
    node_to_root_distances = _tip_distances(branch_lengths, tree, tip_indices)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\diversity\\_phylogenetic.pyx", line 17, in skbio.diversity._phylogenetic._tip_distances
    @cython.boundscheck(False)
ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'long'

======================================================================
ERROR: test_weighted_unifrac_normalized (__main__.UnifracTests.test_weighted_unifrac_normalized)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\tests\test_unifrac.py", line 568, in test_weighted_unifrac_normalized
    actual = weighted_unifrac(
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\_unifrac.py", line 317, in weighted_unifrac
    node_to_root_distances = _tip_distances(branch_lengths, tree, tip_indices)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\diversity\\_phylogenetic.pyx", line 17, in skbio.diversity._phylogenetic._tip_distances
    @cython.boundscheck(False)
ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'long'

======================================================================
ERROR: test_weighted_unifrac_symmetry_normalized (__main__.UnifracTests.test_weighted_unifrac_symmetry_normalized)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\tests\test_unifrac.py", line 525, in test_weighted_unifrac_symmetry_normalized
    actual = weighted_unifrac(
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\_unifrac.py", line 317, in weighted_unifrac
    node_to_root_distances = _tip_distances(branch_lengths, tree, tip_indices)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\diversity\\_phylogenetic.pyx", line 17, in skbio.diversity._phylogenetic._tip_distances
    @cython.boundscheck(False)
ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'long'

======================================================================
ERROR: test_weighted_unifrac_zero_counts_normalized (__main__.UnifracTests.test_weighted_unifrac_zero_counts_normalized)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\tests\test_unifrac.py", line 549, in test_weighted_unifrac_zero_counts_normalized
    actual = weighted_unifrac(
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\maton\qlab\skbio\windows\github\scikit-bio\skbio\diversity\beta\_unifrac.py", line 317, in weighted_unifrac
    node_to_root_distances = _tip_distances(branch_lengths, tree, tip_indices)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "skbio\\diversity\\_phylogenetic.pyx", line 17, in skbio.diversity._phylogenetic._tip_distances
    @cython.boundscheck(False)
ValueError: Buffer dtype mismatch, expected 'DTYPE_t' but got 'long'

----------------------------------------------------------------------

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant