Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] r.confusionmatrix testsuite error #831

Open
neteler opened this issue Nov 4, 2022 · 1 comment
Open

[Bug] r.confusionmatrix testsuite error #831

neteler opened this issue Nov 4, 2022 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@neteler
Copy link
Member

neteler commented Nov 4, 2022

Name of the addon
r.confusionmatrix

Describe the bug
After the recent updates to r.kappa (OSGeo/grass#2573) the r.confusionmatrix testsuite fails.

To Reproduce

Install the addon and run the testsuite script of it:

GRASS nc_spm_08_grass7/user1:testsuite >

g.extension r.confusionmatrix
cd grass-addons/src/raster/r.confusionmatrix/testsuite/

python test_r_confusionmatrix.py 
['', '', 'Reference Map', 'landclass96']
['', '', 'developed', 'agriculture', 'herbaceous', 'shrubland', 'forest', 'water', 'sediment', '8', '9', '10', '11', '15', '18', '20', '21', 'User Accuracy', 'Commission Error']
['Classified Map', 'High Intensity Developed', 34931, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100.0, 0.0]
['landuse96_28m', 'Low Intensity Developed', 38146, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Cultivated', 0, 2137, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Managed Herbaceous Cover', 0, 0, 25488, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Upland Herbaceous', 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Riverine/Estuarine Herbaceous', 0, 0, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Evergreen Shrubland', 0, 0, 0, 16464, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Deciduous Shrubland', 0, 0, 0, 320, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Mixed Shrubland', 0, 0, 0, 45, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Mixed Hardwoods', 0, 0, 0, 0, 7893, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Bottomland Hardwoods/Hardwood Swamps', 0, 0, 0, 0, 19733, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Southern Yellow Pine', 0, 0, 0, 0, 64774, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Mixed Hardwoods/Conifers', 0, 0, 0, 0, 33865, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Water Bodies', 0, 0, 0, 0, 0, 5303, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Unconsolidated Sediment', 0, 0, 0, 0, 0, 0, 194, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 100.0]
['', 'Producer Accuracy', 47.8, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 'Overall Accuracy', 14.01]
['', 'Omission Error', 52.2, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 100.0, 'Kappa coefficient', 0.09]

Cleaning up...

F['', '', 'Reference Map', 'landclass96_class1only']
['', '', '1', 'User Accuracy', 'Commission Error']
['Classified Map', '1', 73077, 100.0, 0.0]
['landuse96_28m_class1only', '', '', '', '']
['', 'Producer Accuracy', 100.0, 'Overall Accuracy', 100.0]
['', 'Omission Error', 0.0, 'Kappa coefficient', nan]

/home/mneteler/.grass8/addons/scripts/r.confusionmatrix:386: RuntimeWarning: invalid value encountered in double_scalars
  kappa = (overall_accuracy / 100 - pc) / (1 - pc)
WARNING: Only one class in reference dataset.
Cleaning up...

F
======================================================================
FAIL: test_confusionmatrix_with_raster_reference (__main__.Testconfusionmatrix)
Test confusionmatrix with raster reference
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mneteler/software/grass-addons/src/raster/r.confusionmatrix/testsuite/test_r_confusionmatrix.py", line 71, in test_confusionmatrix_with_raster_reference
    self.assertFilesEqualMd5(
  File "/home/mneteler/software/grass_main/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 733, in assertFilesEqualMd5
    self.fail(self._formatMessage(msg, stdmsg))
AssertionError: Output file is not equal to reference file 
Files output_confusionmatrix.csv and data/confusionmatrix_raster_matrix.csv don't have the same MD5 sums

======================================================================
FAIL: test_confusionmatrix_with_singleclass (__main__.Testconfusionmatrix)
Test confusionmatrix with a single class reference and classification
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mneteler/software/grass-addons/src/raster/r.confusionmatrix/testsuite/test_r_confusionmatrix.py", line 90, in test_confusionmatrix_with_singleclass
    self.assertFilesEqualMd5(
  File "/home/mneteler/software/grass_main/dist.x86_64-pc-linux-gnu/etc/python/grass/gunittest/case.py", line 733, in assertFilesEqualMd5
    self.fail(self._formatMessage(msg, stdmsg))
AssertionError: Output file is not equal to reference file 
Files output_confusionmatrix.csv and data/confusionmatrix_raster_matrix_oneclass.csv don't have the same MD5 sums

----------------------------------------------------------------------
Ran 2 tests in 1.537s
FAILED (failures=2)

Expected behavior
Successful run of testsuite

System description (please complete the following information):

  • Operating System: Linux Fedora 36
  • latest GRASS GIS version 8.3.dev from Github
g.version -rge
version=8.3.dev
date=2022
revision=8dd3ce7d0c
build_date=2022-11-04
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8
libgis_revision=b35aca1c58
libgis_date=2022-08-28T12:21:16+00:00
proj=8.2.1
gdal=3.4.3
geos=3.10.2
sqlite=3.36.0
@neteler neteler added the bug Something isn't working label Nov 4, 2022
@neteler
Copy link
Member Author

neteler commented Nov 25, 2022

Perhaps also interesting:

Add json output option to r.kappa
OSGeo/grass#2666

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants