Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance increase in writeIFD #3680

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from
Draft

Performance increase in writeIFD #3680

wants to merge 2 commits into from

Conversation

iwbh15
Copy link

@iwbh15 iwbh15 commented Apr 7, 2021

out as “random access stream around the given file” exchange by temporary random access stream around a ByteArrayHandle.

out as “random access stream around the given file” exchange by temporary random access stream around a ByteArrayHandle.
@dgault dgault added this to the 6.7.0 milestone Apr 7, 2021
@dgault
Copy link
Member

dgault commented Apr 7, 2021

From imagesc message:

In TiffSaver.java the function writeIFD() loops over IFD keys in

    for (Integer key : keys) {
      if (key.equals(IFD.LITTLE_ENDIAN) || key.equals(IFD.BIG_TIFF) ||
          key.equals(IFD.REUSE)) continue;

      Object value = ifd.get(key);
      
      writeIFDValue(extraStream, ifdBytes + fp, key.intValue(), value);
    }

In writeIFDValue various write operations are executed. This operations write values into the RandomAccessOutputStream out and thereby force several bounds checks. This bounds checks seems to be slow and take a significant amount of time because out is a “random access stream around the given file”.

If a temporary random access stream around a ByteArrayHandle is used to collect all data before writing it to the file in a single write operation, about 33% of the operation time can be saved.

I got the following times for the
test image loaded in original QuPath without modification in the bioformats package:

Run 1: Write OME-TIFF in : 20332 ms
Run 2: Write OME-TIFF in : 15441 ms
Run 3: Write OME-TIFF in : 15390 ms
Run 4: Write OME-TIFF in : 15211 ms
I got the following times for the
test image loaded in QuPath without my modification in the bioformats package:

Run 1: Write OME-TIFF in : 14764 ms
Run 2: Write OME-TIFF in : 10880 ms
Run 3: Write OME-TIFF in : 10241 ms
Run 4: Write OME-TIFF in : 9754 ms

@dgault dgault added the include label May 27, 2021
@dgault
Copy link
Member

dgault commented Aug 17, 2021

Tested by converting the below public datasets, each run 5 times and averaged (time in seconds):

Dataset With 6.6.1 With PR
tubhiswt-2D 0.9916 0.621
tubhiswt-3D 1.851 1.133
tubhiswt-4D 11.4598 9.8055
MitoCheck 5.73525 6.56225
ND2 karl 8.2606 10.4744
ND2 maxime 2.047 2.4804
Leica-LIF michael 1.8748 1.3592
Zeiss-CZI idr0011 5.2698 5.8498
Olympus-OIR etienne 5.1768 5.6236

Those runs were proving inconclusive with mixed results, as a follow up I ran the Java performance tests that were originally designed for the profiling of ome-files (https://github.com/ome/ome-files-performance) and the results again was not as expected.

NIRHTa-001.ome.tiff - 6.6.1:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read NIRHTa-001.ome.tiff 27425 22690 18037 4653
Java pixeldata.read.init NIRHTa-001.ome.tiff 6827 4309 3298 1011
Java pixeldata.read.pixels NIRHTa-001.ome.tiff 20598 18381 14739 3641
Java pixeldata.write NIRHTa-001.ome.tiff 41230 31559 14488 17070
Java pixeldata.write.init NIRHTa-001.ome.tiff 25 11 10 1
Java pixeldata.write.pixels NIRHTa-001.ome.tiff 27077 23147 11458 11688
Java pixeldata.write.close NIRHTa-001.ome.tiff 14127 8400 3020 5380

NIRHTa-001.ome.tiff - with this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read NIRHTa-001.ome.tiff 26353 20279 15205 5073
Java pixeldata.read.init NIRHTa-001.ome.tiff 6449 3972 2966 1005
Java pixeldata.read.pixels NIRHTa-001.ome.tiff 19903 16306 12239 4067
Java pixeldata.write NIRHTa-001.ome.tiff 230290 192029 19388 172641
Java pixeldata.write.init NIRHTa-001.ome.tiff 19 15 9 6
Java pixeldata.write.pixels NIRHTa-001.ome.tiff 204939 175540 13989 161550
Java pixeldata.write.close NIRHTa-001.ome.tiff 25331 16473 5389 11084

MitoCheck - 6.6.1:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read 00001_01.ome.tiff 2530 2362 2133 228
Java pixeldata.read.init 00001_01.ome.tiff 1543 1401 1309 91
Java pixeldata.read.pixels 00001_01.ome.tiff 987 961 824 136
Java pixeldata.write 00001_01.ome.tiff 3195 2944 2569 375
Java pixeldata.write.init 00001_01.ome.tiff 5 5 4 0
Java pixeldata.write.pixels 00001_01.ome.tiff 869 726 414 312
Java pixeldata.write.close 00001_01.ome.tiff 2320 2212 2149 62

MitoCheck - with this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read 00001_01.ome.tiff 4861 4013 3606 405
Java pixeldata.read.init 00001_01.ome.tiff 3502 2763 2561 200
Java pixeldata.read.pixels 00001_01.ome.tiff 1358 1249 1044 204
Java pixeldata.write 00001_01.ome.tiff 4887 4422 3839 583
Java pixeldata.write.init 00001_01.ome.tiff 10 9 8 1
Java pixeldata.write.pixels 00001_01.ome.tiff 1250 1035 637 397
Java pixeldata.write.close 00001_01.ome.tiff 3626 3377 3193 183

tubhiswt-4D - 6.6.1:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read tubhiswt_C0_TP0.ome.tif 3598 2666 1986 680
Java pixeldata.read.init tubhiswt_C0_TP0.ome.tif 730 619 508 110
Java pixeldata.read.pixels tubhiswt_C0_TP0.ome.tif 2867 2047 1477 570
Java pixeldata.write tubhiswt_C0_TP0.ome.tif 3492 3191 1577 1613
Java pixeldata.write.init tubhiswt_C0_TP0.ome.tif 5 5 4 0
Java pixeldata.write.pixels tubhiswt_C0_TP0.ome.tif 2957 2707 1211 1495
Java pixeldata.write.close tubhiswt_C0_TP0.ome.tif 529 479 362 117

tubhiswt-4D - with this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read tubhiswt_C0_TP0.ome.tif 5479 4361 3358 1002
Java pixeldata.read.init tubhiswt_C0_TP0.ome.tif 1139 975 765 210
Java pixeldata.read.pixels tubhiswt_C0_TP0.ome.tif 4340 3385 2593 792
Java pixeldata.write tubhiswt_C0_TP0.ome.tif 3428 2968 1783 1184
Java pixeldata.write.init tubhiswt_C0_TP0.ome.tif 6 6 5 0
Java pixeldata.write.pixels tubhiswt_C0_TP0.ome.tif 2694 2298 1284 1014
Java pixeldata.write.close tubhiswt_C0_TP0.ome.tif 727 663 493 170

@dgault
Copy link
Member

dgault commented Aug 18, 2021

Pushing this to 6.8.0 to continue testing in order to better understand the impact of the changes across a variety of datasets

@dgault dgault modified the milestones: 6.7.0, 6.8.0 Aug 18, 2021
@sbesson sbesson requested a review from dgault November 15, 2021 14:09
@dgault dgault modified the milestones: 6.8.0, 6.9.0 Dec 7, 2021
@dgault
Copy link
Member

dgault commented Dec 7, 2021

I reran a number of the performance tests again, this time against 6.7.0. The results are again somewhat inconclusive, the PR performs very well against some datasets such as BBBC but is comparable or slightly slower for others. I would still like to dig into this further to understand exactly what situations this PR performs best in. Pushing to 6.9.0 to re investigate in the new year.

public/2016-06/tubhiswt-4D

With 6.7.0:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read tubhiswt_C0_TP0.ome.tif 2019 1718 1275 442
Java pixeldata.read.init tubhiswt_C0_TP0.ome.tif 532 470 388 81
Java pixeldata.read.pixels tubhiswt_C0_TP0.ome.tif 1487 1248 887 360
Java pixeldata.write tubhiswt_C0_TP0.ome.tif 3525 2769 1414 1355
Java pixeldata.write.init tubhiswt_C0_TP0.ome.tif 4 4 3 0
Java pixeldata.write.pixels tubhiswt_C0_TP0.ome.tif 3122 2379 1121 1257
Java pixeldata.write.close tubhiswt_C0_TP0.ome.tif 398 386 288 97
Java metadata.read tubhiswt_C0_TP0.ome.tif 340 287 258 28
Java metadata.write tubhiswt_C0_TP0.ome.tif 687 619 575 43

With this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read tubhiswt_C0_TP0.ome.tif 1960 1741 1304 436
Java pixeldata.read.init tubhiswt_C0_TP0.ome.tif 573 506 414 91
Java pixeldata.read.pixels tubhiswt_C0_TP0.ome.tif 1387 1235 890 345
Java pixeldata.write tubhiswt_C0_TP0.ome.tif 3058 2267 1346 920
Java pixeldata.write.init tubhiswt_C0_TP0.ome.tif 4 4 3 0
Java pixeldata.write.pixels tubhiswt_C0_TP0.ome.tif 2594 1822 1011 810
Java pixeldata.write.close tubhiswt_C0_TP0.ome.tif 458 440 331 109
Java metadata.read tubhiswt_C0_TP0.ome.tif 304 271 243 27
Java metadata.write tubhiswt_C0_TP0.ome.tif 535 512 478 34

public/2016-06/BBBC/NIRHTa-001.ome.tiff

With 6.7.0:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read NIRHTa-001.ome.tiff 84497 20574 14903 5670
Java pixeldata.read.init NIRHTa-001.ome.tiff 53498 5576 2923 2653
Java pixeldata.read.pixels NIRHTa-001.ome.tiff 30999 14997 11980 3017
Java pixeldata.write NIRHTa-001.ome.tiff 498571 319218 17075 302142
Java pixeldata.write.init NIRHTa-001.ome.tiff 9 8 7 0
Java pixeldata.write.pixels NIRHTa-001.ome.tiff 488468 311556 14110 297445
Java pixeldata.write.close NIRHTa-001.ome.tiff 10093 7653 2957 4696
Java metadata.read NIRHTa-001.ome.tiff 714 626 568 57
Java metadata.write NIRHTa-001.ome.tiff 3210 3168 2916 252

With this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read NIRHTa-001.ome.tiff 85328 20564 14818 5745
Java pixeldata.read.init NIRHTa-001.ome.tiff 52732 5499 2916 2582
Java pixeldata.read.pixels NIRHTa-001.ome.tiff 32596 15065 11902 3163
Java pixeldata.write NIRHTa-001.ome.tiff 293042 281321 13875 267446
Java pixeldata.write.init NIRHTa-001.ome.tiff 10 9 7 1
Java pixeldata.write.pixels NIRHTa-001.ome.tiff 284517 274954 10979 263974
Java pixeldata.write.close NIRHTa-001.ome.tiff 8514 6357 2888 3469
Java metadata.read NIRHTa-001.ome.tiff 625 554 500 53
Java metadata.write NIRHTa-001.ome.tiff 2899 2856 2705 151

public/2016-06/sub-resolutions/Fluorescence/LuCa-7color_Scan1.ome.tiff

With 6.7.0:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read LuCa-7color_Scan1.ome.tiff 146199 113315 106156 7158
Java pixeldata.read.init LuCa-7color_Scan1.ome.tiff 549 458 406 51
Java pixeldata.read.pixels LuCa-7color_Scan1.ome.tiff 145650 112856 105749 7106
Java pixeldata.write LuCa-7color_Scan1.ome.tiff 46756 18517 6020 12497
Java pixeldata.write.init LuCa-7color_Scan1.ome.tiff 9 7 5 2
Java pixeldata.write.pixels LuCa-7color_Scan1.ome.tiff 45940 18134 5790 12343
Java pixeldata.write.close LuCa-7color_Scan1.ome.tiff 806 375 224 151
Java metadata.read LuCa-7color_Scan1.ome.tiff 211 203 183 19
Java metadata.write LuCa-7color_Scan1.ome.tiff 376 370 335 34

With this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read LuCa-7color_Scan1.ome.tiff 103991 89697 84476 5220
Java pixeldata.read.init LuCa-7color_Scan1.ome.tiff 386 334 284 49
Java pixeldata.read.pixels LuCa-7color_Scan1.ome.tiff 103604 89363 84192 5171
Java pixeldata.write LuCa-7color_Scan1.ome.tiff 47337 15316 5000 10316
Java pixeldata.write.init LuCa-7color_Scan1.ome.tiff 6 5 4 1
Java pixeldata.write.pixels LuCa-7color_Scan1.ome.tiff 45624 14995 4832 10162
Java pixeldata.write.close LuCa-7color_Scan1.ome.tiff 1706 314 162 151
Java metadata.read LuCa-7color_Scan1.ome.tiff 266 223 194 28
Java metadata.write LuCa-7color_Scan1.ome.tiff 378 359 322 37

public/2016-06/MitoCheck/00001_01.ome.tiff

With 6.7.0:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read 00001_01.ome.tiff 2585 2490 2150 339
Java pixeldata.read.init 00001_01.ome.tiff 1786 1702 1514 187
Java pixeldata.read.pixels 00001_01.ome.tiff 799 787 635 151
Java pixeldata.write 00001_01.ome.tiff 11178 10356 2612 7744
Java pixeldata.write.init 00001_01.ome.tiff 7 6 5 0
Java pixeldata.write.pixels 00001_01.ome.tiff 8845 8068 461 7606
Java pixeldata.write.close 00001_01.ome.tiff 2326 2281 2145 136
Java metadata.read 00001_01.ome.tiff 948 811 748 62
Java metadata.write 00001_01.ome.tiff 8823 8680 8466 213

With this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read 00001_01.ome.tiff 4581 2721 2288 432
Java pixeldata.read.init 00001_01.ome.tiff 2727 1800 1581 218
Java pixeldata.read.pixels 00001_01.ome.tiff 1853 921 706 214
Java pixeldata.write 00001_01.ome.tiff 6483 5747 3276 2470
Java pixeldata.write.init 00001_01.ome.tiff 7 7 6 0
Java pixeldata.write.pixels 00001_01.ome.tiff 3413 2723 493 2230
Java pixeldata.write.close 00001_01.ome.tiff 3062 3015 2777 238
Java metadata.read 00001_01.ome.tiff 963 934 860 73
Java metadata.write 00001_01.ome.tiff 10392 10171 9866 305

public/2016-06/sub-resolutions/Z-stack/retina_large.ome.tiff

With 6.7.0:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read retina_large.ome.tiff 80689 78559 31175 47383
Java pixeldata.read.init retina_large.ome.tiff 1354 609 415 193
Java pixeldata.read.pixels retina_large.ome.tiff 79335 77949 30759 47189
Java pixeldata.write retina_large.ome.tiff 10680 9542 1351 8190
Java pixeldata.write.init retina_large.ome.tiff 9 7 5 1
Java pixeldata.write.pixels retina_large.ome.tiff 9602 9095 1058 8037
Java pixeldata.write.close retina_large.ome.tiff 1067 438 287 151
Java metadata.read retina_large.ome.tiff 267 234 211 22
Java metadata.write retina_large.ome.tiff 571 560 509 51

With this PR:

test.lang test.name test.file treal proc.cpu proc.user proc.system
Java pixeldata.read retina_large.ome.tiff 88764 85119 34105 51013
Java pixeldata.read.init retina_large.ome.tiff 838 815 605 209
Java pixeldata.read.pixels retina_large.ome.tiff 87925 84304 33500 50803
Java pixeldata.write retina_large.ome.tiff 14205 13400 1076 12323
Java pixeldata.write.init retina_large.ome.tiff 5 4 3 0
Java pixeldata.write.pixels retina_large.ome.tiff 13759 13040 843 12197
Java pixeldata.write.close retina_large.ome.tiff 439 355 229 126
Java metadata.read retina_large.ome.tiff 278 267 243 24
Java metadata.write retina_large.ome.tiff 622 610 557 52

@petebankhead petebankhead mentioned this pull request Dec 12, 2021
@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1103. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1104. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@jburel
Copy link
Member

jburel commented Jan 25, 2022

Conflicting PR. Removed from build BIOFORMATS-push#7. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@jburel
Copy link
Member

jburel commented Jan 25, 2022

Conflicting PR. Removed from build BIOFORMATS-push#8. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@jburel
Copy link
Member

jburel commented Jan 25, 2022

Conflicting PR. Removed from build BIOFORMATS-push#9. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@jburel
Copy link
Member

jburel commented Jan 25, 2022

Conflicting PR. Removed from build BIOFORMATS-push#4. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@jburel
Copy link
Member

jburel commented Jan 25, 2022

Conflicting PR. Removed from build BIOFORMATS-push#5. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1120. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@dgault dgault removed this from the 6.9.0 milestone Feb 21, 2022
@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#12. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1150. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@sbesson sbesson added this to the 6.11.1 milestone Oct 3, 2022
@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#239. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@dgault dgault removed this from the 6.11.1 milestone Nov 30, 2022
@dgault dgault modified the milestones: 6.12.0, 6.12.1 Feb 6, 2023
@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#373. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1522. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#374. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@dgault dgault modified the milestones: 6.12.1, 6.14.0 Apr 3, 2023
@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#429. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@khaledk2
Copy link

khaledk2 commented Jun 7, 2023

Conflicting PR. Removed from build BIOFORMATS-push#1. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#503. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@khaledk2
Copy link

khaledk2 commented Jun 8, 2023

Conflicting PR. Removed from build BIOFORMATS-push#2. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#504. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@khaledk2
Copy link

khaledk2 commented Jun 9, 2023

Conflicting PR. Removed from build BIOFORMATS-push#3. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#505. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#5. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#1. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@snoopycrimecop
Copy link
Member

Conflicting PR. Removed from build BIOFORMATS-push#511. See the console output for more details.
Possible conflicts:

  • Upstream changes
    • components/formats-bsd/src/loci/formats/tiff/TiffSaver.java

--conflicts

@ome ome deleted a comment from khaledk2 Jun 12, 2023
@ome ome deleted a comment from khaledk2 Jun 12, 2023
@ome ome deleted a comment from khaledk2 Jun 12, 2023
@ome ome deleted a comment from khaledk2 Jun 12, 2023
@dgault dgault added exclude and removed include labels Jun 12, 2023
@dgault dgault removed this from the 6.14.0 milestone Jul 3, 2023
@dgault
Copy link
Member

dgault commented Jul 3, 2023

Converting this PR to draft for now, this PR is something we may well want to return too as part of wider look at improving write performance

@dgault dgault marked this pull request as draft July 3, 2023 14:48
@melissalinkert melissalinkert added this to the 8.0.0 milestone Jan 15, 2024
@melissalinkert
Copy link
Member

Tested with the commits here, compared with the current state of develop. Note that iwbh/develop has been deleted, so checking out that branch may not include the commits here.

I see similar results to what @dgault observed above. For things with lots of small IFDs, there is some improvement, but not necessarily enough to be noticeable in terms of overall conversion time. For large images with few IFDs, performance is the same or slightly worse. I tested with https://downloads.openmicroscopy.org/images/Hamamatsu-NDPI/openslide/CMU-1/CMU-1.ndpi and 4D_test&sizeZ=30&sizeT=1000&pixelType=uint16.fake in particular.

Given the inconclusive testing results, I propose to close this PR but add #3983, #3480, and ome/ome-common-java#78 to the 8.0.0 milestone (so that we prioritize measurable progress on this topic), unless any objections in the next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants