20230227 developer call notes

[hammer]

Pre-notes

• Notes from previous meeting

PRs

• sgkit#1021 Generate VCF header when writing, if no header is explicitly supplied

Issues

• sgkit-publication
  • sgkit-publication#14 Building locally

Notes

Attendees

1. Jeff Hammerbacher
2. Tom White
3. Jerome Kelleher
4. Ben Jeffery
5. Rafal Wojdyla

Discussion

• sgkit#1021
  • TW feels GitHub discussion sufficient, will proceed with JK’s suggestion
• Oxford working on reading large VCFs
  • BJ working on 78k diploid genomes from Genomics England (GEL)
  • BJ hitting Dask scheduler issues; bigger blocks the answer?
  • Good read performance w/ Xarray API: 3 days w/ vcftools → 15s
  • Good storage performance: Zarr 10% - 15% on disk
• sgkit-publication PR#13
  • JK has some text!
  • Pushed PR
  • Best way to comment: GitHub issues
  • JH: let’s emphasize n-dimensional array model
  • JH: columnar good for I/O elimination + sequential access. What does columnar mean in n-dimensional arrays?
  • TW: compression improvements from columns too!
  • TW: OME Zarr paper came out today! OME-Zarr: a cloud-optimized bioimaging file format with international community support (2023)
  • RW: Let's mention GPUs! JK: heterogeneous computing.
• sgkit-publication#14
  • Manubot not solving problems but also creating problems
  • Write in LaTeX. GitHub first, Overleaf second.