Skip to content

Releases: janelia-flyem/dvid

Version 1.0 Release

18 Aug 18:26
Compare
Choose a tag to compare

A backward-compatible release that can be used for previously created databases. A future 2.0 version DVID will have a metadata overhaul to remove unused legacy datatypes and simplify management.

Changes include:

  • Many neuronjson datatype improvements.
    • Allows multiple in-memory databases for both committed and branch HEAD versions of neuronjson datatype.
    • Automatic _user and _time fields that won't automatically change if field values aren't changed even if replacement requested.
  • Improvements in handling of proxied uint8blk grayscale image volumes from neuroglancer precomputed data via gridstore interface.
  • Allow simpler tag-style identifiers as assigned UUID (e.g., v1.0)
  • Allow disabling /split operations via configuration
  • Much faster /sparsevol operations if requesting supervoxels.
  • Faster range query scans in /blocks for annotation datatype

Neuronjson Release

25 Oct 03:39
Compare
Choose a tag to compare

Significant changes:

  • New neuronjson datatype that is tailored for a small JSON doc (value) for each neuron ID (key). It is a replacement for neuron annotations that had previously been held within keyvalue instances and uses in-memory store for the HEAD version and persists data to chosen backend.
  • Upgrade from Badger v2 to v3. This will require you to upgrade your Badger database directories if you have an older version.
  • Switch to Go modules from older "go get" install scripts.
  • Simplified conda builds and release due to switch to Go modules. (@stuarteberg)
  • Support for AWS S3 in the ngprecomputed backend. (Contributed by @pgunn, Flatiron Institute)
  • Expanded number of versions for in-memory mapping in labelmap datatype. Limit before was 256 versions, now expanded to uint32.
  • Ability to renumber labels in labelmap datatype, as well as keeping track of split or renumbered labels by inserting 0 into in-memory mapping.
  • Can now specify Xth parent of a branch leaf using uuid:branch~X for UUID.
  • Lots of fixes.

Maintenance Release

26 Mar 19:45
Compare
Choose a tag to compare
  • fix labelmap /history and improve documentation for it
  • add neuronjson datatype with Google Firestore persistence
  • add optimized ingest supervoxel blocks
  • fix bug in install-developer-dependencies
  • Kafka-related changes:
    • switch to GoCloud SDK pubsub for Kafka support
    • convert to sarama kafka driver; send Key for compacted log issue?
    • use async producer for kafka
    • remove deprecated kafka properties from testing
  • Optimizations for concurrent interactions with backend for labelmap blocks (leveldb)
    • remove Cgo tracking from read hotpaths; not necessary
    • simplify active CGo write tracking by removing channel in favor of atomic write
    • restrict concurrency in labelmap /specificblocks to transcoding

Maintenance Release

09 Jan 20:49
Compare
Choose a tag to compare
  • Remove unused, old storage drivers
  • Various build script changes
  • Now compiled with go-1.16

Maintenance Release

16 May 19:11
Compare
Choose a tag to compare

Improved tooling for repository handling and fixes.

Maintenance Release

28 Feb 22:13
Compare
Choose a tag to compare

Low-memory mode disabled

Previous binary release tarballs (from v0.9.3 through v0.9.10) were built with DVID_LOW_MEMORY=1, which forced badger databases to operate in a special low-memory mode. This is not desirable for production uses. Release builds published here on our github releases page had sub-optimal performance. Our build procedure has changed to avoid using DVID_LOW_MEMORY in release builds.

Other changes

  • add update of max label on label index ingestion; fixes #352
  • return 404 instead of 204 for non-existant key in keyvalue instance; fixes #353
  • allow changing branch names as part of flatten-metadata rpc command
  • fix possible race condition on labelsz test, which might lead to test fails on resource-limited build machines

Maintenance Release

20 Feb 20:41
Compare
Choose a tag to compare
  • add /listlabels to labelmap
  • do fsync on logs to guarantee writes in exchange of throughput
  • add public versions and Google id auth
  • add ability to set next labels in labelmap
  • add more efficient /indices-compressed to labelmap
  • force directory creation for json mutation log
  • don't let server start if mutations jsonstore can't be created when specified
  • By default, release distribution tarball uses dvid-web-console 2.1.7
  • allow repo-level rename/new instance rpc on locked node

Note: No Mac tarball yet.

Maintenance Release

16 Jan 00:25
Compare
Choose a tag to compare
  • allow setting readonly and fullwrite modes in TOML config
  • add error check for analyze-block command
  • add scale option for analyze-block cmd
  • add json mutation logging for other labelmap ops
  • fix jpeg uncompression on ngprecomputed
  • add unsharded ng precomputed support; allow auto instances for ngprecomputed
  • fix rounding error on ng precomputed indexing
  • add POST /api/repo/{uuid}/info
  • further error checking on ngprecomputed block encodes
  • make ngprecomputed more resilient to various JPEG encoding styles
  • revert imageblk GetVoxels before gridstore support
  • fixes number of issues for ngprecomputed driver including clipped chunks
  • try another approach to wildcard origin handler
  • fix bug introduced patching issue #347
  • fix initialization of ngprecomputed extents
  • handle metadata and improve doc for ngprecomputed uint32blk
  • fix inconsistent CORS handling based on config

Maintenance Release

10 Aug 22:33
501fc1c
Compare
Choose a tag to compare
  • JSON marshaling of data instances' versioned props in repos/info assumes leaf of master branch
  • fix buggy extents update; fixes #301
  • add POST /extents to labelmap; fixes #344
  • add documentation for POST /extents to labelmap
  • add uncompressed data return in subvolblocks endpoint; fixes #347
  • minor init logging formatting change
  • allow admin-enabled mutations to bypass read-only mode
  • upgrade to lz4 1.9.2

Maintenance Release

05 Jun 04:31
Compare
Choose a tag to compare
  • fix annotation block size at 64. Note this is dangerous if you have legacy annotation at 32
  • add POST /blocks to annotation
  • fixes to flatten-metadata
  • add admin token querystring to allow selective full write
  • add version-specific migration and full test suite
  • allow version range for mutation log flattening
  • add migrate-batch command
  • enhance flatten metadata command to multi-version use
  • fix multiversion flattening
  • remove debug that can fill logs for large queries
  • decouple CORS from Authorization
  • fix dag duplication on version subset; add test
  • remove transactionDB/gbucket mutex and patch
  • prevent any metadata writes in read-only mode
  • fix copy kv pairs to also transmit version 0 metadata per data instance
  • prevent package doc in generated protobuf
  • allow wildcard auth and block based on HTTP Method
  • add annotation /all-elements request
  • add auth file reload without reboot
  • fix branch ancestry calc, initial mapping load
  • add labelmap mutation JSON log
  • prevent absolute path conversion from creating config entries
  • add ngprecomputed backend for static Neuroglancer Precomputed GCS store
  • add /keyrangevalues endpoint for keyvalue data type
  • add support for confluent cloud kafka auth
  • add kakfa topic creation
  • update confluent kafka libs
  • add auth option for just token verification
  • update brainmaps API for googlevoxels
  • allow CORS to be shutoff in case of nginx fronting
  • return better error responses for /api/server/token
  • allow wildcard access for CORS issues if auth turned off
  • Ship both versions of the dvid console, leave the old version active by default