Packaging Update v10.4.1

Please note that this is NOT a release for general availability. Those running Heketi as a server application should continue to use v10.4.0. This release exists to fix issues with Go modules which only affects code that is importing Heketi as a library.

Release 10.4

This is a security/dependency related release.

Changelog

• Update dependency tool to go modules and refresh base dependencies (PR #1867)
• Update JWT library dependency from form3-tech/jwt-go to golang-jwt/jwt (PR #1822)
• Display the Go version used in build when running heketi --version (PR #1868)

Release builds are now built on CentOS7. This should resolve glibc symbol versioning issues on some distributions.

Release 10.3

This is a security/dependency related release.

Changelog

• Makefile improvement when executing glide (PR #1834)
• Enforce specific version of PyJWT (PR #1830)
• Update JWT library dependency from vulnerable dgrijalva/jwt-go to form3-tech/jwt-go (PR #1822)

Release 10.2

This is a bugfix and client API improvement release.

Changelog

• Fix an issue with the enumeration of pending operations (PR #1815) [1]
• Fix a memory leak caused by use of incompatible gorilla context (PR #1799)
• Fix a case when gluster command returns without a proper error code (PR #1795)
• Fix an issue in heketi-cli when the server does not set content-length (PR #1789)
• Allow consumers of the heketi client API (golang only) to customize how the http client is configured (PR #1771)

[1] - In some cases related to this issue stale pending operations may not be automatically cleaned, due to a identification of the operation type. If you think you have encountered this condition, please file an issue. The original issue was introduced in Heketi version v10.0.0.

Release 10.1

Release 10.1.0

This is a security and bugfix release.

An information-disclosure flaw was found in the way Heketi logs sensitive
information. This flaw allows an attacker with access to the Heketi server logs
to read potentially sensitive information, such as the CHAP passwords for
gluster-block volumes (CVE-2020-10763).

Administrators may want to check old logs for gluster-block passwords if they
created block volumes with CHAP authentication enabled. Restrict access or
remove old logs that retain the passwords.

Thanks to Prasanna Kumar Kalever of Red Hat for finding and fixing this issue.

Changelog

• Fix CVE-2020-10763
• Fix an issue removing/replacing devices on unrecoverable failed nodes
• Add a flag to skip a gluster heal check when gluster can not report on heals (when a node has failed or unable to perform the required action).

Release 10

Major additions in this release

• Heketi now tracks additional metadata for devices, based on LVM PV UUID. This allows Heketi to manage devices even if the device path (for example /dev/sda) changes if the node is rebooted.
• Authentication is now always enabled. [1]
• Heketi now provides a tool to patch endpoints. This can be used when nodes have changed to update endpoints to match.
• Heketi now sets up a strict set of volume options for the 'heketidbstorage' volume by default
• Heketi now provides tooling to update the volume options for existing 'heketidbstorage' volumes
• Heketi can now be run with specific ssh known hosts
• A system for matching devices with volumes has been added. This tag matching [2] system allows admisistrators to create devices tiers or similar scenarios.
• LVM commands can now be executed via a wrapper script. For container use cases, this wrapper script can be used to execute LVM commands on the host. However, the script can be used for other purposes as needed.
• Added a new "brick evict" feature: brick eviction removes a single specified brick from a volume, and then automatically replaces it.
• Device remove is now based on brick eviction. By basing device remove on brick eviction device removal should now be more reliable and debuggable.
• Heketi can now expand existing block volumes via the heketi-cli blockvolume expand subcommand.

Changelog

Among numerous other stability and bug fixes:

• Improved the behavior and logging for executed commands
• Only apply volume limit to block volumes that create a block hosting volume
• Perform device resync updates in a single db transaction
• Do not perform device resync for devices with a pending operation
• Fix issue where old BHVs could not be cleaned up
• Add a '--cluster' boolean flag to the device resync command.
• Add metrics for operations
• For commands that work on multiple clusters, errors that occur on one or more clusters can be reported by cluster id
• Fixed a panic within the operations cleanup subsystem
• Topology prints "removed" rather than "failed" state
• The kubernetes executor can now time out "stuck" commands like the SSH executor could
• Improve error reporting when device setup fails
• Support specifying a node selector for the heketi-storage-copy-job command
• Fixed an issue with internal db inconsistencies after expanding a volume
• Extended device metrics
• Fix a panic within the diagnostic code used to help debug unmount failures
• Allow specifying mount options for bricks in heketi.json or an environment variable.
• The selection of valid block hosting volumes is now randomized
• Prevent characters in a "db dump" from being treated like go format strings

[1] - Authentication can no longer be turned off via the configuration JSON file. Authentication can be disabled for testing and development purposes with a command line option to the server only.

[2] - https://github.com/heketi/heketi/blob/release/10/docs/design/tag-matching.md

Release 9

This is the new stable version of Heketi.

Major additions in this release

• We are no longer building darwin binaries as part of our release process. If you need darwin support please build from source
• Limit volumes per Gluster cluster
• Prevent server from starting if db has unknown dbattributes
• Support a default admin mode option
• Add an option to enable strict zone checking on volume creation
• Add automatic pending operation clean-up functionality
• Configurable device formatting parameters
• Add consistency check feature and state examiner debugging tools
• The faulty and non-functional "db delete-pending-entries" command has been removed

Changelog

Among numerous other stability and bug fixes:

• Refactored ssh and kubernetes executor core code
• container: suppress stat errors in heketi-start.sh
• apps: use "last takes precedence" when extracting heketi opts
• apps: add the heketi volume id to the gluster volume meta data
• apps: refactor app creation and setup
• apps: improve error text when unmarshaling from db fails
• client: for completeness support volume options on heketidbstorage
• Upgrade negroni version to 1.0.0 and update glide.lock
• metrics: Add block volume count to metrics
• heketi-cli: add the gid to the pvc file generated with --persistent-volume-file
• cli: print distribute & replica counts in volume info
• executors: allow specifying a default value for gluster-block prealloc
• client: add error handling to server operations client funcs
• logging: grab output of 'lsof /path/to/brick' in case unmounting fails
• client: add tags output to nodes and devices
• apps: update total counter when checking pending ops in db
• apps: add cluster level error reporting to cluster selection
• apps: capture all pending file and block volumes from operation types
• Metrics: convert device info from KiB to bytes.
• apps: on startup, do not try to "fix" pending BHVs
• heketi: add support for profiling with pprof
• cmdexec: introduce a new config setting gluster_cli_timeout
• metrics: add device metrics in bytes format
• pkg/utils: relicense to dual Apache2 and LGPLv3+
• apps/glusterfs: allow default pre and post volume options
• Shorten file path in logs by redundant "/src/github.com/heketi/" part
• client: add --force-forget option to device delete cli
• executors: do not ignore failures to remove vg/pv
• executors: properly handle output of gluster-block create

Release 8

Release 8.0.0

This is the new stable version of Heketi.

Major additions in this release

• Resumable delete of Volumes and Block Volumes
• Server administrative modes
• Throttling of concurrent operations
• Support configuration of block hosting volume options
• Heketi cli command to fetch operation counters
• Support setting restrictions on block hosting volume; to prevent block hosting volumes from taking new block volumes
• Add an option to destroy data while adding a device to a node
• Heketi Container: load an initial topology if HEKETI_TOPOLOGY_FILE is set

Changelog

Among numerous other stability and bug fixes:

• Support colon characters in device paths
• arbiter: fix volume creation with too big average file size
• Fix issue where Heketi allowed multiple block volume delete operations to be stored in the db
• executors: do not create backups in /etc/lvm/archive by default
• Add grace period to IAT claim in JWT
• Fix incorrectly updating the block hosting volume free size on rollback
• Prevent "Id not found" errors on various functions
• Reserve a small percent of raw capacity of block hosting volumes for metadata
• Correct the usage of number of nodes involved in Gluster block HA
• Shuffle ha nodes to avoid relying on same node
• Correctly update the free size in block info on expansion
• Improve device resync behavior
• Correct block hosting volume free size counters when error conditions are hit
• Prevent creating extra Block Hosting Volumes for concurrent block hosting volume create requests
• Automatically correct some known incorrect block hosting volume values on server restart
• Heketi Container: use a non-blocking flock lock to avoid issue in some versions of Glusterfs
• Fix formatting parameters of arbiter bricks

Release 7

Release 7.0.0

This is the new stable version of Heketi.

Major additions in this release

• support for arbiter volumes
• support for cloning of volumes
• support for prometheus metrics
• native TLS support

Changelog

• add support for arbiter volumes
• support for cloning volumes through a new clone operation on the source volume
• add support for fetching topology info in prometheus format through a new /metrics endpoint
• support using https to access heketi through native TLS
• add a "destroy-existing-data" option to clear devices when added / topology is loaded
• add HEKETI_DB_PATH environment variable to override heketi.json setting
• enable rebalance on volume expansion by default
• make heketi operations more resilient against node outages by adding a node health monitor and a retry mechanism
• cli: show number of bricks in 'node info' output
• deny creation of block-hosting volumes if disabled
• add support to get and set log level to API and heketi-cli
• add a command to clean pending operations from the database
• add environment variable HEKETI_IGNORE_STALE_OPERATIONS
• doc: add design doc for snapshot and cloning
• several other bugfixes
• several other documentation updates and enhancements
• github: add an issue template
• github: add a pull request template

Release 6

Release 6.0.0

This is the new stable version of Heketi.

The main additions in this release are the block-volume API, a great deal of stabilization to prevent inconsistent database and out-of-sync situations, and tooling to do disaster recovery when the database is bad.

Changelog

• Add support for gluster-block volumes
• Add device resync API.
• A lot of internal restructuring and code cleanup.
• Greatly improved robustness, preventing inconsistent database state.
• Add a database import and export feature.
• Add a database repair mode (cleaning orphaned bricks).
• Allow to set heketi's log level through the HEKETI_GLUSTERAPP_LOGLEVEL environment variable.
• Many other bug fixes and improvements.