Releases: TileDB-Inc/TileDB
Releases · TileDB-Inc/TileDB
TileDB 2.19.0
TileDB v2.19.0 Release Notes
Announcements
- TileDB 2.19, targeted for release in December 2023, includes performance improvements to the aggregate pushdown APIs.
Breaking behavior
- The Windows release artifacts no longer include static libraries. #4469
New features
- Allow aggregates to pipeline the processing of full tiles. #4465
- Add class TemporaryDirectory to generate a PRNG-seeded unique directory. #4498
- Add sm.consolidation.total_buffer_size setting. #4550
- Use tile metadata in aggregates. #4468
- Load validity tiles only for null count aggregates when possible. #4513
- Prevent reading tiles when using fragment metadata in aggregates. #4481
Improvements
- Refactor the fragment info loading code to reduce coupling. #4453
- Fragment metadata: adding get_tile_metadata. #4466
- Improve error reporting when performing too large GCS direct uploads. #4047
- Refactor filestore API so that it uses internal TileDB library calls. #3763
- Encode Win32 error messages in UTF-8. #3987
- Remove unused compression defines. #4480
- Integrate baseline object library into the main build. #4446
- Internal clean-up of array timestamps. #3633
- Add base class for VFS filesystems. #4497
- Remove extraneous TILEDB_EXPORT tags. #4507
- Query Plan REST support - part 1: handler. #4516
- Use StorageFormat APIs to generate timestamped uris. #4530
- Refactor the group code. #4378
- Consolidation plan REST support - part1: handler. #4534
- Use constexpr to check for filesystem support. #4548
- Add vfs.gcs.max_direct_upload_size config option. #4047
- Migrate fragment_name parsing into StorageFormat. #4520
- Add setting to disable install_sigpipe_handler on S3. #4573
Defects removed
- Group metadata doesn't get serialized. #3147
- Added
#include <array>
totiledb/sm/cpp_api/type.h
. #4504 - Fix broken linking tests against a shared WebP library. #4525
- Fix delta filter deserialization for format ver 19. #4541
- Fix array schema selection when time traveling. #4549
Documentation
- Add a table with the history of the storage format versions. #4487
- Add timestamped_name file to the Format Spec. #4533
API changes
C API
- Add tiledb_group_get_member_by{index|name}v2 (and deprecate tiledb_group_get_member_by{index|name}). #4019
tiledb_group_create
is no longer marked as deprecated. #4522
C++ API
- Improve error message in C++ API for bad filter option type. #4503
Build System Changes
- Improve building the Windows binaries for the R API. #4448
- Update curl to version 8.4.0. #4523
- Clean-up unused build system code. #4527
- Cache more information when configuring the CMake project. #4449
- Remove the vcpkg submodule and download it automatically when configuring. #4484
- Use the provided CMake packages to import the compression libraries when vcpkg is enabled. #4500
- Remove bundling our static library dependencies; they are now found from the CMake config file. #4505
- Remove the CMake and bootstrap script options related to abseil and crc32c linkage tests. #4527
- Move documentation generation and source formatting into dedicated CMake modules. #4538
- Fix regression when libraries were sometimes installed in the
lib64
directory. #4572
Test only changes
- Remove UnitTestConfig::array_encryption_key_length. #4482
TileDB 2.19.0-rc1
TileDB v2.19.0 Release Notes
Announcements
- TileDB 2.19, targeted for release in December 2023, includes performance improvements to the aggregate pushdown APIs.
Breaking behavior
- The Windows release artifacts no longer include static libraries. #4469
New features
- Allow aggregates to pipeline the processing of full tiles. #4465
- Add class TemporaryDirectory to generate a PRNG-seeded unique directory. #4498
- Add sm.consolidation.total_buffer_size setting. #4550
- Use tile metadata in aggregates. #4468
- Load validity tiles only for null count aggregates when possible. #4513
- Prevent reading tiles when using fragment metadata in aggregates. #4481
Improvements
- Refactor the fragment info loading code to reduce coupling. #4453
- Fragment metadata: adding get_tile_metadata. #4466
- Improve error reporting when performing too large GCS direct uploads. #4047
- Refactor filestore API so that it uses internal TileDB library calls. #3763
- Encode Win32 error messages in UTF-8. #3987
- Remove unused compression defines. #4480
- Integrate baseline object library into the main build. #4446
- Internal clean-up of array timestamps. #3633
- Add base class for VFS filesystems. #4497
- Remove extraneous TILEDB_EXPORT tags. #4507
- Query Plan REST support - part 1: handler. #4516
- Use StorageFormat APIs to generate timestamped uris. #4530
- Refactor the group code. #4378
- Consolidation plan REST support - part1: handler. #4534
- Use constexpr to check for filesystem support. #4548
- Add vfs.gcs.max_direct_upload_size config option. #4047
- Migrate fragment_name parsing into StorageFormat. #4520
- Add setting to disable install_sigpipe_handler on S3. #4573
Defects removed
- Group metadata doesn't get serialized. #3147
- Added
#include <array>
totiledb/sm/cpp_api/type.h
. #4504 - Fix broken linking tests against a shared WebP library. #4525
- Fix delta filter deserialization for format ver 19. #4541
- Fix array schema selection when time traveling. #4549
Documentation
- Add a table with the history of the storage format versions. #4487
- Add timestamped_name file to the Format Spec. #4533
API changes
C API
- Add tiledb_group_get_member_by{index|name}v2 (and deprecate tiledb_group_get_member_by{index|name}). #4019
tiledb_group_create
is no longer marked as deprecated. #4522
C++ API
- Improve error message in C++ API for bad filter option type. #4503
Build System Changes
- Improve building the Windows binaries for the R API. #4448
- Update curl to version 8.4.0. #4523
- Clean-up unused build system code. #4527
- Cache more information when configuring the CMake project. #4449
- Remove the vcpkg submodule and download it automatically when configuring. #4484
- Use the provided CMake packages to import the compression libraries when vcpkg is enabled. #4500
- Remove bundling our static library dependencies; they are now found from the CMake config file. #4505
- Remove the CMake and bootstrap script options related to abseil and crc32c linkage tests. #4527
- Move documentation generation and source formatting into dedicated CMake modules. #4538
- Fix regression when libraries were sometimes installed in the
lib64
directory. #4572
Test only changes
- Remove UnitTestConfig::array_encryption_key_length. #4482
TileDB 2.18.3
TileDB v2.18.3 Release Notes
New features
- Add setting to disable install_sigpipe_handler on S3. #4573
TileDB 2.19.0-rc0
TileDB v2.19.0 Release Notes
Announcements
- TileDB 2.19, targeted for release in December 2023, includes performance improvements to the aggregate pushdown APIs.
Breaking behavior
- The Windows release artifacts no longer include static libraries. #4469
New features
- Allow aggregates pipeline the processing of full tiles. #4465
- Add class TemporaryDirectory to generate a PRNG-seeded unique directory. #4498
- Add sm.consolidation.total_buffer_size setting. #4550
- Use tile metadata in aggregates. #4468
- Load validity tiles only for null count aggregates when possible. #4513
- Prevent reading tiles when using fragment metadata in aggregates. #4481
Improvements
- Refactor the fragment info loading code to reduce coupling. #4453
- Fragment metadata: adding get_tile_metadata. #4466
- Improve experience when performing too large GCS direct uploads. #4047
- Refactor filestore API so that it uses internal TileDB library calls. #3763
- Encode Win32 error messages in UTF-8. #3987
- Remove unused compression defines. #4480
- Integrate baseline object library into the main build. #4446
- Internal clean-up of array timestamps. #3633
- Add base class for VFS filesystems. #4497
- Remove extraneous TILEDB_EXPORT tags. #4507
- Query Plan REST support - part 1: handler. #4516
- Use StorageFormat APIs to generate timestamped uris. #4530
- Refactor the group code. #4378
- Consolidation plan REST support - part1: handler. #4534
- Use constexpr to check for filesystem support. #4548
- Add vfs.gcs.max_direct_upload_size config option. #4047
- Migrate fragment_name parsing into StorageFormat. #4520
Defects removed
- Group metadata doesn't get serialized. #3147
- Added
#include <array>
totiledb/sm/cpp_api/type.h
. #4504 - Fix broken linking tests against a shared WebP library. #4525
- Fix delta filter deserialization for format ver 19. #4541
- Fix array schema selection when time traveling. #4549
Documentation
- Add a table with the history of the storage format versions. #4487
- Add timestamped_name file to the Format Spec. #4533
API changes
C API
- Add new C API functions to get group members and deprecate the existing ones. #4019
tiledb_group_create
is no longer marked as deprecated. #4522
C++ API
- Improve error message in C++ API for bad filter option type. #4503
Build System Changes
- Improve building the Windows binaries for the R API. #4448
- Update curl to version 8.4.0. #4523
- Clean-up unused build system code. #4527
- Cache more information when configuring the CMake project. #4449
- Remove the vcpkg submodule and download it automatically when configuring. #4484
- Use the provided CMake packages to import the compression libraries when vcpkg is enabled. #4500
- Remove bundling our static library dependencies; they are now found from the CMake config file. #4505
- Remove the CMake and bootstrap script options related to abseil and crc32c linkage tests. #4527
- Move documentation generation and source formatting into dedicated CMake modules. #4538
Test only changes
- Remove UnitTestConfig::array_encryption_key_length. #4482
TileDB 2.18.2
TileDB 2.17.5
TileDB v2.17.5 Release Notes
Defects removed
- Fix delta filter deserialization for format ver 19. #4541
TileDB 2.18.1
TileDB v2.18.1 Release Notes
Build System Changes
- Fix errors when building from a directory that is not a Git repository. #4529
TileDB 2.18.0-rc4
TileDB v2.18.0 Release Notes
Announcements
- TileDB 2.18, targeted for release in November 2023, includes a preview set of aggregate pushdown APIs. The APIs will be finalized in 2.19 with performance improvements.
Disk Format
- Fix the format specification for group members. #4380
- Update fragment format spec for info on tile sizes and tile offsets. #4416
Configuration changes
- Remove vfs.file.max_parallel_ops config option. #3964
Breaking C API changes
- Behavior breaking change:
tiledb_group_remove_member
cannot remove named members by URI if the URI is duplicated. #4391
New features
- Support custom headers on s3 requests. #4400
- Add support for Google Cloud Storage on Windows. #4138
- Query aggregates REST support. #4415
- Add random number generator global singleton. #4437
Improvements
- Add an aspect template argument to
class CAPIFunction
. #4430 - Fix enumeration value not found error message. #4339
- Allow forward slashes in enumeration names. #4454
- Adding extra in memory offset for var tiles on read path. #4294
- Aggregators: adding mean aggregate. #4292
- Remove parallel file I/O. #3964
- C.41 constructor: S3. #4368
- Changed the logic of the main loop in
Domain::cell_order_cmp_impl<char>
. #3847 - Add REST client support for delete_fragments and delete_fragments_list. #3923
- Remove the internal
ValidityVector::init_bytemap
method. #4071 - Use exceptions for error handling in the Metadata code. #4012
- Move min/max computation to aggregate_with_count. #4394
- Add memory tracking to Enumeration loading. #4395
- Logs for nested exceptions now incorporate all messages. #4393
- Implement read logging modes. #4386
- Move counts computation to aggregate_with_count. #4401
- Fix delete_fragments handlers according to new model. #4414
- Aggregates: change to use size instead of min cell/max cell. #4436
- Allow for empty enumerations. #4423
- Aggregates: combine sum/mean aggregators. #4440
Documentation
- Update docs for set data buffer APIs. #4381
Defects removed
- Fix serialization of empty enumerations. #4472
- Patch curl to fix CVE-2023-38545. #4408
- Update libwebp to version 1.3.2 to fix CVE-2023-5129. #4384
- Fix missing WebP::sharpyuv installation command. #4325
- Fix re-loading enumerations from the REST server. #4335
- Save error on the context before returning error code in vfs_open. #4347
- Filter pipeline: fix coord filter type. #4351
- Fix creation of Enumerations with
std::vector<bool>
. #4362 - Revert change to S3 is_object, is_bucket checks. #4370
- Specify whether the HDFS backend is enabled in the string returned by
tiledb_as_built_dump
. #4364 - Fix fragment corruption in the GlobalOrderWriter. #4383
- Fix bug in FragmentMetadata::get_footer_size. #4212
- Global order writer: don't try to delete the current fragment if it had failed to be created. #4052
- Fix bug in ordered label reader for unsigned integers. #4404
- Add missing include object.h in group_experimental.h. #4411
- Support Azure Storage SAS tokens that don't start with a question mark. #4418
- Avoid empty Enumeration REST requests. #4369
- Make the
initializer_list
overload ofSubarray::set_subarray
zero-copy. #4062
API changes
C API
- New: Aggregate pushdown C API. #4176
- Implement tiledb_handle_load_array_schema_request. #4399
- Add ArraySchemaEvolution::extend_enumeration. #4445
- Allow null cfg in group consolidate_metadata and vacuum_metadata. #4412
C++ API
- Fix UBSAN error in the Enumeration CPP API. #4357
- Remove default operation type from query condition set membership API (introduced in 2.17). #4390
- Fix CPP set buffer APIs to work with aggregates. #4434
- Adding CPP example for consolidation plan. #4433
- New: Aggregate API for MEAN and NULL_COUNT capi/cppapi wraps. #4443
Build System Changes
- Fix static linking failures and build ExampleExe_static on Windows. #4336
- Run Windows nightly debug build on C:/ drive to avoid out of space errors. #4352
- Manage dependencies with vcpkg by default. #4348
- Find the OpenSSL dependency of the AWS SDK on non-Windows only. #4359
- Use generator expressions when setting compile options for the benchmarks. #4328
- Bump default compiler to GCC 10 and macos images from 11 to 12. #4367
- Fix and optimize the Release CI workflow. #4373
- Stop building gRPC when GCS is enabled. #4435
- Require only ucrt build from Rtools. #4361
- Update the AWS SDK to version 1.11.160. #4214
- Remove warnings from benchmarks. #4396
- Update
clang-format
to version 16. #4397 - Update the Google Cloud Storage SDK to its latest version (2.15.1). #4031
- Update capnproto to version 1.0.1. #4405
- Fix compilation with GCC 13. #4331
- Compilation fix for GCC 9.4 of Ubuntu 20.04. #4355
- Don't error out if DELETE symbol is defined. #4365
Test only changes
- Aggregates pushdown: adding tests for var size buffer overflow. #4315
- Aggregates pushdown: adding incomplete tests. #4301
- Add group backwards compatibility tests. #3996
- Aggregates: Adding tests for computation policies. #4421
- Aggregators: adding benchmark for aggregate_with_count class. #4420
Experimental Changes
TileDB 2.18.0
TileDB v2.18.0 Release Notes
Announcements
- TileDB 2.18, targeted for release in November 2023, includes a preview set of aggregate pushdown APIs. The APIs will be finalized in 2.19 with performance improvements.
Disk Format
- Fix the format specification for group members. #4380
- Update fragment format spec for info on tile sizes and tile offsets. #4416
Configuration changes
- Remove vfs.file.max_parallel_ops config option. #3964
Breaking C API changes
- Behavior breaking change:
tiledb_group_remove_member
cannot remove named members by URI if the URI is duplicated. #4391
New features
- Support custom headers on s3 requests. #4400
- Add support for Google Cloud Storage on Windows. #4138
- Query aggregates REST support. #4415
- Add random number generator global singleton. #4437
Improvements
- Add an aspect template argument to
class CAPIFunction
. #4430 - Fix enumeration value not found error message. #4339
- Allow forward slashes in enumeration names. #4454
- Adding extra in memory offset for var tiles on read path. #4294
- Aggregators: adding mean aggregate. #4292
- Remove parallel file I/O. #3964
- C.41 constructor: S3. #4368
- Changed the logic of the main loop in
Domain::cell_order_cmp_impl<char>
. #3847 - Add REST client support for delete_fragments and delete_fragments_list. #3923
- Remove the internal
ValidityVector::init_bytemap
method. #4071 - Use exceptions for error handling in the Metadata code. #4012
- Move min/max computation to aggregate_with_count. #4394
- Add memory tracking to Enumeration loading. #4395
- Logs for nested exceptions now incorporate all messages. #4393
- Implement read logging modes. #4386
- Move counts computation to aggregate_with_count. #4401
- Fix delete_fragments handlers according to new model. #4414
- Aggregates: change to use size instead of min cell/max cell. #4436
- Allow for empty enumerations. #4423
- Aggregates: combine sum/mean aggregators. #4440
Documentation
- Update docs for set data buffer APIs. #4381
Defects removed
- Fix serialization of empty enumerations. #4472
- Patch curl to fix CVE-2023-38545. #4408
- Update libwebp to version 1.3.2 to fix CVE-2023-5129. #4384
- Fix missing WebP::sharpyuv installation command. #4325
- Fix re-loading enumerations from the REST server. #4335
- Save error on the context before returning error code in vfs_open. #4347
- Filter pipeline: fix coord filter type. #4351
- Fix creation of Enumerations with
std::vector<bool>
. #4362 - Revert change to S3 is_object, is_bucket checks. #4370
- Specify whether the HDFS backend is enabled in the string returned by
tiledb_as_built_dump
. #4364 - Fix fragment corruption in the GlobalOrderWriter. #4383
- Fix bug in FragmentMetadata::get_footer_size. #4212
- Global order writer: don't try to delete the current fragment if it had failed to be created. #4052
- Fix bug in ordered label reader for unsigned integers. #4404
- Add missing include object.h in group_experimental.h. #4411
- Support Azure Storage SAS tokens that don't start with a question mark. #4418
- Avoid empty Enumeration REST requests. #4369
- Make the
initializer_list
overload ofSubarray::set_subarray
zero-copy. #4062
API changes
C API
- New: Aggregate pushdown C API. #4176
- Implement tiledb_handle_load_array_schema_request. #4399
- Add ArraySchemaEvolution::extend_enumeration. #4445
- Allow null cfg in group consolidate_metadata and vacuum_metadata. #4412
C++ API
- Fix UBSAN error in the Enumeration CPP API. #4357
- Remove default operation type from query condition set membership API (introduced in 2.17). #4390
- Fix CPP set buffer APIs to work with aggregates. #4434
- Adding CPP example for consolidation plan. #4433
- New: Aggregate API for MEAN and NULL_COUNT capi/cppapi wraps. #4443
Build System Changes
- Fix static linking failures and build ExampleExe_static on Windows. #4336
- Run Windows nightly debug build on C:/ drive to avoid out of space errors. #4352
- Manage dependencies with vcpkg by default. #4348
- Find the OpenSSL dependency of the AWS SDK on non-Windows only. #4359
- Use generator expressions when setting compile options for the benchmarks. #4328
- Bump default compiler to GCC 10 and macos images from 11 to 12. #4367
- Fix and optimize the Release CI workflow. #4373
- Stop building gRPC when GCS is enabled. #4435
- Require only ucrt build from Rtools. #4361
- Update the AWS SDK to version 1.11.160. #4214
- Remove warnings from benchmarks. #4396
- Update
clang-format
to version 16. #4397 - Update the Google Cloud Storage SDK to its latest version (2.15.1). #4031
- Update capnproto to version 1.0.1. #4405
- Fix compilation with GCC 13. #4331
- Compilation fix for GCC 9.4 of Ubuntu 20.04. #4355
- Don't error out if DELETE symbol is defined. #4365
Test only changes
- Aggregates pushdown: adding tests for var size buffer overflow. #4315
- Aggregates pushdown: adding incomplete tests. #4301
- Add group backwards compatibility tests. #3996
- Aggregates: Adding tests for computation policies. #4421
- Aggregators: adding benchmark for aggregate_with_count class. #4420
Experimental Changes
TileDB 2.18.0-rc3
TileDB v2.18.0 Release Notes
Announcements
- TileDB 2.18, targeted for release in November 2023, includes a preview set of aggregate pushdown APIs. The APIs will be finalized in 2.19 with performance improvements.
Disk Format
- Fix the format specification for group members. #4380
- Update fragment format spec for info on tile sizes and tile offsets. #4416
Configuration changes
- Remove vfs.file.max_parallel_ops config option. #3964
Breaking C API changes
- Behavior breaking change:
tiledb_group_remove_member
cannot remove named members by URI. #4391
New features
- Support custom headers on s3 requests. #4400
- Add support for Google Cloud Storage on Windows. #4138
- Query aggregates REST support. #4415
- Add random number generator global singleton. #4437
Improvements
- Add an aspect template argument to
class CAPIFunction
. #4430 - Fix enumeration value not found error message. #4339
- Allow forward slashes in enumeration names. #4454
- Adding extra in memory offset for var tiles on read path. #4294
- Aggregators: adding mean aggregate. #4292
- Remove parallel file I/O. #3964
- C.41 constructor: S3. #4368
- Changed the logic of the main loop in
Domain::cell_order_cmp_impl<char>
. #3847 - Add REST client support for delete_fragments and delete_fragments_list. #3923
- Remove the internal
ValidityVector::init_bytemap
method. #4071 - Use exceptions for error handling in the Metadata code. #4012
- Move min/max computation to aggregate_with_count. #4394
- Add memory tracking to Enumeration loading. #4395
- Logs for nested exceptions now incorporate all messages. #4393
- Implement read logging modes. #4386
- Move counts computation to aggregate_with_count. #4401
- Fix delete_fragments handlers according to new model. #4414
- Aggregates: change to use size instead of min cell/max cell. #4436
- Allow for empty enumerations. #4423
- Aggregates: combine sum/mean aggregators. #4440
Documentation
- Update docs for set data buffer APIs. #4381
Defects removed
- Fix serialization of empty enumerations. #4472
- Patch curl to fix CVE-2023-38545. #4408
- Update libwebp to version 1.3.2 to fix CVE-2023-5129. #4384
- Fix missing WebP::sharpyuv installation command. #4325
- Fix re-loading enumerations from the REST server. #4335
- Save error on the context before returning error code in vfs_open. #4347
- Filter pipeline: fix coord filter type. #4351
- Fix creation of Enumerations with
std::vector<bool>
. #4362 - Revert change to S3 is_object, is_bucket checks. #4370
- Specify whether the HDFS backend is enabled in the string returned by
tiledb_as_built_dump
. #4364 - Fix fragment corruption in the GlobalOrderWriter. #4383
- Fix bug in FragmentMetadata::get_footer_size. #4212
- Global order writer: don't try to delete the current fragment if it had failed to be created. #4052
- Fix bug in ordered label reader for unsigned integers. #4404
- Add missing include object.h in group_experimental.h. #4411
- Support Azure Storage SAS tokens that don't start with a question mark. #4418
- Avoid empty Enumeration REST requests. #4369
- Make the
initializer_list
overload ofSubarray::set_subarray
zero-copy. #4062
API changes
C API
- New: Aggregate pushdown C API. #4176
- Implement tiledb_handle_load_array_schema_request. #4399
- Add ArraySchemaEvolution::extend_enumeration. #4445
- Allow null cfg in group consolidate_metadata and vacuum_metadata. #4412
C++ API
- Fix UBSAN error in the Enumeration CPP API. #4357
- Remove default operation type from query condition set membership API (introduced in 2.17). #4390
- Fix CPP set buffer APIs to work with aggregates. #4434
- Adding CPP example for consolidation plan. #4433
- New: Aggregate API for MEAN and NULL_COUNT capi/cppapi wraps. #4443
Build System Changes
- Fix static linking failures and build ExampleExe_static on Windows. #4336
- Run Windows nightly debug build on C:/ drive to avoid out of space errors. #4352
- Manage dependencies with vcpkg by default. #4348
- Find the OpenSSL dependency of the AWS SDK on non-Windows only. #4359
- Use generator expressions when setting compile options for the benchmarks. #4328
- Bump default compiler to GCC 10 and macos images from 11 to 12. #4367
- Fix and optimize the Release CI workflow. #4373
- Stop building gRPC when GCS is enabled. #4435
- Require only ucrt build from Rtools. #4361
- Update the AWS SDK to version 1.11.160. #4214
- Remove warnings from benchmarks. #4396
- Update
clang-format
to version 16. #4397 - Update the Google Cloud Storage SDK to its latest version (2.15.1). #4031
- Update capnproto to version 1.0.1. #4405
- Fix compilation with GCC 13. #4331
- Compilation fix for GCC 9.4 of Ubuntu 20.04. #4355
- Don't error out if DELETE symbol is defined. #4365
Test only changes
- Aggregates pushdown: adding tests for var size buffer overflow. #4315
- Aggregates pushdown: adding incomplete tests. #4301
- Add group backwards compatibility tests. #3996
- Aggregates: Adding tests for computation policies. #4421
- Aggregators: adding benchmark for aggregate_with_count class. #4420