v1.0.0

After more than 3 years since its initial public release...

🎉 lakeFS 1.0 is now generally available 🎉

---

This is a huge milestone for the lakeFS maintainers, contributors and community and we can't be more excited!

Benefits

1.0 is more than just a cosmetic change - it includes a few important benefits and guarantees to anyone running or using lakeFS:

• lakeFS 1.x ensures both backwards and forwards compatibility with any future 1.x release, as per Semver.
• Not just the lakeFS Open API: This is also true for the new lakeFS Python SDK, the Java SDK and the lakectl CLI tool.
• Bug fixes will be released for lakeFS 1.x even after the release of future major version, and at least until October 2024
• Security fixes will be released (and if needed, backported) even after the release of future major version, and at least until June 2025(!).
• While lakeFS is already trusted in production in hundreds of organizations around the world, 1.x ensures a stable API going forward and it is highly recommended for all lakeFS users.

Feature Highlights

The lakeFS 1.0 release actually doesn't introduce any new capabilities other than the guarantees given above. However,
for users of slightly older 0.x lakeFS versions, here are a few highlights from the last few months that are included with this version:

• lakectl local - Allows syncing a local directory with remote data in lakeFS, including full Git integration (read announcement)
• iceberg catalog - Making it possible to branch, commit and merge multiple Iceberg tables together and in tandem with other formats (read announcement)
• lua based hooks - Run your own custom logic when commits, merges and other events occur. No need to host a webhook server or deploy anything! (read announcement)
• enhanced security with pre-signed URLs - Allows lakeFS to version data it cannot even read! This enhanced security by leveraging pre-signed URLs supported on all common object stores, to allow lakeFS to authorize access to data without having to proxy that data through the lakeFS server (read more)
• Many many bug fixes and performance improvements across the board

Notes on upgrading from lakeFS 0.x

While most of the API hasn't changed much, there are a few notable API changes between 0.x and 1.x.

The migration path is usually pretty simple and not many code changes are required - but please read through the list of breaking changes: lakeFS 1.0 - Code Migration Guide

For more information on the benefits of upgrading to 1.0 and the migration path, please refer to the lakeFS documentation or reach out on Slack.

🐛 Bugs fixed:

• Increased the maximum attempts for DynamoDB with configuration support (#6804)
• Amended Azure import to utilize content MD5 instead of entry Etag (#6802)
• Corrected issue where the UI upload failed due to conversion of undefined or null values (#6818)

v0.113.0

Changelog

🆕 What's new:

• Reintroduce recursive flag for lakectl fs download/upload (#6777)
• Spark client update to use the latest lakefs SDK package (#6742)

🐛 Bugs fixed:

• Fix lakectl upload with pre-sign for Azure (#6660)
• Fix lakeFSFS ETag value when using pre-signed mode (#6751)
• Fix LinkPhysicalAddress trim quotes and spaces from checksum (#6749)
• Fix lakectl upload using pre-signed use ContentMD5 header for ETag (#6750)
• Fix LinkPhysicalAddress response with full path (#6748)
• Fix lakectl download error with bad path traversal blocked (#6775)
• Fix UI Azure pre-sign upload (#6764)
• Fix UI upload azure pre-signed URL checksum as hex md5 (#6770)
• Fix UI copy pre-signed URL not working when pre-sign UI disabled (#6776)
• Fix delete branch protection prevent commit because invalid state (#6788)

v0.112.1

Changelog

🆕 What's new:

• New Java SDK (#6656)
• New Pyhon SDK (#6562)
• Export Hooks: Glue Catalog exporter (#6653)
• Remove recursive flag from lakectl fs Upload and Download (#6681)
• CosmosDB: Add throughput configuration (#6693)
• lakectl: block fs stage command for objects in the storage namespace (#6696)
• API: Deprecate stageObject (#6690)
• API: linkPhysicalAddress do not verify external addresses (#6667)
• API: GC prepare move operations to internal tag (#6700)
• API: support response Too many requests 420 (#6711)
• API: deprecate upload object If-None-Match and Storage Class (#6708)
• LakeFSFileSystem is now compatible only with lakeFS versions >= 0.108.0 (#6725)

v0.112.0

Changelog

🆕 What's new:

• New Java SDK (#6656)
• New Pyhon SDK (#6562)
• Export Hooks: Glue Catalog exporter (#6653)
• Remove recursive flag from lakectl fs Upload and Download (#6681)
• CosmosDB: Add throughput configuration (#6693)
• lakectl: block fs stage command for objects in the storage namespace (#6696)
• API: Deprecate stageObject (#6690)
• API: linkPhysicalAddress do not verify external addresses (#6667)
• API: GC prepare move operations to internal tag (#6700)
• API: support response Too many requests 420 (#6711)
• API: deprecate upload object If-None-Match and Storage Class (#6708)
• LakeFSFileSystem is now compatible only with lakeFS versions >= 0.108.0 (#6725)

v0.111.1

Changelog

🐛 Bugs fixed:

• Fix email display bug when using external auth provider (#6680)
• Fix repository creation bug in quickstart mode (#6682)

v0.111.0

Changelog

🆕 What's new:

• lakectl endpoint configure and endpoint path reuse (#6609)
• Remove UpdateBranchToken API (#6590)
• Table Extractor Hook and _lakefs_tables format (#6589)
• Export Hooks: Symlink Exporter (#6636)
• Move setting endpoints to be under /settings/ (#6649)
• Remove expired commits from prepare_gc_commits response (#6634)
• Remove lakectl dbt deprecated functionality (#6632)
• Remove lakectl direct deprecated functionality (#6623)
• API cleanup: remove emailer service implementation (#6661)
• API cleanup: consolidate lakefs level information API (#6647)
• API cleanup: make createSymlinkFile internal (#6630)
• API cleanup: Mark refs dump/restore current API as internal (#6640)

🐛 Bug fixed:

• Fix: lua s3 client endpoint configure if set (#6629)
• Fix: UI alert if "quickstart" repository failed on local (#6622)
• Fix: Lua runtime - marshaling arrays (#6655)
• Fix: Presigned URL Cache - change to Default expiry due to bug in package (#6651)

v0.110.0

🆕 : What's new:

• Upgrade code to use AWS SDK go v2 (#6486)
• Actions: Lua package to parse URL (#6597)
• UI: Commit info consistent across screens (#6593)
• API: Update APIs tag for internal use (#6582)
• lakectl: dbt and metastore commands marked as deprecated (#6565)
• Removed deprecated: expand template API (#6587)
• Removed deprecated: remove the update and forgot password APIs (#6591)

🐛 : Bug fixed:

• Fix lakectl local verify bad path error on Windows (#6602)
• Fix Azure pre-signed URL for blob imported from different account (#6594)

v0.109.0

🆕 What's new:

• Restrict lakectl local to common prefixes only (#6510)
• lakectl local gitignore end marker (#6522)
• Remove logBranchCommits (#6528)
• Lua package for yaml support (#6545)
• lakectl skip stats for unrelased version (#6551)
• Remove OTF diff APIs from non experimental tag/category (#6563)
• S3 API: Propogate slowdown response from cosmosdb to client (#6556)
• logging and lakectl NO_COLOR support (#6569)

🐛 Bug fixed:

• Fix: lakectl import with Azure Data Lake Storage (#6515)
• Fix: UI pdf viewer by X-Frame-Options set to SAMEORIGIN (#6520)
• Fix: Lua gs client close (#6547)
• Fix: Lua path.Join to handle concatenation of two separators (#6549)
• Fix latest version cache and remove the use of go-github (#6572)
• Fix: DynamoDB SeekGE bug (#6561)

v0.108.0

🆕 What's new:

• Support vertex AI (Datasets + Fuse) (#6492)
• A better way to handle interrupts in lakectl local (#6455)
• When creating Repository: dummy file location changed from <storage-namespace>/dummy to <storage-namespace>/_lakefs/dummy (#6497)
• Removed support for migration from lakeFS version < v0.50.0 (#6497)
• Deprecate lakectl fs "--direct" flag (#6480)

🐛 Bug fixed:

• Fix: Ensure that metadata is included in merge operations (#6500)
• Fix: Increase DynamoDB concurrent requests (#6489)
• Fix: UI ignores presigned URL configuration for GetObject requests (#6477)
• Fix: lakectl local sync parallelism (#6485)
• Fix: UI object information slides outside the modal dialog (#6501)
• Fix: CosmosDB SeekGE query from iterator start key (#6503)
• Fix: UI ADLS import validity regex (#6509)

v0.107.1

Changelog

🐛 Bug fixed:

• Fix: Allow disabling pre-signed URLs for google storage block adapter (#6462)
• Fix: lakectl local with windows (#6464)