v0.21.1

Changelog

93b4222 Add UI delete branch #956 (#991)
ec39a82 Add initial implementation of Graveler (#1019)
11a13f8 Allow lakectl to run without a config file (#1040)
48ff288 Cataloger create branch by ref (#1020)
153bf85 Change graveler iterator interface (#1014)
f25a4e5 Rename forest package and move it to graveler (#1035)
78dea4f Simplify commit iterator logic (#1017)
182bb7a Upgrade webui ini package to fix GHSA-qqgx-2p2h-9c37 (#1051)
8aada34 Added graveler tree interface (#1043)
211afbb Avoid int cast without limit check (#1038)
4895fae Fix gateway branch name validation (#1024)
a8dc344 Fix javascript lint issues (#1039)
12e854c List commits limit back default amount (#1045)
d8f2060 Use default amount for ui list branches amount (#1021)

v0.21.0

Note

This release requires running database migration.
The lakeFS service will not run if the migration version isn't compatible with the binary.
Before running the new version you will require to run migrate, with the new version:

$ lakefs migrate up

Changelog

d19107d Add test for refs diff (#1012)
4ce4df1 Fix gitleaks configuration (#950)
68cdf0c Generate {application,text}/xml headers on gateway by client's choice (#992)
c5a4955 Set default value for all amounts in swagger (#1011)
5ee62e0 pg refs manager for graveler

v0.20.0

Note

This release requires running database migration.
The lakeFS service will not run if the migration version isn't compatible with the binary.
Before running the new version you will require to run migrate, with the new version:

$ lakefs migrate up

Changelog

e86f416 bug fix Merge may apply partial changes due to concurrency issues (#1007)
99e89ec bug fix Committing an object deletion to master erases object from all commits to master - #997 (#1000)
5e34175 Change sstable interface to match new graveler package (#1003)
3084355 Feature/staging mgr postgres (#981)
b1b3858 Graveler transform catalog interface (#993)
e88f3c7 Immutable tiered storage (#962)
cf041f8 Check and fix the use of errors.As (#1004)

v0.19.0

Note

This release requires running database migration.
The lakeFS service will not run if the migration version isn't compatible with the binary.
Before running the new version you will require to run migrate, with the new version:

$ lakefs migrate up

Changelog

3aa396f Add "lakectl fs upload --recursive" flag (#979)
18bf790 Add logs for gs nessie failures (#990)
a57e068 Extract multipart upload implementation from mvcc cataloger (#989)

v0.18.0

Note

This release requires running database migration.
The lakeFS service will not run if the migration version isn't compatible with the binary.
Before running the new version you will require to run migrate, with the new version:

$ lakefs migrate up

Changelog

7b08240 rollback committed changes blocked by child branches (#980)
4f89db9 Add interface for async handling of sstable writer flushing and closing (#978)
83478b7 Catalog rollback to commit branch verification (#988)
9623170 Merge direction #955 (#968) (thanks @shamikakumar)
06806c8 Rocks catalog interface (#959)
c152e2c add installation id after init (#983)
90600f9 allow empty installation id (#942)
4b28f2e catalog interface, change Dereference to RevParse (#972)
dc69cc4 cataloger rocks initial connect to catalog (#976)
fdd0ca1 check before building and pushing an existing image in benchmark flow (#974)
24fb08c sstable interface for committed data (#971)
b8bdcc0 sstable writer to accept a single rocks.EntryRecord arg (#975)

v0.17.0

Note

This release requires running database migration.
The lakeFS service will not run if the migration version isn't compatible with the binary.
Before running the new version you will require to run migrate, with the new version:

$ lakefs migrate up

Changelog

8f6365e Add API key specification to "lakefs setup" and use that in Nessie
1ee9a61 Add continuous export configuration
331f771 Allow access key configuration from "lakefs superuser" command
db02f63 Avoid "magic" number in exit(2)
24974e3 Call gen_random_uuid directly from the public schema (#944)
35a8ecb Clean up time.Duration type multiplication in Tx retries
a4b3ec0 Continuous export: start a new export after each commit or merge
4c888fa Create cataloger based on configuration (#951)
25b500e Fetch lakeFS envariables in nessie auth config
b3acce7 Improve logging and clean up flows
5740599 Log db params in nessie auth_test
38c4763 Make "lakefs init" work when not specifying access_key_id
99b0c86 Add ability to specify Access Key ID and Secret Key in setup endpoint (#908)
4526447 Export lakeFS branch to S3 continuously (#534)
2c137c6 local adapter - respect namespace in objects full path (#938)
95e2125 lakefs init broken: "access-key-id: flag accessed but not defined: access-key-id" (#940)
147788a Add repair to export (#943)
84a52ee Continuous export (#949)
6f8a0fe Migration validate version should not create schema_migration table on check (#953)
5cea104 Move cataloger mvcc implementation under a different package (#946)
04e6499 Return installation ID as part of health check response (#964)
5e77681 Run hooks outside of transactions
b810243 Share lakefs config values between nessie and lakefs containers
f32712f Test (empty) diff between a ref and itself
b35b36e [CR] Flatten if/elses in catalog/mvcc/cataloger_merge_test.go
3e63322 [CR] GetPrimitive doesn't call pgxscan, use pgx directly there
40fec4b [CR] Refactor merge & commit hooks
25d2bec [CR] Revert envariables change in nessie GH action + compose files
6985dba [CR] Use cobra.ExactArgs, SQL IF [NOT] EXISTS
5976728 [CR] Validate nonempty access key ID, secrets in Swagger
99770f8 [CR] pass hook values by value
5cd4d16 [GH actions] re-run lakeFS in nessie action
84a50ab [bug] verify required positional args are present
634e85a [bugfix] return ErrNotFound correctly (#952)
4871e49 [checks] use a shorter error line to pass err113 golangci check
4ebed6c [nessie] allow nessie to connect to postgres container
d55eb5d [post-rebase] Revert "status" back to "state" in field names
79ff970 add repair option to export when running export with repair it will consider the previous errors as repaired and continue from last commit
8782fd4 change ExportState to work with one function that calls a callback with the current status and updates the status regarding to the values returned from the callback
577388e change requests SQL fixes export callback return newRef add endpoint for repair
af3c650 fix calling to testing.T inside a goroutine
2e2fc4f fix duplicate migration
74ac713 helm docs: add extra env vars (#937)
0f3084a lakefs diagnostics query db version (#939)
1183d77 local adapter - respect namespace in objects full path
b69c89a make postgres connection string optional (#948)
28e4052 rename lakefs init to setup and keep init as alias (#961)

v0.16.2

Changelog

34185da Diff performance improvement by trim lineage (#921)
148b8e8 Revert "Share lakefs config values between nessie and lakefs containers"
e23adc4 S3 reference and example parameters to work with MinIO
46c7a57 Share lakefs config values between nessie and lakefs containers
a83dc02 add integration test for export

v0.16.1

Changelog

78b6d8f Fix fail export in case export is currently in progress #914
c6674e6 Document export and unhide from CLI #915
825006f WebUI get config called only on create repository #917
19a4357 lakefs init and setup handler share code #919
1d79c2d Fix export to s3 not working due to wrong paths #920
c3fd501 Update ddl/000009_export_current.down.sql

v0.16.0

Note

This release requires running database migration.
The lakeFS service will not run if the migration version isn't compatible with the binary.
Before running the new version you will require to run migrate, with the new version:

$ lakefs migrate up

• The older version of lakeFS will not work with the new migration version

Changelog

7a78820 Diff working in a separate goroutine from the merge
365cc9f Fix wrong UI error when repo name contains underscore (on repo creation) 833 (#879) (thanks @Sufiyan1997)
dd4a6fb When uploading a file, the path in the repo should be visible #835 (#866) (thanks @shamikakumar)
24f18e4 Configure AWS clients to retry S3 ops when possible
0416813 "mkdir -p" destination directories for local adapter Put, Copy
8e579d0 Add export state logic and tests
328b134 Align repo id to repo name (#874)
9dddbac Change export tasks generation to be iterative
da896de Create extension pgcrypto in "public" schema
5b4d41d DB scanner options by value with an option to update AdditionalWhere
9ebd4d7 Enable GetCommit to work when branch name, returns the last commit on branch
cccb491 Explicitly use PUBLIC schema
f7c0dbc Fail a task if it retries too many times
9f593f3 Fix repo id to name in the UI (#881)
063ce17 Fixed and responded to all review comments
9dd26be Hide "export" command
725e181 Refactor database to use sql + scany instead of sqlx
c82c549 Remove SQLX usage from parade
bf99b95 Restore a name for dreaded magic number of 5 retries
40f916b SQL - replaced "values" and string concat with array and unnest
a7affc3 Use pgx instead of sql
1eeb2a4 [CR] DROP TABLE... IF EXISTS
72c25aa [CR] Remove probably-unused fields from monitoring
7201f13 [CR] Rename Get -> GetPrimitive, GetStruct -> Get
3ff1f62 [CR] Use standard db:"-" tag, copy all input to convertRawCommit
cf229a5 [CR] cleanups: "go mod tidy", deferred Close wrapped in unneeded func
ccf7ccf [bug] file tasks depend on nothing
4ec6f1d [export] Add current export status to DB
1a369aa [lint] missing error return caught!
5394c77 [lint] use proper Go-style comment for TaskDataColumnNames
1fd05a6 [parade] Move DDL under ddl/
08c9a5c [parade] count failures in direct predecessors, pass them on tasks
e4dbf3b [rebase] Fix dbPool closure in new superuser command
d35ed36 add endpoint for export
88f41e9 add export state - run start and end with a callback between without the need to open a transaction outside of the cataloger fix - missing if error db.ERRNotFound instead of ErrEntryNotFound
d440053 add filtering by relevant commits to branch scanned
332f7c4 add tests for lastKeysInPrefixRegexp
e4fb6ee after merging with changes that happened in the code, and debugging it
04c9489 change ExportConfig action remove sizes from varchar in parade
ae06cf6 change authorization for export commands change error with debug in case parade migration didn't run yet
6df779c change merge isolation level in the merge transaction from serializable to ReadCommitted
c8dd42d change operationID from executeContinuousEcport to RunExport return export ID in API
98a9132 change permissions for executing continuous export
f8cb19b change task_generator tests to work with export ID generator
1916c12 check context just in query execute
78f0f21 count records, table sizes and log errors
77120b3 default output filename for lakefs diagnostics
df06d15 dev compose file - use local dockerfile for our docker-compose
9c0cff6 extended max commits in brench, added limit to sql
de39e35 fix diff same branch run on left side lineage (#875)
fff50f2 fix div thanks to Shamika Kumar (#867) (thanks @shamikakumar)
f85b6e8 fix same branch diff should compare by a checksum (#878)
47bb5af fixed bug in sorting commits, and in deciding when to use "between" instead of
7574553 generate a status file on done
a41923f increase read buffer for branch
a49707e lakefs diagnostics command (#889)
021b307 limit number of commitIDs on branch scanner select
96e6a03 merge from parent - remove accesor branches that did not change from the parent branch
00d3eb2 migrate goto with force support (#872)
be26f36 modify installation documentation to require 1.25.4+ docker-compose version
8355201 modify sql script for commits index name
8a38e7d moving merge activity to doMerge routine
f5c43ae no need to return export status and message in case of error
e54749b preformed gofmt of diff scanner
665ea06 release checklist (#870)
26e2160 remove select * from big tables and use statistics
342ea0f remove size constraint from task id
f53b27c remove trimmedLineage from DBScannerOtions, and move them to lineage scanner
386e103 remove unused GCS parameters
892f092 require export-path flag on export set config command
cce32b6 scanner skip to next path
6cf71d5 show lakefs log on Nessie error (#876)
f8f59fd test code to call collector and verify expected files are collected
c9de871 turn diff into a scanner, and make merge apply the diff changes incrementally
923be9f upgrade react-scripts to v4.0.0 (#886)

v0.15.0

Changelog

8527b2a Add new ReadConfigAction auth action for reading config (#837) (thanks @daniel-shuy )
2a2a8d5 Callhome to collect heartbeats (#841)
4032a01 Configure branch continuous export: Swagger defs and API handlers
1626f37 Diffs are now on references not branches (#863)
89b8b7d Feature/db read entry - read entry/s using simple union sql, instead of the complex views/sqEntriesLineage (#783)
0bba0ca Fixing terraform version bug (#860)
0f923de Implement cataloger current diff using scanners (#790)
0b2f3ba Validate dependencies licenses (#836)
e26816a add option to set a fixed installation ID
bd52e00 bug fix conflict result compare type (#829)
8d333ed change environment variable for installation id to be with underscore instead of dash
1ab4684 change handler to actor document actor
6c6351b child diff simplify check - compare delete and content first (#845)
99e1bd1 diff between two references on the same branch (#842)
b7f7cbc diff with additional fields support (#851)
4a51df8 lakefs new repositories with import branch as parent branch by default (#810)
00c7c08 lakefs superuser command for adding more admin users (#864)
7d7d611 move action_manager to parade document action_manager add data to logs
3021a8a move catalog errors to errors.go (#847)
1270083 remove extra space from error message (#820)
1f3cd72 replace current branching model with a per-use-case set of models (#824)