{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":409655014,"defaultBranch":"main","name":"qbeast-spark","ownerLogin":"Qbeast-io","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2021-09-23T15:54:12.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/64087078?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1714982422.0","currentOid":""},"activityList":{"items":[{"before":"d37d23824ae3bce702085a09f2462ea0b915fba0","after":"f56491fc0e0d4686511ff66c9960aa7c41371864","ref":"refs/heads/main","pushedAt":"2024-05-28T12:16:43.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Jiaweihu08","name":"jiawei","path":"/Jiaweihu08","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47899566?s=80&v=4"},"commit":{"message":"Issue 327: 1/4 Document Delete limitations for Qbeast (#329)","shortMessageHtmlLink":"Issue 327: 1/4 Document Delete limitations for Qbeast (#329)"}},{"before":"607535537d743a8fec500bc7fcb6d96f1dfbdd91","after":"d37d23824ae3bce702085a09f2462ea0b915fba0","ref":"refs/heads/main","pushedAt":"2024-05-02T13:32:27.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Jiaweihu08","name":"jiawei","path":"/Jiaweihu08","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47899566?s=80&v=4"},"commit":{"message":"Issue 325: Change build version to 0.7.0 (#324)","shortMessageHtmlLink":"Issue 325: Change build version to 0.7.0 (#324)"}},{"before":"6a8e1ca5472b9a8132738f78957a733ecba73a53","after":"607535537d743a8fec500bc7fcb6d96f1dfbdd91","ref":"refs/heads/main","pushedAt":"2024-05-02T10:49:01.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 302: Add Logging information to the Contributing guide (#308)\n\nAdds logging documentation with examples into the CONTRIBUTING.md file.","shortMessageHtmlLink":"Issue 302: Add Logging information to the Contributing guide (#308)"}},{"before":"d38a94c3b51ccaabcc10e90a3a6483c0165c2a5f","after":"6a8e1ca5472b9a8132738f78957a733ecba73a53","ref":"refs/heads/main","pushedAt":"2024-05-02T10:24:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 309: Update documentation for 0.6.0 release (#310)\n\n- Updates to README\r\n- Updates to CONTRIBUTING\r\n- Updates to docs/, including QbeastFormat, QbeastTable and Quickstart.","shortMessageHtmlLink":"Issue 309: Update documentation for 0.6.0 release (#310)"}},{"before":"ea0a26dda3d9a96deaef1bfb7af94902f057f4ae","after":"d38a94c3b51ccaabcc10e90a3a6483c0165c2a5f","ref":"refs/heads/main","pushedAt":"2024-04-18T06:07:44.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Issue 317: Reduce optimization overhead (#318)\n\n* Broadcast rollup map and cube max weights","shortMessageHtmlLink":"Issue 317: Reduce optimization overhead (#318)"}},{"before":"7457882136a10083603eb66bbe9ec49000bd7ca1","after":"ea0a26dda3d9a96deaef1bfb7af94902f057f4ae","ref":"refs/heads/main","pushedAt":"2024-04-17T13:39:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Issue 315: Correct cube element counts for rollup during optimization (#316)\n\n* Rollup relies on cube element count from extendedData after indexing","shortMessageHtmlLink":"Issue 315: Correct cube element counts for rollup during optimization ("}},{"before":"48f5af97ffe8ccd0b92b19e6fb0b52c7531da997","after":"7457882136a10083603eb66bbe9ec49000bd7ca1","ref":"refs/heads/main","pushedAt":"2024-04-16T14:48:48.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 312: Optimization should not introduce data changes. (#313)\n\nSets the value of the Delta dataChange flag in relevant parts of the code.","shortMessageHtmlLink":"Issue 312: Optimization should not introduce data changes. (#313)"}},{"before":"b34b3a3e290a20adf2bf8687598a8e5454e255fe","after":"48f5af97ffe8ccd0b92b19e6fb0b52c7531da997","ref":"refs/heads/main","pushedAt":"2024-04-10T12:58:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 296: Unit Test for IndexFiles (#307)\n\nIndexFiles unit testing, tests all the public methods, specially the Json exceptions when trying to parse malformed blocks.","shortMessageHtmlLink":"Issue 296: Unit Test for IndexFiles (#307)"}},{"before":"6e6b5b43f95e5113a869cceb0410a764c45dd7b3","after":"b34b3a3e290a20adf2bf8687598a8e5454e255fe","ref":"refs/heads/main","pushedAt":"2024-04-09T06:46:26.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Issue 285: Versioning policy documentation (#305)\n\n* Add Versioning Policy to CONTRIBUTING guide","shortMessageHtmlLink":"Issue 285: Versioning policy documentation (#305)"}},{"before":"b0fa253b89b6230f74c3c15a692ce1655da77051","after":"6e6b5b43f95e5113a869cceb0410a764c45dd7b3","ref":"refs/heads/main","pushedAt":"2024-04-05T14:34:31.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Issue 274: Fix erroneous idempotent writes (#304)\n\n* Fix comparison when issuing a smaller or equal txnVersion","shortMessageHtmlLink":"Issue 274: Fix erroneous idempotent writes (#304)"}},{"before":"5ac486599abf791475b545b40fc3d49f1aa5bd53","after":"b0fa253b89b6230f74c3c15a692ce1655da77051","ref":"refs/heads/main","pushedAt":"2024-04-04T17:20:14.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 281: Schema Merge (#301)\n\nWith this PR:\r\n\r\n- The QbeastMetadataOperation is no longer setting up canMergeSchema and canOverwriteSchema parameters to true by default. Now they are initialized according to the properties specified in DeltaOptions / QbeastOptions.\r\n- When Creating a Table, the behavior of Delta is to do a first commit in the path with only the Metadata (schema, table id...). In the Qbeast case, no initial schema was committed. This is solved by introducing a first write with an empty data frame and a Convert To Qbeast operation.","shortMessageHtmlLink":"Issue 281: Schema Merge (#301)"}},{"before":"2fedabcad817ba9313de768ce830c52af020398e","after":"5ac486599abf791475b545b40fc3d49f1aa5bd53","ref":"refs/heads/main","pushedAt":"2024-04-04T16:17:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Issue 288: Add log messages (#289)\n\n* Adding log messages to be able to trace activity when saving to a table with qbeast. Logs are separated into DEBUG, INFO, WARN and TRACE depending on the weight of the message and the developer's best practises. (To be added to the documentation in another PR)","shortMessageHtmlLink":"Issue 288: Add log messages (#289)"}},{"before":"277b1701512a9eb82a629a2d2bd8e7e3070b760a","after":"2fedabcad817ba9313de768ce830c52af020398e","ref":"refs/heads/main","pushedAt":"2024-04-04T12:17:10.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Issue 297: Reduce overhead for CubeDomainsBuilder instantiation (#298)\n\n* Reduce overhead for CubeDomainsBuilder instantiation\r\n\r\nFixes #297","shortMessageHtmlLink":"Issue 297: Reduce overhead for CubeDomainsBuilder instantiation (#298)"}},{"before":"065a6b244ba14c8750754dadb1e358d3a5f17417","after":"277b1701512a9eb82a629a2d2bd8e7e3070b760a","ref":"refs/heads/main","pushedAt":"2024-04-02T11:30:02.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"Jiaweihu08","name":"jiawei","path":"/Jiaweihu08","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47899566?s=80&v=4"},"commit":{"message":"Merge pull request #300 from Jiaweihu08/299-conflicting-columns-to-index-comparison\n\nIssue 299: Check columnsToIndex depending on user input","shortMessageHtmlLink":"Merge pull request #300 from Jiaweihu08/299-conflicting-columns-to-in…"}},{"before":"1d1be60cd01980a9417ef3130c6fb38a931e337a","after":"065a6b244ba14c8750754dadb1e358d3a5f17417","ref":"refs/heads/main","pushedAt":"2024-03-27T13:00:07.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 292: Merge main-1.0.0 into main (#284)\n\nMain added features:\r\n- Rollup\r\n- Domain-driven appends\r\n- Auto-indexing\r\n- Multi-block file\r\n\r\nAdditionally, this merge performs the following:\r\n- Updates documentation according to the new version.\r\n- Removes unnecessary classes (e.g., CubeInfo).\r\n- Resolves inconsistencies with the Auto Indexing #247 and CREATE EXTERNAL TABLE without OPTIONS #248 changes.\r\n- compact() is no longer necessary, but we are leaving it to avoid additional changes to the staging area. We have issue 294 open to resolve it later.\r\n\r\n---------\r\n\r\nCo-authored-by: Alexey Akimov\r\nCo-authored-by: Jiawei\r\nCo-authored-by: osopardo1\r\nCo-authored-by: osopardo1\r\nCo-authored-by: SrTangente\r\nCo-authored-by: SrTangente\r\nCo-authored-by: jiawei","shortMessageHtmlLink":"Issue 292: Merge main-1.0.0 into main (#284)"}},{"before":"f496012dd17bb4f13a0a3f786bb4dc50c154e444","after":"ba089b232a79bc34e96b57f9ec52ba76dc34f824","ref":"refs/heads/main-1.0.0","pushedAt":"2024-03-22T17:42:31.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"fpj","name":"Flavio Junqueira","path":"/fpj","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/572920?s=80&v=4"},"commit":{"message":"Issue 253: Add file skipping with delta (#254)\n\nThis is a rework of the query implementation. This PR uses internal Delta query engine always\r\nexcept the queries with sampling clause. For the later, the Qbeast engine is used.\r\n\r\n\r\n\r\n* #253 Initial import of the new implementation of FileIndex\r\n* #253 Logging in DefaultFileIndex and SamplingListFileStrategy are improved\r\n* #253 A test for DefaultFileIndex\r\n* #253 OTreeIndex and test are removed\r\n* #253 EmptyFile index is made serializable","shortMessageHtmlLink":"Issue 253: Add file skipping with delta (#254)"}},{"before":"a800310107b63347aefd93efbd534016df1e1714","after":"1d1be60cd01980a9417ef3130c6fb38a931e337a","ref":"refs/heads/main","pushedAt":"2024-03-21T13:31:15.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #287 from osopardo1/change-build-version\n\nUpdate version to 0.6.0","shortMessageHtmlLink":"Merge pull request #287 from osopardo1/change-build-version"}},{"before":"9bc54ebaab914f653dca549fd28814070650994c","after":"f496012dd17bb4f13a0a3f786bb4dc50c154e444","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-27T16:37:56.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"Jiaweihu08","name":"jiawei","path":"/Jiaweihu08","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47899566?s=80&v=4"},"commit":{"message":"Merge pull request #279 from Jiaweihu08/add-userMetadata-option\n\nAdd userMetadata option","shortMessageHtmlLink":"Merge pull request #279 from Jiaweihu08/add-userMetadata-option"}},{"before":"04b5d18e1ad4f31e41949b4332d041d7b20d8d22","after":"9bc54ebaab914f653dca549fd28814070650994c","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-27T10:40:55.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"Jiaweihu08","name":"jiawei","path":"/Jiaweihu08","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47899566?s=80&v=4"},"commit":{"message":"Merge pull request #277 from Jiaweihu08/275-update-delta-version\n\nUpdate delta lake version","shortMessageHtmlLink":"Merge pull request #277 from Jiaweihu08/275-update-delta-version"}},{"before":"dbc07a4f93422912af68f10386dc8b5d29d7f900","after":"a800310107b63347aefd93efbd534016df1e1714","ref":"refs/heads/main","pushedAt":"2024-02-22T07:22:58.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #273 from fpj/issue-272-notice-file\n\nIssue 272: Add NOTICE file","shortMessageHtmlLink":"Merge pull request #273 from fpj/issue-272-notice-file"}},{"before":"908339665e54a32fa086f5b16036070d91653c8b","after":"dbc07a4f93422912af68f10386dc8b5d29d7f900","ref":"refs/heads/main","pushedAt":"2024-02-19T13:37:04.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #271 from fpj/issue-270-license-headers\n\nUpdate licence headers","shortMessageHtmlLink":"Merge pull request #271 from fpj/issue-270-license-headers"}},{"before":"6a780ea18010c773c14825402edc2c90d18d6cba","after":"04b5d18e1ad4f31e41949b4332d041d7b20d8d22","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-13T11:35:40.000Z","pushType":"pr_merge","commitsCount":5,"pusher":{"login":"Jiaweihu08","name":"jiawei","path":"/Jiaweihu08","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47899566?s=80&v=4"},"commit":{"message":"Merge pull request #268 from Jiaweihu08/267-remove-redundant-metadata\n\n267 Remove redundant metadata creations","shortMessageHtmlLink":"Merge pull request #268 from Jiaweihu08/267-remove-redundant-metadata"}},{"before":"03800e6c7af9fa6974bd62a625bfd78caac64d80","after":"6a780ea18010c773c14825402edc2c90d18d6cba","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-08T11:29:29.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #266 from Jiaweihu08/fix-optimize-file-filtering\n\nConvert Seq to Set to improve time complexity","shortMessageHtmlLink":"Merge pull request #266 from Jiaweihu08/fix-optimize-file-filtering"}},{"before":"881c04e1b5941fe8b06a5a685103a762f45c6a17","after":"03800e6c7af9fa6974bd62a625bfd78caac64d80","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-07T14:14:21.000Z","pushType":"pr_merge","commitsCount":4,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #263 from Jiaweihu08/261-add-block-filtering-when-sampling\n\nAdd block filtering when sampling","shortMessageHtmlLink":"Merge pull request #263 from Jiaweihu08/261-add-block-filtering-when-…"}},{"before":"61b8db42e0745b7d2ce3f0a131ae3a9b248168e3","after":"881c04e1b5941fe8b06a5a685103a762f45c6a17","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-07T14:13:58.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #265 from Jiaweihu08/fix-cubeStatus-type\n\ncubeStatuses should be a SortedMap","shortMessageHtmlLink":"Merge pull request #265 from Jiaweihu08/fix-cubeStatus-type"}},{"before":"8bc87dc8cb839130f2d82478883d9875e1e4fc65","after":"61b8db42e0745b7d2ce3f0a131ae3a9b248168e3","ref":"refs/heads/main-1.0.0","pushedAt":"2024-02-05T09:24:43.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #260 from alexeiakimov/259-add-warning-when-writes-are-ignored-for-idempotence\n\ntxnAppId and txnVersion are logged with warning","shortMessageHtmlLink":"Merge pull request #260 from alexeiakimov/259-add-warning-when-writes…"}},{"before":"1b024c771713801e731b903681e1571cc1a8046c","after":"908339665e54a32fa086f5b16036070d91653c8b","ref":"refs/heads/main","pushedAt":"2024-01-22T10:28:23.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #258 from alexeiakimov/256-backport-refactor-publishing-configuration-for-maven\n\n#256 backport to main publishing to local Ivy and Maven turned on","shortMessageHtmlLink":"Merge pull request #258 from alexeiakimov/256-backport-refactor-publi…"}},{"before":"b5e6defb0606504b3a1a0779184e671c7947895a","after":"8bc87dc8cb839130f2d82478883d9875e1e4fc65","ref":"refs/heads/main-1.0.0","pushedAt":"2024-01-19T14:11:00.000Z","pushType":"pr_merge","commitsCount":3,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #257 from alexeiakimov/256-refactor-publishing-configuration-for-maven\n\nPublishing turned on for the qbeastSpark project","shortMessageHtmlLink":"Merge pull request #257 from alexeiakimov/256-refactor-publishing-con…"}},{"before":"3efc2fc13014b12364de22548d2b0e22772089fe","after":"1b024c771713801e731b903681e1571cc1a8046c","ref":"refs/heads/main","pushedAt":"2024-01-09T13:24:10.000Z","pushType":"pr_merge","commitsCount":2,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #240 from cdelfosse/readme-spark-update\n\nUpdate README.md to use the latest Spark, Delta, and Qbeast format versions","shortMessageHtmlLink":"Merge pull request #240 from cdelfosse/readme-spark-update"}},{"before":"dd71a00a4969207d72d61541330111f12ec65489","after":"b5e6defb0606504b3a1a0779184e671c7947895a","ref":"refs/heads/main-1.0.0","pushedAt":"2024-01-09T13:11:48.000Z","pushType":"pr_merge","commitsCount":10,"pusher":{"login":"osopardo1","name":"Paola Pardo","path":"/osopardo1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/22685017?s=80&v=4"},"commit":{"message":"Merge pull request #251 from osopardo1/upgrade-spark-3.5-delta-3.0\n\nUpgrade Apache Spark 3.5 and Delta Lake 3.0","shortMessageHtmlLink":"Merge pull request #251 from osopardo1/upgrade-spark-3.5-delta-3.0"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEVdArpgA","startCursor":null,"endCursor":null}},"title":"Activity · Qbeast-io/qbeast-spark"}