-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AAE-22075 Fix "detaching an uninitialized collection" logs and improve task search query performance #1441
Merged
tom-dal
merged 27 commits into
develop
from
fix/AAE-22075-fixing_Detaching_an_uninitialized_collection_with_enabled_filters_from_a_session
May 16, 2024
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…azy_load_no_trans=true", if set to false, exception is raised)
This reverts commit 6a148ac.
* AAE-21097 add permission getter on Jwt adapter, implement conversion to granted authority and include permissions in auth flow * AAE-21097 add AuthorizationConfigurerIT, fix omitted methods logic * AAE-21097 fix tests * AAE-21097 add license header * AAE-21097 remove unused import * AAE-21097 CustomAuthorizationManager implemented with different authority types for role and permission * AAE-21097 move CustomAuthorizationManager * AAE-21097 license header
from: https://github.com/Activiti/Activiti Co-authored-by: alfresco-build <>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.23.1 to 3.25.3. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@0b21cf2...d39d31e) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…n task table" This reverts commit 3e35763.
…alized_collection_with_enabled_filters_from_a_session
Quality Gate passedIssues Measures |
igdianov
approved these changes
May 16, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good work!
tom-dal
deleted the
fix/AAE-22075-fixing_Detaching_an_uninitialized_collection_with_enabled_filters_from_a_session
branch
May 16, 2024 07:34
jesty
added a commit
that referenced
this pull request
May 17, 2024
…e task search query performance (#1441) * AAE-22075 Wip initial commit * WIP * AAE-22075 Wip fixed failing test * AAE-22075 reproduce error. ("spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true", if set to false, exception is raised) * AAE-22075 reproduce exact situation of issue * AAE-22075 tentative with DSL filters * Revert "AAE-22075 tentative with DSL filters" This reverts commit 6a148ac. * AAE-22075 test with entity graph * AAE-18428 Fix Swagger API suggest to use the HAL-JSON format (#1306) * AAE-21097 Implement permission-based authorization (#1427) * AAE-21097 add permission getter on Jwt adapter, implement conversion to granted authority and include permissions in auth flow * AAE-21097 add AuthorizationConfigurerIT, fix omitted methods logic * AAE-21097 fix tests * AAE-21097 add license header * AAE-21097 remove unused import * AAE-21097 CustomAuthorizationManager implemented with different authority types for role and permission * AAE-21097 move CustomAuthorizationManager * AAE-21097 license header * AAE-22123 Fix NPE when getting permissions (#1431) * AAE-22123 added key check in KeycloakResourceJwtAdapter * AAE-22123 change test * chore(deps): upgrade Activiti/Activiti to version 8.5.0-alpha.2 (#1432) from: https://github.com/Activiti/Activiti Co-authored-by: alfresco-build <> * build(deps): bump github/codeql-action from 3.23.1 to 3.25.3 (#1429) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.23.1 to 3.25.3. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@0b21cf2...d39d31e) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * AAE-22123 Fix omitted methods (#1433) * AAE-22123 fix omitted methods management * AAE-22123 fix omitted methods management and tests * AAE-22075 perform single queries on join table after select on task table * Revert "AAE-22075 perform single queries on join table after select on task table" This reverts commit 3e35763. * AAE-22075 use jpaquery and fluentyquery * AAE-22075 reduced number of queries, applied sorting parameters * AAE-22075 fix sorting * AAE-22075 fix sorting * AAE-22075 fix * AAE-22075 remove unused imports * AAE-22075 remove sql logs from test * AAE-22075 fix import * AAE-22075 added tests * AAE-22075 refactoring --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Akhilesh Pamidimarthi <akhilesh.pamidimarthi@hyland.com> Co-authored-by: jesty <davidecerbo@gmail.com> Co-authored-by: RahulKumar1606 <126051097+RahulKumar1606@users.noreply.github.com> Co-authored-by: Alfresco Build <8039454+alfresco-build@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
erdemedeiros
pushed a commit
that referenced
this pull request
May 17, 2024
RCorsaletti
pushed a commit
that referenced
this pull request
May 22, 2024
…e task search query performance (#1441) * AAE-22075 Wip initial commit * WIP * AAE-22075 Wip fixed failing test * AAE-22075 reproduce error. ("spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true", if set to false, exception is raised) * AAE-22075 reproduce exact situation of issue * AAE-22075 tentative with DSL filters * Revert "AAE-22075 tentative with DSL filters" This reverts commit 6a148ac. * AAE-22075 test with entity graph * AAE-18428 Fix Swagger API suggest to use the HAL-JSON format (#1306) * AAE-21097 Implement permission-based authorization (#1427) * AAE-21097 add permission getter on Jwt adapter, implement conversion to granted authority and include permissions in auth flow * AAE-21097 add AuthorizationConfigurerIT, fix omitted methods logic * AAE-21097 fix tests * AAE-21097 add license header * AAE-21097 remove unused import * AAE-21097 CustomAuthorizationManager implemented with different authority types for role and permission * AAE-21097 move CustomAuthorizationManager * AAE-21097 license header * AAE-22123 Fix NPE when getting permissions (#1431) * AAE-22123 added key check in KeycloakResourceJwtAdapter * AAE-22123 change test * chore(deps): upgrade Activiti/Activiti to version 8.5.0-alpha.2 (#1432) from: https://github.com/Activiti/Activiti Co-authored-by: alfresco-build <> * build(deps): bump github/codeql-action from 3.23.1 to 3.25.3 (#1429) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.23.1 to 3.25.3. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@0b21cf2...d39d31e) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * AAE-22123 Fix omitted methods (#1433) * AAE-22123 fix omitted methods management * AAE-22123 fix omitted methods management and tests * AAE-22075 perform single queries on join table after select on task table * Revert "AAE-22075 perform single queries on join table after select on task table" This reverts commit 3e35763. * AAE-22075 use jpaquery and fluentyquery * AAE-22075 reduced number of queries, applied sorting parameters * AAE-22075 fix sorting * AAE-22075 fix sorting * AAE-22075 fix * AAE-22075 remove unused imports * AAE-22075 remove sql logs from test * AAE-22075 fix import * AAE-22075 added tests * AAE-22075 refactoring --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Akhilesh Pamidimarthi <akhilesh.pamidimarthi@hyland.com> Co-authored-by: jesty <davidecerbo@gmail.com> Co-authored-by: RahulKumar1606 <126051097+RahulKumar1606@users.noreply.github.com> Co-authored-by: Alfresco Build <8039454+alfresco-build@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue link: AAE-22075
The cause for the task search being slow when process variable have to be fetched is that Hibernate is performing N+1 queries: one query to search the tasks and N queries to fetch process variables for each task. (and also N queries to fetch candidate users and candidate groups).
A first attempt to solve the problem has been made just applying EntityGraph. Using an EntityGraph we are able to specify which field to retrieve eagerly, but the drawback is that, being the table join a cartesian product, Hibernate is not able anymore to handle pagination and sorting in the query. This way, pagination and sorting are performed in-memory which cause OOM error when the task list is huge. Additionally, even for smaller task lists that still exceed the page size, overall performance is not improved because of the large amount of data to retrieve from database.
https://stackoverflow.com/questions/75637639/spring-data-3-fetch-join-with-paging-and-specifications-is-not-working-properly
The proposed solution follows a similar approach to the one already leveraged in this other repository method
activiti-cloud/activiti-cloud-query-service/activiti-cloud-services-query/activiti-cloud-services-query-repo/src/main/java/org/activiti/cloud/services/query/app/repository/CustomizedTaskRepositoryImpl.java
Line 66 in 1527e48
It consists of performing 3 queries:
Additionally, the user Spring @transactional annotation ,instead of Jarkarta one, with readOnly=true attribute, should improve performance by skipping dirty-checking on entities, since DB flush is not needed.
https://medium.com/@jkha7371/is-transactional-readonly-true-a-silver-bullet-1dbf130c97f8
https://vladmihalcea.com/spring-read-only-transaction-hibernate-optimization/
Outcome of simple performance tests performed via Postman:
Jmeter tests are underway.