Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: Use FromRequest for datasource APIs #33039

Merged
merged 4 commits into from May 2, 2024
Merged

chore: Use FromRequest for datasource APIs #33039

merged 4 commits into from May 2, 2024

Conversation

sharat87
Copy link
Member

@sharat87 sharat87 commented Apr 29, 2024 •
edited

This PR removes use of @JsonProperty in Datasource entity so that it encrypted fields can be stored to the database in Postgres version.

EE Unit and Cypress All pass.

/ok-to-test tags="@tag.All"

Tip

馃煝 馃煝 馃煝 All cypress tests have passed! 馃帀 馃帀 馃帀
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/8904366086
Commit: 1546722
Cypress dashboard url: Click here!

@github-actions github-actions bot added the skip-changelog Adding this label to a PR prevents it from being listed in the changelog label Apr 29, 2024
@sharat87 sharat87 added the ok-to-test Required label for CI label Apr 29, 2024
@sharat87 sharat87 added ok-to-test Required label for CI and removed ok-to-test Required label for CI labels Apr 29, 2024
@sharat87 sharat87 added ok-to-test Required label for CI and removed ok-to-test Required label for CI labels Apr 30, 2024
@sharat87 sharat87 marked this pull request as ready for review May 2, 2024 07:37
@coderabbitai coderabbitai
Copy link
Contributor

coderabbitai bot commented May 2, 2024 •
edited

Walkthrough

Walkthrough

The recent updates primarily focus on enhancing the JSON serialization control across various authentication and configuration classes within the AppSmith server interfaces. These changes involve the widespread application of @JsonView annotations to manage data visibility more effectively during JSON serialization and deserialization processes. This is complemented by adjustments in dependency injections and method annotations in controller classes to streamline operations and improve security.

Changes

Files Change Summary
ApiKeyAuth.java, BasicAuth.java, BearerTokenAuth.java, Connection.java, DBAuth.java, Datasource.java, DatasourceConfiguration.java, DatasourceStorageDTO.java, OAuth2.java Added or modified @JsonView annotations for enhanced JSON serialization control.
DatasourceController.java, .../controllers/ce/DatasourceControllerCE.java Removed RateLimitService dependency and updated constructor and method annotations. Enhanced method parameters with @JsonView for targeted data visibility during operations.
Recent Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between ef89875 and 1546722.
Files selected for processing (11)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/ApiKeyAuth.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/BasicAuth.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/BearerTokenAuth.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/Connection.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/DBAuth.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/Datasource.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/DatasourceConfiguration.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/DatasourceStorageDTO.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/OAuth2.java (3 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/DatasourceController.java (2 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/DatasourceControllerCE.java (5 hunks)
Files not reviewed due to errors (8)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/BearerTokenAuth.java (no review received)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/Connection.java (no review received)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/BasicAuth.java (no review received)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/DBAuth.java (no review received)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/DatasourceController.java (no review received)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/ApiKeyAuth.java (no review received)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/DatasourceConfiguration.java (no review received)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/DatasourceStorageDTO.java (no review received)
Additional comments not posted (22)
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/OAuth2.java (10)

47-47: Ensure that the grantType field's new @JsonView annotation aligns with the intended visibility and access control.

56-56: The clientId field's @JsonView annotation is correctly applied for public and request-specific views.

59-59: The clientSecret field correctly combines @Encrypted with @JsonView to ensure secure handling during serialization.

62-62: The authorizationUrl field's @JsonView annotation is appropriately set for public and request-specific views.

67-67: The accessTokenUrl field's @JsonView annotation is correctly applied, ensuring controlled access.

71-71: The scopeString field's @JsonView annotation is correctly set for public and request-specific views.

74-74: The scope field's @JsonView annotation is appropriately applied, aligning with the intended visibility.

81-81: The headerPrefix field's @JsonView annotation ensures that it is only accessible in appropriate contexts.

86-86: The audience field's @JsonView annotation is correctly set for public and request-specific views.

89-89: The resource field's @JsonView annotation is correctly applied, ensuring controlled access.

app/server/appsmith-interfaces/src/main/java/com/appsmith/external/models/Datasource.java (7)

42-42: The pluginName field's @JsonView annotation is set to public, which is appropriate for non-sensitive data.

47-47: The organizationId field's @JsonView annotation is set to public, ensuring it is accessible where needed.

53-53: The templateName field's @JsonView annotation is set to public, aligning with its intended use.

61-61: The datasourceStorages field's @JsonView annotation is correctly applied for public and request-specific views.

92-92: The isRecentlyCreated field's @JsonView annotation is set to public, which is appropriate for this non-sensitive data.

100-100: The isTemplate field's @JsonView annotation is set to public, ensuring it is accessible where needed.

107-107: The isMock field's @JsonView annotation is set to public, aligning with its intended use.

app/server/appsmith-server/src/main/java/com/appsmith/server/controllers/ce/DatasourceControllerCE.java (5)

59-59: The getAll method's @JsonView annotation is correctly applied, ensuring that the response structure is controlled.

71-71: The create method's @JsonView annotation on the request body ensures that only the necessary data is deserialized from the request.

82-82: The update method's @JsonView annotation on the request body is appropriately used to control data deserialization.

49-49: The removal of RateLimitService from the constructor is noted. Ensure that rate limiting is handled elsewhere if necessary.

Verification successful

The verification process has shown that RateLimitService is extensively used across various parts of the application, including critical areas like authentication and user services. This widespread use suggests that the application maintains a robust rate limiting mechanism, even though it has been removed from the DatasourceControllerCE. Therefore, it seems that the removal from this specific controller does not compromise the overall rate limiting strategy of the application.

  • AuthenticationSuccessHandlerCE.java and AuthenticationSuccessHandler.java use it to handle rate limiting on authentication success.
  • SecurityConfig.java integrates it into security configurations.
  • DatasourceServiceImpl.java and DatasourceServiceCEImpl.java include it, indicating that datasource operations might still be rate-limited through these services.
  • LoginRateLimitFilter.java applies rate limiting specifically to login attempts.

Given this distribution, the removal from DatasourceControllerCE appears to be compensated by its application in other critical areas. Thus, the initial review comment is consistent with the current state of the codebase.

Scripts Executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify if rate limiting is handled in other parts of the application after the removal from DatasourceControllerCE.

# Search for rate limiting logic in other controllers or services.
rg --files-with-matches --type java 'RateLimitService' | xargs rg 'RateLimitService'

Length of output: 5201

49-49: The updateDatasourceStorages method's @JsonView annotation is correctly applied, ensuring that the response structure is controlled.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@sharat87 sharat87 requested a review from trishaanand May 2, 2024 09:49
@sharat87 sharat87 merged commit 258e92e into release May 2, 2024
111 of 118 checks passed
@sharat87 sharat87 deleted the chore/datasource-jsonview branch May 2, 2024 10:46
@NilanshBansal NilanshBansal mentioned this pull request May 6, 2024
Merged
2 tasks
sharat87 pushed a commit that referenced this pull request May 6, 2024
@NilanshBansal
fix: authenticationType field saved to database (#33196)
a5edfb0 
## Description
With the introduction of FromRequest for the create datasource API in
[this](#33039) PR, we have
restricted the fields that will be accepted from the client in the
request body. Due to this the `authenticationType` field was not getting
accepted from the client in the create datasource API, which lead to the
failure of airtable queries.

This PR allows the acceptance of `authenticationType` field from the
client.
Discussions
[here](https://theappsmith.slack.com/archives/CTHN8GX5Y/p1714974437746959).

Fixes #33184   

## Automation

/ok-to-test tags="@tag.Datasource"

### 馃攳 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!CAUTION]
> 馃敶 馃敶 馃敶 Some tests have failed.
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/8966382606>
> Commit: a292aa0
> Cypress dashboard: <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=8966382606&attempt=3&selectiontype=test&testsstatus=failed&specsstatus=fail"
target="_blank"> Click here!</a>
> The following are new failures, please fix them before merging the PR:
<ol>
> <li>cypress/e2e/Regression/ServerSide/QueryPane/DSDocs_Spec.ts </ol>
> To know the list of identified flaky tests - <a
href="https://internal.appsmith.com/app/cypress-dashboard/identified-flaky-tests-65890b3c81d7400d08fa9ee3?branch=master"
target="_blank">Refer here</a>

<!-- end of auto-generated comment: Cypress test results  -->






## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No
NilanshBansal added a commit that referenced this pull request May 6, 2024
@NilanshBansal
fix: authenticationType field saved to database (#33196)
a054e8c 
## Description
With the introduction of FromRequest for the create datasource API in
[this](#33039) PR, we have
restricted the fields that will be accepted from the client in the
request body. Due to this the `authenticationType` field was not getting
accepted from the client in the create datasource API, which lead to the
failure of airtable queries.

This PR allows the acceptance of `authenticationType` field from the
client.
Discussions
[here](https://theappsmith.slack.com/archives/CTHN8GX5Y/p1714974437746959).

Fixes #33184

## Automation

/ok-to-test tags="@tag.Datasource"

### 馃攳 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!CAUTION]
> 馃敶 馃敶 馃敶 Some tests have failed.
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/8966382606>
> Commit: a292aa0
> Cypress dashboard: <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=8966382606&attempt=3&selectiontype=test&testsstatus=failed&specsstatus=fail"
target="_blank"> Click here!</a>
> The following are new failures, please fix them before merging the PR:
<ol>
> <li>cypress/e2e/Regression/ServerSide/QueryPane/DSDocs_Spec.ts </ol>
> To know the list of identified flaky tests - <a
href="https://internal.appsmith.com/app/cypress-dashboard/identified-flaky-tests-65890b3c81d7400d08fa9ee3?branch=master"
target="_blank">Refer here</a>

<!-- end of auto-generated comment: Cypress test results  -->

## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No

(cherry picked from commit a5edfb0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nidhi-nair nidhi-nair nidhi-nair approved these changes

@mohanarpit mohanarpit Awaiting requested review from mohanarpit

@nsarupr nsarupr Awaiting requested review from nsarupr

@abhvsn abhvsn Awaiting requested review from abhvsn

@sondermanish sondermanish Awaiting requested review from sondermanish sondermanish is a code owner

@trishaanand trishaanand Awaiting requested review from trishaanand

Assignees
No one assigned
Labels
ok-to-test Required label for CI skip-changelog Adding this label to a PR prevents it from being listed in the changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@sharat87 @nidhi-nair