Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests executed on a real deployment as part of the CICD #1220

Open
dlpzx opened this issue Apr 25, 2024 · 1 comment
Open

Integration tests executed on a real deployment as part of the CICD #1220

dlpzx opened this issue Apr 25, 2024 · 1 comment

Comments

@dlpzx
Copy link
Contributor

dlpzx commented Apr 25, 2024

Is your feature request related to a problem? Please describe.
I would like data.all to be tested "in reality" once it is deployed. That allows us to verify code changes on real infrastructure. It speeds up development because it increases the confidence that developers and maintainers have that a feature does not introduce bugs.

Describe the solution you'd like
A way to execute locally and in a CICD pipeline tests that run against a real API of data.all.

Describe alternatives you've considered
Defining the tests using pytest. Once #950 is complete we will be able to use the SDK directly to configure the test clients and the API call definitions.

Additional context
Add any other context or screenshots about the feature request here.

P.S. Please Don't attach files. Add code snippets directly in the message body instead.

@dlpzx
Copy link
Contributor Author

dlpzx commented Apr 25, 2024

In PR #1219 the design for integration tests in AWS is introduced. In that PR only tests for Organizations are included. We slowly need to add tests for other core and modules APIs
Core:

  • Organizations
  • Environments - effort:large
  • Groups - effort:low
  • Permissions - effort:low
  • Stacks - effort:low
  • VPC -effort:low

Modules. ---> they depend on core modules, it might be difficult to test them without environments

  • Catalog (glossaries) - effort:medium
  • Dashboards - effort:medium
  • DataPipelines - effort:medium
  • DataSharing - effort:large
  • Datasets - effort:large
  • Feed - effort:low
  • MLStudio - effort:medium
  • Notebooks - effort:medium
  • Notifications - effort:low
  • Vote - effort:low
  • Worksheet - effort:medium

dlpzx added a commit that referenced this issue Apr 30, 2024
… CICD (#1219)

### Feature or Bugfix
- Feature

### Detail
Add integration tests that use a real Client to execute different
validation actions.

- Define the Client and the way API calls are posted to API Gateway in
the conftest
- Define the Cognito users and the different fixtures needed for all
tests
- Write tests for the Organization core module as example
- Add feature flag in `cdk.json` called `with_approval_tests` that can
be defined at the deployment environment level. If set to True, a
CodeBuild stage running the tests is created.

### Relates
- #1220

### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).

- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
  - Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
  - Is injection prevented by parametrizing queries?
  - Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
  - Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
  - Do you use a standard proven implementations?
  - Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
  - Have you used the least-privilege principle? How?


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant