Sync with Pro (TEST) #10249
Closed
Sync with Pro (TEST) #10249
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
IagoAbal
commented
May 15, 2024
IagoAbal
commented
Depends on #10216 To query the AWS STS endpoint, some extra data is needed to compute authorization headers. Currently these can be included in an `http` validator using the optional `auth` field. Since we should only ever need to query one endpoint for AWS tokens, this PR adds a new `aws` validator, where the rule writer can include much less information manually. Before an AWS validator looked like this: ``` validators: - http: request: auth: access_key_id: $KEY region: us-east-1 secret_access_key: $SECRET service: sts type: sigv4 headers: Host: sts.amazonaws.com User-Agent: Semgrep method: GET url: https://sts.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15 response: ... ``` Now it can look like this: ``` validators: - aws: request: access_key_id: $KEY region: us-east-1 secret_access_key: $SECRET response: ... ``` synced from 64c9364fbeea7c56bbf26e8804566867e21477b8
During the merge of Pro+OSS we renamed `semgrep/` to `OSS/` and that
broke the benchmarks.
Had to remove an expected finding in _aws-doc-sdk-examples_:
Error running benchmark semgrep.bench.aws-doc-sdk-examples
Missing 1 findings:
{"check_id":
"OSS.perf.bench.rules_cache.aws-doc-sdk-examples.tainted-sql-string",
"end": {"col": 243, "line": 57, "offset": 2421}, "extra": "<masked in
benchmarks>", "path": "<masked in
benchmarks>/OSS/perf/bench/aws-doc-sdk-examples/input/aws-doc-sdk-examples.git/javav2/usecases/CreatingSpringRedshiftRest/src/main/java/com/aws/rest/InjectWorkService.java",
"start": {"col": 35, "line": 57, "offset": 2213}}
I tried reproducing this with a pre-merge version (but using directly
_semgrep-core-proprietary_) and I could not get that finding either.
Also deleted some files that referred to Emma's HOME folder which I'm
assuming should not have been committed.
test plan:
CI
synced from f089de8d20f6143034fe5a63a3b81649ab490317
test plan: modify OSS/src/main/Main.ml indentation and run pre-commit run -a => now it fixes the file synced from 0fb525f415fcc7bfe13cda0e0c0b9b1bd86c090b
Previously we only consumed a single "global" ignore list from the App. With this, we'll use the new `product_ignored_files` (See semgrep/semgrep-interfaces#248) and only use the deprecated `ignored_files` as a fallback if `product_ignored_files` is not present. Existing plumbing makes it so that this merely requires a swap out for the `ignore_patterns` property. Test plan: updated pytest coverage. synced from 014272b1ee69731f74af0aff2a59cbd50e2f26c8
test plan: wait for sync script of iago and see if precommit now pass for #10241 synced from 55840a1d105aece6cb5d054dbd3f08696ca40d49
test plan: pre-commit run -a See that it fixes some of the CI failures in #10241 precommit check synced from 740a985a8b024fec103f7d96a9d542302e07a4ff
test plan: made similar PR in semgrep: https://github.com/semgrep/semgrep/pull/10243/files and it passes synced from b273346673bfa331418ec6457bfec701b64cf5ba
Adds support for parsing git urls without an explicit protocol or being terminated in ".git" that contain dashes in the owner or repo. closes [saf-988](https://linear.app/semgrep/issue/SAF-988/limitation-in-git-url-parser-for-the-format-gitserverownerrepo-with-no) ## test plan Adds tests, CI should continue to pass, plus the following commands. * `make` * `make install` * `cd semgrep/cli` * `make setup` * `make test` synced from df1b33caeb13087b6f206b3a95ff4d1b8fe60bc6
test plan: pre-commit run -a synced from 7476ab566890701c85fe1fe1c370437dca2b2fbb
|
PR checklist:
If you're unsure about any of this, please see: |
aryx
reviewed
May 15, 2024
| @@ -52,8 +52,8 @@ handler.setFormatter( | |||
| ) | |||
| logger.addHandler(handler) | |||
|
|
|||
| PERF_PATH = "semgrep/perf" | |||
| PERF_RULE_PATH = "semgrep.perf" | |||
| PERF_PATH = "OSS/perf" | |||
| @@ -1,4003 +0,0 @@ | |||
| [ | |||
There was a problem hiding this comment.
I deleted it because given that all paths referred to Emma's HOME path then I assume this was committed by mistake. I warned and asked Emma to review the PR.
| "spans": [ | ||
| { | ||
| "end": { | ||
| "col": 46, | ||
| "line": 4, | ||
| "offset": 45 | ||
| }, | ||
| "file": "/Users/emma/workspace/semgrep-proprietary/semgrep/perf/bench/kotlinPrefilterTest/input/leakcanary/shark/shark/src/main/java/shark/ReferenceReader.kt", | ||
| "file": "/Users/emma/workspace/semgrep-proprietary/OSS/perf/bench/kotlinPrefilterTest/input/leakcanary/shark/shark/src/main/java/shark/ReferenceReader.kt", |
There was a problem hiding this comment.
@emjin why do we have /Users/emma/... in snapsot files?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.