Possible bug: hasCodeWithoutSemanticChanges not behaving as expected

[adrianoapmartins]

Describe the bug

hasCodeWithoutSemanticChanges seems to be true to code that clearly has semantic changes.

This breaks:

• https://github.com/reviewpad/marketing/issues/85
• https://github.com/reviewpad/website/pull/192

How to reproduce

Using the following reviewpad.yml configuration:

Current config:

workflows:
  - name: automating-safe-approvals-for-low-risk-changes
    if:
      - rule: '!$isDraft()
            && $hasCodeWithoutSemanticChanges()
            && !$hasFilePattern("src/package.json")'
    then:
      - '$approve("LGTM: this PR contains only low-risk code changes")'

and suggest the following changes in a PR:

- var name = 'World';
- console.log(`Hello ${name}`);
+ var first = 'Hello';
+ var second = 'World';
+ var sentence = first + " " + second;
+ console.log(sentence);

Live examples:
adrianoapmartins/layout#6
adrianoapmartins/layout#7

Expected behavior

hasCodeWithoutSemanticChanges returns false and fails the rule on the reviewpad.yml configuration

Screenshots

Conversation

Files changed

Reviewpad Log

2023-03-30T16:10:26Z [info] [START] request received
2023-03-30T16:10:26Z [info] Request details: app=gh_app_196990, event_type=pull_request, event_action=opened, delivery_id=5ec11b50-cf15-11ed-8637-d13f7f3270ce, repo_url=https://github.com/adrianoapmartins/layout
2023-03-30T16:10:27Z [info] Processing pull_request event
2023-03-30T16:10:27Z [info] Found pull request 7
2023-03-30T16:10:27Z [info] [REVIEWPAD-START] running reviewpad on pull request https://github.com/adrianoapmartins/layout/pull/7
2023-03-30T16:10:29Z [info] Analyzing workflow 'automating-safe-approvals-for-low-risk-changes'
2023-03-30T16:10:31Z [info] Evaluating workflow 'automating-safe-approvals-for-low-risk-changes'
2023-03-30T16:10:31Z [info] File code/hello-world.js is not ignored
2023-03-30T16:10:32Z [info] No semantic changes detected
2023-03-30T16:10:32Z [info] Rule '!$isDraft() && $hasCodeWithoutSemanticChanges() && !$hasFilePattern("src/package.json")' activated
2023-03-30T16:10:32Z [info] Executing program
2023-03-30T16:10:33Z [info] Creating review APPROVE with body LGTM: this PR contains only low-risk code changes
2023-03-30T16:10:34Z [info] Action $approve("LGTM: this PR contains only low-risk code changes") executed
2023-03-30T16:10:34Z [info] Execution done
2023-03-30T16:10:34Z [info] Generating report

[reviewpad]

AI-Generated Pull Request Summary: This issue describes a bug where the function hasCodeWithoutSemanticChanges incorrectly returns true when there are clearly semantic changes in the code. This leads to problems such as wrongly approving pull requests that shouldn't be approved. The issue provides a code example to reproduce the issue, which includes the use of the reviewpad.yml configuration file and suggested code changes in a pull request. Additionally, the issue provides screenshots of the problem and the relevant logs for further investigation. The expected behavior should be for hasCodeWithoutSemanticChanges to return false and fail the rule in the reviewpad.yml configuration.

[adrianoapmartins]

@marcelosousa I need you to unblock this for me:

1. I'll check it out soon
2. Put this article on hold for an indefinite time, I'm not sure when I'll be able to check it.

The article: https://github.com/reviewpad/marketing/issues/85