njsscan fails to scan projects containing symlinks

[disposedtrolley]

Hello!

I'm running njsscan v0.3.3 which appears to crash whenever scanning a directory that contains a symlink. This is an issue for any version of njsscan that can pull libsast 1.5.2.

For example, running njsscan . within this repository will yield:

- Pattern Match ████████████████████████████████████████████████████████████ 6
Exception in thread Thread-1 (myrunner):
Traceback (most recent call last):
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/semgrep/semgrep_main.py", line 340, in main
    target_manager = TargetManager(
  File "<attrs generated init semgrep.target_manager.TargetManager>", line 24, in __init__
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/semgrep/target_manager.py", line 486, in __attrs_post_init__
    self.targets = [
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/semgrep/target_manager.py", line 487, in <listcomp>
    Target(
  File "<attrs generated init semgrep.target_manager.Target>", line 7, in __init__
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/semgrep/target_manager.py", line 341, in validate_path
    raise FilesNotFoundError(paths=tuple([value]))
semgrep.error.FilesNotFoundError: File not found: main_sym.js

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/libsast/common.py", line 45, in myrunner
    ret[0] = function(*args, **kwargs)
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/libsast/core_sgrep/helpers.py", line 58, in invoke_semgrep
    ) = semgrep_main.main(
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/semgrep/semgrep_main.py", line 351, in main
    raise SemgrepError(e)
semgrep.error.SemgrepError: File not found: main_sym.js
- Semantic Grep  8
Traceback (most recent call last):
  File "/Users/james/.asdf/installs/python/3.10.4/bin/njsscan", line 8, in <module>
    sys.exit(main())
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/njsscan/__main__.py", line 77, in main
    ).scan()
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/njsscan/njsscan.py", line 44, in scan
    result = scanner.scan()
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/libsast/scanner.py", line 65, in scan
    self.options).scan(valid_paths)
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/libsast/core_sgrep/semantic_sgrep.py", line 41, in scan
    self.format_output(sgrep_out)
  File "/Users/james/.asdf/installs/python/3.10.4/lib/python3.10/site-packages/libsast/core_sgrep/semantic_sgrep.py", line 46, in format_output
    self.findings['errors'] = results['errors']
TypeError: 'NoneType' object is not subscriptable

Note the line semgrep.error.SemgrepError: File not found: main_sym.js.

I believe this is the underlying issue:

1. njsscan v0.3.3 uses libsast 1.5.2, which depends on semgrep 0.104.0. njsscan v0.3.1 used libsast 1.5.0 which depended on semgrep 0.80.0.
2. semgrep 0.104.0 includes validation logic that raises an exception for paths that are explicitly provided to scan, which libsast is providing here. semgrep 0.80.0 appeared to only filter out invalid paths, without raising an exception.

I also dug up a previous Semgrep issue where symlink filtering was a bit too aggressive, but I don't think this is a related issue.

I think one possible solution would be to proactively filter out symlinks prior to invoking Semgrep here.

Let me know if this makes sense to you, and thanks for all of your work on njsscan!

[ajinabraham]

Thanks for the detailed report. I will take a look at this when I have some bandwidth.