Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve diagnostics deduplication #58220

Merged
merged 5 commits into from Apr 26, 2024
Merged

Improve diagnostics deduplication #58220

merged 5 commits into from Apr 26, 2024

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Apr 17, 2024

Fixes #58207.

Before, we only considered two diagnostics the same if everything about them was equal, including the message chain and related information. With this PR, we now consider diagnostics at the same location and with the same head message as being the same, regardless of the rest of the message chain and related information.
That is necessary because, as the new test shows, we can generate the "same" diagnostic at different moments in type checking, and they will have different elaboration (different message chains and related information).

This PR also makes it so deduplication keeps the diagnostic with more elaboration between two diagnostics considered to be equal.

Note that this doesn't completely solve the issue, because not all diagnostics that should be considered equal have the same code and head message.
Edit: follow-up PR #58318 fixes this.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Apr 17, 2024
@gabritto
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 17, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the user tests comparing main and refs/pull/58220/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@gabritto
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 297,009k (± 0.01%) 297,007k (± 0.00%) ~ 296,984k 297,022k p=0.873 n=6
Parse Time 2.71s (± 0.54%) 2.71s (± 0.40%) ~ 2.69s 2.72s p=0.805 n=6
Bind Time 0.82s (± 0.00%) 0.82s (± 0.00%) ~ 0.82s 0.82s p=1.000 n=6
Check Time 8.35s (± 0.36%) 8.32s (± 0.39%) ~ 8.26s 8.35s p=0.683 n=6
Emit Time 7.06s (± 0.39%) 7.07s (± 0.39%) ~ 7.05s 7.12s p=0.871 n=6
Total Time 18.93s (± 0.11%) 18.93s (± 0.28%) ~ 18.84s 19.00s p=1.000 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,019k (± 0.98%) 192,420k (± 0.79%) ~ 191,766k 195,511k p=1.000 n=6
Parse Time 1.63s (± 1.53%) 1.66s (± 1.13%) ~ 1.62s 1.67s p=0.121 n=6
Bind Time 0.87s (± 1.40%) 0.87s (± 0.94%) ~ 0.86s 0.88s p=1.000 n=6
Check Time 11.32s (± 0.37%) 11.31s (± 0.20%) ~ 11.29s 11.35s p=0.572 n=6
Emit Time 3.15s (± 0.37%) 3.13s (± 0.73%) ~ 3.10s 3.16s p=0.164 n=6
Total Time 16.97s (± 0.33%) 16.96s (± 0.28%) ~ 16.88s 17.01s p=1.000 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,749,594k (± 0.00%) 1,749,611k (± 0.00%) ~ 1,749,549k 1,749,680k p=0.521 n=6
Parse Time 9.99s (± 0.64%) 9.97s (± 0.45%) ~ 9.91s 10.04s p=0.688 n=6
Bind Time 3.35s (± 0.61%) 3.35s (± 0.64%) ~ 3.33s 3.39s p=0.805 n=6
Check Time 82.00s (± 0.26%) 81.92s (± 0.41%) ~ 81.26s 82.23s p=0.936 n=6
Emit Time 0.20s (± 2.06%) 0.20s (± 2.02%) ~ 0.20s 0.21s p=0.218 n=6
Total Time 95.54s (± 0.24%) 95.43s (± 0.36%) ~ 94.77s 95.68s p=1.000 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,307,395k (± 0.01%) 2,307,221k (± 0.04%) ~ 2,305,962k 2,308,279k p=0.810 n=6
Parse Time 7.42s (± 0.56%) 7.43s (± 0.45%) ~ 7.39s 7.49s p=1.000 n=6
Bind Time 2.74s (± 0.36%) 2.73s (± 0.69%) ~ 2.71s 2.76s p=0.332 n=6
Check Time 49.58s (± 0.93%) 49.41s (± 0.63%) ~ 49.05s 49.81s p=0.471 n=6
Emit Time 3.96s (± 1.34%) 4.08s (± 2.73%) +0.13s (+ 3.16%) 3.95s 4.24s p=0.045 n=6
Total Time 63.69s (± 0.62%) 63.67s (± 0.39%) ~ 63.27s 63.99s p=0.810 n=6
self-build-src-public-api - node (v18.15.0, x64)
Memory used 2,382,448k (± 0.01%) 2,382,950k (± 0.03%) ~ 2,381,858k 2,383,619k p=0.128 n=6
Parse Time 7.65s (± 0.84%) 7.66s (± 0.59%) ~ 7.61s 7.72s p=0.936 n=6
Bind Time 2.51s (± 0.91%) 2.51s (± 0.83%) ~ 2.49s 2.54s p=0.809 n=6
Check Time 49.73s (± 0.72%) 49.65s (± 0.65%) ~ 49.27s 50.15s p=0.748 n=6
Emit Time 3.89s (± 2.49%) 3.91s (± 2.78%) ~ 3.81s 4.11s p=0.936 n=6
Total Time 63.79s (± 0.60%) 63.76s (± 0.58%) ~ 63.24s 64.28s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Memory used 419,464k (± 0.01%) 419,484k (± 0.01%) ~ 419,436k 419,535k p=0.471 n=6
Parse Time 4.23s (± 0.25%) 4.21s (± 0.46%) ~ 4.18s 4.23s p=0.250 n=6
Bind Time 1.60s (± 1.93%) 1.60s (± 1.23%) ~ 1.58s 1.63s p=0.809 n=6
Check Time 22.34s (± 0.38%) 22.34s (± 0.23%) ~ 22.26s 22.40s p=0.809 n=6
Emit Time 1.73s (± 1.03%) 1.69s (± 2.93%) ~ 1.64s 1.77s p=0.107 n=6
Total Time 29.89s (± 0.33%) 29.84s (± 0.20%) ~ 29.77s 29.94s p=0.336 n=6
ts-pre-modules - node (v18.15.0, x64)
Memory used 368,996k (± 0.02%) 369,004k (± 0.02%) ~ 368,931k 369,168k p=1.000 n=6
Parse Time 3.66s (± 0.90%) 3.66s (± 0.36%) ~ 3.64s 3.68s p=0.870 n=6
Bind Time 1.90s (± 1.58%) 1.91s (± 1.61%) ~ 1.87s 1.94s p=0.677 n=6
Check Time 19.38s (± 0.26%) 19.37s (± 0.31%) ~ 19.28s 19.43s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 24.94s (± 0.37%) 24.95s (± 0.38%) ~ 24.78s 25.04s p=0.936 n=6
vscode - node (v18.15.0, x64)
Memory used 2,914,727k (± 0.00%) 2,914,826k (± 0.00%) ~ 2,914,737k 2,914,879k p=0.173 n=6
Parse Time 13.44s (± 0.40%) 13.43s (± 0.37%) ~ 13.34s 13.48s p=0.629 n=6
Bind Time 4.13s (± 2.57%) 4.10s (± 2.06%) ~ 4.05s 4.27s p=0.618 n=6
Check Time 72.23s (± 0.52%) 72.59s (± 0.32%) ~ 72.30s 72.91s p=0.093 n=6
Emit Time 20.79s (±10.20%) 20.15s (± 7.82%) ~ 19.42s 23.36s p=1.000 n=6
Total Time 110.58s (± 1.81%) 110.26s (± 1.32%) ~ 109.48s 113.19s p=0.688 n=6
webpack - node (v18.15.0, x64)
Memory used 409,388k (± 0.01%) 409,384k (± 0.01%) ~ 409,312k 409,441k p=0.936 n=6
Parse Time 3.27s (± 0.94%) 3.24s (± 0.58%) ~ 3.21s 3.26s p=0.145 n=6
Bind Time 1.38s (± 0.59%) 1.38s (± 0.54%) ~ 1.37s 1.39s p=0.729 n=6
Check Time 14.39s (± 0.16%) 14.38s (± 0.20%) ~ 14.34s 14.41s p=0.622 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 19.04s (± 0.18%) 19.00s (± 0.20%) ~ 18.96s 19.05s p=0.170 n=6
xstate-main - node (v18.15.0, x64)
Memory used 458,720k (± 0.01%) 458,689k (± 0.01%) ~ 458,603k 458,786k p=0.471 n=6
Parse Time 3.22s (± 0.68%) 3.22s (± 0.39%) ~ 3.21s 3.24s p=1.000 n=6
Bind Time 1.17s (± 0.88%) 1.17s (± 0.64%) ~ 1.16s 1.18s p=0.931 n=6
Check Time 18.08s (± 0.42%) 18.11s (± 0.25%) ~ 18.03s 18.16s p=0.375 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.48s (± 0.35%) 22.50s (± 0.24%) ~ 22.40s 22.55s p=0.336 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@gabritto Here are the results of running the top 400 repos comparing main and refs/pull/58220/merge:

Everything looks good!

@jakebailey
Copy link
Member

@typescript-bot run dt

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 17, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
run dt ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@@ -304,7 +305,7 @@ export function isExternalModuleNameRelative(moduleName: string): boolean {
}

export function sortAndDeduplicateDiagnostics<T extends Diagnostic>(diagnostics: readonly T[]): SortedReadonlyArray<T> {
return sortAndDeduplicate<T>(diagnostics, compareDiagnostics);
return sortAndDeduplicate<T>(diagnostics, compareDiagnostics, diagnosticsEqualityComparer);
Copy link
Member

@weswigham weswigham Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, DiagnosticCollection's .add method is still using compareDiagnosticsSkipRelatedInformation, so it's just first-in-wins (with the same root message) - should that also get updated to prefer the "lesser" message like this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compareDiagnosticsSkipRelatedInformation is still the same in terms of what it considers equal - two diagnostics with the same location and head message but different elaboration will still be considered different by that function, and so both will be added via DiagnosticCollection.add. Only later, when we call sortAndDeduplicateDiagnostics, will we use diagnosticsEqualityComparer and get rid of all but one of the diagnostics that we now consider equal.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I think about it, if we want to keep using functions like sortAndDeduplicate and insertSorted, I think we need to have two separate functions for comparison and equality: one that fully compares diagnostics for purposes of sorting (and sorting a diagnostic with more elaboration before one with less), and another for purposes of deduplication that is more permissive and only compares location and head message for equality. Otherwise we'd have to implement a function that somehow says "yes, those two diagnostics are equal, but one of them is preferred".

@gabritto gabritto merged commit ebcb09d into main Apr 26, 2024
25 checks passed
@gabritto gabritto deleted the gabritto/issue58207 branch April 26, 2024 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Duplicate diagnostics
5 participants