Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check unresolved callables during partial eval #1497

Merged
merged 7 commits into from May 13, 2024

Conversation

swernli
Copy link
Collaborator

@swernli swernli commented May 8, 2024

This change updates the RCA check pass to not treat unresolved callables as errors and instead relies on a new check at partial eval time that checks the resolved callable with the known arguments to return RCA check errors at that point.

This change updates the RCA check pass to not treat unresolved callables as errors and instead relies on a new check at partial eval time that checks the resolved callable with the known arguments to return RCA check errors at that point.
Copy link

github-actions bot commented May 9, 2024

Benchmark for 8ff8b98

Click to view benchmark
Test Base PR %
Array append evaluation 340.0±2.10µs 345.2±3.00µs +1.53%
Array literal evaluation 179.9±2.61µs 171.3±5.23µs -4.78%
Array update evaluation 421.6±1.91µs 420.2±0.77µs -0.33%
Core + Standard library compilation 17.3±0.47ms 17.3±0.83ms 0.00%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.0±0.04ms 0.00%
Large file parity evaluation 33.8±0.10ms 33.8±0.49ms 0.00%
Large input file compilation 12.2±0.66ms 12.1±0.68ms -0.82%
Large input file compilation (interpreter) 46.0±1.27ms 44.7±1.23ms -2.83%
Large nested iteration 33.2±0.19ms 33.3±0.22ms +0.30%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1573.3±96.38µs 1545.7±34.27µs -1.75%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.12ms 7.7±0.11ms -1.28%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1410.0±32.96µs 1417.4±48.01µs +0.52%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.3±0.41ms 27.4±0.25ms +0.37%
Teleport evaluation 87.3±4.10µs 86.9±3.93µs -0.46%

Copy link

Benchmark for 4b8418e

Click to view benchmark
Test Base PR %
Array append evaluation 360.5±20.95µs 340.4±4.00µs -5.58%
Array literal evaluation 199.0±1.35µs 171.3±1.12µs -13.92%
Array update evaluation 438.8±1.59µs 423.9±15.41µs -3.40%
Core + Standard library compilation 19.9±0.96ms 17.1±0.39ms -14.07%
Deutsch-Jozsa evaluation 5.1±0.10ms 5.1±0.07ms 0.00%
Large file parity evaluation 33.8±0.10ms 33.9±0.22ms +0.30%
Large input file compilation 13.6±0.78ms 11.7±0.30ms -13.97%
Large input file compilation (interpreter) 51.3±1.75ms 47.6±1.72ms -7.21%
Large nested iteration 35.3±0.51ms 33.5±1.57ms -5.10%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1627.6±213.31µs 1544.3±34.94µs -5.12%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.1±0.23ms 7.7±0.08ms -4.94%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1500.8±216.15µs 1416.0±55.90µs -5.65%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.8±0.43ms 27.6±0.62ms -4.17%
Teleport evaluation 89.0±3.56µs 88.1±3.57µs -1.01%

Copy link

Benchmark for 21a9106

Click to view benchmark
Test Base PR %
Array append evaluation 345.0±8.99µs 342.9±12.88µs -0.61%
Array literal evaluation 179.4±1.03µs 171.2±0.80µs -4.57%
Array update evaluation 424.6±5.92µs 419.7±2.94µs -1.15%
Core + Standard library compilation 17.7±0.84ms 17.2±1.29ms -2.82%
Deutsch-Jozsa evaluation 5.1±0.07ms 5.1±0.05ms 0.00%
Large file parity evaluation 33.7±0.16ms 34.0±1.40ms +0.89%
Large input file compilation 12.3±0.87ms 11.4±0.30ms -7.32%
Large input file compilation (interpreter) 46.1±2.77ms 48.8±4.69ms +5.86%
Large nested iteration 33.6±0.90ms 33.3±0.35ms -0.89%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1536.2±34.70µs 1554.9±34.22µs +1.22%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.23ms 7.8±0.25ms +1.30%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1404.2±27.51µs 1438.0±62.67µs +2.41%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.4±0.24ms 27.7±0.35ms +1.09%
Teleport evaluation 87.6±3.67µs 87.7±4.04µs +0.11%

Copy link

Benchmark for 071746f

Click to view benchmark
Test Base PR %
Array append evaluation 353.9±2.36µs 339.2±1.92µs -4.15%
Array literal evaluation 198.7±0.74µs 170.7±2.50µs -14.09%
Array update evaluation 425.3±4.49µs 418.1±1.49µs -1.69%
Core + Standard library compilation 17.3±0.60ms 17.7±1.12ms +2.31%
Deutsch-Jozsa evaluation 5.0±0.04ms 5.0±0.04ms 0.00%
Large file parity evaluation 34.0±0.16ms 34.1±0.11ms +0.29%
Large input file compilation 12.0±0.35ms 11.7±0.23ms -2.50%
Large input file compilation (interpreter) 47.4±1.62ms 46.2±1.43ms -2.53%
Large nested iteration 33.7±0.55ms 33.2±0.29ms -1.48%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1566.7±83.04µs 1557.5±53.41µs -0.59%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.08ms 7.9±0.23ms +2.60%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1429.2±92.57µs 1426.5±56.71µs -0.19%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.5±0.59ms 27.8±0.26ms +1.09%
Teleport evaluation 89.1±3.44µs 87.4±3.56µs -1.91%

@swernli swernli enabled auto-merge May 13, 2024 18:49
Copy link

Benchmark for c88d3aa

Click to view benchmark
Test Base PR %
Array append evaluation 343.5±8.47µs 369.1±2.95µs +7.45%
Array literal evaluation 198.6±1.55µs 184.2±14.30µs -7.25%
Array update evaluation 424.6±2.88µs 421.4±1.74µs -0.75%
Core + Standard library compilation 18.2±0.86ms 17.9±0.98ms -1.65%
Deutsch-Jozsa evaluation 5.1±0.04ms 5.2±0.05ms +1.96%
Large file parity evaluation 34.1±0.16ms 34.0±0.18ms -0.29%
Large input file compilation 12.1±0.56ms 12.1±0.39ms 0.00%
Large input file compilation (interpreter) 49.0±2.02ms 49.1±1.61ms +0.20%
Large nested iteration 33.6±0.10ms 33.4±0.76ms -0.60%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1560.9±121.90µs 1571.1±107.26µs +0.65%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.17ms 7.8±0.14ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1438.0±88.85µs 1435.3±72.36µs -0.19%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.7±0.61ms 28.0±0.42ms +1.08%
Teleport evaluation 89.4±3.48µs 91.8±4.48µs +2.68%

Copy link

Benchmark for d09aded

Click to view benchmark
Test Base PR %
Array append evaluation 349.9±15.12µs 340.5±4.38µs -2.69%
Array literal evaluation 198.6±1.21µs 171.1±1.85µs -13.85%
Array update evaluation 429.1±8.79µs 422.8±4.50µs -1.47%
Core + Standard library compilation 17.2±0.70ms 18.9±0.72ms +9.88%
Deutsch-Jozsa evaluation 5.1±0.05ms 5.2±0.11ms +1.96%
Large file parity evaluation 33.9±0.13ms 33.7±0.43ms -0.59%
Large input file compilation 11.8±0.49ms 12.8±0.34ms +8.47%
Large input file compilation (interpreter) 47.7±2.79ms 50.1±1.92ms +5.03%
Large nested iteration 33.6±1.16ms 33.5±0.34ms -0.30%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1555.3±68.87µs 1567.0±129.01µs +0.75%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.08ms 7.9±0.15ms +2.60%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1444.5±116.13µs 1416.2±36.82µs -1.96%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.5±0.81ms 27.5±0.43ms 0.00%
Teleport evaluation 88.2±4.28µs 90.2±4.40µs +2.27%

@swernli swernli added this pull request to the merge queue May 13, 2024
Merged via the queue into main with commit ff57a3c May 13, 2024
16 checks passed
@swernli swernli deleted the swernli/partial-eval-unresolved-call-check branch May 13, 2024 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants