Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Activate abortUnless in svcomp conf #1464

Merged
merged 1 commit into from
May 27, 2024
Merged

Activate abortUnless in svcomp conf #1464

merged 1 commit into from
May 27, 2024

Conversation

sim642
Copy link
Member

@sim642 sim642 commented May 14, 2024

This is on top of #1462 to fix #1453, but the following benchmarking was also done with #1450 included.

In #875 (comment), when the abortUnless analysis was added, it didn't seem to pay off. Now with Apron and autotuning, it seems that it might now.

sv-benchmarks no-overflow

With 60s timeout, we gain 92 new correct trues. There are also 3 new TIMEOUTs to be investigated.

CPU time linear scale

Always activating abortUnless has ~1% overhead, which is negligible.
image

CPU time log scale

Visible differences are only at the low end and the slowdown cases are precisely those nla-digbench tasks where we become more precise because of this, which is expected.
image

TODO

New TIMEOUTs to investigate:

@sim642 sim642 added feature sv-comp precision pr-dependency Depends or builds on another PR, which should be merged before labels May 14, 2024
@sim642 sim642 added this to the SV-COMP 2025 milestone May 14, 2024
@sim642
Copy link
Member Author

sim642 commented May 17, 2024

The timeouts are related to __VERIFIER_assert also being identified as an abortUnless function. Apron analysis refining according to that somehow breaks things.

@sim642
Copy link
Member Author

sim642 commented May 21, 2024

The recursified_knuth timeout is acceptable. It is caused by new refinement from assume_abort_if_not(a > 0); which somehow causes an explosion of contexts. This happens even with just def_exc, no intervals or anything of infinite height is needed. Tracing shows contexts increasing like Not {160}, ..., Not {240}, etc. So one exclusion in the context leads to a different exclusion in the recursive call.

This is a more fundamental problem that's not caused by abortUnless. Context widening fixes the timeout, but our svcomp conf doesn't use it (probably for more precision on recursive tasks that finish in time).

@sim642 sim642 marked this pull request as ready for review May 21, 2024 13:33
@michael-schwarz
Copy link
Member

Context widening fixes the timeout, but our svcomp conf doesn't use it (probably for more precision on recursive tasks that finish in time).

This might the sort of thing where the context gas ⛽ may pay off.

@michael-schwarz michael-schwarz self-requested a review May 21, 2024 15:49
Copy link
Member

@michael-schwarz michael-schwarz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see that this old thing now pays off!

@sim642 sim642 removed their assignment May 23, 2024
@michael-schwarz
Copy link
Member

What's the holdup here? If no one else wants to review it, I'd suggest we merge it.

@sim642
Copy link
Member Author

sim642 commented May 27, 2024

The holdup is that this depends on #1462 to prevent everything from just crashing. And that in itself depends on #1470 being in place first to avoid other regressions.

Base automatically changed from issue-1453 to master May 27, 2024 13:16
@sim642 sim642 removed the pr-dependency Depends or builds on another PR, which should be merged before label May 27, 2024
@sim642 sim642 merged commit 487412c into master May 27, 2024
21 checks passed
@sim642 sim642 deleted the svcomp-abortUnless branch May 27, 2024 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Witness generation crashes in combine_env with abortUnless analysis
2 participants