Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address Query Parsing Performance #4395

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Address Query Parsing Performance #4395

wants to merge 4 commits into from

Conversation

jogibear9988
Copy link
Member

@jogibear9988 jogibear9988 commented Jan 26, 2024

Fixes #4394

@MaceWindu MaceWindu added this to the In-progress milestone Jan 27, 2024
@viceroypenguin viceroypenguin changed the title Issue/4394 Address Query Parsing Performance Jan 27, 2024
@MaceWindu MaceWindu marked this pull request as draft January 28, 2024 17:34
@MaceWindu
Copy link
Contributor

/azp run test-metrics

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@MaceWindu
Copy link
Contributor

MaceWindu commented Jan 28, 2024

that's in debug with overridetostrng
image

@MaceWindu
Copy link
Contributor

and in release
image

@jogibear9988
Copy link
Member Author

jogibear9988 commented Jan 28, 2024

As Info: I merged #4339 in this pull request, cause without the query would not run at all.

@MaceWindu
Copy link
Contributor

MaceWindu commented Jan 28, 2024

I doubt it makes sense to do anything here before new parser lends. Profiler shows about half of time spent in this method https://github.com/linq2db/linq2db/blob/master/Source/LinqToDB/SqlQuery/SelectQueryOptimizer.cs#L453-L482

@sdanyliv ?

@MaceWindu
Copy link
Contributor

image

@MaceWindu
Copy link
Contributor

new parser shows similar numbers currently, but as I said - it's better leave it for later as slow areas will be heavy refactored there

@linq2dbot
Copy link

Test baselines changed by this PR. Don't forget to merge/close baselines PR after this pr merged/closed.

@igor-tkachev
Copy link
Member

I like this test. Profiler shows a lot of points to optimize.

@sdanyliv
Copy link
Member

sdanyliv commented Feb 2, 2024

Profiler adds additional delays. Without profiler in Release, whole query is executed under 550-600 milliseconds. Second run with cache hit is 8 msec. There are a lot of ways how to improve SelectQuery optimization and I will make research for new LINQ Translator but after all tests are passed. Not too much left.

Note that SelectQuery optimization is fully REWRITTEN, making cardinal changes in current version will be discarded .

@MaceWindu MaceWindu modified the milestones: In-progress, 6.0.0 Feb 3, 2024
@jogibear9988
Copy link
Member Author

jogibear9988 commented Feb 5, 2024

@sdanyliv

Your coments were already resolved in pull req for issue #4337, wich I had merged into this, without this the query didn't run at all

@jogibear9988
Copy link
Member Author

The Test results in an endless call stack of this 4 methods, till it crashes...

image

@@ -9,6 +9,8 @@

using NUnit.Framework;

using static Tests.UserTests.Issue4394Tests;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove that

@sdanyliv
Copy link
Member

sdanyliv commented Feb 5, 2024

Tested on V6
In Release second query executes approx 8 seconds without Stack Overflow. In Debug it crashes with SO, Probably too much debug information

Good test for optimizing translator, bad for production.

Your goal to kill linq2db or make job done?

@jogibear9988
Copy link
Member Author

Tested on V6 In Release second query executes approx 8 seconds without Stack Overflow. In Debug it crashes with SO, Probably too much debug information

Good test for optimizing translator, bad for production.

Your goal to kill linq2db or make job done?

I tried in Release mode....

I did not write this query, also in out software it was only for 3Shelfs, now someone needed it for 10...
the Performance is not really very critical, so if it works, it does not matter.

But for now, we created a view wich we map to the same object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Complex Query Takes Forever - but I think it's query generation
5 participants