Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize the grouping behavior when finding by large path #347

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Optimize the grouping behavior when finding by large path #347

wants to merge 2 commits into from

Conversation

nbekirov
Copy link

@nbekirov nbekirov commented May 3, 2019

Hello!

While testing how closure_tree is handling deep nesting for a project I noticed that at some point it started to issue an unreasonably large number of SQL queries. Looking at the max_join_tables (currently 50) I assumed that for 55 levels of nesting there will be something like 2 queries and not 50!

I tracked down the issue to the use of in_groups which splits the path array in "(the given) number of groups" and replaced it with in_groups_of which would rather split "in groups of (the given) size".

Check out the updated spec example SQL output:

  • before (20 queries)
# max_join_tables = 20
# path = [{"name"=>"1"}, {"name"=>"2"}, {"name"=>"3"}, {"name"=>"4"}, {"name"=>"5"}, {"name"=>"6"}, {"name"=>"7"}, {"name"=>"8"}, {"name"=>"9"}, {"name"=>"10"}, {"name"=>"11"}, {"name"=>"12"}, {"name"=>"13"}, {"name"=>"14"}, {"name"=>"15"}, {"name"=>"16"}, {"name"=>"17"}, {"name"=>"18"}, {"name"=>"19"}, {"name"=>"20"}, {"name"=>"21"}, {"name"=>"22"}, {"name"=>"23"}, {"name"=>"24"}, {"name"=>"25"}, {"name"=>"26"}, {"name"=>"27"}, {"name"=>"28"}, {"name"=>"29"}, {"name"=>"30"}, {"name"=>"31"}, {"name"=>"32"}, {"name"=>"33"}, {"name"=>"34"}, {"name"=>"35"}, {"name"=>"36"}, {"name"=>"37"}, {"name"=>"38"}, {"name"=>"39"}, {"name"=>"40"}, {"name"=>"41"}, {"name"=>"42"}, {"name"=>"43"}, {"name"=>"44"}, {"name"=>"45"}, {"name"=>"46"}, {"name"=>"47"}, {"name"=>"48"}, {"name"=>"49"}, {"name"=>"50"}, {"name"=>"51"}, {"name"=>"52"}, {"name"=>"53"}, {"name"=>"54"}, {"name"=>"55"}, {"name"=>"56"}, {"name"=>"57"}, {"name"=>"58"}, {"name"=>"59"}, {"name"=>"60"}, {"name"=>"61"}, {"name"=>"62"}, {"name"=>"63"}, {"name"=>"64"}, {"name"=>"65"}, {"name"=>"66"}, {"name"=>"67"}, {"name"=>"68"}, {"name"=>"69"}, {"name"=>"70"}]
# subpath = [{"name"=>"1"}, {"name"=>"2"}, {"name"=>"3"}, {"name"=>"4"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" IS NULL ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"5"}, {"name"=>"6"}, {"name"=>"7"}, {"name"=>"8"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"9"}, {"name"=>"10"}, {"name"=>"11"}, {"name"=>"12"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"13"}, {"name"=>"14"}, {"name"=>"15"}, {"name"=>"16"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"17"}, {"name"=>"18"}, {"name"=>"19"}, {"name"=>"20"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"21"}, {"name"=>"22"}, {"name"=>"23"}, {"name"=>"24"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"25"}, {"name"=>"26"}, {"name"=>"27"}, {"name"=>"28"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"29"}, {"name"=>"30"}, {"name"=>"31"}, {"name"=>"32"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"33"}, {"name"=>"34"}, {"name"=>"35"}, {"name"=>"36"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"37"}, {"name"=>"38"}, {"name"=>"39"}, {"name"=>"40"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p2"."parent_id" = $5 ORDER BY "tags"."id" ASC LIMIT $6

# subpath = [{"name"=>"41"}, {"name"=>"42"}, {"name"=>"43"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"44"}, {"name"=>"45"}, {"name"=>"46"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"47"}, {"name"=>"48"}, {"name"=>"49"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"50"}, {"name"=>"51"}, {"name"=>"52"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"53"}, {"name"=>"54"}, {"name"=>"55"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"56"}, {"name"=>"57"}, {"name"=>"58"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"59"}, {"name"=>"60"}, {"name"=>"61"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"62"}, {"name"=>"63"}, {"name"=>"64"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"65"}, {"name"=>"66"}, {"name"=>"67"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5

# subpath = [{"name"=>"68"}, {"name"=>"69"}, {"name"=>"70"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p1"."parent_id" = $4 ORDER BY "tags"."id" ASC LIMIT $5
  • and after the change (4 queries)
# max_join_tables = 20
# path = [{"name"=>"1"}, {"name"=>"2"}, {"name"=>"3"}, {"name"=>"4"}, {"name"=>"5"}, {"name"=>"6"}, {"name"=>"7"}, {"name"=>"8"}, {"name"=>"9"}, {"name"=>"10"}, {"name"=>"11"}, {"name"=>"12"}, {"name"=>"13"}, {"name"=>"14"}, {"name"=>"15"}, {"name"=>"16"}, {"name"=>"17"}, {"name"=>"18"}, {"name"=>"19"}, {"name"=>"20"}, {"name"=>"21"}, {"name"=>"22"}, {"name"=>"23"}, {"name"=>"24"}, {"name"=>"25"}, {"name"=>"26"}, {"name"=>"27"}, {"name"=>"28"}, {"name"=>"29"}, {"name"=>"30"}, {"name"=>"31"}, {"name"=>"32"}, {"name"=>"33"}, {"name"=>"34"}, {"name"=>"35"}, {"name"=>"36"}, {"name"=>"37"}, {"name"=>"38"}, {"name"=>"39"}, {"name"=>"40"}, {"name"=>"41"}, {"name"=>"42"}, {"name"=>"43"}, {"name"=>"44"}, {"name"=>"45"}, {"name"=>"46"}, {"name"=>"47"}, {"name"=>"48"}, {"name"=>"49"}, {"name"=>"50"}, {"name"=>"51"}, {"name"=>"52"}, {"name"=>"53"}, {"name"=>"54"}, {"name"=>"55"}, {"name"=>"56"}, {"name"=>"57"}, {"name"=>"58"}, {"name"=>"59"}, {"name"=>"60"}, {"name"=>"61"}, {"name"=>"62"}, {"name"=>"63"}, {"name"=>"64"}, {"name"=>"65"}, {"name"=>"66"}, {"name"=>"67"}, {"name"=>"68"}, {"name"=>"69"}, {"name"=>"70"}]
# subpath = [{"name"=>"1"}, {"name"=>"2"}, {"name"=>"3"}, {"name"=>"4"}, {"name"=>"5"}, {"name"=>"6"}, {"name"=>"7"}, {"name"=>"8"}, {"name"=>"9"}, {"name"=>"10"}, {"name"=>"11"}, {"name"=>"12"}, {"name"=>"13"}, {"name"=>"14"}, {"name"=>"15"}, {"name"=>"16"}, {"name"=>"17"}, {"name"=>"18"}, {"name"=>"19"}, {"name"=>"20"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" INNER JOIN "tags" AS p3
             ON p3."id" =
"p2"."parent_id" INNER JOIN "tags" AS p4
             ON p4."id" =
"p3"."parent_id" INNER JOIN "tags" AS p5
             ON p5."id" =
"p4"."parent_id" INNER JOIN "tags" AS p6
             ON p6."id" =
"p5"."parent_id" INNER JOIN "tags" AS p7
             ON p7."id" =
"p6"."parent_id" INNER JOIN "tags" AS p8
             ON p8."id" =
"p7"."parent_id" INNER JOIN "tags" AS p9
             ON p9."id" =
"p8"."parent_id" INNER JOIN "tags" AS p10
             ON p10."id" =
"p9"."parent_id" INNER JOIN "tags" AS p11
             ON p11."id" =
"p10"."parent_id" INNER JOIN "tags" AS p12
             ON p12."id" =
"p11"."parent_id" INNER JOIN "tags" AS p13
             ON p13."id" =
"p12"."parent_id" INNER JOIN "tags" AS p14
             ON p14."id" =
"p13"."parent_id" INNER JOIN "tags" AS p15
             ON p15."id" =
"p14"."parent_id" INNER JOIN "tags" AS p16
             ON p16."id" =
"p15"."parent_id" INNER JOIN "tags" AS p17
             ON p17."id" =
"p16"."parent_id" INNER JOIN "tags" AS p18
             ON p18."id" =
"p17"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p3"."name" = $5 AND "p4"."name" = $6 AND "p5"."name" = $7 AND "p6"."name" = $8 AND "p7"."name" = $9 AND "p8"."name" = $10 AND "p9"."name" = $11 AND "p10"."name" = $12 AND "p11"."name" = $13 AND "p12"."name" = $14 AND "p13"."name" = $15 AND "p14"."name" = $16 AND "p15"."name" = $17 AND "p16"."name" = $18 AND "p17"."name" = $19 AND "p18"."name" = $20 AND "p18"."parent_id" IS NULL ORDER BY "tags"."id" ASC LIMIT $21

# subpath = [{"name"=>"21"}, {"name"=>"22"}, {"name"=>"23"}, {"name"=>"24"}, {"name"=>"25"}, {"name"=>"26"}, {"name"=>"27"}, {"name"=>"28"}, {"name"=>"29"}, {"name"=>"30"}, {"name"=>"31"}, {"name"=>"32"}, {"name"=>"33"}, {"name"=>"34"}, {"name"=>"35"}, {"name"=>"36"}, {"name"=>"37"}, {"name"=>"38"}, {"name"=>"39"}, {"name"=>"40"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" INNER JOIN "tags" AS p3
             ON p3."id" =
"p2"."parent_id" INNER JOIN "tags" AS p4
             ON p4."id" =
"p3"."parent_id" INNER JOIN "tags" AS p5
             ON p5."id" =
"p4"."parent_id" INNER JOIN "tags" AS p6
             ON p6."id" =
"p5"."parent_id" INNER JOIN "tags" AS p7
             ON p7."id" =
"p6"."parent_id" INNER JOIN "tags" AS p8
             ON p8."id" =
"p7"."parent_id" INNER JOIN "tags" AS p9
             ON p9."id" =
"p8"."parent_id" INNER JOIN "tags" AS p10
             ON p10."id" =
"p9"."parent_id" INNER JOIN "tags" AS p11
             ON p11."id" =
"p10"."parent_id" INNER JOIN "tags" AS p12
             ON p12."id" =
"p11"."parent_id" INNER JOIN "tags" AS p13
             ON p13."id" =
"p12"."parent_id" INNER JOIN "tags" AS p14
             ON p14."id" =
"p13"."parent_id" INNER JOIN "tags" AS p15
             ON p15."id" =
"p14"."parent_id" INNER JOIN "tags" AS p16
             ON p16."id" =
"p15"."parent_id" INNER JOIN "tags" AS p17
             ON p17."id" =
"p16"."parent_id" INNER JOIN "tags" AS p18
             ON p18."id" =
"p17"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p3"."name" = $5 AND "p4"."name" = $6 AND "p5"."name" = $7 AND "p6"."name" = $8 AND "p7"."name" = $9 AND "p8"."name" = $10 AND "p9"."name" = $11 AND "p10"."name" = $12 AND "p11"."name" = $13 AND "p12"."name" = $14 AND "p13"."name" = $15 AND "p14"."name" = $16 AND "p15"."name" = $17 AND "p16"."name" = $18 AND "p17"."name" = $19 AND "p18"."name" = $20 AND "p18"."parent_id" = $21 ORDER BY "tags"."id" ASC LIMIT $22

# subpath = [{"name"=>"41"}, {"name"=>"42"}, {"name"=>"43"}, {"name"=>"44"}, {"name"=>"45"}, {"name"=>"46"}, {"name"=>"47"}, {"name"=>"48"}, {"name"=>"49"}, {"name"=>"50"}, {"name"=>"51"}, {"name"=>"52"}, {"name"=>"53"}, {"name"=>"54"}, {"name"=>"55"}, {"name"=>"56"}, {"name"=>"57"}, {"name"=>"58"}, {"name"=>"59"}, {"name"=>"60"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" INNER JOIN "tags" AS p3
             ON p3."id" =
"p2"."parent_id" INNER JOIN "tags" AS p4
             ON p4."id" =
"p3"."parent_id" INNER JOIN "tags" AS p5
             ON p5."id" =
"p4"."parent_id" INNER JOIN "tags" AS p6
             ON p6."id" =
"p5"."parent_id" INNER JOIN "tags" AS p7
             ON p7."id" =
"p6"."parent_id" INNER JOIN "tags" AS p8
             ON p8."id" =
"p7"."parent_id" INNER JOIN "tags" AS p9
             ON p9."id" =
"p8"."parent_id" INNER JOIN "tags" AS p10
             ON p10."id" =
"p9"."parent_id" INNER JOIN "tags" AS p11
             ON p11."id" =
"p10"."parent_id" INNER JOIN "tags" AS p12
             ON p12."id" =
"p11"."parent_id" INNER JOIN "tags" AS p13
             ON p13."id" =
"p12"."parent_id" INNER JOIN "tags" AS p14
             ON p14."id" =
"p13"."parent_id" INNER JOIN "tags" AS p15
             ON p15."id" =
"p14"."parent_id" INNER JOIN "tags" AS p16
             ON p16."id" =
"p15"."parent_id" INNER JOIN "tags" AS p17
             ON p17."id" =
"p16"."parent_id" INNER JOIN "tags" AS p18
             ON p18."id" =
"p17"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p3"."name" = $5 AND "p4"."name" = $6 AND "p5"."name" = $7 AND "p6"."name" = $8 AND "p7"."name" = $9 AND "p8"."name" = $10 AND "p9"."name" = $11 AND "p10"."name" = $12 AND "p11"."name" = $13 AND "p12"."name" = $14 AND "p13"."name" = $15 AND "p14"."name" = $16 AND "p15"."name" = $17 AND "p16"."name" = $18 AND "p17"."name" = $19 AND "p18"."name" = $20 AND "p18"."parent_id" = $21 ORDER BY "tags"."id" ASC LIMIT $22

# subpath = [{"name"=>"61"}, {"name"=>"62"}, {"name"=>"63"}, {"name"=>"64"}, {"name"=>"65"}, {"name"=>"66"}, {"name"=>"67"}, {"name"=>"68"}, {"name"=>"69"}, {"name"=>"70"}]
SELECT  "tags".* FROM "tags" INNER JOIN "tags" AS p0
             ON p0."id" =
"tags"."parent_id" INNER JOIN "tags" AS p1
             ON p1."id" =
"p0"."parent_id" INNER JOIN "tags" AS p2
             ON p2."id" =
"p1"."parent_id" INNER JOIN "tags" AS p3
             ON p3."id" =
"p2"."parent_id" INNER JOIN "tags" AS p4
             ON p4."id" =
"p3"."parent_id" INNER JOIN "tags" AS p5
             ON p5."id" =
"p4"."parent_id" INNER JOIN "tags" AS p6
             ON p6."id" =
"p5"."parent_id" INNER JOIN "tags" AS p7
             ON p7."id" =
"p6"."parent_id" INNER JOIN "tags" AS p8
             ON p8."id" =
"p7"."parent_id" WHERE "tags"."name" = $1 AND "p0"."name" = $2 AND "p1"."name" = $3 AND "p2"."name" = $4 AND "p3"."name" = $5 AND "p4"."name" = $6 AND "p5"."name" = $7 AND "p6"."name" = $8 AND "p7"."name" = $9 AND "p8"."name" = $10 AND "p8"."parent_id" = $11 ORDER BY "tags"."id" ASC LIMIT $12

@seuros seuros self-requested a review May 3, 2019 15:32
@seuros
Copy link
Member

seuros commented May 3, 2019

That some pretty good optimization. Did you check the cost of the both queries ?

@nbekirov
Copy link
Author

nbekirov commented May 3, 2019

@seuros, no actually.

For me, it was enough that path size of 49 was not calling find_by_large_path thus keeping it all to a single query (with ~50 inner-joins) and then going up by just 2 to 51 levels of nesting suddenly started spamming the DB in an unexpected manner.

I may check the performance of the big SQL under Postgres for example if we want to reduce the max_join_tables. But again, IMO those are 2 different optimizations.

@seuros
Copy link
Member

seuros commented May 3, 2019

Since you have the data already populated, can you append EXPLAIN to the query in psql. You will get the query cost there.
Else i will try to do it this weekend to release a minor version on rubygems.

@nbekirov
Copy link
Author

nbekirov commented May 3, 2019 via email •
edited

@nbekirov
Copy link
Author

nbekirov commented May 3, 2019 via email •
edited

@nbekirov
Copy link
Author

nbekirov commented May 7, 2019

@seuros, here's the promised data:

@seuros
Copy link
Member

seuros commented May 7, 2019

Can you add a Changelog entry to this PR ? Add version 7.1.0

@nbekirov
Copy link
Author

nbekirov commented May 7, 2019

Done :)

@seuros seuros self-assigned this May 14, 2019
@nbekirov
Copy link
Author

Hello!

Can I help with anything more on this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seuros seuros Awaiting requested review from seuros

Assignees

@seuros seuros

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@nbekirov @seuros @NBekirovAcronis