Unify optimize calls between top-level select and sub-select

[mfussenegger]

Side effect of the change is that this now enables fetch optimization
for sub-queries.

Given the impact of this, I'd hold this off for 5.8

[matriv]

Shouldn't we have a changes entry for this?
(Maybe some benchmarks to see if it actually improves performance)

[mfussenegger]

Good call. Looks like this makes it worse in some cases:

Three node cluster

Q: select * from articles where id in (select id from colors where coolness > 0) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |       88.784 ±   24.866 |     76.193 |     84.462 |     89.902 |    327.791 |
|   V2    |       92.580 ±   27.771 |     80.273 |     88.493 |     91.625 |    335.170 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               +   4.19%                           +   4.66%
There is a 84.93% probability that the observed difference is not random, and the best estimate of that difference is 4.19%
The test has no statistical significance

Q: select * from articles where id in (select id from colors where coolness > 0 limit 100) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |        4.599 ±    2.740 |      3.106 |      4.121 |      4.438 |     29.793 |
|   V2    |        5.395 ±    1.312 |      4.142 |      5.122 |      5.517 |     14.346 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               +  15.93%                           +  21.65%
There is a 99.98% probability that the observed difference is not random, and the best estimate of that difference is 15.93%
The test has statistical significance

Q: select * from articles where id in (select id from colors where coolness > 0 limit 10) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |        3.274 ±    1.143 |      2.350 |      3.161 |      3.458 |     16.142 |
|   V2    |        2.901 ±    1.261 |      2.202 |      2.683 |      2.993 |     17.781 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               -  12.08%                           -  16.36%
There is a 99.79% probability that the observed difference is not random, and the best estimate of that difference is 12.08%
The test has statistical significance

Q: select * from articles where id not in (select id from colors where coolness > 0) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |      414.548 ±    9.325 |    401.679 |    412.644 |    417.640 |    462.819 |
|   V2    |      424.616 ±   11.068 |    408.753 |    423.037 |    427.560 |    478.425 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               +   2.40%                           +   2.49%
There is a 100.00% probability that the observed difference is not random, and the best estimate of that difference is 2.40%
The test has statistical significance

Single node cluster

# Results (server side duration in ms)
V1: 5.8.0-e0a9b0b6782cc893871e9cdf1a19d49729e85a1f
V2: 5.8.0-f388324fba1b2ea85de6a3a52d3a17e9476189d5

Q: select * from articles where id in (select id from colors where coolness > 0) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |       85.310 ±   22.863 |     79.978 |     82.517 |     84.617 |    399.319 |
|   V2    |       89.104 ±   23.844 |     83.179 |     85.835 |     87.945 |    410.937 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               +   4.35%                           +   3.94%
There is a 89.49% probability that the observed difference is not random, and the best estimate of that difference is 4.35%
The test has no statistical significance

Q: select * from articles where id in (select id from colors where coolness > 0 limit 100) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |        1.902 ±    1.807 |      1.224 |      1.567 |      1.959 |     22.871 |
|   V2    |        2.988 ±    0.906 |      2.306 |      2.761 |      3.176 |     13.415 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               +  44.41%                           +  55.17%
There is a 100.00% probability that the observed difference is not random, and the best estimate of that difference is 44.41%
The test has statistical significance

Q: select * from articles where id in (select id from colors where coolness > 0 limit 10) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |        1.062 ±    0.147 |      0.854 |      1.045 |      1.106 |      2.450 |
|   V2    |        1.185 ±    0.167 |      0.998 |      1.157 |      1.230 |      2.832 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               +  10.93%                           +  10.19%
There is a 100.00% probability that the observed difference is not random, and the best estimate of that difference is 10.93%
The test has statistical significance

Q: select * from articles where id not in (select id from colors where coolness > 0) limit 10000
C: 1
| Version |         Mean ±    Stdev |        Min |     Median |         Q3 |        Max |
|   V1    |      429.877 ±    9.959 |    421.066 |    428.179 |    431.825 |    546.776 |
|   V2    |      416.736 ±   10.045 |    408.934 |    414.852 |    418.107 |    525.040 |
├---------┴-------------------------┴------------┴------------┴------------┴------------┘
|               -   3.10%                           -   3.16%
There is a 100.00% probability that the observed difference is not random, and the best estimate of that difference is 3.10%
The test has statistical significance

Relates a bit to:

• Optimization of queries with "LIMIT" clause (fetch vs. no-fetch, source-lookup) #11285 and we could also consider having a cost model for fetch/non-fetch (Introduce Cost Model for Query Optimization #14146)

I'll put this on hold for now. After #15548 where we may eliminate some of the differences we can decide on how to proceed here.