Expand databases benchmarked

[russellpierce]

Athena and Redshift Spectrum are BigQuery-like. One wonders hope they would stack up in this comparison.

[georgewfraser]

I definitely want to add these. The tricky part of this is that you have to make a bunch of choices when you put the data in S3. Even if we just decide to go with Parquet, we have to decide how big to make the files and the blocks within the files. So it will take some fiddling around to be sure we're being fair to Athena/Spectrum.

[russellpierce]

I agree. Apples to apples is hard - if not impossible. Maybe I'm atypical, but as far as I'm concerned defaults or a (very) limited parameter search is reasonable. Unlike with Redshift's sort keys, where Amazon makes a big deal about optimizing for your query-type, Athena and Spectrum both seem to be advertised more or less as 'drop on S3 and go'. It seems fair to take them more or less at their word for the benchmark or a very limited search of the parameter space.

[mike-weinberg]

@russellpierce as I mentioned in #13 Athena/Spectrum have been shown elsewhere to be so sensitive to file types that getting performance to approach that of normal warehouses requires significant work, as well as that Athena and Spectrum are at best par with bigquery after all optimization is done (as you were hinting at when you said "bigquery-like")

For that reason I will be closing this issue (for now!)