Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some experements with sparse minmax indexes #6667

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft

Commits on Feb 16, 2024

  1. Use compressed chunk tupdesc to check which metadata columns we have

    The compressed qual pushdown will look at presence of metadata columns
    in the compressed chunk, not at whether the uncompressed column is an
    orderby. This way, the decision on which columns have metadata is made
    only during the creation of compressed chunk.
    
    This doesn't introduce any behavior changes, just changes the decision
    about which columns have metadata to happen in a single place. This will
    simplify experimenting with configurable metadata columns.
    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    d8b8753 View commit details
    Browse the repository at this point in the history
  2. do the same in row compressor

    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    9a974bc View commit details
    Browse the repository at this point in the history
  3. Optimize the order of compressed chunk metadata columns

    Put global metadata columns first, segmentby next, orderby metadata
    before the compressed orderby columns. This matches the order in which
    they are likely to be accessed when reading from a compressed chunk.
    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    a38cc0d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    728fc1a View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    55b71b3 View commit details
    Browse the repository at this point in the history
  6. increase rows per batch to 4k

    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    3376f46 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    547b33b View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    5355cd3 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    958e7dc View commit details
    Browse the repository at this point in the history
  10. ref

    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    f678e1f View commit details
    Browse the repository at this point in the history
  11. Disable bulk decompression for small limits

    It is about 10x faster than row-by-row decompression, so if we're going
    to decompress 10x less rows than the batch size, it's more efficient to
    decompress row-by-row.
    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    9c25407 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    425fdc1 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    aa541a3 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    633eeb7 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    3bc6041 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    bc1dcec View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    36def7c View commit details
    Browse the repository at this point in the history
  18. Merge remote-tracking branches 'akuzm/origin/bulk-limit', 'akuzm/orde…

    …r' and 'akuzm/rows' into HEAD
    akuzm committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    724c25c View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    22e5a6a View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    6f1c0a7 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    e504682 View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    a592f02 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. Configuration menu
    Copy the full SHA
    4a872ab View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    5ee5786 View commit details
    Browse the repository at this point in the history