about rpms for llvm

[atorik]

Hi bulkloaders,

I have some questions about llvm support for pg_bulkload.

[Question 1]
As you know, pg_bulkloader releases pg_bulkload-llvmjit-*.rpm, which contain bitcode of llvm.
I noticed that these rpms only had pg_bulkload.bc, and there were no  *.index.bc.

# rpm2cpio pg_bulkload-llvmjit-3.1.16-1.pg12.rhel7.x86_64.rpm | cpio -itv
drwxr-xr-x   2 root     root            0 Jan 17 13:44 ./usr/pgsql-12/lib/bitcode/pg_bulkload
-rw-r--r--   1 root     root        26228 Jan 17 13:44 ./usr/pgsql-12/lib/bitcode/pg_bulkload/pg_bulkload.bc

As far as I read the manual,  it seems pg_bulkload.index.bc is necessary. How do you think?

The relevant files have to be installed into $pkglibdir/bitcode/$extension/ and a summary of them into $pkglibdir/bitcode/$extension.index.bc,

[Question 2]
When I built pg_bulkload manually, I found many *.bc files.

12.2_llvm/lib/bitcode/pg_bulkload/
├── binary
├─ logger.bc
├── parser_binary
├── parser_csv
├── parser_function
├── parser_tuple
├── pg_btree
├── pg_bulkload
├── pg_strutil
├── p
│   ├── pgut-b
│   └── pgut-ip
├── reader
├── source.bc
├── writer
├── writer_binary
├── writer_buffered
├── writer_direct
└── writer_parallel

I'm a little bit wondering it is appropriate just containing pg_bulkload.bc in pg_bulkload-llvmjit-*.rpm.

[Question 3]
PGDG also releases rpms for pg_bulkload, however, they don't release rpms just for llvmjit, and the pg_bulkload-*.rpm contains both bitcode of llvm and other pg_bulkload materials.

I think it's a design choice, but I imagine separating llvm materials may be good for some use cases.

  1) When users don't need llvm support, they can do the minimum installation.
  2) When users don't want llvm for pg_bulkload, but need llvm for others generally, they can do this by not installing "pg_bulkload-llvm*.rpm".

Are there other reasons for the separation?

Thanks.

[MoonInsung]

Dear atorik
Thank you for reporting!

First, a quick answer to explain questions 1 and 2. currently released llvm.rpm is the wrong file.
As you said, the JIT module is all includes functions C file need a bc file and an index.bc(a summary of all bc file) file to make them available.
So far, I've misunderstood that I need BC files for only SQL-related functions.
I'll update the revised RPMs, maybe next week.

Question 3 is the same as what you think.
For users who do not use jit function (users who do not install postgresqlXX-llvmjit-XX.X.rpm), the BC files are not necessary, so they are separated into rpm files.
In the case of PGDG(yum.postgresql.org), bc is included in the main rpm, but our idea is to design without installing unnecessary files.

Thank you for the report, and I'll update it as soon as possible.

Best regards.
Moon.

[atorik]

Moon-san,

Thanks for your investigations!
I appreciate your corporation.