PyKaldi Pip whl packages

[bmilde]

Currently its not very easy to install pykaldi as outlined on the main page of this repo and the compilation is error prone depending on your compiler and Python version. It also takes many hours even on fast CPUs to compile everything even if you encounter no errors.

Just a FYI that we have created whl packages for Python 3.6 / 3.7 / 3.8 for pykaldi 0.1.2 and are hosting them at our university in this folder: https://ltdata1.informatik.uni-hamburg.de/pykaldi/

We have installation instructions in: https://ltdata1.informatik.uni-hamburg.de/pykaldi/README.txt

Kaldi needs to be compiled separately, but this can be done much quicker than compiling llvm/protobuf/clif etc. Also it can be linked against MKL, making it quite a bit faster if you have an Intel CPU, see the README.

Most users will find these much easier to work with, maybe you can link to our university folder from the main page? Or host these packages on PyPI?

[yaleimeng]

Thanks ! This is what I want.

[tjysdsg]

Still, it would be great if we can install pykaldi just using pip install pykaldi. Also, the anaconda package has been outdated for quite a while.

[bmilde]

Working on that and already registered the project with PyPI. But they have to increase the size limit for us, since the whl packages are quite large. We'll hopefully get "pip install pykaldi" working soon.

…

On Tue, Jul 27, 2021, 1:24 PM Jiyang Tang ***@***.***> wrote: Still, it would be great if we can install pykaldi just using pip install pykaldi. Also, the anaconda package has been outdated for quite a while. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#261 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACKGA6VEGQJY5BJQIGXIM4TTZ2JP5ANCNFSM4Y5Z2TQQ> .

[mauvilsa]

How big are your wheel packages? I have created a wheel package that includes all libfst*.so and libkaldi*.so such that to use in Ubuntu 20.04 the only requirement is to do apt-get install python3-pip intel-mkl && pip3 install pykaldi*.whl. The whl file is only 40M. I achieved this by stripping out debug symbols from the *.so files. Would this be of interest? Anyway if in pypi the whl files are without debugging symbols, the respective whl file including debugging symbols could be hosted elsewhere for when people need to debug.

You can find a built package in https://drive.google.com/file/d/1BcMVUdpUakgmutPEAWoAE39P6dfsHFif/view?usp=sharing. It has the linux_x86_64 tag so it can't be uploaded to pypi. But the stripping of symbols could be done for any package. If there is interest in this I could share how I did it.

[srdecny]

@mauvilsa Could you please share how you managed to include the libkaldi*.so files into the wheel?

[mauvilsa]

@srdecny find attached. Just run build_mkl_wheel.sh.
build-pykaldi-whl.zip

[srdecny]

@mauvilsa Thanks for the quick response! ❤️