Recreating examples with GPU

[iFimbrethil]

Hey team-- working through the colab example here, but trying to recreate on my own. Can't get it to recognize the GPUs here:
res = ezkl.gen_settings(model_path, settings_path, py_run_args=py_run_args)

so this line hangs.
I see you guys use halo2-- how exactly is that integrated on the backend? Looking to reproduce this locally :-)

[jasonmorton]

Could you say more about the environment you are running this in?

[alexander-camuto]

1. gen-settings wouldn't use GPUs anyways -- can you give us some more details on the model ? that command doesn't generated proofs.

2. further it sounds like you're just installing non gpu ezkl. are you certain you've installed ezkl-gpu with pip and also setting the env variable ENABLE_ICICLE_GPU=true ? See the readme for details.

[iFimbrethil]

trying with a small transformers model: https://huggingface.co/Writer/palmyra-small
just as an example.

installed ezkl-gpu yes, but missed setting that env variable. good callout!

[alexander-camuto]

ah its a pretty large model (128Million) so generating settings may take a while tbh. Its a one off cost though

[iFimbrethil]

interesting— is generating settings something that can be loaded on gpu, or is there nothing parallel there? just curious

[alexander-camuto]

def parallelizable -- but we focused on getting proving on GPUs first -- as its the biggest bottleneck