Modified export.py to add the ability to export fp16 weights.

[rdentato]

Added a --version 3 to the parameters that is like legacy_export but exports data in fp16 format.

Warning! needs test. I have too little memory (64GB are needed for a 7B parameters model and I couldn't find a smaller one)

[kroggen]

I am against creating a new version. It should export to existing versions, but choosing the output format (fp32, fp16, int8...)

Current code still expect the freq_cis to be there in the final file

I also think you mean 64 GB instead of MB. BTW this OOM problem is tracked on #341

[rdentato]

It will be the "legacy version" but with fp16 weights.
Because of how export.py works, you need to give a version number to it:

usage: export.py [-h] [--version VERSION] (--checkpoint CHECKPOINT | --meta-llama META_LLAMA | --hf HF) filepath

Changing the command line parameters of export.py could be the topic of another PR. For example having a "name" and a "version" for each model format that can be exported.

[rdentato]

Just thought of another way, but I'm not sure I like it: use the extension of the output file to determine the fp32/fp16 size.
For example:

python export.py --hf mrm8488/llama-2-coder-7b llama-2-coder-7b.f16

could signal --version 0 (the default) but with weights in fp16 format. The code would be a little bit uglier but if this could serve to avoid confusion ...

[kroggen]

Why your serialize_half is not named serialize_fp16 to be similar with the existing serialize_fp32?

Why your function is not together with the other 2 in the beginning of the file?

[kroggen]

My attempt is at #347

[rdentato]

About the half, is to stay consistent with the torch.half in the code. I liked it better (I ad fp16 initially).

for the position, I don't like to disrupt existing code (at least not for something so minimal) and just added at the end of the block. But in this case I agree I should have put it earlier, together with the other two, moved up.