[compiler] Support large models

[seanshpark]

Issue to gather issues, task for large models, such as

• flatbuffer size > 2G
• quantize < 8 bit
• ...

---

• support 4bit weights: [luci] Support INT4/UINT4 weights #12717
• quantize with fake-quantize: Quantize fake-quantized w4a16 model from onnx #12738
• end-to-end validation for int4: [compiler] end-to-end validation with INT4/UINT4 model #12797
• onnx with INT16: [compiler] Upgrade ONNXRuntime #12816

---

things to do with our LLM model

• convert model to circle -> done with internal onnx2circle
• optimize circle model -> conversion seems ok
• ...

[hseok-oh]

We may need to support external data file on circle spec (like onnx spec) to save large weight.

[seanshpark]

We may need to support external data file

circle schema from 0.7 has this in changes from tflite

(for table Operator)
  // When an op is using custom_options in a model that is larger than 2GB, then
  // we instead use the following attributes to find the buffer location which
  // is stored outside of flatbuffers, the offset is calculated relative to the
  // beginning of the file and is only valid if > 1
  large_custom_options_offset: ulong;
  large_custom_options_size: ulong;

(for table Buffer)
  // In a model that is larger than 2GB, then buffers instead uses the following
  // attributes to find stored data, which is outside of flatbuffers
  // the offset is calculated relative to the beginning of the file and is only
  // valid if > 1.
  offset: ulong;
  size: ulong;

I need to look inside tflite implementation but,
it seems large data are added at the end of the file, flat buffer managed part,
as single file.