nncase is a cross-platform neural network optimization toolkit for fast inference.
Download prebuilt binaries from Release.
ncc -i <input format> -o <output format> [--dataset <dataset path>] [--postprocess <dataset postprocess>] [--weights-bits <weights quantization bits>] <input path> <output path>
-iInput format
| value | description |
|---|---|
| tflite | .tflite TFLite model |
| paddle | __model__ PaddlePaddle model |
| caffe | .caffemodel Caffe model |
| k210model | .kmodel K210 model (Only supported in inference mode) |
-oOutput format
| value | description |
|---|---|
| k210model | .kmodel K210 model |
| tf | .pb TensorFlow model |
| tflite | .tflite TFLite model |
| inference | .bin Model's raw output (Only support k210model input) |
--inference-typeInference type
| value | description |
|---|---|
| uint8 | Use quantized kernels (default) |
| float | Use float kernels |
-
--datasetDataset path, required when the output format isinferenceork210modelwith inference type equalsuint8. -
--postprocessDataset postprocess method
| value | description |
|---|---|
| 0to1 | normalize images to [0, 1] |
| n1to1 | normalize images to [-1, 1] |
--weights-bitsWeights quantization bits
| value | description |
|---|---|
| 8 | 8bit quantization [0, 255] |
| 16 | 16bit quantization [0, 65535] |
-
--float-fcUse float fullyconnected kernels. -
--channelwise-outputUse channelwise quantization for output layers.
-
Convert TFLite model to K210 model.
ncc -i tflite -o k210model --dataset ./images ./mbnetv1.tflite ./mbnetv1.kmodel -
Convert PaddlePaddle model to TensorFlow model.
ncc -i paddle -o tf ./MobileNetV1_pretrained ./mbnetv1.pb -
Inference K210 model and get output binaries.
ncc -i k210model -o inference --dataset ./images ./mbnetv1.kmodel ./output -
Tutorials
- 20 classes object detection
- Iris flowers classification
| layer | parameters |
|---|---|
| Conv2d | kernel={3x3,1x1} stride={1,2} padding=same * |
| DepthwiseConv2d | kernel={3x3,1x1} stride={1,2} padding=same * |
| FullyConnected | |
| Add | |
| MaxPool2d | |
| AveragePool2d | |
| GlobalAveragePool2d | |
| BatchNormalization | |
| BiasAdd | |
| Relu | |
| Relu6 | |
| LeakyRelu | |
| Concatenation | |
| L2Normalization | |
| Sigmoid | |
| Softmax | |
| Flatten | |
| ResizeNearestNeighbor |
* When using TensorFlow Conv2d/DepthwiseConv2d kernel=3x3 stride=2 padding=same, you must first use tf.pad([[0,0],[1,1],[1,1],[0,0]]) to pad the input and then use Conv2d/DepthwiseConv2d with valid padding.