Skip to content
forked from microsoft/LQ-Nets

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

License

Notifications You must be signed in to change notification settings

lokender/LQ-Nets

 
 

Repository files navigation

LQ-Nets

By Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua.

Microsoft Research Asia (MSRA).

Introduction

This repository contains the training code of LQ-Nets introduced in our ECCV 2018 paper:

D. Zhang*, J. Yang*, D. Ye* and G. Hua. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks. ECCV 2018 (*: Equal contribution) PDF

Dependencies

  • Python 2.7 or 3.3+
  • Python bindings for OpenCV
  • TensorFlow >= 1.3.0
  • TensorPack

Usage

Download the ImageNet dataset and decompress into the structure like

dir/
  train/
    n01440764/
      n01440764_10026.JPEG
      ...
    ...
  val/
    ILSVRC2012_val_00000001.JPEG
    ...

To train a quantized "pre-activation" ResNet-18, simply run

python imagenet.py --gpu 0,1,2,3 --data /PATH/TO/IMAGENET --mode preact --depth 18 --qw 1 --qa 2 --logdir_id w1a2 

After the training, the result model will be stored in ./train_log/w1a2.

For more options, please refer to python imagenet.py -h.

Results

ImageNet Experiments

Quantizing both weight and activation

Model Bit-width(W/A) Top-1(%) Top-5(%)
ResNet-18 1/2 62.6 84.3
ResNet-18 2/2 64.9 85.9
ResNet-18 3/3 68.2 87.9
ResNet-18 4/4 69.3 88.8
ResNet-34 1/2 66.6 86.9
ResNet-34 2/2 69.8 89.1
ResNet-34 3/3 71.9 90.2
ResNet-50 1/2 68.7 88.4
ResNet-50 2/2 71.5 90.3
ResNet-50 3/3 74.2 91.6
ResNet-50 4/4 75.1 92.4
AlexNet 1/2 55.7 78.8
AlexNet 2/2 57.4 80.1
DenseNet-121 2/2 69.6 89.1
VGG-Variant 1/2 67.1 87.6
VGG-Variant 2/2 68.8 88.6
GoogLeNet-Variant 1/2 65.6 86.4
GoogLeNet-Variant 2/2 68.2 88.1

Quantizing weight only

Model Bit-width(W/A) Top-1(%) Top-5(%)
ResNet-18 2/32 68.0 88.0
ResNet-18 3/32 69.3 88.8
ResNet-18 4/32 70.0 89.1
ResNet-50 2/32 75.1 92.3
ResNet-50 4/32 76.4 93.1
AlexNet 2/32 60.5 82.7

More results can be found in the paper.

Citation

If you use our code or models in your research, please cite our paper with

@inproceedings{ZhangYangYeECCV2018,
    author = {Zhang, Dongqing and Yang, Jiaolong and Ye, Dongqiangzi and Hua, Gang},
    title = {LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks},
    booktitle = {European Conference on Computer Vision (ECCV)},
    year = {2018}
}

About

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%