Borrowing Weights from a Pretrained Network

Weights

What are Weights

Weights are learnable parameters in the network, which are tuned by the back propagation phase.

Who Makes the Weights?

Weights are generated by caffe when your network in constructed.

Borrowing Weights from a Pre-Trained Network

To borrow the weights of an already trained model, we need to do two things:

• Rename our layer to match the name of the original model's layer.

For example, let say the original model had a layer name ip1, then we should name our layer ip1:

layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool2"
  top: "ip1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 500
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}

• Train our new hybrid model declaring the location of the weights:

caffe train —solver ourSolver.prototxt —weights theirModel.caffemodel