mxnet

Rational Activation Functions for MxNet

This package contains an implementation of Rational Activation Functions for the machine learning framework MxNet.

Dependencies

Depending on your CUDA version, you should install mxnet-cu102 (CUDA 10.2) or mxnet-cu101 (CUDA 10.1). E.g.:

pip3 install -U pip wheel
pip3 install mxnet-cu102  rational-activations

Integrating Rational Activation Functions into Neural Networks

In MxNet, you can instantiate a Rational Activation Function by running

from rational.mxnet import Rational

my_fun = Rational()

This instantiates a HybridBlock, supporting both symbolic and imperative execution.

Customizing Rational

If you wish to customize your Rational instance, feel free to play around with its parameters.

from rational.mxnet import Rational

my_costum_fun = Rational(approx_func='tanh')

Integrating a Rational instance into a neural network

You can integrate a Rational instance into a neural network as follows.

import mxnet as mx
from rational.mxnet import Rational

my_fun = Rational()

# create small neural network and add my_fun as layer
net = mx.gluon.nn.HybridSequential()
with net.name_scope():
    net.add(my_fun)
net.initialize()

# for symbolic computation call 'hybridize'
net.hybridize()

Documentation

Please find more documentation on ReadTheDocs.