fix bugs and publish 0.1.0

tnbar · Apr 5, 2021 · 4f218f4 · 4f218f4
1 parent e056712
commit 4f218f4
Show file tree

Hide file tree

Showing 10 changed files with 311 additions and 29 deletions.
diff --git a/README.md b/README.md
@@ -14,3 +14,7 @@ pip install tednet
 ```
 
 More information could be found in [Document](https://tednet.readthedocs.io/en/latest/index.html).
+
+---
+### Quick Start
+
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -29,6 +29,7 @@ tednet - A Toolkit for Tensor Decomposition Networks
     :caption: Tutorials
 
     tutorials/tr_cnn
+    tutorials/tr_rnn
 
 Tensor, also known as a multi-way array, can be viewed as a higher-order extension of a vector (i.e., an order-1 tensor) and a matrix (i.e., an order-2 tensor).  Like rows and columns in a matrix, an order-N tensor :math:`{\mathcal X}\in\mathbb R^{I_1\times I_2 \ldots\times I_N}` has N-modes (or ways, orders, indices) whose lengths (or dimensions) are represented by :math:`I_1, \ldots, I_N` respectively. Interestingly, tensors are graphically represented in Tensor network diagrams. As following illustration, a black node denotes a tensor and a edge connected to the node means a tensor mode.
 

diff --git a/docs/source/quick_start.ipynb b/docs/source/quick_start.ipynb
@@ -17,12 +17,185 @@
     "id": "oz-X6ZrGVwgu"
    },
    "source": [
-    "In this quick starting guide we show the basics of working with t3f library. The main concept of the library is a TensorTrain object – a compact (factorized) representation of a tensor (=multidimensional array). This is generalization of the matrix low-rank decomposition."
+    "In this section, we would like to show a overview to give a quick start."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Operation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "There are some operations supported in tednet, and it is convinient to use them."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import tednet as tdt"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Create matrix whose diagonal elements are ones**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor([[1., 0., 0., 0., 0.],\n",
+      "        [0., 1., 0., 0., 0.],\n",
+      "        [0., 0., 1., 0., 0.],\n",
+      "        [0., 0., 0., 1., 0.],\n",
+      "        [0., 0., 0., 0., 1.]])\n"
+     ]
+    }
+   ],
+   "source": [
+    "diag_matrix = tdt.eye(5, 5)\n",
+    "print(diag_matrix)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Take Pytorch tensor to Numpy narray**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<class 'torch.Tensor'>\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(type(diag_matrix))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "AttributeError",
+     "evalue": "module 'tednet' has no attribute 'to_numpy'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
+      "\u001b[0;32m<ipython-input-4-7453e51b14b4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdiag_matrix\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtdt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_numpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdiag_matrix\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+      "\u001b[0;31mAttributeError\u001b[0m: module 'tednet' has no attribute 'to_numpy'"
+     ]
+    }
+   ],
+   "source": [
+    "diag_matrix = tdt.to_numpy(diag_matrix)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<class 'torch.Tensor'>\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(type(diag_matrix))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Take Numpy narray to Pytorch tensor**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "AttributeError",
+     "evalue": "module 'tednet' has no attribute 'to_tensor'",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
+      "\u001b[0;32m<ipython-input-6-1701b2504789>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdiag_matrix\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtdt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdiag_matrix\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+      "\u001b[0;31mAttributeError\u001b[0m: module 'tednet' has no attribute 'to_tensor'"
+     ]
+    }
+   ],
+   "source": [
+    "diag_matrix = tdt.to_tensor(diag_matrix)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<class 'torch.Tensor'>\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(type(diag_matrix))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Tensor Decomposition Networks (Tensor Ring for Sample)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**To use tensor ring decomposition models, simply calling the tensor ring module is enough.**"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
    "metadata": {
     "colab": {
      "base_uri": "https://localhost:8080/",
@@ -34,14 +207,20 @@
    },
    "outputs": [],
    "source": [
-    "\n",
     "import tednet as tdt\n",
     "import tednet.tnn.tensor_ring as tr"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Here, we would like to give a case of building the TR-LeNet5.**"
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 9,
    "metadata": {
     "colab": {},
     "colab_type": "code",
@@ -65,17 +244,18 @@
    ]
   },
   {
-   "cell_type": "markdown",
-   "metadata": {
-    "colab_type": "text",
-    "id": "i2QmXYqeVwhQ",
-    "pycharm": {
-     "name": "#%% md\n"
-    }
-   },
-   "source": [
-    "It is easy to define a tensor ring model.\n"
-   ]
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {

diff --git a/docs/source/tutorials/tr_rnn.ipynb b/docs/source/tutorials/tr_rnn.ipynb
@@ -0,0 +1,106 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 0,
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.5"
+  },
+  "colab": {
+   "name": "riemannian.ipynb",
+   "provenance": [],
+   "collapsed_sections": []
+  }
+ },
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "gqwrcHFAVwgs",
+    "colab_type": "text"
+   },
+   "source": [
+    "# Tensor Ring Convolutional Neural Network"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "oz-X6ZrGVwgu",
+    "colab_type": "text"
+   },
+   "source": [
+    "By replacing convolutional kernel with tensor ring cores, tensor ring CNN is constructed.\n",
+    "\n",
+    "Here is an example to use a TR-based model with `tednet`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {
+    "id": "ri9QCNEAVwgw",
+    "colab_type": "code",
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 34
+    },
+    "outputId": "efd8c64b-836e-449b-a385-dd3a63a5b4a2"
+   },
+   "source": [
+    "\n",
+    "import tednet as tdt\n",
+    "import tednet.tnn.tensor_ring as tr"
+   ],
+   "execution_count": 1,
+   "outputs": [
+    {
+     "output_type": "stream",
+     "text": [
+      "TensorFlow 2.x selected.\n"
+     ],
+     "name": "stdout"
+    }
+   ]
+  },
+  {
+   "cell_type": "code",
+   "metadata": {
+    "id": "5Awp7wdwVwg3",
+    "colab_type": "code",
+    "colab": {}
+   },
+   "source": [
+    "# Define a TR-LeNet5\n",
+    "model = tr.TRLeNet5(10, [6, 6, 6, 6])"
+   ],
+   "execution_count": 0,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "i2QmXYqeVwhQ",
+    "colab_type": "text",
+    "pycharm": {
+     "name": "#%% md\n"
+    }
+   },
+   "source": [
+    "It is easy to define a tensor ring model.\n"
+   ]
+  }
+ ]
+}
diff --git a/tednet/__init__.py b/tednet/__init__.py
@@ -1,8 +1,7 @@
 # -*- coding: UTF-8 -*-
 
-
 __author__ = "Perry"
-__version__ = "0.0.9"
+__version__ = "0.1.0"
 
 from ._ops import *
 

diff --git a/tednet/_ops.py b/tednet/_ops.py
@@ -53,7 +53,7 @@ def eye(n: int, m: int, device: torch.device = "cpu", requires_grad: bool=False)
 
 
 def to_numpy(tensor: torch.Tensor) -> np.ndarray:
-    """Convert torch.Tensor to numpy variable.
+    """Convert torch.Tensor to numpy.ndarray.
 
     Parameters
     ----------
@@ -74,7 +74,7 @@ def to_numpy(tensor: torch.Tensor) -> np.ndarray:
 
 
 def to_tensor(arr: np.ndarray) -> torch.Tensor:
-    """Convert numpy variable to torch.Tensor.
+    """Convert numpy.ndarray to torch.Tensor.
 
     Parameters
     ----------