Sparse Tensor Regression

This repository implements the core function of paper "Sparse Multi-Response Tensor Regression for Alzheimer's Disease Study With Multivariate Clinical Assessments". Firstly, we generate the patterns and the datasets. Then we use Sparse Tensor Regression method to estimate our model. The training results are compared with the generated patterns.

How to Use

1. Install Matlab tensor_toolbox.
2. Download the release package and unzip it in the folder.
3. Run the script demo_reg in the Matlab environment.

Introduction

Multivariate Tensor Regression Model

The multivariate tensor regression model is Eq.(3)

Datasets

Firstly, we generate the model B following specified patterns and simulate X and e following a normal distribution. The response variables Y are calcualted by B, X and e. Given n independent and identically distributed samples

following chapter IV. We sperate them into three datasets: training set, validation set and testing set.

Objective Function

Then we minimize the objective function Eq.(5) using mini-batch gradient descending algorithm to estimate the models' parameters.

where

Minimize Objective Function

Here, we have two ways to find the minimum of the objective function. One is naive mini-batch gradient descending (TrainModelGradDesc.m), the other one is updating the models by closed form solution (TrainModelDerivative.m), proposed in the paper. The first one is simple and obvious, but its performance is inferior to the second one. We use the second method and alternatively update the parameters as below.

where

Experiments

The experiment results are below. We compare two training methods: one is naive mini-batch gradient descending (TrainModelGradDesc.m), the other one is updating the model by closed form solution (TrainModelDerivative.m), proposed in the paper. Obviously, the first training method is time-consuming and its results are less accurate than the second training method.

Mini-batch gradient descending method needs about 24 hours to reach convergence, while derivative closed form updating method needs about 5 minutes to reach convergence. We set the rank = 3 and lambda = 100.

Documents

• demo_reg.m demo script contains three components: 1. Generate the patterns. 2. Generate the datasets. 3. Estimate the models.
• CalcObjFunc.m Calculate the value of objective function Eq.(5).
• ComposeTensor.m Compose the components to a tensor. This is the inverse operation of CP decomposition.
• DecomposeTensor.m Decompose the tensor to the components by the input argument rank. The actual decomposition method is CP decomposition cp_als.
• DrawTrainingResults.m Draw the training results that were saved in the training/ folder and compare between the generated pattern and the estimated patterns.
• GenerateData.m Generate the data by input arguments and store them in the ./data/ folder.
• GenerateDataset.m Generate all three datasets: training, validation and testing. It calls GenerateData.m to generate data.
• GeneratePattern.m Generate the default patterns and store them in the ./data/ folder.
• InitModels.m Initialize the models by random values.
• LoadModels.m Load the model from files. These files may be the snapshot of the last training, saved by SaveTrainingStatus.m.
• SaveTrainingStatus.m Save the snapshot of the training process, including temporal models values.
• TrainModelDerivative.m We let the partial derivative to be zero and update the model B by its closed form solution.
• TrainModelGradDesc.m Estimate the model by mini-batch gradient descending.

License

• Author: Space Liang
• License: Apache License 2.0

References

1. Li, Z., Suk, H. I., Shen, D., & Li, L. (2016). Sparse Multi-Response Tensor Regression for Alzheimer's Disease Study With Multivariate Clinical Assessments. IEEE transactions on medical imaging, 35(8), 1927-1936.
2. Brett W. Bader, Tamara G. Kolda and others. MATLAB Tensor Toolbox Version 2.6, Available online, February 2015. URL: http://www.sandia.gov/~tgkolda/TensorToolbox/.