Skip to content

hiroyuki-kasai/NMFLibrary

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NMFLibrary: Non-negative Matrix Factorization Library

Authors: Hiroyuki Kasai

Last page update: July 22, 2022

Latest library version: 2.1 (see Release notes for more info)


Announcement

We are very welcome to your contribution. Please tell us

  • NMF solvers written by MATLAB,
  • appplication MATLAB flies using NMF solvers, and
  • your comments and suggestions.

Introduction

The NMFLibrary is a pure-Matlab library of a collection of algorithms of non-negative matrix factorization (NMF). The solvers can be also called from python (see demo.py).


Bibliograph

If this library is useful for you, please cite this as presented below:

@misc{kasai_NMFLibrary_2017,
    Author = {Kasai, Hiroyuki},
     Title = {{NMFLibrary}: MATLAB library for non-negative matrix factorization (NMF)},
     Year  = {2017},
     Howpublished = {\url{https://github.com/hiroyuki-kasai/NMFLibrary}}
}


Algorithm configurations

Category Name in example codes function options.alg other options
Frobenius-norm Fro-MU fro_mu_nmf mu metric='euc'
Modified Fro-MU fro_mu_nmf mod_mu
Accelerated Fro-MU fro_mu_nmf acc_mu
PGD pgd_nmf pgd
Direct PGD pgd_nmf direct_pgd
Adaptive-step PGD pgd_nmf adp_step_pgd
ALS als_nmf als
Hierarchical ALS als_nmf hals_mu
Accelerated Hierarchical ALS als_nmf acc_hals_mu
ASGROUP anls_nmf anls_asgroup
ASGIVENS anls_nmf anls_asgivens
BPP anls_nmf anls_bpp
Divergence Div-MU-KL div_mu_nmf metric='kl-div'
Div-MU-ALPHA div_mu_nmf metric='alpha-div'
Div-MU-BETA div_mu_nmf metric='beta-div'
Div-MU-IS div_mu_nmf metric='beta-div' d_beta=0
Div-MU-KL div_mu_nmf metric='beta-div' d_beta=1
Div-ADMM-IS div_admm_nmf metric='beta-div' d_beta=0
Div-ADMM-KL div_admm_nmf metric='beta-div' d_beta=1
KL-FPA kl_fpa_nmf
KL-BMD kl_bmd_nmf
Semi Semi-MU semi_mu_nmf
Semi-BCD semi_bcd_nmf
Variant NeNMF nenmf
GNMF GNMF
SDNMF SDNMF
Robust Robust-MU robust_mu_nmf
Sparse Sparse-MU-EUC sparse_mu_nmf metric='euc'
Sparse-MU-KL sparse_mu_nmf metric='kl-div'
sparseNMF sparse_nmf
SC-NMF sc_nmf
Nonsmooth-NMF ns_nmf metric='euc', update_alg='apg'
Proj-Sparse proj_sparse_nmf
PALM-Sparse-Smooth palm_sparse_smooth_nmf
Orthogonal DTPP dtpp_nmf
Orth-MU orth_mu_nmf
NMF-HALS-SO hals_so_nmf
ALT-ONMF alternating_onmf
Symmetric SymmANLS symm_anls
SymmHALS symm_halsacc
SymmNewton symm_newton
Online Incremental-NMF incremental_mu_nmf
Online-MU online_mu_nmf
Accelerated Online-MU acc_online_mu_nmf
SPG spg_nmf
Robust-Online-MU robust_online_mu_nmf
ASAG-MU-NMF asag_mu_nmf
Stochastic-MU smu_nmf
SVRMU svrmu_nmf
R-SVRMU svrmu_nmf robust=true
SAGMU sagmu_nmf
Probabilistic PNMF-VB vb_pro_nmf
PNMF-VB-ARD vb_pro_nmf ard=true
Prob-NM prob_nmf
Deep Deep-Semi deep_semi_nmf
Deep-Bidir-Semi deep_bidirectional_nmf
Deep-nsNMF deep_ns_nmf
Deep-Multiview-Semi deep_multiview_semi_nmf
Convex Convex-MU convex_mu_nmf sub_mode='std'
Kernel-Convex-MU convex_mu_nmf sub_mode='kernel'
Separable SPA spa
SNPA snpa
Convolutive MU-Conv mu_conv_nmf
Heur-MU-Conv heuristic_mu_conv_nmf
ADMM-Y-Conv admm_y_conv_nmf
ADMM-Seq-Conv admm_seq_conv_nmf
Rank2 Rank2-NMF rank2nmf
Nonnegative matrix tri-factorization Sep-Symm-NMTF sep_symm_nmtf
Nonnegative under-approximation recursive_nmu recursive_nmu
Minimum-volume minvol-NMF minvol_nmf
Weighted Low-Rank matrix approximation WLRA wlra

Folders and files

./                              - Top directory.
./README.md                     - This readme file.
./run_me_first.m                - The scipt that you need to run first.
./demo.m                        - Demonstration script to check and understand this package easily. 
./demo_face.m                   - Demonstration script to check and understand this package easily. 
./demo.py                       - Demonstration script to use this package easily from python. 
|plotter/                       - Contains plotting tools to show convergence results and various plots.
|auxiliary/                     - Some auxiliary tools for this project.
|solver/                        - Contains various optimization algorithms.
    |--- frobenius_norm/        - NMF solvers with Frobenius norm metric.
    |--- divergence/            - NMF solvers with various divertence metrics (KL, beta, alpha, IS).
    |--- online/                - Online/stochstic NMF solvers.
    |--- sparse/                - Sparse NMF solvers.
    |--- robust/                - Robust NMF solvers.
    |--- orthogonal/            - Orthogonal NMF solvers.
    |--- symmetric/             - Symmetric NMF solvers.
    |--- semi/                  - Semi NMF solvers.
    |--- deep/                  - Deep NMF solvers.
    |--- probabilistic/         - Probabilistic NMF solvers.
    |--- convex/                - Convex NMF solver.
    |--- convolutive/           - Convolutive NMF solvers.
    |--- minvol/                - Minimum-volume rank-deficient NMF.
    |--- nm_under_approx/       - Recursive non-negative matrix underapproximation.
    |--- nmtf/                  - Separable symmetric nonnegative matrix tri-factorization.
    |--- projective_nmf/        - Projective NMF solver.
    |--- rank2/                 - rank-two NMF solver.
    |--- weight_lowrank_aprox/  - Weighted Low-Rank matrix Approximation algorithm.
    |--- nenmf/                 - Nesterov's accelerated NMF solver.
    |--- nnls/                  - Solvers for nonnegativity-constrained least squares.
    |--- 3rd_party/             - Solvers provided by 3rd_party.
    |--- solver_health_check.m  - Health check scripts for solvers.
|applications/                  - Some appplications using NMF.

First to do

Run run_me_first for path configurations.

%% First run the setup script
run_me_first; 

Simplest usage example: 4 steps!

Just execute demo for the simplest demonstration of this package. .

%% Execute the demonstration script
demo; 

The "demo.m" file contains below.

%% generate synthetic data non-negative matrix V size of (mxn)
m = 500;
n = 100;
V = rand(m,n);
    
%% Initialize rank to be factorized
rank = 5;

%% perform factroization
% Fro-MU
options.alg = 'mu';
[w_mu, infos_mu] = fro_mu_nmf(V, rank, options);
% Hierarchical ALS
options.alg = 'hals';
[w_hals, infos_hals] = als_nmf(V, rank, options);        
    
%% plot
display_graph('epoch','cost', {'MU', 'HALS'}, {w_mu, w_hals}, {infos_mu, infos_hals});

Let's take a closer look at the code above bit by bit. The procedure has only 4 steps!

Step 1: Generate data

First, we generate synthetic data of V of size (mxn).

m = 500;
n = 100;
V = rand(m,n);

Step 2: Define rank

We set the rank value.

rank = 5;

Step 3: Perform solver

Now, you can perform nmf solvers, e.g., Frobenius-norm MU and Hierarchical ALS (HALS), calling solver functions, i.e., fro_mu_nmf() function and als_nmf() function after setting some optimization options.

% Fro-MU
options.alg = 'mu';
[w_mu, infos_mu] = fro_mu_nmf(V, rank, options);
% Hierarchical ALS
options.alg = 'hals';
[w_hals, infos_hals] = als_nmf(V, rank, options); 

They return the final solutions of w and the statistics information that include the histories of epoch numbers, cost values, norms of gradient, the number of gradient evaluations and so on.

Step 4: Show result

Finally, display_graph() provides output results of decreasing behavior of the cost values in terms of the number of iterrations (epochs) and time [sec].

display_graph('epoch','cost', {'Fro-MU', 'HALS'}, {w_mu, w_hals}, {infos_mu, infos_hals});
display_graph('time','cost', {'Fro-MU', 'HALS'}, {w_mu, w_hals}, {infos_mu, infos_hals});

That's it!


More plots

"demo_face.m" illustrates the learned basis (dictrionary) in case of CBCL face datasets.

The dataset is first loaded into V instead of generating synthetic data in Step 1.

V = importdata('./data/CBCL_face.mat');

Then, we can display basis elements (W: dictionary) obtained with different algorithms additionally in Step 4.

plot_dictionnary(w_mu.W, [], [7 7]); 
plot_dictionnary(w_hals.W, [], [7 7]); 


How to use NMFLibrary from python

Step 1: Find the path to the MATLAB folder

Run matlabroot in the MATLAB command window.

matlabroot; 

Step 2: Install the Engine API

To install the engine API, choose one of the following. You must call this python install command in the specified folder. The followings are examples in case of R2022a.

  • Windows
cd "c:\Program Files\MATLAB\R2022a\extern\engines\python"
python setup.py install
  • Linux
cd "/usr/local/MATLAB/R2022a/bin/matlab/extern/engines/python"
python setup.py install
  • macOS
cd "/Applications/MATLAB_R2022a.app/extern/engines/python"
python setup.py install

Step 3: Run demonstration code

python demo.py

As for Steps 1 and 2, see more details here.


License

  • The NMFLibrary is free, non-commercial and open source.
  • The code provided iin NMFLibrary should only be used for academic/research purposes.
  • Third party files are ported and included as is.
    • Many solvers (fro_mu_nmf.m, als_nmf.m, wlra.m, spa.m, snpa.m, proj_sparse_nmf.m, rank2nmf.m, projective_nmf.m, alternating_onmf.m, recursive_nmu.m, sep_symm_nmtf.m, minvol_nmf.m, nnls_*.m, semi_bcd_nmf.m and others) are ported from the codes of NMF book written by Nicolas Gillis.
    • For ANLS algorithms: nnlsm_activeset.m, nnls1_asgivens.m, nnlsm_blockpivot.m, and normalEqComb.m written by Jingu Kim.
    • For PGD algorithm: nlssubprob.m.
    • For GNMF algorithm: GNMF.m, GNMF_Multi.m, constructW.m and litekmeans.m writtnen by Deng Cai.
    • For SDNMF algorithm: SDNMF.m and SDNMF_Multi.m writtnen by Wei Qian.
    • For Symmetric algorithms writtnen by D.Kang et al. and Z. Zhu et al.
    • For KL-FPA algorithm: kl_fpa_nmf.m writtnen by Felipe Yanez.
    • For KL-BMD algorithm: BMD.m writtnen by by LTK Hien.
    • For Deep algorithm: deep_semi_nmf.m, deep_bidirectional_nmf.m` writtnen by G.Trigeorgis, and 'deep_multiview_semi_nmf.m' writtnen by H.Zhao.
    • For PALM-Sparse-Smooth algorithm: palm_sparse_smooth_nmf.m writtnen by Raimon Fabregat.
    • For Convex-MU algorithm: convex_mu_nmf.m writtnen by Yifeng Li.
    • For Convolutive algorithm: mu_conv_nmf.m, heuristic_mu_conv_nmf.m, admm_y_conv_nmf.m, and admm_seq_conv_nmf.m writtnen by lyn202206.
    • For Probabilistic algorithm: prob_nmf.m by NMF DTU Toolbox, Lars Kai Hansen.
    • For Probabilistic algorithm: vb_pro_nmf.m` is ported from the Python code originally written by T. Brouwer et. al..
    • For dictionaly visualization: plot_dictionnary.m, rescale.m, and getoptions.m.

Acknowledge

  • Thank you for big contributions to this library to
    • Haonan Huang
    • Mitsuhiko Horie
    • Takumi Fukunaga

Problems or questions

If you have any problems or questions, please contact the author: Hiroyuki Kasai (email: hiroyuki dot kasai at waseda dot jp)


Release notes

  • Version 2.1 (July 22, 2022)
    • No-option is allowed in all solvers.
    • A python demonstration script is added.
    • Application scripts are added.
  • Version 2.0 (July 18, 2022)
    • Major update.
    • New NMF models are added.
    • New NMF solvers are added.
    • Code structure of solvers is refactored.
    • Else
      • Solver/user-defined stopping function is supported.
      • Statistics display module is refactored.
      • Acceleration algorithms on frobenius-norm based methods are added.
      • Divergence-based methods are separated from frobenius-norm based methods.
      • Version update checking mechanism is supported.
      • Demonstration script to use this package from python is included.
  • Version 1.8.1 (Oct. 14, 2020)
    • Bug fixed in sc_nmf.m and semi_mu_nmf, and added the LPinitSemiNMF algorithm into generate_init_factors.m (Thanks to Haonan Huang).
  • Version 1.7.0 (June 27, 2019)
    • Symmetic solvers are added.
    • Clustering quality measurements are integrated into store_nmf_info.m.
  • Version 1.7.0 (May 21, 2019)
    • PNMF-VB and NeNMF are added.
    • Fixed some bugs.
  • Version 1.6.0 (May 16, 2019)
    • DTPP is added.
  • Version 1.5.1 (Apr. 22, 2019)
    • Some solvers are modified to fix bugs.
  • Version 1.5.0 (July 30, 2018)
    • fnsNMF and NMF-HALS-SO are added.
  • Version 1.4.0 (July 24, 2018)
    • sparseMU and orthMU are added.
    • MU with Kullback-Leibler divergence (KL), Amari alpha divergence, and beta divergenceare added.
  • Version 1.3.0 (July 23, 2018)
    • SC-NMF, scNMF and csNMF are added.
  • Version 1.2.0 (July 21, 2018)
    • GNMF, Semi-NMF and SDNMF are added.
  • Version 1.1.0 (Apr. 17, 2018)
    • Online/stochastic solvers are added.
  • Version 1.0.0 (Apr. 04, 2017)
    • Initial version.