Skip to content

R package for penalized factor analysis via trust-region algorithm and automatic multiple tuning parameter selection

License

Notifications You must be signed in to change notification settings

egeminiani/penfa

Repository files navigation

penfa

minimal R version lifecycle Last-changedate Website Licence R-CMD-check

Overview

An R package for estimating single- and multiple-group penalized factor models via a trust-region algorithm with integrated automatic multiple tuning parameter selection (Geminiani et al., 2021). Supported penalties include lasso, adaptive lasso, scad, mcp, and ridge.

Installation

You can install the released version of penfa from CRAN with:

install.packages("penfa")

And the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("egeminiani/penfa")

Example

This is a basic example showing how to fit a PENalized Factor Analysis model with the alasso penalty and the automatic tuning procedure. A shrinkage penalty is applied to the whole factor loading matrix.

Let’s load the data (see ?ccdata for details).

library(penfa)
data(ccdata)

Step 1 : specify the model syntax

syntax = 'help  =~   h1 + h2 + h3 + h4 + h5 + h6 + h7 + 0*v1 + v2 + v3 + v4 + v5
          voice =~ 0*h1 + h2 + h3 + h4 + h5 + h6 + h7 +   v1 + v2 + v3 + v4 + v5'

Step 2: fit the model

alasso_fit <- penfa(model  = syntax,
                    data   = ccdata,
                    std.lv = TRUE,
                    pen.shrink = "alasso")
#> Computing weights for alasso (ML estimates)... done.
#> 
#> Automatic procedure: 
#> Iteration  1 : 0.00298271 
#> Iteration  2 : 0.00452604 
#> 
#> Largest absolute gradient value: 12.76355181
#> Fisher information matrix is positive definite
#> Eigenvalue range: [180.2917, 9189645]
#> Trust region iterations: 15 
#> Factor solution: admissible 
#> Effective degrees of freedom: 27.12936
alasso_fit
#> penfa 0.1.1 reached convergence
#> 
#>   Number of observations                                    767
#>                                                                
#>   Estimator                                                PMLE
#>   Optimization method                              trust-region
#>   Information                                            fisher
#>   Strategy                                                 auto
#>   Number of iterations (total)                               58
#>   Number of two-steps (automatic)                             2
#>   Effective degrees of freedom                           27.129
#>                                                                
#>   Penalty function:                                            
#>     Sparsity                                             alasso
#>                                                                
#> 

Step 3: inspect the results

summary(alasso_fit)
#> penfa 0.1.1 reached convergence
#> 
#>   Number of observations                                    767
#>   Number of groups                                            1
#>   Number of observed variables                               12
#>   Number of latent factors                                    2
#>                                                                
#>   Estimator                                                PMLE
#>   Optimization method                              trust-region
#>   Information                                            fisher
#>   Strategy                                                 auto
#>   Number of iterations (total)                               58
#>   Number of two-steps (automatic)                             2
#>   Influence factor                                            4
#>   Number of parameters:                                        
#>     Free                                                     13
#>     Penalized                                                22
#>   Effective degrees of freedom                           27.129
#>   GIC                                                 17222.980
#>   GBIC                                                17348.928
#>                                                                
#>   Penalty function:                                            
#>     Sparsity                                             alasso
#>                                                                
#>   Additional tuning parameter                                  
#>     alasso                                                    1
#>                                                                
#>   Optimal tuning parameter:                                    
#>     Sparsity                                                   
#>      - Factor loadings                                    0.005
#>                                                                
#> 
#> Parameter Estimates:
#> 
#> Latent Variables:
#>                     Type    Estimate  Std.Err     2.5%    97.5%
#>   help =~                                                      
#>     h1               pen       0.766    0.030    0.707    0.825
#>     h2               pen       0.858    0.028    0.803    0.913
#>     h3               pen       0.775    0.030    0.717    0.834
#>     h4               pen       0.921    0.038    0.847    0.995
#>     h5               pen       0.810    0.040    0.732    0.887
#>     h6               pen       0.782    0.044    0.696    0.868
#>     h7               pen       0.523    0.050    0.426    0.620
#>     v1             fixed       0.000             0.000    0.000
#>     v2               pen       0.000                           
#>     v3               pen       0.000                           
#>     v4               pen       0.000                           
#>     v5               pen      -0.000                           
#>   voice =~                                                     
#>     h1             fixed       0.000             0.000    0.000
#>     h2               pen      -0.000                           
#>     h3               pen       0.000                           
#>     h4               pen      -0.041                           
#>     h5               pen       0.053    0.031   -0.008    0.114
#>     h6               pen       0.104    0.038    0.029    0.180
#>     h7               pen       0.341    0.049    0.246    0.437
#>     v1               pen       0.851    0.028    0.795    0.906
#>     v2               pen       0.871    0.028    0.817    0.926
#>     v3               pen       0.842    0.029    0.786    0.898
#>     v4               pen       0.843    0.029    0.787    0.899
#>     v5               pen       0.805    0.029    0.747    0.862
#> 
#> Covariances:
#>                     Type    Estimate  Std.Err     2.5%    97.5%
#>   help ~~                                                      
#>     voice           free       0.877    0.011    0.855    0.900
#> 
#> Variances:
#>                     Type    Estimate  Std.Err     2.5%    97.5%
#>    .h1              free       0.388    0.021    0.346    0.429
#>    .h2              free       0.233    0.014    0.205    0.261
#>    .h3              free       0.372    0.021    0.332    0.413
#>    .h4              free       0.184    0.012    0.160    0.209
#>    .h5              free       0.235    0.014    0.207    0.263
#>    .h6              free       0.201    0.012    0.177    0.225
#>    .h7              free       0.264    0.015    0.235    0.293
#>    .v1              free       0.245    0.015    0.216    0.275
#>    .v2              free       0.208    0.014    0.182    0.235
#>    .v3              free       0.261    0.016    0.230    0.292
#>    .v4              free       0.259    0.016    0.228    0.290
#>    .v5              free       0.324    0.019    0.287    0.361
#>     help           fixed       1.000             1.000    1.000
#>     voice          fixed       1.000             1.000    1.000

Vignettes and Tutorials

  • See vignette("automatic-tuning-selection") for the estimation of a penalized factor model with lasso and alasso penalties. The tuning parameter producing the optimal amount of sparsity in the factor loading matrix is found through the automatic tuning procedure.

  • See vignette("grid-search-tuning-selection") for the estimation of a penalized factor model with scad and mcp penalties. A grid search is conducted, and the optimal tuning parameter is the one generating the penalized model with the lowest GBIC (Generalized Bayesian Information Criterion).

  • See “multiple-group-analysis” for the estimation of a multiple-group penalized factor model with the alasso penalty. This model encourages sparsity in the loading matrices and cross-group invariance of loadings and intercepts. The automatic multiple tuning parameter procedure is employed for finding the optimal tuning parameter vector.

  • See “plotting-penalty-matrix” for details on how to produce interactive plots of the penalty matrices.

Literature

  • Geminiani, E., Marra, G., & Moustaki, I. (2021). “Single- and Multiple-Group Penalized Factor Analysis: A Trust-Region Algorithm Approach with Integrated Automatic Multiple Tuning Parameter Selection.” Psychometrika, 86(1), 65-95. https://doi.org/10.1007/s11336-021-09751-8

  • Geminiani, E. (2020). “A Penalized Likelihood-Based Framework for Single and Multiple-Group Factor Analysis Models.” PhD thesis, University of Bologna. http://amsdottorato.unibo.it/9355/.

How to cite

#> 
#> To cite penfa in publications use:
#> 
#>   Geminiani, E., Marra, G., & Moustaki, I. (2021). Single- and
#>   Multiple-Group Penalized Factor Analysis: A Trust-Region Algorithm
#>   Approach with Integrated Automatic Multiple Tuning Parameter
#>   Selection.  Psychometrika, 86(1), 65-95.
#>   https://doi.org/10.1007/s11336-021-09751-8
#> 
#> A BibTeX entry for LaTeX users is
#> 
#>   @Article{,
#>     title = {Single- and Multiple-Group Penalized Factor Analysis: A Trust-Region Algorithm Approach with Integrated Automatic Multiple Tuning Parameter Selection},
#>     author = {Geminiani Elena and Marra Giampiero and Moustaki Irini},
#>     journal = {Psychometrika},
#>     year = {2021},
#>     volume = {86},
#>     number = {1},
#>     pages = {65-95},
#>     url = {https://doi.org/10.1007/s11336-021-09751-8},
#>   }
#> 
#>   Elena Geminiani, Giampiero Marra and Irini Moustaki (2021). penfa:
#>   Single- And Multiple-Group Penalized Factor Analysis. R package
#>   version 0.1.1.
#> 
#> A BibTeX entry for LaTeX users is
#> 
#>   @Manual{,
#>     title = {penfa: Single- And Multiple-Group Penalized Factor Analysis},
#>     author = {Elena Geminiani and Giampiero Marra and Irini Moustaki},
#>     year = {2021},
#>     note = {R package version 0.1.1},
#>   }