Use of Genetic Algorithm for Selecting Number of Non-Zero Loadings in Sparse Principal Component Analysis

Use of genetic algorithm for selecting number of non-zero loadings in sparse principal component analysis

 

Shriram Gajjar*, Murat Kulahci**, Ahmet Palazoglu*

*University of California, Davis, CA 95616, USA

(Tel: 530-752-8774; e-mail: anpalazoglu@ucdavis.edu).

**Technical University of Denmark, Lyngby, Denmark and Luleå University of Technology, Luleå, Sweden

(e-mail:muku@dtu.dk)

Background: With the advances in communication technologies and the emergence of smart factories, large volumes of data are routinely collected and stored at high sampling rates. The data generation and collection are so fast-paced that humans have to rely on computers for consuming as well as processing the data. This necessitates development of algorithms and methods that are able to extract significant information from large datasets to reduce process variability, discover the underlying process dynamics and improve decision-making.

Method: Principal component analysis (PCA) is among the most commonly and widely used multivariate technique with various applications ranging from facial recognition to data dimension reduction to clustering (Cinar et al., 2007). One of the drawbacks of using PCA for dimension reduction is that most variable loadings are typically non-zero. Such non-zero loadings make it difficult to interpret the derived PCs and may confound subsequent analyses. To address this challenge, Zou et al. (2006) proposed Sparse Principal Component Analysis (SPCA) but specifying number of non-zero variable loadings (NNZL) for each sparse principal component (SPC) is a numerically hard combinatorial problem. To tackle such combinatorial optimization problems, Evolutionary Algorithms (EA) based on the principle of evolution were introduced in the past decade. Here we propose the use of genoud (GENetic Optimization Using Derivatives) function in R that effectively combines EA methods with a derivative based (quasi-Newton) method to solve difficult optimization problems (Holland and Goldberg, 1989). The genoud function maximizes the Index of Sparseness (IS) to find the best combination of NNZL (Trendafilov, 2014).

Results: A major challenge in using SPCA is in deciding the penalty parameters or selecting the NNZL. One can take an exhaustive search approach that goes through all possible combinations of NNZL in each SPC, then choose a solution that meets the required criteria. However, such approach is computationally intensive and, in scenarios with large variables, it may be impractical and even infeasible to go through all possible combinations. Nevertheless, we use the exhaustive search as a comparative benchmark reference. The use of genoud provides an unsupervised process of selecting NNZL to maximize IS and provides a more intuitive solution for understanding the physical meaning of variables monitored in chemical processes. Ultimately, NNZL on each SPC can be specified without any loss of information. We used a synthetic example, pitprops data and Tennessee Eastman benchmark process to compare the PCA and SPCA results and showcase NNZL selection technique. The proposed approach allows to recover underlying latent structure and improve fault detection and diagnosis.

References

Cinar, A., Palazoglu, A. and Kayihan, F. (2007) 'Multivariate Statistical Monitoring Techniques', Chemical Process Performance Evaluation Chemical Industries: CRC Press, pp. 37-71.

Holland, J. and Goldberg, D. 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading, MA.

Trendafilov, N. T. (2014) 'From simple structure to sparse components: a review', Computational Statistics,29(3-4), pp. 431-454.

Zou, H., Hastie, T. and Tibshirani, R. (2006) 'Sparse Principal Component Analysis', Journal of Computational and Graphical Statistics, 15(2), pp. 265-286.