Article

Overview and recent advances in partial least squares

Authors:

Nicole KrämerAuthors Info & Claims

SLSFS'05: Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection

Pages 34 - 51

https://doi.org/10.1007/11752790_2

Published: 23 February 2005 Publication History

Abstract

Partial Least Squares (PLS) is a wide class of methods for modeling relations between sets of observed variables by means of latent variables. It comprises of regression and classification tasks as well as dimension reduction techniques and modeling tools. The underlying assumption of all PLS methods is that the observed data is generated by a system or process which is driven by a small number of latent (not directly observed or measured) variables. Projections of the observed data to its latent structure by means of PLS was developed by Herman Wold and coworkers [48,49,52].

References

[1]

T. Almøy. A simulation study on comparison of prediction models when only a few components are relevant. Computational Statistics and Data Analysis, 21:87-107, 1996.

Digital Library

[2]

N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68:337-404, 1950.

[3]

G. Baffi, E.B. Martin, and A.J. Morris. Non-linear projection to latent structures revisited (the neural network PLS algorithm). Computers Chemical Engineering, 23:1293-1307, 1999.

[4]

M. Barker and W.S. Rayens. Partial least squares for discrimination. Journal of Chemometrics, 17:166-173, 2003.

[5]

M.S. Bartlett. Further aspects of the theory of multiple regression. In Proceedings of the Cambridge Philosophical Society, volume 34, pages 33-40, 1938.

[6]

T. De Bie, N. Cristianini, and R. Rosipal. Eigenproblems in Pattern Recognition. In E. Bayro-Corrochano, editor, Handbook of Geometric Computing: Applications in Pattern Recognition, Computer Vision, Neuralcomputing, and Robotics, pages 129-170. Springer, 2005.

[7]

N.A. Butler and M.C. Denham. The peculiar shrinkage properties of partial least squares regression. Journal of the Royal Statistical Society: B, 62:585-593, 2000.

[8]

S. de Jong. SIMPLS: an alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems, 18:251-263, 1993.

[9]

S. de Jong, B.M. Wise, and N.L. Ricker. Canonical partial least squares and continuum power regression. Journal of Chemometrics, 15:85-100, 2001.

[10]

I.E. Frank. A nonlinear PLS model. Chemolab, 8:109-119, 1990.

[11]

I.E. Frank and J.H. Friedman. A Statistical View of Some Chemometrics Regression Tools. Technometrics, 35:109-147, 1993.

[12]

C. Goutis. Partial least squares yields shrinkage estimators. The Annals of Statistics, 24:816-824, 1996.

[13]

T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2001.

[14]

I.S. Helland. Some theoretical aspects of partial least squares regression. Chemometrics and Intelligent Laboratory Systems, 58:97-107, 1999.

[15]

M. Hestenes and E. Stiefel. Methods for conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49:409-436, 1952.

[16]

A.E. Hoerl and R.W. Kennard. Ridge regression: bias estimation for nonorthogonal problems. Technometrics, 12:55-67, 1970.

[17]

A. Höskuldsson. PLS Regression Methods. Journal of Chemometrics, 2:211-228, 1988.

[18]

J.S. Hulland. Use of partial least squares (PLS) in strategic management research: A review of four recent studies. Strategic Management Journal, 20:195-204, 1999.

[19]

C. Lanczos. An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards, 45:225-280, 1950.

[20]

P.J. Lewi. Pattern recognition, reflection from a chemometric point of view. Chemometrics and Intelligent Laboratory Systems, 28:23-33, 1995.

[21]

O.C. Lingjærde and N. Christophersen. Shrinkage Structure of Partial Least Squares. Scandinavian Journal of Statistics, 27:459-473, 2000.

[22]

N.J. Lobaugh, R. West, and A.R. McIntosh. Spatiotemporal analysis of experimental differences in event-related potential data with partial least squares. Psychophysiology, 38:517-530, 2001.

[23]

R. Manne. Analysis of Two Partial-Least-Squares Algorithms for Multivariate Calibration. Chemometrics and Intelligent Laboratory Systems, 2:187-197, 1987.

[24]

K.V. Mardia, J.T. Kent, and J.M. Bibby. Multivariate Analysis. Academic Press, 1997.

[25]

M. Martens and H. Martens. Partial Least Squares Regression. In J.R. Piggott, editor, Statistical Procedures in Food Research, pages 293-359. Elsevier Applied Science, London, 1986.

[26]

W.F. Massy. Principal components regression in exploratory statistical research. Journal of the American Statistical Association, 60:234-256, 1965.

[27]

M. Momma. Efficient Computations via Scalable Sparse Kernel Partial Least Squares and Boosted Latent Features. In Proceedings of SIGKDD International Conference on Knowledge and Data Mining, pages 654-659, Chicago, IL, 2005.

Digital Library

[28]

D.V. Nguyen and D.M. Rocke. Tumor classification by partial least squares using microarray gene expression data. Bioinformatics, 18:39-50, 2002.

[29]

J. Nilsson, S. de Jong, and A.K. Smilde. Multiway Calibration in 3D QSAR. Journal of Chemometrics, 11:511-524, 1997.

[30]

B. Parlett. The symmetric eigenvalue problem. SIAM, 1998.

Digital Library

[31]

A. Phatak and F. de Hoog. Exploiting the connection between PLS, Lanczos, and conjugate gradients: Alternative proofs of some properties of PLS. Journal of Chemometrics, 16:361-367, 2003.

[32]

A. Phatak, P.M. Rilley, and A. Penlidis. The asymptotic variance of the univariate PLS estimator. Linear Algebra and its Applications, 354:245-253, 2002.

[33]

S. Rännar, F. Lindgren, P. Geladi, and S. Wold. A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and algorithm. Chemometrics and Intelligent Laboratory Systems, 8:111-125, 1994.

[34]

R. Rosipal and L.J. Trejo. Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. Journal of Machine Learning Research, 2:97-123, 2001.

Digital Library

[35]

R. Rosipal, L.J. Trejo, and B. Matthews. Kernel PLS-SVC for Linear and Nonlinear Classification. In Proceedings of the Twentieth International Conference on Machine Learning, pages 640-647, Washington, DC, 2003.

[36]

P.D. Sampson, A. P. Streissguth, H.M. Barr, and F.L. Bookstein. Neurobehavioral effects of prenatal alcohol: Part II. Partial Least Squares analysis. Neurotoxicology and tetralogy, 11:477-491, 1989.

[37]

C. Saunders, A. Gammerman, and V. Vovk. Ridge Regression Learning Algorithm in Dual Variables. In Proceedings of the 15th International Conference on Machine Learning, pages 515-521, Madison, WI, 1998.

Digital Library

[38]

B. Schölkopf and A. J. Smola. Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press, 2002.

Digital Library

[39]

A. Smilde, R. Bro, and P. Geladi. Multi-way Analysis: Applications in the Chemical Sciences. Wiley, 2004.

[40]

M. Stone and R.J. Brooks. Continuum Regression: Cross-validated Sequentially Constructed Prediction Embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression. Journal of the Royal Statistical Society: B, 52:237-269, 1990.

[41]

H. D. Vinod. Canonical ridge and econometrics of joint production. Journal of Econometrics, 4:147-166, 1976.

[42]

G. Wahba. Splines Models of Observational Data, volume 59 of Series in Applied Mathematics. SIAM, Philadelphia, 1990.

[43]

J. Wallerius, L.J. Trejo, R. Matthew, R. Rosipal, and J.A. Caldwell. Robust feature extraction and classification of EEG spectra for real-time classification of cognitive state. In Proceedings of 11th International Conference on Human Computer Interaction, Las Vegas, NV, 2005.

[44]

L.E. Wangen and B.R. Kowalsky. A multiblock partial least squares algorithm for investigating complex chemical systems. Journal of Chemometrics, 3:3-20, 1989.

[45]

J.A. Wegelin. A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case. Technical report, Department of Statistics, University of Washington, Seattle, 2000.

[46]

J. Westerhuis, T. Kourti, and J. MacGregor. Analysis of multiblock and hierarchical PCA and PLS models. Journal of Chemometrics, 12:301-321, 1998.

[47]

H. Wold. Path models with latent variables: The NIPALS approach. In H.M. Blalock et al., editor, Quantitative Sociology: International perspectives on mathematical and statistical model building, pages 307-357. Academic Press, 1975.

[48]

H. Wold. Soft modeling: the basic design and some extensions. In J.-K. Jöreskog and H. Wold, editor, Systems Under Indirect Observation, volume 2, pages 1-53. North Holland, Amsterdam, 1982.

[49]

H.Wold. Partial least squares. In S. Kotz and N.L. Johnson, editors, "Encyclopedia of the Statistical Sciences", volume 6, pages 581-591. John Wiley & Sons, 1985.

[50]

S. Wold. Nonlinear partial least squares modeling II, Spline inner relation. Chemolab, 14:71-84, 1992.

[51]

S. Wold, N. Kettaneh-Wold, and B. Skagerberg. Nonlinear PLS modelling. Chemometrics and Intelligent Laboratory Systems, 7:53-65, 1989.

[52]

S. Wold, H. Ruhe, H. Wold, and W.J. Dunn III. The collinearity problem in linear regression. The partial least squares (PLS) approach to generalized inverse. SIAM Journal of Scientific and Statistical Computations, 5:735-743, 1984.

Digital Library

[53]

K.J. Worsley. An overview and some new developments in the statistical analysis of PET and fMRI data. Human Brain Mapping, 5:254-258, 1997.

Cited By

Isachenko RStrijov V(2022)Quadratic programming feature selection for multicorrelated signal decoding with partial least squaresExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.117967207:COnline publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.117967
Aldieri ATerzini MAudenino ABignardi CMorbiducci U(2022)Combining shape and intensity dxa-based statistical approaches for osteoporotic HIP fracture risk assessmentComputers in Biology and Medicine10.1016/j.compbiomed.2020.104093127:COnline publication date: 3-Jan-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2020.104093
Robinson DChen QXue BKilleen DGordon KZhang M(2022)A New Genetic Algorithm for Automated Spectral Pre-processing in Nutrient AssessmentApplications of Evolutionary Computation10.1007/978-3-031-02462-7_19(283-298)Online publication date: 20-Apr-2022
https://dl.acm.org/doi/10.1007/978-3-031-02462-7_19
Show More Cited By

Recommendations

An overview on the shrinkage properties of partial least squares regression

The aim of this paper is twofold. In the first part, we recapitulate the main results regarding the shrinkage properties of partial least squares (PLS) regression. In particular, we give an alternative proof of the shape of the PLS shrinkage factors. It ...
Partial Least Squares Path Modeling: Basic Concepts, Methodological Issues and Applications
Comparison of Principal Component Regression and Partial Least Squares Regression by R
ICEICE '12: Proceedings of the 2012 Second International Conference on Electric Information and Control Engineering - Volume 04

Principle component regression (PCR) and partial least squares regression (PLSR) are two methodologies commonly used to solve dimension reduction and the multi-co linearity problem. The goal of this paper is to analyze and predict dependent variable ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

SLSFS'05: Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection

February 2005

208 pages

ISBN:3540341374

Editors:
Craig Saunders
ISIS Research Group, University of Southampton, U.K.
,
Marko Grobelnik
Dept. of Knowledge Technologies, Jozef Stefan Institute, Jamova 39, Ljubljana, Slovenia
,
Steve Gunn
School of Electronics and Computer Science, University of Southampton, Building 1, Highfield Campus, Southampton, UK
,
John Shawe-Taylor
The Centre for Computational Statistics and Machine Learning Department of Computer Science, University College London, Gower St., London, UK

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 23 February 2005

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

105
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Isachenko RStrijov V(2022)Quadratic programming feature selection for multicorrelated signal decoding with partial least squaresExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.117967207:COnline publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.117967
Aldieri ATerzini MAudenino ABignardi CMorbiducci U(2022)Combining shape and intensity dxa-based statistical approaches for osteoporotic HIP fracture risk assessmentComputers in Biology and Medicine10.1016/j.compbiomed.2020.104093127:COnline publication date: 3-Jan-2022
https://dl.acm.org/doi/10.1016/j.compbiomed.2020.104093
Robinson DChen QXue BKilleen DGordon KZhang M(2022)A New Genetic Algorithm for Automated Spectral Pre-processing in Nutrient AssessmentApplications of Evolutionary Computation10.1007/978-3-031-02462-7_19(283-298)Online publication date: 20-Apr-2022
https://dl.acm.org/doi/10.1007/978-3-031-02462-7_19
Guo JShi MZhu XHuang WHe YZhang WTang Z(2021)Improving human action recognition by jointly exploiting video and WiFi cluesNeurocomputing10.1016/j.neucom.2020.11.074458:C(14-23)Online publication date: 11-Oct-2021
https://dl.acm.org/doi/10.1016/j.neucom.2020.11.074
Liu LDong XWang T(2020)Semi-Supervised Cross-Modal Retrieval Based on Discriminative ComappingComplexity10.1155/2020/14624292020Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1155/2020/1462429
Fairuzzahira FZagloel TArdi R(2020)Conceptual Modelling of Supplier Loyalty and Buyer-Supplier Relationship for MediationProceedings of the 3rd Asia Pacific Conference on Research in Industrial and Systems Engineering10.1145/3400934.3400988(295-299)Online publication date: 16-Jun-2020
https://dl.acm.org/doi/10.1145/3400934.3400988
Guan JZhang MLu Z(2020)Large-Scale Cross-Domain Few-Shot LearningComputer Vision – ACCV 202010.1007/978-3-030-69535-4_29(474-491)Online publication date: 30-Nov-2020
https://dl.acm.org/doi/10.1007/978-3-030-69535-4_29
Batra VHaldar AHe YFerhatosmanoglu HVogiatzis GGuha T(2020)Variational Recurrent Sequence-to-Sequence Retrieval for Stepwise IllustrationAdvances in Information Retrieval10.1007/978-3-030-45439-5_4(50-64)Online publication date: 14-Apr-2020
https://dl.acm.org/doi/10.1007/978-3-030-45439-5_4
Li YPan YYao TChao HRui YMei T(2019)Learning Click-Based Deep Structure-Preserving Embeddings with Visual AttentionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/332899415:3(1-19)Online publication date: 8-Aug-2019
https://dl.acm.org/doi/10.1145/3328994
Patel YGomez LRusiñol MKaratzas DJawahar CEl Saddik ADel Bimbo AZhang ZHauptmann ACandan KBertini MXie LWei X(2019)Self-Supervised Visual Representations for Cross-Modal RetrievalProceedings of the 2019 on International Conference on Multimedia Retrieval10.1145/3323873.3325035(182-186)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3323873.3325035
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents