Principal Variables as an Alternative to Principal Components for Trait Selection in Agricultural Research

1. Introduction

The rapid expansion of data collection and storage across scientific disciplines has increased the need for statistical methods that can extract meaningful information from high-dimensional datasets. In multivariate studies, many variables are measured for each experimental unit, often with strong correlations and overlapping information. Therefore, dimensionality reduction has become essential for simplifying complex datasets while preserving their key characteristics.

Principal Component Analysis (PCA) is one of the most widely used techniques for dimensionality reduction, data exploration, and feature extraction in multivariate analysis. Introduced by Karal Pearson in 1901 and further developed by Hotelling in 1933, PCA transforms a set of correlated variables into a smaller number of uncorrelated linear combinations known as principal components. These components are constructed so that the first principal component accounts for the maximum possible variation in the data, while each subsequent component explains the maximum remaining variation subject to being uncorrelated with the preceding components. By concentrating most of the total variability into a few components, PCA provides an efficient summary of the original data structure.

Despite its usefulness, PCA does not achieve dimensionality reduction in terms of the original variables because each principal component is generally a linear combination of all the variables under study. As a result, all original variables remain necessary for the computation and interpretation of the principal components. In many practical applications, particularly in agriculture and biology, researchers are interested in identifying a subset of the original variables that adequately represents the information contained in the complete dataset. Such variable selection procedures not only reduce the cost and effort associated with data collection but also improve interpretability and facilitate decision-making.

Several methods have been proposed for selecting representative variables from a multivariate dataset. Important contributions to this problem include the work of Jolliffe (1972, 1973), who suggested variable selection procedures based on principal components, McCabe (1984), who introduced the concept of principal variables, and Krzanowski (1987), who further developed principal variable selection methods. Cadima and Jolliffe (2001) provided additional refinements and comparisons of variable selection techniques. Considering information theoretic approach, Hooda and Hooda (2002, 2006) proposed and used entropy-based criteria for variable selection under the assumption of multivariate normality using Shannon’s measure of entropy. These approaches aim to identify a subset of variables that retains most of the information present in the original dataset while minimizing redundancy. The present study reviews and illustrates the major principal variable analysis techniques for selecting representative variables from multivariate data. The article demonstrates the usefulness of principal variable analysis as an effective tool for trait reduction and information preservation in agricultural research.

2. Principal Variable Analysis Methods

Principal Variable Analysis (PVA) comprises a class of variable selection procedures designed to identify a subset of the original variables that retains most of the information contained in the complete dataset. Unlike Principal Component Analysis, which replaces the original variables with linear combinations of those variables, PVA seeks to preserve the interpretability of the data by selecting actual observed variables. The basic objective is to obtain a reduced set of variables that adequately reproduces the variability and correlation structure of the original multivariate data while minimizing information loss. Several approaches to principal variable selection have been proposed in the literature. McCabe (1984) introduced the concept of principal variables based on maximizing the proportion of total variation explained by the selected subset. Krzanowski (1987) extended this approach by developing methods that identify variables capable of reproducing the principal component structure of the original data. Alternative criteria based on covariance structure, information loss, and entropy measures have also been suggested. In general, these methods rank variables according to their contribution to the overall information content of the dataset and iteratively eliminate redundant variables. The resulting subset of principal variables provides a parsimonious representation of the original data and facilitates efficient data collection, interpretation, and subsequent statistical analyses.

2.1 Jolliffe’s PCA-Based Variable Selection Methods

The four PCA-based variable selection procedures proposed by Jolliffe (1972) are commonly referred to as the B1, B2, B3, and B4 methods. The B1, B2, and B3 methods are based on backward elimination, whereas the B4 method follows a forward selection strategy by retaining variables that exhibit high absolute loadings on the first few principal components. The underlying principle of these methods is that variables making substantial contributions to specific principal components should either be retained or discarded depending on the amount of information represented by those components.

The rationale behind the B1 method is that small eigenvalues correspond to near-constant relationships among subsets of variables. Consequently, little information is lost when variables involved in such relationships are removed. The B1 procedure is computationally intensive because it requires repeated principal component analyses following each variable deletion. In contrast, the remaining three methods are computationally more efficient, as they require only a single PCA of the complete dataset.

The B2 method is based on the loadings associated with the smallest ((p-q)) eigenvalues, where (p) is the total number of variables and (q) is the number of variables to be retained. Beginning with the component corresponding to the smallest eigenvalue, the variable having the largest absolute loading and not previously associated with any other component is identified. This process is repeated for each of the ((p-q)) components, and the associated variables are subsequently discarded.

The B3 method also utilizes the components corresponding to the smallest (p-q) eigenvalues obtained from a PCA of all p variables. For each variable, the sum of squared loadings across these (p-q) components is calculated. The (p-q) variables with the largest sums are considered to contribute most strongly to the low-variance components and are therefore eliminated.

In contrast, the B4 method focuses on the (q) principal components corresponding to the largest eigenvalues. Starting with the first principal component, one variable having the largest absolute loading is associated with each component. The (q) variables thus identified are retained, while the remaining (p-q) variables are discarded. Since B4 emphasizes variables that contribute most to the major sources of variation in the data, it provides a direct and intuitive forward-selection approach to variable reduction.

2.2 McCabe’s Generalized Variance Criterion

Let X be an n×p data matrix with n observations on a p-dimensional measurement vector X. The data matrix X is decomposed into an nq submatrix X1 and an n(p-q) sub matrix X2 (1 ≤ q ≤ p). Denote the covariance matrix of the X by

S = .

Where S11 denote the covariance matrix of the selected variables and S22 that of the redundant variables and S12 = S´21. We wish to select a subset of q variables, which best represents the original p variables then Writing the generalized variance |S| as:

S = |S11|. |S22|-S21S-111S12| = |S11| . | S22.1|

Here, S22.1, denote the conditional covariance matrix of discarded variables given the selected variables. Then |S11| contains the variability retained by the subset of the selected variables, while, | S22.1| contains the variability left over in the discarded variables subset given the selected subset. Hence, the best subset of variables is one, which maximizes |S11| or equivalently minimizes |S22.1|.

2.3 Hooda’s Method Based on Information Theory

Let H(X) and H(Y) be Shannon’s entropies for the distributions of X and Y respectively where, Y = AqX, (q≤ p). Then, information loss is given by the difference in Shannon’s entropies of the two distributions as:

Information loss = H(X) – H(Y)

The information loss is nonnegative and attains the lower bound if and only if Aq is non-singular. For reduction in dimensions Aq must be singular and according to the entropy criterion (Hooda and Hooda, 2006), we find Aq for which information loss is minimum or equivalently, the entropy of Y = (Y1,Y2,…, Yq) is maximum for a given value of q.

Under the assumption of multivariate normality, X ~ Np (μ, ) and so

Y ~ Np ( Aq μ, ). Shannon’s entropies of the distribution of X and Y are respectively given by:

and

Under the assumption of multivariate normality, H(Y) can be computed for all possible combinations of p variables taken q at a time. The subset with the maximum H(Y), or equivalently the minimum information loss, is considered the best subset of size q.

REFERENCES

Cadima, J., and I. T. Jolliffe. 2001. “Variable Selection and the Interpretation of Principal Subspaces.” Journal of Agricultural, Biological, and Environmental Statistics 6 (1): 62–79.

Hooda, B. K., and D. S. Hooda. 2006. “Dimension Reduction in Multivariate Analysis Using Maximum Entropy Criterion.” Journal of Statistics and Management Systems 9 (1): 175–183.

Hooda, D. S., and B. K. Hooda. 2001. “On Measurement of Stochastic Dependence in Multivariate Data.” Indian Journal of Pure and Applied Mathematics 32 (6): 801–815.

Jolliffe, I. T. 1972. “Discarding Variables in a Principal Component Analysis. I: Artificial Data.” Applied Statistics 21: 160–173.

Jolliffe, I. T. 1973. “Discarding Variables in a Principal Component Analysis. II: Real Data.” Applied Statistics 22: 21–31.

Jolliffe, I. T. 2002. Principal Component Analysis. 2nd ed. New York: Springer-Verlag.

Krzanowski, W. J. 1987. “Selection of Variables to Preserve Multivariate Data Structure, Using Principal Components.” Applied Statistics 36: 22–33.

McCabe, G. P. 1984. “Principal Variables.” Technometrics 26 (2): 137–144.