Principal Component Analysis (PCA): What It Is, How It Works and a Free Online PCA Tool
Principal component analysis (PCA) is a statistical technique for dimensionality reduction. It rewrites many correlated variables as a smaller set of uncorrelated principal components that keep as much of the original variation as possible. In Indonesian, PCA adalah “analisis komponen utama”, with the same meaning. This guide explains PCA analysis step by step, with the formula, an example, the PCA plots, how to interpret them, PCA vs factor analysis, R and Python code, and where to run a PCA analysis online for free.
1. What is PCA analysis?
Imagine a questionnaire with 30 items or a dataset with 50 gene-expression measures. Many of these variables move together, so they carry overlapping information. PCA finds new axes, the principal components (PCs), that are linear combinations of the original variables. The first component (PC1) captures the most variance, PC2 captures the most of what remains while being uncorrelated with PC1, and so on. You then keep only the first few components.
2. Why use PCA? Dimensionality reduction and machine learning
- Dimensionality reduction: compress many variables into 2–5 components with little loss of information.
- Principal component analysis in machine learning: remove redundant features, speed up models, reduce overfitting and visualize high-dimensional data in 2D or 3D.
- Multicollinearity: components are uncorrelated, so they can replace highly correlated predictors in regression.
- Exploration: reveal clusters, outliers and the main patterns in a dataset.
- Scale development: screen questionnaire items before a confirmatory factor analysis (see the SEM guide).
3. How PCA works: the formula
| Step | Operation | Formula |
|---|---|---|
| 1. Standardize | Centre each variable (and scale it if units differ). | z = (x − μ) / σ |
| 2. Covariance or correlation matrix | Measure how variables vary together. | C = ZᵀZ / (n − 1) |
| 3. Eigendecomposition | Find eigenvectors (directions) and eigenvalues (variance). | C v = λ v |
| 4. Rank components | Sort by eigenvalue; share of variance explained. | λᵢ / Σλ |
| 5. Scores | Project the data onto the kept components. | T = Z V |
| 6. Loadings | Correlation between a variable and a component. | loading = vᵢ √λᵢ |
Correlation or covariance? Use the correlation matrix (standardized data) when variables have different units, for example age in years and income in dollars. Use the covariance matrix when all variables share one unit and their raw variance matters, such as spectra or gene expression.
4. PCA example
A researcher measures six survey items on N = 300 participants. Before running PCA, check sampling adequacy: KMO = .82 (above .60 is acceptable) and Bartlett’s test χ²(15) = 612.4, p < .001, so the items are correlated enough.
| Component | Eigenvalue | % variance | Cumulative % |
|---|---|---|---|
| PC1 | 2.85 | 47.5 | 47.5 |
| PC2 | 1.52 | 25.3 | 72.8 |
| PC3 | 0.68 | 11.3 | 84.2 |
| PC4 | 0.44 | 7.3 | 91.5 |
| PC5 | 0.31 | 5.2 | 96.7 |
| PC6 | 0.20 | 3.3 | 100.0 |
Two eigenvalues exceed 1 and together explain 72.8% of the variance, so two components are retained.
| Variable | PC1 | PC2 | Communality |
|---|---|---|---|
| Item 1 | .82 | .10 | .68 |
| Item 2 | .78 | .15 | .63 |
| Item 3 | .74 | .08 | .55 |
| Item 4 | .12 | .85 | .74 |
| Item 5 | .09 | .80 | .65 |
| Item 6 | −.10 | .72 | .53 |
5. PCA plots
The PCA plot comes in three main forms. The scree plot (Figure 2) shows eigenvalues to decide how many components to keep. The score plot maps each observation on PC1 and PC2 to reveal clusters and outliers. The loading plot or biplot (Figure 3) shows how variables relate to the components.
6. PCA analysis interpretation
| Question | Guideline |
|---|---|
| How many components? | Eigenvalue > 1 (Kaiser, 1960), the scree elbow, parallel analysis (Horn, 1965; Hayton et al., 2004), or 60–80% cumulative variance. |
| Is PCA suitable? | KMO ≥ .60, significant Bartlett’s test, most correlations above |.30|. |
| What does a loading mean? | ≥ .40 is meaningful; ≥ .70 is strong. Cross-loadings above .30 on two components need attention. |
| How well is a variable represented? | Communality: share of its variance explained by the kept components (aim for ≥ .40). |
| What do scores show? | Position of each case on a component; distant points are potential outliers. |
| Naming components | Name each PC after the variables loading highly on it, and note that this is an interpretation. |
In the example, Items 1–3 define PC1 and Items 4–6 define PC2, a clean two-component structure.
7. PCA vs factor analysis
| PCA | Factor analysis | |
|---|---|---|
| Goal | Data reduction and summary | Find latent constructs |
| Variance used | Total variance | Shared (common) variance |
| Measurement error | Not modeled | Separated as unique variance |
| Direction | Components are built from variables | Variables are caused by factors |
| Typical use | Preprocessing, visualization, machine learning | Theory, scale validation |
For a detailed discussion of when to choose each, see Fabrigar et al. (1999). Use PCA when you only need fewer variables; use factor analysis when you believe unobserved constructs cause the item responses.
8. PCA in R and Python
PCA in R (base R, see the prcomp documentation):
pca <- prcomp(df, scale. = TRUE) # correlation-based PCA
summary(pca) # variance explained
screeplot(pca, type = "lines") # scree plot
biplot(pca) # PCA plotPCA in Python (scikit-learn, see the PCA documentation):
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
X = StandardScaler().fit_transform(df)
pca = PCA(n_components=2).fit(X)
print(pca.explained_variance_ratio_)
scores = pca.transform(X)9. Free PCA software and online tools
| Option | Type | Best for |
|---|---|---|
| StatFas PCA calculator | Free online, no installation | Quick PCA with APA7-ready results |
R (prcomp, FactoMineR) | Free download | Full control and reproducible scripts |
| Python (scikit-learn) | Free download | Machine-learning pipelines |
| JASP, jamovi | Free download, point-and-click | SPSS-style menus |
| Learnbin Lab PCA tool | Free online | Interactive 2D and 3D score plots |
For a PCA analysis online free workflow, paste your data into the StatFas PCA calculator, choose the variables, and read the eigenvalues, loadings and plots. Want a PDF copy of this guide? Use your browser’s Print option and choose “Save as PDF”.
10. Reporting PCA in APA7
Also state the correlation or covariance matrix used, the retention rule, the rotation (or none) and the software.
11. Principal component analysis: a review and recent developments
PCA goes back to Pearson (1901) and Hotelling (1933). Jolliffe and Cadima (2016) reviewed how it has been extended, and Abdi and Williams (2010) give a clear technical tutorial. For a short, accessible introduction, read Ringnér (2008). Modern variants include sparse PCA (fewer non-zero loadings for easier reading), kernel PCA (non-linear structure), robust PCA (resistant to outliers), incremental PCA (very large data) and PCA for compositional or functional data. Classic PCA remains the right first step for most research datasets.
12. FAQ
What is PCA analysis?
A method that turns many correlated variables into a few uncorrelated components that keep most of the variance.
How many components should I keep?
Combine the scree plot, eigenvalues above 1, parallel analysis and 60–80% cumulative variance.
Is PCA the same as factor analysis?
No. PCA summarizes total variance; factor analysis models shared variance and error (Table 5).
Is there a free online PCA tool?
Yes: the StatFas calculator, plus R, Python, JASP and jamovi (Table 6).