Random Forest Explained: How It Works and a Free Online Tool

1. What it is

A random forest is an ensemble of many decision trees. Each tree is grown on a bootstrap sample of the rows and, at every split, considers only a random subset of the predictors. For classification the forest takes a majority vote; for regression it averages the trees’ predictions.

Averaging many diverse trees cuts the variance of a single tree, so the forest predicts better and overfits less, with almost no tuning.

2. How the random forest algorithm works (in machine learning)

1. Bootstrap: draw B random samples of the rows, with replacement.
2. Grow a tree on each sample. At every split, pick the best split among a random subset of mtry predictors (lowest Gini impurity for classification, lowest variance for regression).
3. Do not prune: let each tree grow deep, so each has low bias and high variance.
4. Aggregate: majority vote (classifier) or average (regressor). Averaging cancels much of the variance.
5. Validate with the out-of-bag rows, which each tree never saw.

The two sources of randomness, bootstrap rows and random predictors per split, make the trees different from one another, which is what makes the average work.

3. Random forest in 7 minutes: the short version

• A random forest is many decision trees voting together.
• Each tree sees a different random sample of the data and predictors.
• Classifier: the most common vote wins. Regressor: the average wins.
• OOB error gives a free accuracy estimate; variable importance shows which predictors help.
• It needs little tuning, handles non-linear effects and interactions, and rarely overfits badly.

4. Random forest classifier vs random forest regressor

5. When to use it

• Prediction of a categorical outcome (classification) or a continuous one (regression).
• Many predictors, non-linear effects or interactions you do not want to specify in advance.
• Mixed numeric and categorical predictors, with little preprocessing.
• You want variable importance as a by-product.
• You need effect sizes, coefficients or p values: use linear regression or logistic regression instead.

6. Data required

7. Assumptions and limits

• Rows are independent (no repeated measures on the same subject unless handled in the split).
• No normality, linearity or equal-variance assumptions.
• It cannot extrapolate beyond the range of the training data.
• Importance scores are biased towards predictors with many categories or levels, and are not causal effects.
• Heavily imbalanced classes need class weights or resampling.

8. Core ideas and formulas

About one third of the rows are left out of each bootstrap sample. Predicting them with the trees that never saw them gives the out-of-bag (OOB) error, a built-in validation estimate. A common default is mtry ≈ √p predictors per split for classification and p/3 for regression.

9. How to interpret each part of the output

10. A worked example

The classic iris data: 150 flowers, 3 species, 4 measurements. A forest of 500 trees (seed 42, mtry = 2) gives:

Reading it. The forest classifies about 95% of the flowers correctly on rows it never trained on. Petal length and petal width carry about 86% of the importance, so they separate the species best. Exact values change a little with the random seed, so set it and report it.

11. A solved numerical example by hand

Step 1: Gini of a split. A node has 6 cases: 4 of class A and 2 of class B. Gini = 1 − (4/6)² − (2/6)² = 1 − .444 − .111 = .444. A candidate split sends 3 A cases left (Gini = 0) and 1 A + 2 B cases right (Gini = 1 − 1/9 − 4/9 = .444). Weighted Gini = (3/6)(0) + (3/6)(.444) = .222, so the split reduces impurity by .444 − .222 = .222. The tree chooses the split with the largest reduction.

Step 2: Voting. Five trees predict A, A, B, A, B for a new case. A gets 3 of 5 votes, so the forest predicts A (estimated probability .60).

Step 3: Regression. Three trees predict 12, 14 and 13. The forest predicts (12 + 14 + 13) ÷ 3 = 13.

Step 4: OOB error. If 5 of the 150 cases are misclassified by the trees that did not train on them, OOB error = 5 ÷ 150 = 3.3%.

12. What a random forest can be used to do

• Classification: diagnosing disease from lab values, spam filtering, customer churn.
• Regression: predicting prices, yields, exam scores.
• Variable selection: ranking predictors by importance before a regression.
• Remote sensing: land-cover and crop mapping, for example classifying satellite images in Google Earth Engine with ee.Classifier.smileRandomForest().
• Imputation and outlier detection: via proximity measures and related methods.

13. How to do it in SPSS

14. How to do it in R

library(randomForest)
set.seed(42)

rf <- randomForest(Species ~ ., data = iris, ntree = 500, importance = TRUE)
print(rf)            # OOB estimate of error rate: 5.33%
importance(rf)       # MeanDecreaseAccuracy and MeanDecreaseGini
varImpPlot(rf)

# prediction on new data
predict(rf, newdata = iris[1:5, ])

# faster alternative with many rows
library(ranger)
ranger(Species ~ ., data = iris, num.trees = 500, importance = "permutation")

15. How to do it in Python

from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance

X, y = load_iris(return_X_y=True)
rf = RandomForestClassifier(n_estimators=500, oob_score=True, random_state=42)
rf.fit(X, y)

print(rf.oob_score_)                 # 0.947
print(rf.feature_importances_)       # petal length ~0.44, petal width ~0.42

# permutation importance is more reliable than the default Gini importance
pi = permutation_importance(rf, X, y, n_repeats=20, random_state=42)
print(pi.importances_mean)

# random forest regressor (sklearn)
from sklearn.ensemble import RandomForestRegressor
reg = RandomForestRegressor(n_estimators=500, oob_score=True, random_state=42)
reg.fit(X[:, :3], X[:, 3])           # predict petal width from the other three
print(reg.oob_score_)                # out-of-bag R-squared

16. How to do it on StatFas

17. APA 7 reporting

Report the number of trees, mtry, the seed, how performance was validated (OOB or a held-out test set) and the metric (accuracy, AUC, RMSE). Do not interpret importance as a causal effect.

18. Common mistakes

• Reporting training-set accuracy, which is nearly perfect, instead of OOB or test accuracy.
• Reading Gini importance as an effect size or as causal.
• Ignoring class imbalance.
• Using random k-fold splits on clustered or time-ordered data.
• Not setting or reporting the random seed.

19. When a random forest is not the right tool

• You need inference (coefficients, odds ratios, p values): use regression.
• Very small samples: simpler models are usually more stable.
• Extrapolation: use a parametric model.
• Data-reduction goal: use PCA.

Frequently Asked Questions

Run It Now

Fit a random forest and see accuracy and variable importance.

Run Random Forest Free

Related Guides

• Statistical Analysis: Types, Purpose and How to Do Them Online for Free
• Principal Component Analysis (PCA): Explained
• Structural Equation Modeling: How to Report SEM
• Pearson Correlation Coefficient: How to Interpret and Report

See the full overview of 10 core statistical analyses, browse the step-by-step tutorials or explore all free statistical tools.