Gradient Boosting (XGBoost, LightGBM)

Summary

An ensemble method that builds trees sequentially, each one correcting the errors of the previous — XGBoost and LightGBM are the industry-standard implementations, often winning tabular-data competitions.

How to Learn This

1Fit an XGBoost or LightGBM model and compare it to your random forest baseline.
2Learn the key hyperparameters: learning rate, number of estimators, max depth.
3Understand at a high level why boosting often outperforms bagging (random forests) on tabular data.

Random Forests

K-Nearest Neighbors

More topics in Machine Learning Fundamentals

Supervised vs Unsupervised vs Reinforcement Learning Train / Test / Validation Split Overfitting & Underfitting Bias-Variance Tradeoff Cross-Validation Linear Regression Logistic Regression Decision Trees Random Forests K-Nearest Neighbors Support Vector Machines Naive Bayes

Stuck on this topic? Ask an Insider

Get 1:1 guidance from people who've walked this exact path — free on the InsideEdge app.

Download