ML Algorithm Deep-Dive Questions
Part of the Data Science Roadmap
In short
'How does a random forest actually work?' Anyone can call .fit(). Interviewers want to see you understand what happens inside, when it breaks, and why you'd pick it over something else.
'Explain how random forests work' or 'when would you use logistic regression over a decision tree?' — testing genuine understanding of the algorithms from Phases 6-9, not just that you've called the library function.
Think of it like this
The lab viva
In a practical viva, the examiner doesn't care that you saw the colour change. She asks why it changed, what happens with more acid, and what could go wrong. ML deep-dives work the same way: they test the mechanism, not the button you pressed.
Watch it happen
Explaining a decision tree, then a forest
1/4A tree picks the question that best separates defaulters from repayers, measured by Gini impurity or entropy.
The big ideas
What you need to know
Intuition first
Start with a one-line picture, then go to maths only if they push. Clear beats clever.
Know where it breaks
For each algorithm, know its assumptions and one case where it fails badly.
Talk in trade-offs
Logistic regression is simple to explain; boosted trees often score higher but need more tuning. Say when each wins.
Bias and variance
Most 'why does X help?' questions come down to reducing bias, variance, or both.
See it in code
Walk through the code
1from sklearn.datasets import load_breast_cancer2from sklearn.model_selection import cross_val_score3from sklearn.tree import DecisionTreeClassifier4from sklearn.ensemble import RandomForestClassifier5 6X, y = load_breast_cancer(return_X_y=True)7 8tree = DecisionTreeClassifier(random_state=0)9forest = RandomForestClassifier(n_estimators=300, max_features="sqrt", random_state=0)10 11for name, m in [("tree", tree), ("forest", forest)]:12 scores = cross_val_score(m, X, y, cv=5)13 print(name, scores.mean().round(3), scores.std().round(3))1/3One fully grown tree: fits training data closely, so its scores jump around between folds.
Run this before your interview and explain the result out loud. Being able to show the idea in code is a strong signal.
Save yourself the pain
Common mistakes
Oops
Reciting a textbook definition and stopping there.
Do this instead
Explain the idea, give a small example, then mention one strength and one weakness.
Oops
Saying 'random forests can't overfit'.
Do this instead
Adding trees doesn't make it worse, but deep trees on noisy data can still overfit. Tune depth and leaf size.
Your mission
Three 60-second explanations
Quick check
Why does a random forest usually generalise better than a single deep decision tree?
Go deeper
- 1Review Phases 6-9 and practice explaining each major algorithm without notes.
- 2Prepare to discuss trade-offs between algorithms for a given problem type.
- 3Practice drawing a simple diagram (e.g. a decision tree split) while explaining out loud.
More ml & statistics at this stop
Statistics & Probability Questions
Model Evaluation Questions
Stuck on this topic? Ask an Insider
Get 1:1 guidance from people who've walked this exact path — on the InsideEdge app.
