11. Tune hyperparameters with a plan
Tune learning rates, tree depth, regularization strength, batch size, neighbors, and other controls with grid search, random search, and Bayesian search. You will run tuning as an experiment, not as trial-and-error guessing.