Category: Linear Algebra
-
Histograms, Gradient Boosted Trees, Group-By Queries and One-Hot Encoding
This post shows how filling histograms can be done in very different ways thereby connecting very different areas: from gradient boosted trees to SQL queries to one-hot encoding. Let’s jump into it! Modern gradient boosted trees (GBT) like LightGBM, XGBoost and the HistGradientBoostingRegressor of scikit-learn all use two techniques on top of standard gradient boosting:…
-
From Least Squares Benchmarks to the Marchenko–Pastur Distribution
In this blog post, I tell the story how I learned about a theorem for random matrices of the two Ukrainian🇺🇦 mathematicians Vladimir Marchenko and Leonid Pastur. It all started with benchmarking least squares solvers in scipy. Setting the Stage for Least Squares Solvers Least squares starts with a matrix and a vector and one…
-
Least Squares Minimal Norm Solution
Beyond the least squares TLDR: an illustration of the minimal norm solution in Python.