{"id":638,"date":"2021-10-21T16:38:26","date_gmt":"2021-10-21T14:38:26","guid":{"rendered":"https:\/\/lorentzen.ch\/?p=638"},"modified":"2021-10-21T16:38:27","modified_gmt":"2021-10-21T14:38:27","slug":"personal-highlights-of-scikit-learn-1-0","status":"publish","type":"post","link":"https:\/\/lorentzen.ch\/index.php\/2021\/10\/21\/personal-highlights-of-scikit-learn-1-0\/","title":{"rendered":"Personal Highlights of Scikit-Learn 1.0"},"content":{"rendered":"\n

Yes! After more than 10 years, scikit-learn<\/a> released its 1.0 version on 24 September 2021. In this post, I’d like to point out some personal highlights apart from the release highlights<\/a>.<\/p>\n\n\n\n

1. Feature Names<\/h2>\n\n\n\n

This one is listed in the release highlights, but deserves to be mentioned again.<\/p>\n\n\n\n

from sklearn.compose import ColumnTransformer\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import OneHotEncoder, StandardScaler\nimport pandas as pd\n\ndf = pd.DataFrame({\n    "pet": ["dog", "cat", "fish"],\n    "age": [3, 7, 1],\n    "noise": [-99, pd.NA, 1e-10],\n    "target": [1, 0, 1],\n})\ny = df.pop("target")\nX = df\n\npreprocessor = ColumnTransformer(\n    [\n        ("numerical", StandardScaler(), ["age"]),\n        ("categorical", OneHotEncoder(), ["pet"]),\n    ],\n    verbose_feature_names_out=False,\n    remainder="drop",\n)\n\npipe = make_pipeline(preprocessor, LogisticRegression())\npipe.fit(X, y)\npipe[:-1].get_feature_names_out()<\/pre><\/div>\n\n\n\n
array(['age', 'pet_cat', 'pet_dog', 'pet_fish'], dtype=object)<\/code><\/pre>\n\n\n\n
This is not yet available for all estimators and transformers, but it is a big step towards SLEP007<\/a>.<\/p>\n\n\n\n
2. ColumnTransformer allows changed order of columns<\/h2>\n\n\n\n
Before this release, ColumnTransformer<\/em><\/a> recorded the order of the columns of a dataframe during the fit<\/em> method and required that a dataframe X<\/em> passed to transform<\/em> had the exact same columns and in the exact same order.<\/p>\n\n\n\n
This was a big pain point in productive settings because fit<\/em> and predict<\/em> of a model pipeline, both calling transform<\/em>, often get data from different sources, and, for instance, SQL does not care about the order of columns. On top, remainder=”drop”<\/em> forced you to have also all dropped columns in transform<\/em>. This contradicted at least my modelling workflow as I often specify all meaningful features explicitly and drop the rest by the remainder<\/em> option. This then led to unwanted surprises when applying predict<\/em> to new data in the end. It might also happen, that one forgets to remove the target variable from the training X<\/em> and relies on the drop option. Usually, the application of the predictive model pipeline is on new data without the target variable. The error in this case, however, might be considered a good thing.<\/p>\n\n\n\n
With pull request (PR) #19263<\/a>, the ColumnTransformer<\/em> only cares about the presence and names of the columns, not about their order. With remainder=”drop”<\/em>, it only cares about the specified columns and ignores all other columns, even no matter if the dropped ones are different in fit and transform. Note that this only works with pandas dataframes as input (or an object that quacks alike).<\/p>\n\n\n\n
df_new = pd.DataFrame({\n    "age": [1, 9, 3],\n    "another_noise": [pd.NA, -99, 1e-10],\n    "pet": ["cat", "dog", "fish"],\n})\npipe.predict(df_new)<\/pre><\/div>\n\n\n\n
You find these little code snippets as notebook at the usual place: https:\/\/github.com\/lorentzenchr\/notebooks\/blob\/master\/blogposts\/2021-10-21%20scikit-learn_v1_release_highlights.ipynb<\/a>.<\/p>\n\n\n\n
3. Poisson criterion for random forests<\/h2>\n\n\n\n
Scikit-learn v0.24 shipped with the new option criterion=”poisson”<\/em> for DecisionTreeRegressor<\/a> to split nodes based on the reduction of Poisson deviance. Version 1.0 passed this option further to the RandomForestRegressor<\/a> in PR #19836<\/a>. Random forests are often used models and valued for their ease of use. We even like to write blog posts about them:<\/p>\n\n\n\n