{"id":920,"date":"2022-07-11T10:33:41","date_gmt":"2022-07-11T08:33:41","guid":{"rendered":"https:\/\/lorentzen.ch\/?p=920"},"modified":"2022-07-11T10:33:42","modified_gmt":"2022-07-11T08:33:42","slug":"shapviz-goes-h2o","status":"publish","type":"post","link":"https:\/\/lorentzen.ch\/index.php\/2022\/07\/11\/shapviz-goes-h2o\/","title":{"rendered":"shapviz goes H2O"},"content":{"rendered":"\n

In a recent post<\/a>, I introduced the initial version of the “shapviz” package<\/a>. Its motto: do one thing, but do it well: visualize SHAP values.<\/p>\n\n\n\n

The initial community feedback was very positive, and a couple of things have been improved in version 0.2.0. Here the main changes:<\/p>\n\n\n\n

  1. “shapviz” now works with tree-based models of the h2o<\/code><\/a> package in R. <\/li>
  2. Additionally, it wraps the shapr<\/code><\/a> package, which implements an improved version of Kernel SHAP taking into account feature dependence.<\/li>
  3. A simple interface to collapse SHAP values of dummy variables was added.<\/li>
  4. The default importance plot is now a bar plot, instead of the (slower) beeswarm plot. In later releases, the latter might be moved to a separate function sv_summary()<\/code> for consistency with other packages.<\/li>
  5. Importance plot and dependence plot now work neatly with ggplotly()<\/code>. The other plot types cannot be translated with ggplotly()<\/code> because they use geoms from outside ggplot. At least I do not know how to do this…<\/li><\/ol>\n\n\n\n

    Example<\/h3>\n\n\n\n

    Let’s build an H2O gradient boosted trees model to explain diamond prices. Then, we explain the model with our “shapviz” package. Note that H2O itself also offers some SHAP plots. “shapviz” is directly applied to the fitted H2O model. This means you don’t have to write a single superfluous line of code.<\/p>\n\n\n

    \n\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t
    R<\/div>\n\t\t\t<\/div><\/div>\n\t\t\t<\/div>\n\t\t\t
    \n\n
    library(shapviz)\nlibrary(tidyverse)\nlibrary(h2o)\n\nh2o.init()\n\nset.seed(1)\n\n# Get rid of that darn ordinals\nord <- c(\"clarity\", \"cut\", \"color\")\ndiamonds[, ord] <- lapply(diamonds[, ord], factor, ordered = FALSE)\n\n# Minimally tuned GBM with 260 trees, determined by early-stopping with CV\ndia_h2o <- as.h2o(diamonds)\nfit <- h2o.gbm(\n  c(\"carat\", \"clarity\", \"color\", \"cut\"),\n  y = \"price\",\n  training_frame = dia_h2o,\n  nfolds = 5,\n  learn_rate = 0.05,\n  max_depth = 4,\n  ntrees = 10000,\n  stopping_rounds = 10,\n  score_each_iteration = TRUE\n)\nfit\n\n# SHAP analysis on about 2000 diamonds\nX_small <- diamonds %>%\n  filter(carat <= 2.5) %>%\n  sample_n(2000) %>%\n  as.h2o()\n\nshp <- shapviz(fit, X_pred = X_small)\n\nsv_importance(shp, show_numbers = TRUE)\nsv_importance(shp, show_numbers = TRUE, kind = \"bee\")\nsv_dependence(shp, \"color\", \"auto\", alpha = 0.5)\nsv_force(shp, row_id = 1)\nsv_waterfall(shp, row_id = 1)<\/pre><\/div>\n\n<\/div><\/div>\n\t\t<\/div>\n\n\n
    Summary and importance plots<\/h3>\n\n\n\n
    The SHAP importance and SHAP summary plots clearly show that carat is the  most important variable. On average, it impacts the prediction by 3247 USD. The effect of “cut” is much smaller. Its impact on the predictions, on average, is plus or minus 112 USD. <\/p>\n\n\n\n
    \"\"
    SHAP summary plot<\/figcaption><\/figure>\n\n\n\n
    \"\"
    SHAP importance plot<\/figcaption><\/figure>\n\n\n\n
    SHAP dependence plot<\/h3>\n\n\n\n
    The SHAP dependence plot shows the effect of “color” on the prediction: The better the color (close to “D”), the higher the price. Using a correlation based heuristic, the plot selected carat on the color scale to show that the color effect is hightly influenced by carat in the sense that the impact of color increases with larger diamond weight. This clearly makes sense!<\/p>\n\n\n\n
    \"\"
    Dependence plot for “color”<\/figcaption><\/figure>\n\n\n\n
    Waterfall and force plot<\/h2>\n\n\n\n
    Finally, the waterfall and force plots show how a single prediction is decomposed into contributions from each feature. While this does not tell much about the model itself, it might be helpful to explain what SHAP values are and to debug strange predictions.<\/p>\n\n\n\n
    \"\"
    Waterfall plot<\/figcaption><\/figure>\n\n\n\n
    \"\"
    Force plot<\/figcaption><\/figure>\n\n\n\n
    Short wrap-up<\/h3>\n\n\n\n