{"id":2096,"date":"2026-09-21T05:00:00","date_gmt":"2026-09-21T03:00:00","guid":{"rendered":"https:\/\/lorentzen.ch\/?p=2096"},"modified":"2026-09-20T00:18:54","modified_gmt":"2026-09-19T22:18:54","slug":"when-chain-ladder-factors-misbehave-smoothing-reserves-with-polars-and-whittaker-henderson","status":"publish","type":"post","link":"https:\/\/lorentzen.ch\/index.php\/2026\/09\/21\/when-chain-ladder-factors-misbehave-smoothing-reserves-with-polars-and-whittaker-henderson\/","title":{"rendered":"When Chain-Ladder Factors Misbehave: Smoothing Reserves with Polars and Whittaker-Henderson"},"content":{"rendered":"<div style=\"background-color: #d9edf7; color: #31708f; border-left-color: #31708f; \" class=\"ub-styled-box ub-notification-box wp-block-ub-styled-box\" id=\"ub-styled-box-bef7e861-d08d-4714-9cba-9fab98f5f233\">\n<p id=\"ub-styled-box-notification-content-\"><strong>TD;DR<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>We run a full Chain-Ladder reserving calculation for 233 insurers in a handful of <code><strong><a href=\"https:\/\/pola.rs\/\">polars<\/a><\/strong><\/code> expressions, cross-checked against R&#8217;s <code><strong><a href=\"https:\/\/cran.r-project.org\/package=ChainLadder\">ChainLadder<\/a><\/strong><\/code> package.<\/li>\n\n\n\n<li>One insurer&#8217;s incurred development pattern turns out non-monotonic \u2014 the kind of pattern that trips up standard parametric curve fits.<\/li>\n\n\n\n<li><strong>Whittaker-Henderson<\/strong> smoothing, newly available as <code><a href=\"https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.signal.whittaker_henderson.html\">scipy.signal.whittaker_henderson<\/a><\/code>, handles it gracefully and shifts the estimated reserve by about 3%.<br><\/li>\n<\/ul>\n\n\n<\/div>\n\n\n<h2 class=\"wp-block-heading\" id=\"0-a-reserving-problem-solved-without-excel-\"><strong>A Reserving Problem, Solved Without Excel<\/strong><\/h2>\n\n\n\n<p>Ask a reserving actuary how they run a Chain-Ladder and you&#8217;ll usually hear &#8220;Excel&#8221; or the name of a pricey specialized tool. It turns out a modern dataframe library handles it just as well \u2014 in a few lines, for hundreds of companies at once.<\/p>\n\n\n\n<p>We use the <a href=\"https:\/\/www.casact.org\/publications-research\/research\/research-resources\/loss-reserving-data-pulled-naic-schedule-p\">CAS loss reserving data<\/a>, specifically the <em>other liability<\/em> line of business (LoB): 233 US insurers (&#8220;GRNAME&#8221;), 10 accident years (1998\u20132007), paid and incurred losses at every development lag (1-10). We treat 2007 as our reporting year, i.e. we simulate a year-end closing.<\/p>\n\n\n\n<p>After reading in the data, we prepare it a bit<\/p>\n\n\n\n<div class=\"wp-block-codemirror-blocks-code-block code-block\"><pre class=\"CodeMirror\" data-setting=\"{&quot;showPanel&quot;:true,&quot;languageLabel&quot;:&quot;language&quot;,&quot;fullScreenButton&quot;:true,&quot;copyButton&quot;:true,&quot;mode&quot;:&quot;python&quot;,&quot;mime&quot;:&quot;text\/x-python&quot;,&quot;theme&quot;:&quot;material&quot;,&quot;lineNumbers&quot;:false,&quot;styleActiveLine&quot;:false,&quot;lineWrapping&quot;:false,&quot;readOnly&quot;:true,&quot;fileName&quot;:&quot;Python&quot;,&quot;language&quot;:&quot;Python&quot;,&quot;maxHeight&quot;:&quot;400px&quot;,&quot;modeName&quot;:&quot;python&quot;}\">import polars as pl\n\n# Other Liability Data Set, December 2025\ndf = pl.read_csv(\n    &quot;https:\/\/www.casact.org\/sites\/default\/files\/2026-03\/othliab_pos_98-07.csv&quot;\n)\ndf_triangle = (\n    df\n    .group_by(&quot;GRNAME&quot;, &quot;AccidentYear&quot;, &quot;DevelopmentLag&quot;)\n    .agg(pl.sum(&quot;IncurredLosses&quot;).alias(&quot;Incurred&quot;), pl.sum(&quot;CumPaidLoss&quot;).alias(&quot;Paid&quot;))\n    .sort(&quot;GRNAME&quot;, &quot;AccidentYear&quot;, &quot;DevelopmentLag&quot;)\n)<\/pre><\/div>\n\n\n\n<p>This is still the &#8220;full triangle&#8221;, i.e. all 10 development lags for all 10 accident years. Note that <code>df_triangle<\/code> is not in a triangle format but in a long data format, see <a href=\"https:\/\/doi.org\/10.18637\/jss.v059.i10\">Tidy Data by Hadley Wickham<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"1-chain-ladder-in-a-few-lines-of-polars-\"><strong>Chain-Ladder in a Few Lines of Polars<\/strong><\/h2>\n\n\n\n<p>The mechanics are the textbook ones: aggregate claims by accident year i and development lag j (starts at 1, not 0), keep only the upper-left (already observed) triangle, and compute CL development factors<\/p>\n\n\n\n<div class=\"wp-block-katex-display-block katex-eq\" data-katex-display=\"true\"><pre>f^{CL}_j = \\frac{\\sum_{i=1998}^{2008-j} C_{i,j}}{\\sum_{i=1998}^{2008-j} C_{i,j-1}}<\/pre><\/div>\n\n\n\n<p>separately for every company. In polars this is just a <code>group_by<\/code> plus a couple of window (<code>.over(...)<\/code>) expressions \u2014 no manual loops over triangles required:<\/p>\n\n\n\n<div class=\"wp-block-codemirror-blocks-code-block code-block\"><pre class=\"CodeMirror\" data-setting=\"{&quot;showPanel&quot;:true,&quot;languageLabel&quot;:&quot;language&quot;,&quot;fullScreenButton&quot;:true,&quot;copyButton&quot;:true,&quot;mode&quot;:&quot;python&quot;,&quot;mime&quot;:&quot;text\/x-python&quot;,&quot;theme&quot;:&quot;material&quot;,&quot;lineNumbers&quot;:false,&quot;styleActiveLine&quot;:false,&quot;lineWrapping&quot;:false,&quot;readOnly&quot;:true,&quot;fileName&quot;:&quot;Python&quot;,&quot;language&quot;:&quot;Python&quot;,&quot;maxHeight&quot;:&quot;400px&quot;,&quot;modeName&quot;:&quot;python&quot;}\">def df_2_cl_factors(df, group_by=None):\n    g = [] if group_by is None else group_by\n\n    cl_factors = (\n        df\n        .group_by(g + [&quot;AccidentYear&quot;, &quot;DevelopmentLag&quot;])\n        .agg(pl.sum(&quot;Paid&quot;), pl.sum(&quot;Incurred&quot;))\n        # Filter upper left triangle\n        .filter(REPORTING_YEAR &gt;= pl.col(&quot;AccidentYear&quot;) + pl.col(&quot;DevelopmentLag&quot;) - 1)\n        .sort(g + [&quot;AccidentYear&quot;, &quot;DevelopmentLag&quot;])\n        .with_columns(\n            PreviousPaid=pl.col(&quot;Paid&quot;).shift(1).over(g + [&quot;AccidentYear&quot;]),\n            PreviousIncurred=pl.col(&quot;Incurred&quot;).shift(1).over(g + [&quot;AccidentYear&quot;]),\n        )\n        # Calculate volume-weighted factors per lag\n        .group_by(g + [&quot;DevelopmentLag&quot;])\n        .agg(\n            number_of_years=pl.len(),\n            Paid=pl.col(&quot;Paid&quot;).sum(),\n            PreviousPaid=pl.col(&quot;PreviousPaid&quot;).sum(),\n            Incurred=pl.col(&quot;Incurred&quot;).sum(),\n            PreviousIncurred=pl.col(&quot;PreviousIncurred&quot;).sum(),\n        )\n        .with_columns(\n            f_CL_paid=pl.when(pl.col(&quot;PreviousPaid&quot;) == 0).then(1).otherwise(pl.col(&quot;Paid&quot;) \/ pl.col(&quot;PreviousPaid&quot;)),\n            f_CL_inc=pl.when(pl.col(&quot;PreviousIncurred&quot;) == 0).then(1).otherwise(pl.col(&quot;Incurred&quot;) \/ pl.col(&quot;PreviousIncurred&quot;)),\n        )\n        .sort(g + [&quot;DevelopmentLag&quot;])\n    )\n    return cl_factors\n\ncl_factors = df_2_cl_factors(df_triangle, group_by=[&quot;GRNAME&quot;])<\/pre><\/div>\n\n\n\n<p>We validated the result against R&#8217;s <code>ChainLadder<\/code> package for one insurer, Grinnell Mut Grp, and the incurred factors matched.<\/p>\n\n\n\n<p>Next, we calculate the Chain-Ladder development factors, again for both paid and incurred, this time separately for each company. Note that we take care to only account for the <em>upper left<\/em> triangle, which is what&#8217;s usually available in practice.<\/p>\n\n\n\n<div class=\"wp-block-codemirror-blocks-code-block code-block\"><pre class=\"CodeMirror\" data-setting=\"{&quot;showPanel&quot;:true,&quot;languageLabel&quot;:&quot;language&quot;,&quot;fullScreenButton&quot;:true,&quot;copyButton&quot;:true,&quot;mode&quot;:&quot;python&quot;,&quot;mime&quot;:&quot;text\/x-python&quot;,&quot;theme&quot;:&quot;material&quot;,&quot;lineNumbers&quot;:false,&quot;styleActiveLine&quot;:false,&quot;lineWrapping&quot;:false,&quot;readOnly&quot;:true,&quot;fileName&quot;:&quot;Python&quot;,&quot;language&quot;:&quot;Python&quot;,&quot;maxHeight&quot;:&quot;400px&quot;,&quot;modeName&quot;:&quot;python&quot;}\">def df_2_cl_factors(df, group_by=None):\n    g = [] if group_by is None else group_by\n\n    cl_factors = (\n        df\n        .group_by(g + [&quot;AccidentYear&quot;, &quot;DevelopmentLag&quot;])\n        .agg(pl.sum(&quot;Paid&quot;), pl.sum(&quot;Incurred&quot;))\n        # Filter upper left triangle\n        .filter(REPORTING_YEAR &gt;= pl.col(&quot;AccidentYear&quot;) + pl.col(&quot;DevelopmentLag&quot;) - 1)\n        .sort(g + [&quot;AccidentYear&quot;, &quot;DevelopmentLag&quot;])\n        .with_columns(\n            PreviousPaid=pl.col(&quot;Paid&quot;).shift(1).over(g + [&quot;AccidentYear&quot;]),\n            PreviousIncurred=pl.col(&quot;Incurred&quot;).shift(1).over(g + [&quot;AccidentYear&quot;]),\n        )\n        # Calculate volume-weighted factors per lag\n        .group_by(g + [&quot;DevelopmentLag&quot;])\n        .agg(\n            number_of_years=pl.len(),\n            Paid=pl.col(&quot;Paid&quot;).sum(),\n            PreviousPaid=pl.col(&quot;PreviousPaid&quot;).sum(),\n            Incurred=pl.col(&quot;Incurred&quot;).sum(),\n            PreviousIncurred=pl.col(&quot;PreviousIncurred&quot;).sum(),\n        )\n        .with_columns(\n            f_CL_paid=pl.when(pl.col(&quot;PreviousPaid&quot;) == 0).then(1).otherwise(pl.col(&quot;Paid&quot;) \/ pl.col(&quot;PreviousPaid&quot;)),\n            f_CL_inc=pl.when(pl.col(&quot;PreviousIncurred&quot;) == 0).then(1).otherwise(pl.col(&quot;Incurred&quot;) \/ pl.col(&quot;PreviousIncurred&quot;)),\n        )\n        .sort(g + [&quot;DevelopmentLag&quot;])\n    )\n    return cl_factors\n\ncl_factors = df_2_cl_factors(df_triangle, group_by=[&quot;GRNAME&quot;])\ncl_factors.filter(pl.col(&quot;GRNAME&quot;) == &quot;Grinnell Mut Grp&quot;)<\/pre><\/div>\n\n\n\n<div class=\"wp-block-codemirror-blocks-code-block code-block\"><pre class=\"CodeMirror\" data-setting=\"{&quot;showPanel&quot;:true,&quot;languageLabel&quot;:&quot;language&quot;,&quot;fullScreenButton&quot;:true,&quot;copyButton&quot;:true,&quot;mode&quot;:&quot;null&quot;,&quot;mime&quot;:&quot;text\/plain&quot;,&quot;theme&quot;:&quot;material&quot;,&quot;lineNumbers&quot;:false,&quot;styleActiveLine&quot;:false,&quot;lineWrapping&quot;:false,&quot;readOnly&quot;:true,&quot;fileName&quot;:&quot;Plain Text&quot;,&quot;language&quot;:&quot;Plain Text&quot;,&quot;maxHeight&quot;:&quot;400px&quot;,&quot;modeName&quot;:&quot;text&quot;}\">\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 GRNAME     \u2506 Developmen \u2506 number_of_ \u2506 Paid   \u2506 \u2026 \u2506 Incurred \u2506 PreviousIn \u2506 f_CL_paid \u2506 f_CL_inc \u2502\n\u2502 ---        \u2506 tLag       \u2506 years      \u2506 ---    \u2506   \u2506 ---      \u2506 curred     \u2506 ---       \u2506 ---      \u2502\n\u2502 str        \u2506 ---        \u2506 ---        \u2506 i64    \u2506   \u2506 i64      \u2506 ---        \u2506 f64       \u2506 f64      \u2502\n\u2502            \u2506 i64        \u2506 u32        \u2506        \u2506   \u2506          \u2506 i64        \u2506           \u2506          \u2502\n\u255e\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u256a\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2561\n\u2502 Grinnell   \u2506 1          \u2506 10         \u2506 59563  \u2506 \u2026 \u2506 191044   \u2506 0          \u2506 1.0       \u2506 1.0      \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 2          \u2506 9          \u2506 89554  \u2506 \u2026 \u2506 172605   \u2506 166116     \u2506 1.727141  \u2506 1.039063 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 3          \u2506 8          \u2506 107323 \u2506 \u2026 \u2506 151543   \u2506 151052     \u2506 1.38351   \u2506 1.003251 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 4          \u2506 7          \u2506 106828 \u2506 \u2026 \u2506 129664   \u2506 131101     \u2506 1.135997  \u2506 0.989039 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 5          \u2506 6          \u2506 96179  \u2506 \u2026 \u2506 105967   \u2506 106851     \u2506 1.088687  \u2506 0.991727 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 6          \u2506 5          \u2506 82541  \u2506 \u2026 \u2506 86134    \u2506 86800      \u2506 1.05192   \u2506 0.992327 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 7          \u2506 4          \u2506 64703  \u2506 \u2026 \u2506 66151    \u2506 66159      \u2506 1.019475  \u2506 0.999879 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 8          \u2506 3          \u2506 48924  \u2506 \u2026 \u2506 49409    \u2506 49457      \u2506 1.008015  \u2506 0.999029 \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 9          \u2506 2          \u2506 32840  \u2506 \u2026 \u2506 33029    \u2506 32969      \u2506 1.006343  \u2506 1.00182  \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2502 Grinnell   \u2506 10         \u2506 1          \u2506 15785  \u2506 \u2026 \u2506 15915    \u2506 15908      \u2506 1.002413  \u2506 1.00044  \u2502\n\u2502 Mut Grp    \u2506            \u2506            \u2506        \u2506   \u2506          \u2506            \u2506           \u2506          \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518<\/pre><\/div>\n\n\n\n<p>We validated the result against R&#8217;s <code>ChainLadder<\/code> package for one insurer, Grinnell Mut Grp, and the incurred factors matched, see the linked notebook.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"442\" src=\"https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-1-1024x442.png\" alt=\"\" class=\"wp-image-2111\" srcset=\"https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-1-1024x442.png 1024w, https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-1-300x129.png 300w, https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-1-768x331.png 768w, https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-1.png 1050w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Chain-Ladder factors for the 5 largest companies of LoB other liability, plus Grinnell Mut Grp and Virginia Mut Ins Co.<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"2-a-pattern-that-doesnt-play-nice-\"><strong>A Pattern That Doesn&#8217;t Play Nice<\/strong><\/h2>\n\n\n\n<p>Zooming into the CL factors of individual companies, two stood out: the paid pattern of Virginia Mut Ins Co and the incurred pattern of Grinnell Mut Grp. Neither is monotonic \u2014 a red flag, since most commercial reserving tools only offer parametric curve fits that <em>are<\/em> strictly monotonic (to be fair, parametric curve fits are more for tail factor estimation).<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"576\" height=\"453\" src=\"https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-3.png\" alt=\"\" class=\"wp-image-2113\" srcset=\"https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-3.png 576w, https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-3-300x236.png 300w\" sizes=\"auto, (max-width: 576px) 100vw, 576px\" \/><\/figure>\n\n\n\n<p>Grinnell&#8217;s incurred CL factors rise above 1 early on, dip below 1 (case reserves getting released, maybe subrogation), then climb back above 1 with some zig-zag in later years. A parametric curve simply can&#8217;t represent that saddle-shaped pattern.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"3-enter-whittaker-henderson-\"><strong>Enter Whittaker-Henderson<\/strong><\/h2>\n\n\n\n<p><a href=\"https:\/\/en.wikipedia.org\/wiki\/Whittaker%E2%80%93Henderson_smoothing\">Whittaker-Henderson (WH) smoothing<\/a> is a non-parametric, penalized smoother \u2014 no functional form assumed, just a trade-off between fitting the data and penalizing roughness. That trade-off is controlled by two knobs: the penalty <code>order<\/code> (we use the standard <code>order=2<\/code>, i.e. penalizing curvature) and the penalty strength <code>lamb<\/code>, which we set by eye.<\/p>\n\n\n\n<p>It happens to be a perfect match here: development factors are a discrete-time signal with equal time steps, exactly what WH smoothing was designed for (it dates back to Georg Bohlmann in 1899 \u2014 arguably it should be called Bohlmann-Whittaker-Henderson). As of scipy 1.18, it ships out of the box as <code><a href=\"https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.signal.whittaker_henderson.html\">scipy.signal.whittaker_henderson<\/a><\/code> \u2014 full disclosure, I contributed that implementation, so I might be a little biased towards finding excuses to use it \ud83d\ude04<\/p>\n\n\n\n<div class=\"wp-block-codemirror-blocks-code-block code-block\"><pre class=\"CodeMirror\" data-setting=\"{&quot;showPanel&quot;:true,&quot;languageLabel&quot;:&quot;language&quot;,&quot;fullScreenButton&quot;:true,&quot;copyButton&quot;:true,&quot;mode&quot;:&quot;python&quot;,&quot;mime&quot;:&quot;text\/x-python&quot;,&quot;theme&quot;:&quot;material&quot;,&quot;lineNumbers&quot;:false,&quot;styleActiveLine&quot;:false,&quot;lineWrapping&quot;:false,&quot;readOnly&quot;:true,&quot;fileName&quot;:&quot;Python&quot;,&quot;language&quot;:&quot;Python&quot;,&quot;maxHeight&quot;:&quot;400px&quot;,&quot;modeName&quot;:&quot;python&quot;}\">from scipy.signal import whittaker_henderson\n\nf_smooth = whittaker_henderson(\n  signal=f_cl_inc, weights=previous_incurred, lamb=1e4\n).x<\/pre><\/div>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"576\" height=\"453\" src=\"https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image.png\" alt=\"\" class=\"wp-image-2125\" srcset=\"https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image.png 576w, https:\/\/lorentzen.ch\/wp-content\/uploads\/2026\/09\/image-300x236.png 300w\" sizes=\"auto, (max-width: 576px) 100vw, 576px\" \/><\/figure>\n\n\n\n<p>One neat property of WH smoothing: it preserves the weighted in-sample sum of the signal, so the smoothed factors reproduce the observed incurred losses exactly on the fitted range. Out-of-sample \u2014 i.e. for the not-yet-observed lower-right triangle \u2014 the smoothed and raw factors diverge, which is exactly where it matters for reserving.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"4-does-it-change-the-reserve-\"><strong>Does It Change the Reserve?<\/strong><\/h2>\n\n\n\n<p>Yes, a bit:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td>accident year<\/td><td class=\"has-text-align-right\" data-align=\"right\">reserve CL incurred<\/td><td class=\"has-text-align-right\" data-align=\"right\">reserve smoothed<\/td><\/tr><tr><td>1998<\/td><td class=\"has-text-align-right\" data-align=\"right\">0.0<\/td><td class=\"has-text-align-right\" data-align=\"right\">0.0<\/td><\/tr><tr><td>1999<\/td><td class=\"has-text-align-right\" data-align=\"right\">7.5<\/td><td class=\"has-text-align-right\" data-align=\"right\">20.6<\/td><\/tr><tr><td>2000<\/td><td class=\"has-text-align-right\" data-align=\"right\">37.2<\/td><td class=\"has-text-align-right\" data-align=\"right\">39.0<\/td><\/tr><tr><td>2001<\/td><td class=\"has-text-align-right\" data-align=\"right\">21.5<\/td><td class=\"has-text-align-right\" data-align=\"right\">38.3<\/td><\/tr><tr><td>2002<\/td><td class=\"has-text-align-right\" data-align=\"right\">23.3<\/td><td class=\"has-text-align-right\" data-align=\"right\">13.7<\/td><\/tr><tr><td>2003<\/td><td class=\"has-text-align-right\" data-align=\"right\">-124.9<\/td><td class=\"has-text-align-right\" data-align=\"right\">-118.9<\/td><\/tr><tr><td>2004<\/td><td class=\"has-text-align-right\" data-align=\"right\">-336.1<\/td><td class=\"has-text-align-right\" data-align=\"right\">-358.5<\/td><\/tr><tr><td>2005<\/td><td class=\"has-text-align-right\" data-align=\"right\">-522.0<\/td><td class=\"has-text-align-right\" data-align=\"right\">-525.9<\/td><\/tr><tr><td>2006<\/td><td class=\"has-text-align-right\" data-align=\"right\">-482.1<\/td><td class=\"has-text-align-right\" data-align=\"right\">-456.4<\/td><\/tr><tr><td>2007<\/td><td class=\"has-text-align-right\" data-align=\"right\">394.4<\/td><td class=\"has-text-align-right\" data-align=\"right\">398.2<\/td><\/tr><tr><td><strong>TOTAL<\/strong><\/td><td class=\"has-text-align-right\" data-align=\"right\"><strong>-981.1<\/strong><\/td><td class=\"has-text-align-right\" data-align=\"right\"><strong>-950.0<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>The total reserve shifts by about 3% (+31.1) \u2014 modest for the total, though individual accident years move more (some by double digits in percentage terms), since smoothing lets a factor&#8217;s neighbors pull it away from its own noisy ratio.<\/p>\n\n\n\n<p>If we compare against the incurred losses after all 10 years of development \u2014 the closest proxy we have to the true ultimate loss \u2014 both CL and the WH-smoothed CL turn out to overestimate it. The un-smoothed CL is only marginally closer.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td>incurred<\/td><td class=\"has-text-align-right\" data-align=\"right\">ultimate CL<\/td><td class=\"has-text-align-right\" data-align=\"right\">ultimate smoothed<\/td><\/tr><tr><td>189,901<\/td><td class=\"has-text-align-right\" data-align=\"right\">194,067<\/td><td class=\"has-text-align-right\" data-align=\"right\">194,098<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>So for this one company, smoothing made the development pattern easier to reason about, but it didn&#8217;t make the forecast more accurate \u2014 the difference (about 31) is small compared to the 4,582 standard error Mack&#8217;s method reports for this triangle, so neither method is clearly better here. A good reminder to check against a holdout whenever one is available.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"5-takeaways-\"><strong>Takeaways<\/strong><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>polars makes Chain-Ladder wrangling compact and fast, even across hundreds of companies at once.<\/li>\n\n\n\n<li>Volume-weighted CL factors computed in polars match R&#8217;s <code>ChainLadder<\/code> package exactly \u2014 always reassuring when switching tools.<\/li>\n\n\n\n<li>Whittaker-Henderson smoothing is a flexible alternative to parametric curve fitting whenever development factors are noisy or non-monotonic, and it&#8217;s now built into scipy. A smoother pattern isn&#8217;t automatically a more accurate one, though \u2014 always check against a holdout when you can.<\/li>\n<\/ul>\n\n\n\n<p>Natural next steps could be to add a tail factor, estimate reserve uncertainty (\u00e0 la Mack&#8217;s method), or let REML pick <code>lamb<\/code> automatically instead of choosing it by eye.<\/p>\n\n\n\n<p>The full notebook \u2014 all the polars code, the charts, and the R comparison \u2014 is on <a href=\"https:\/\/github.com\/lorentzenchr\/notebooks\/blob\/master\/blogposts\/2026-09-21%20Chain%20Ladder.ipynb\">GitHub<\/a>. This post as well as the notebook was AI reviewed.<\/p>\n\n\n\n<p>Spotted a bug, or have a favorite way to smooth development factors? Let me know in the comments!<br><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Chain-Ladder reserving in polars, validated against R and Whittaker-Henderson smoothing for a difficult development pattern.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[42,17],"tags":[43,45,46,6,5,44,47],"class_list":["post-2096","post","type-post","status-publish","format-standard","hentry","category-actuarial","category-programming","tag-actuarial","tag-chain-ladder","tag-polars","tag-python","tag-r","tag-reserving","tag-scipy"],"featured_image_src":null,"author_info":{"display_name":"Christian Lorentzen","author_link":"https:\/\/lorentzen.ch\/index.php\/author\/christian\/"},"_links":{"self":[{"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/posts\/2096","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/comments?post=2096"}],"version-history":[{"count":31,"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/posts\/2096\/revisions"}],"predecessor-version":[{"id":2134,"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/posts\/2096\/revisions\/2134"}],"wp:attachment":[{"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/media?parent=2096"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/categories?post=2096"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lorentzen.ch\/index.php\/wp-json\/wp\/v2\/tags?post=2096"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}