Predictive Modelling: Actuarial Meets AI

By Jonas Osman Abdelghafour, actuary and quantitative risk expert.

Insurance pricing is one of the oldest applications of predictive modelling in commercial use. It is also, in the space of a decade, one of the most transformed. The pricing engine of a well-run personal-lines insurer today is a hybrid: an actuarial GLM structure that carries the interpretability and regulatory defensibility built up over decades, augmented by gradient-boosted or neural components that extract the signal the GLM leaves behind.

That hybrid is where the profession has landed, and it is the right place to be for now. What matters is how the two halves are integrated, how the resulting model is governed, and how the firm handles the regulatory scrutiny that follows any use of AI in a consumer-facing pricing process.

Why the GLM is still the backbone

The generalised linear model earned its place as the pricing standard for defensible reasons. It produces coefficients that can be read as multiplicative rating factors. It handles the exposure offset naturally through the log-link Poisson and Gamma structure. It integrates cleanly into rating engines, reinsurance treaties, and reserving GLMs of the same form. And it survives regulator questions because every rating factor traces to a specific coefficient with a specific standard error.

The critiques are also well-known. GLMs cannot capture non-linear interactions unless they are explicitly parameterised. They can be sensitive to correlated covariates in ways that machine-learning methods handle more gracefully. And they leave predictive lift on the table — routinely five to fifteen percent of Gini improvement — that gradient-boosted alternatives can extract.

The pragmatic response, adopted by most sophisticated pricing teams, is not to replace the GLM but to structure the modelling process so the GLM captures the interpretable core and a machine-learning component captures the residual.

The residual model pattern

The pattern that has become standard in personal-lines pricing looks like this. Fit a GLM on the exposure-adjusted target — claim frequency, severity, or a combined loss cost. Score the GLM out-of-sample. Fit a gradient-boosted model on the residual, using the same features plus additional ones the GLM structure cannot accommodate. Score the composite. Iterate.

Two design choices matter. First, the residual model must be constrained — usually through interaction-depth and learning-rate limits — so that it captures signal the GLM missed rather than re-fitting the signal the GLM already found. Second, the residual model's contribution must be monitored per rating factor. If the gradient-boosted component consistently up-weights or down-weights a specific segment, that is either a rating factor the GLM should absorb or a fairness concern that needs attention before deployment.

For a broader treatment of where machine learning helps in loss modelling, see the companion note on modern loss reserving. The AI and quantitative risk models service is how I structure this work with clients.

Feature engineering: still where the wins are

Practitioners new to insurance modelling often expect the algorithm to be where the lift comes from. Ten years of published pricing studies say the opposite: the largest gains come from features, not from methods.

Telematics data, external weather and traffic data, credit-adjacent signals where jurisdiction permits, and — increasingly — geospatial fixed-effects at postcode-plus-adjacency granularity, all deliver more predictive uplift than the choice between XGBoost and LightGBM ever will. The methodological choice becomes important once the feature set has stabilised.

The corresponding governance risk is that new features introduce protected-characteristic proxies without the modelling team noticing. Every new feature should trigger a fairness audit — not a checkbox, but an actual test that the feature's marginal effect does not disproportionately price groups whose price the regulator would object to. This is where the analytical work meets the enterprise risk management framework and, ultimately, the board.

Neural networks: warranted in narrow places

Deep learning in insurance pricing has been slower to arrive than in some other verticals, for good reasons. Structured tabular data of the kind pricing teams work with does not usually benefit from the representational power of neural networks in ways gradient-boosted trees cannot match. Where neural networks earn their keep is in unstructured data.

The three current use cases with real evidence behind them are damage-image assessment for motor claims (convolutional architectures fine-tuned on garage photographs), text extraction from claims notes and adjuster narratives (transformer-based models that turn free text into structured features usable by the pricing GLM), and time-series telematics feature learning (recurrent or attention-based models that summarise driving traces into pricing-usable features). In each case the neural component feeds the actuarial model — it does not replace it.

Governance that survives the regulator

Every European insurer using AI in a consumer-facing process is now living with three overlapping frameworks: the model-risk expectations that pre-date AI (broadly aligned with the US SR 11-7 discipline), the EU AI Act's classification of insurance pricing as high-risk, and — for life and pensions — the existing Solvency II model change and validation regime. These frameworks agree on more than they disagree.

Inventory. Every model in production has an owner, a purpose, a materiality tier, and a documented risk assessment. Machine-learning components inside a pricing chain are not exempt.

Documentation. For every model: purpose, data lineage, training and validation methodology, performance metrics, calibration, monitoring plan, and remediation triggers. Documentation cannot be generated after the fact when the model is challenged — it must be part of the deployment process.

Monitoring. Concept drift, calibration drift, feature drift, and — critically — fairness drift, all measured on a defined cadence, with pre-committed remediation thresholds. See the credit risk model monitoring dashboard note for a compatible pattern on the credit side.

Independent challenge. The same second-line model risk function that reviews GLMs reviews the machine-learning components, using the independent validation approach I use with clients. If the reviewer cannot follow the model, that is a documentation deficiency, not a limitation the reviewer has to accept.

Explainability. For any model that produces a rating decision affecting a consumer, an explanation of the drivers behind that decision must be recoverable. SHAP and related attribution methods are usually sufficient, provided they are computed at prediction time and stored, not reconstructed post-hoc.

Where the profession still gets it wrong

Three recurring failure modes are worth calling out.

Optimising for lift, not for business value. A model with a higher Gini index that renews at a lower retention rate is not a better model. Business-value backtests — expected loss ratio, expected renewal, expected lifetime value — need to sit alongside the statistical metrics. This is a governance point as much as a modelling one, and it is where the modelling team's incentives and the pricing committee's incentives need to be aligned.

Ignoring adverse selection dynamics. A pricing model that improves your loss ratio in year one can be destabilising the segment in year three, if competitors are pricing differently on the same signals. Portfolio-level monitoring of retention, competitor rate index, and mix change should be part of every pricing model's ongoing review — this connects to the wider ORSA scenario design conversation.

Underinvesting in the transition path. Replacing an old rating structure with a new pricing model is a business change, not a technical deployment. The transition path — parallel running, controlled rollout by geography or segment, defined kill-switch conditions — is often what determines whether the new model beds in or is reversed under commercial pressure.

Closing

Predictive modelling in insurance is neither a solved problem nor an AI-hype front line. It is a mature discipline that has absorbed two generations of methodological advance — GLMs, then gradient boosting, now selective use of neural networks — and it is now absorbing a third generation of governance and regulation around that same technical stack. The firms that navigate that governance layer well will be the ones that can deploy new modelling techniques quickly and defensibly. The firms that treat governance as a downstream compliance task will find themselves either falling behind or being forced by the regulator to reverse decisions after the fact.

If you are building or reviewing a hybrid pricing model, or preparing for AI Act scrutiny, see the AI and quantitative risk service line or contact me.