Claim Narratives Into Actuarial Variables

By Jonas Osman Abdelghafour.

Commentary on research published by the Casualty Actuarial Society in June 2026. The original paper, by Rob Lieberthal and Richard Tran, is available from the CAS and is well worth reading in full.

In June 2026 the Casualty Actuarial Society published research on applying large language models to unstructured claims data. In summary: the authors built a proof-of-concept two-stage framework that converts narrative claims information into 36 structured actuarial variables, separating document-level extraction from claim-level synthesis, and validated the extraction against independent clinical expert review using a human-in-the-loop approach. Open-source code and a synthetic dataset accompany the work.

That description is short, and it undersells what is interesting about the paper. The headline — "LLMs can read claim files" — is not the contribution. The contribution is the architecture, and specifically one design decision that most practitioners building similar tools get wrong.

Let me explain what that decision is and why it matters.

---

The Problem, For Anyone Who Hasn't Lived It

A claim file is mostly text. Adjuster notes, medical records, correspondence, legal filings, police reports, call transcripts, photographs with captions. A P&C insurer holds enormous quantities of it.

An actuarial model, meanwhile, consumes structured variables: injury type, body part, treatment category, litigation status, comorbidity indicators, return-to-work status. The gap between the two is bridged today by one of three things — a coding scheme applied by adjusters at first notice, a set of keyword rules, or nothing at all.

All three lose information. First-notice coding is captured before the facts are known and is rarely revised. Keyword rules are brittle: "no evidence of fracture" and "evidence of fracture" differ by one word and completely by meaning. And the third option, doing nothing, means the richest data the insurer holds never enters a model.

This is why reserving actuaries have long suspected that the residual variance in their models is partly readable — sitting in the file, in prose, invisible to the triangle.

Why Two Stages, and Why That Is the Whole Point

The intuitive way to build this is one stage: give the model the whole claim file, ask for the variables, get a table back.

That approach fails in production for reasons that are not obvious until you have watched it happen.

Context dilution. A large claim file can run to hundreds of pages. Even with a long context window, model attention degrades across very long inputs, and a detail on page 180 that contradicts page 12 may simply be missed.

Untraceable outputs. If the model returns "litigation status: represented," you cannot tell which document supported that. When it is wrong, you cannot find out why, and you cannot fix it systematically.

Correlated errors. A single misreading early in a long prompt propagates. One misidentified injury flows through severity, treatment and duration variables simultaneously, producing a coherent and entirely wrong claim record — which is far more dangerous than an obviously broken one.

The two-stage separation addresses all three:

Stage one — document-level extraction. Each document is processed independently. What does this medical report say about injury type? What does this adjuster note say about litigation? Each extraction is short, focused, and traceable to a source document.

Stage two — claim-level synthesis. The extracted facts are reconciled into a single claim record. This is where the conflicts get resolved: the initial report said soft tissue, the follow-up imaging said fracture, the later note said surgery. Which governs?

The separation matters because the two stages fail differently and therefore need different controls. Stage one errors are reading errors — testable against source documents, correctable with better prompting or extraction schemas. Stage two errors are reasoning errors — testable against expert judgement, correctable with better reconciliation rules.

Collapse them into one step and you get a single opaque error rate that tells you nothing about where to intervene. This is the same principle as trajectory validation in agentic systems: validate the sequence, not just the endpoint.

The Validation Design Deserves More Attention Than It Will Get

The research validated extraction against independent clinical expert review, human-in-the-loop.

That sentence is doing a lot of work, and it is the part most firms will skip when they build their own version.

Here is why it is hard. To validate an extraction system you need ground truth — a set of claims where you know the correct answer. For structured data, ground truth is cheap. For narrative extraction, ground truth is another expert reading the same documents, which is slow, expensive and itself variable. Two clinicians reading the same file will not always agree on injury classification.

This has a consequence practitioners consistently underestimate: your validation ceiling is the inter-rater reliability of your human experts. If two clinicians agree 85% of the time on a variable, an extraction system that agrees with one of them 85% of the time is performing at human level. Reporting that as "85% accurate" without the human baseline makes a strong result look mediocre — and, more dangerously, makes a genuinely poor result on an easy variable look acceptable.

If you take one operational point from the CAS work, take this: measure inter-rater reliability on your validation sample before you measure model performance. Without it, your accuracy figure has no denominator.

Where This Genuinely Changes Actuarial Work

Three applications, in ascending order of value and difficulty.

Reserving segmentation. The immediate use. Structured variables extracted from narrative allow case reserves to be grouped more homogeneously — separating claims that look identical in the triangle but differ in litigation status, comorbidity or treatment trajectory. Better homogeneity means lower process variance, which means tighter reserve ranges without any change in methodology.

Early identification of development drivers. Long-tail claims develop for reasons visible in the file long before they appear in paid or incurred data. Attorney involvement, treatment escalation, a comorbidity that predicts extended disability. Extracting these systematically at an early duration converts a lagging indicator into a leading one.

Ratemaking feature engineering. The hardest and most sensitive. Variables extracted from claim narrative can inform pricing — but the moment they do, they enter a regulated decision and attract the full apparatus of unfair discrimination scrutiny. An extracted variable that correlates with a protected characteristic is a rating factor problem regardless of how it was derived. This application needs a fairness assessment before it needs a lift chart.

Four Things to Get Right Before Deploying Something Like This

Version-pin the model and re-validate on change. Extraction behaviour is a property of a specific model version. A provider update can alter classification boundaries without any change to your code. If your validation is not tied to a pinned version, your evidence expires silently.

Keep the extraction schema in the model inventory. The schema — the 36 variables, their definitions and permitted values — is as much a modelling artefact as the algorithm. Definitional drift in a variable like "litigation status" will invalidate historical comparisons in exactly the way a changed reserving assumption would.

Retain the source citation. Every extracted value should carry a pointer to the document and passage that supported it. This is what makes the system auditable rather than merely accurate, and it costs almost nothing to implement at build time and a great deal to retrofit.

Test on adversarial inputs. Claim files contain text written by parties with financial interests in the outcome. Demand letters, provider narratives, submissions from represented claimants. An extraction system reading text that was drafted to be persuasive is operating in an adversarial environment, and prompt injection through submitted documents is a live risk in claims specifically.

The Broader Signal

The CAS has framed this work as part of equipping actuaries to lead in an AI-enabled risk landscape, with professional judgement and governance central rather than incidental. That framing is the right one, and it is worth noting what the research did not do: it did not automate a decision. It converted text into variables that an actuary then uses.

That is the correct place to start. The extraction layer is where AI adds the most value in actuarial work and creates the least governance exposure — because the output is an input, and every existing control on how actuaries use inputs still applies.

The firms that get value from this in 2026 will be the ones that build the extraction layer properly and leave the decision layer alone.

---

Key Takeaways

• The CAS-funded research converts narrative claims documents into 36 structured actuarial variables via a two-stage LLM framework, with open-source code and a synthetic dataset.
• Separating document-level extraction from claim-level synthesis is the key design decision: the two stages fail differently and require different controls.
• Validation ceiling is set by inter-rater reliability among human experts — measure the human baseline before reporting model accuracy.
• Highest-value applications are reserving segmentation and early identification of development drivers; ratemaking use triggers unfair discrimination scrutiny.
• Version-pin the model, inventory the extraction schema, retain source citations, and test against adversarial claimant-supplied documents.

---

Frequently Asked Questions

Can large language models be used for reserving? Not to set reserves directly, but increasingly to extract structured variables from unstructured claim documents that then feed conventional reserving methods. The CAS research demonstrates a proof-of-concept framework for this, converting narrative claims information into structured actuarial variables usable in reserving, ratemaking and claims management.

How do you validate an LLM extraction system for actuarial use? Through independent expert review of a sample, with human-in-the-loop comparison against the model's extractions. Critically, the inter-rater reliability of the human reviewers should be measured first, since it sets the practical ceiling on achievable agreement and provides the baseline against which model performance should be reported.

Is extracted claims data safe to use in pricing? It requires additional care. Once an extracted variable influences a rating decision it becomes a rating factor subject to unfair discrimination review, regardless of how it was derived. Fairness assessment and regulatory review should precede any pricing use, whereas reserving segmentation applications carry substantially lower regulatory exposure.

---

Jonas Osman Abdelghafour is the founder of Quantica Risk Model, which builds and independently validates actuarial and AI models for insurers. This article is commentary on publicly available CAS research; the original paper should be consulted directly.